This project has retired. For details please refer to its Attic page.
Uses of Class org.apache.crunch.DoFn (Apache Crunch 0.7.0 API)

Uses of Class
org.apache.crunch.DoFn

Packages that use DoFn
org.apache.crunch Client-facing API and core abstractions. 
org.apache.crunch.contrib.bloomfilter Support for creating Bloom Filters. 
org.apache.crunch.fn Commonly used functions for manipulating collections. 
org.apache.crunch.lib Joining, sorting, aggregating, and other commonly used functionality. 
org.apache.crunch.lib.join Inner and outer joins on collections. 
org.apache.crunch.lib.sort   
org.apache.crunch.types Common functionality for business object serialization. 
 

Uses of DoFn in org.apache.crunch
 

Subclasses of DoFn in org.apache.crunch
 class CombineFn<S,T>
          A special DoFn implementation that converts an Iterable of values into a single value.
static class CombineFn.AggregatorCombineFn<K,V>
          Deprecated. Use the Aggregators.toCombineFn(org.apache.crunch.Aggregator) adapter
 class FilterFn<T>
          A DoFn for the common case of filtering the members of a PCollection based on a boolean condition.
static class FilterFn.AndFn<S>
          Deprecated. Use FilterFns.and(FilterFn...)
static class FilterFn.NotFn<S>
          Deprecated. Use FilterFns.not(FilterFn)
static class FilterFn.OrFn<S>
          Deprecated. Use FilterFns.or(FilterFn...)
 class MapFn<S,T>
          A DoFn for the common case of emitting exactly one value for each input record.
 

Methods in org.apache.crunch with parameters of type DoFn
<K,V> PTable<K,V>
PCollection.parallelDo(DoFn<S,Pair<K,V>> doFn, PTableType<K,V> type)
          Similar to the other parallelDo instance, but returns a PTable instance instead of a PCollection.
<T> PCollection<T>
PCollection.parallelDo(DoFn<S,T> doFn, PType<T> type)
          Applies the given doFn to the elements of this PCollection and returns a new PCollection that is the output of this processing.
<K,V> PTable<K,V>
PCollection.parallelDo(String name, DoFn<S,Pair<K,V>> doFn, PTableType<K,V> type)
          Similar to the other parallelDo instance, but returns a PTable instance instead of a PCollection.
<K,V> PTable<K,V>
PCollection.parallelDo(String name, DoFn<S,Pair<K,V>> doFn, PTableType<K,V> type, ParallelDoOptions options)
          Similar to the other parallelDo instance, but returns a PTable instance instead of a PCollection.
<T> PCollection<T>
PCollection.parallelDo(String name, DoFn<S,T> doFn, PType<T> type)
          Applies the given doFn to the elements of this PCollection and returns a new PCollection that is the output of this processing.
<T> PCollection<T>
PCollection.parallelDo(String name, DoFn<S,T> doFn, PType<T> type, ParallelDoOptions options)
          Applies the given doFn to the elements of this PCollection and returns a new PCollection that is the output of this processing.
 

Uses of DoFn in org.apache.crunch.contrib.bloomfilter
 

Subclasses of DoFn in org.apache.crunch.contrib.bloomfilter
 class BloomFilterFn<S>
          The class is responsible for generating keys that are used in a BloomFilter
 

Uses of DoFn in org.apache.crunch.fn
 

Subclasses of DoFn in org.apache.crunch.fn
 class CompositeMapFn<R,S,T>
           
 class ExtractKeyFn<K,V>
          Wrapper function for converting a MapFn into a key-value pair that is used to convert from a PCollection<V> to a PTable<K, V>.
 class IdentityFn<T>
           
 class MapKeysFn<K1,K2,V>
          Deprecated. Use PTable.mapKeys(org.apache.crunch.MapFn, org.apache.crunch.types.PType)
 class MapValuesFn<K,V1,V2>
          Deprecated. Use PTable.mapValues(org.apache.crunch.MapFn, org.apache.crunch.types.PType)
 class PairMapFn<K,V,S,T>
           
 

Uses of DoFn in org.apache.crunch.lib
 

Subclasses of DoFn in org.apache.crunch.lib
static class Aggregate.TopKCombineFn<K,V>
           
static class Aggregate.TopKFn<K,V>
           
 

Methods in org.apache.crunch.lib with parameters of type DoFn
static
<K,V1,V2,U,V>
PTable<U,V>
SecondarySort.sortAndApply(PTable<K,Pair<V1,V2>> input, DoFn<Pair<K,Iterable<Pair<V1,V2>>>,Pair<U,V>> doFn, PTableType<U,V> ptype)
          Perform a secondary sort on the given PTable instance and then apply a DoFn to the resulting sorted data to yield an output PTable<U, V>.
static
<K,V1,V2,U,V>
PTable<U,V>
SecondarySort.sortAndApply(PTable<K,Pair<V1,V2>> input, DoFn<Pair<K,Iterable<Pair<V1,V2>>>,Pair<U,V>> doFn, PTableType<U,V> ptype, int numReducers)
          Perform a secondary sort on the given PTable instance and then apply a DoFn to the resulting sorted data to yield an output PTable<U, V>, using the given number of reducers.
static
<K,V1,V2,T>
PCollection<T>
SecondarySort.sortAndApply(PTable<K,Pair<V1,V2>> input, DoFn<Pair<K,Iterable<Pair<V1,V2>>>,T> doFn, PType<T> ptype)
          Perform a secondary sort on the given PTable instance and then apply a DoFn to the resulting sorted data to yield an output PCollection<T>.
static
<K,V1,V2,T>
PCollection<T>
SecondarySort.sortAndApply(PTable<K,Pair<V1,V2>> input, DoFn<Pair<K,Iterable<Pair<V1,V2>>>,T> doFn, PType<T> ptype, int numReducers)
          Perform a secondary sort on the given PTable instance and then apply a DoFn to the resulting sorted data to yield an output PCollection<T>, using the given number of reducers.
 

Uses of DoFn in org.apache.crunch.lib.join
 

Subclasses of DoFn in org.apache.crunch.lib.join
 class FullOuterJoinFn<K,U,V>
          Used to perform the last step of an full outer join.
 class InnerJoinFn<K,U,V>
          Used to perform the last step of an inner join.
 class JoinFn<K,U,V>
          Represents a DoFn for performing joins.
 class LeftOuterJoinFn<K,U,V>
          Used to perform the last step of an left outer join.
 class RightOuterJoinFn<K,U,V>
          Used to perform the last step of an right outer join.
 

Methods in org.apache.crunch.lib.join with parameters of type DoFn
static
<K,U,V,T> PCollection<T>
OneToManyJoin.oneToManyJoin(PTable<K,U> left, PTable<K,V> right, DoFn<Pair<U,Iterable<V>>,T> postProcessFn, PType<T> ptype)
          Performs a join on two tables, where the left table only contains a single value per key.
 

Uses of DoFn in org.apache.crunch.lib.sort
 

Subclasses of DoFn in org.apache.crunch.lib.sort
static class SortFns.AvroGenericFn<V extends Tuple>
          Pulls a composite set of keys from an Avro GenericRecord instance.
static class SortFns.SingleKeyFn<V extends Tuple,K>
          Extracts a single indexed key from a Tuple instance.
static class SortFns.TupleKeyFn<V extends Tuple,K extends Tuple>
          Extracts a composite key from a Tuple instance.
 

Uses of DoFn in org.apache.crunch.types
 

Subclasses of DoFn in org.apache.crunch.types
static class PGroupedTableType.PairIterableMapFn<K,V>
           
 

Methods in org.apache.crunch.types that return DoFn
static
<M extends com.google.protobuf.Message>
DoFn<String,M>
Protos.lineParser(String sep, Class<M> msgClass)
           
 



Copyright © 2013 The Apache Software Foundation. All Rights Reserved.