|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.crunch.DoFn<Pair<S,Iterable<T>>,Pair<S,T>> org.apache.crunch.CombineFn<S,T>
public abstract class CombineFn<S,T>
A special DoFn
implementation that converts an Iterable
of
values into a single value. If a CombineFn
instance is used on a
PGroupedTable
, the function will be applied to the output of the map
stage before the data is passed to the reducer, which can improve the runtime
of certain classes of jobs.
Note that the incoming Iterable
can only be used to create an
Iterator
once. Calling Iterable.iterator()
method a second
time will throw an IllegalStateException
.
Constructor Summary | |
---|---|
CombineFn()
|
Methods inherited from class org.apache.crunch.DoFn |
---|
cleanup, configure, initialize, process, scaleFactor, setContext |
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static CombineFn.AggregatorFactory<Long> SUM_LONGS
Aggregators.SUM_LONGS()
public static CombineFn.AggregatorFactory<Integer> SUM_INTS
Aggregators.SUM_INTS()
public static CombineFn.AggregatorFactory<Float> SUM_FLOATS
Aggregators.SUM_FLOATS()
public static CombineFn.AggregatorFactory<Double> SUM_DOUBLES
Aggregators.SUM_DOUBLES()
public static CombineFn.AggregatorFactory<BigInteger> SUM_BIGINTS
Aggregators.SUM_BIGINTS()
public static CombineFn.AggregatorFactory<Long> MAX_LONGS
Aggregators.MAX_LONGS()
public static CombineFn.AggregatorFactory<Integer> MAX_INTS
Aggregators.MAX_INTS()
public static CombineFn.AggregatorFactory<Float> MAX_FLOATS
Aggregators.MAX_FLOATS()
public static CombineFn.AggregatorFactory<Double> MAX_DOUBLES
Aggregators.MAX_DOUBLES()
public static CombineFn.AggregatorFactory<BigInteger> MAX_BIGINTS
Aggregators.MAX_BIGINTS()
public static CombineFn.AggregatorFactory<Long> MIN_LONGS
Aggregators.MIN_LONGS()
public static CombineFn.AggregatorFactory<Integer> MIN_INTS
Aggregators.MIN_INTS()
public static CombineFn.AggregatorFactory<Float> MIN_FLOATS
Aggregators.MIN_FLOATS()
public static CombineFn.AggregatorFactory<Double> MIN_DOUBLES
Aggregators.MIN_DOUBLES()
public static CombineFn.AggregatorFactory<BigInteger> MIN_BIGINTS
Aggregators.MIN_BIGINTS()
Constructor Detail |
---|
public CombineFn()
Method Detail |
---|
public static final <K,V> CombineFn<K,V> aggregator(CombineFn.Aggregator<V> aggregator)
Aggregators.toCombineFn(Aggregator)
public static final <K,V> CombineFn<K,V> aggregatorFactory(CombineFn.AggregatorFactory<V> aggregator)
PGroupedTable.combineValues(Aggregator)
which doesn't require a factory.
public static final <K,V1,V2> CombineFn<K,Pair<V1,V2>> pairAggregator(CombineFn.AggregatorFactory<V1> a1, CombineFn.AggregatorFactory<V2> a2)
Aggregators.pairAggregator(Aggregator, Aggregator)
public static final <K,A,B,C> CombineFn<K,Tuple3<A,B,C>> tripAggregator(CombineFn.AggregatorFactory<A> a1, CombineFn.AggregatorFactory<B> a2, CombineFn.AggregatorFactory<C> a3)
Aggregators.tripAggregator(Aggregator, Aggregator, Aggregator)
public static final <K,A,B,C,D> CombineFn<K,Tuple4<A,B,C,D>> quadAggregator(CombineFn.AggregatorFactory<A> a1, CombineFn.AggregatorFactory<B> a2, CombineFn.AggregatorFactory<C> a3, CombineFn.AggregatorFactory<D> a4)
Aggregators.quadAggregator(Aggregator, Aggregator, Aggregator, Aggregator)
public static final <K> CombineFn<K,TupleN> tupleAggregator(CombineFn.AggregatorFactory<?>... factories)
Aggregators.tupleAggregator(Aggregator...)
public static final <K> CombineFn<K,Long> SUM_LONGS()
Aggregators.SUM_LONGS()
public static final <K> CombineFn<K,Integer> SUM_INTS()
Aggregators.SUM_INTS()
public static final <K> CombineFn<K,Float> SUM_FLOATS()
Aggregators.SUM_FLOATS()
public static final <K> CombineFn<K,Double> SUM_DOUBLES()
Aggregators.SUM_DOUBLES()
public static final <K> CombineFn<K,BigInteger> SUM_BIGINTS()
Aggregators.SUM_BIGINTS()
public static final <K> CombineFn<K,Long> MAX_LONGS()
Aggregators.MAX_LONGS()
public static final <K> CombineFn<K,Long> MAX_LONGS(int n)
Aggregators.MAX_LONGS(int)
public static final <K> CombineFn<K,Integer> MAX_INTS()
Aggregators.MAX_INTS()
public static final <K> CombineFn<K,Integer> MAX_INTS(int n)
Aggregators.MAX_INTS(int)
public static final <K> CombineFn<K,Float> MAX_FLOATS()
Aggregators.MAX_FLOATS()
public static final <K> CombineFn<K,Float> MAX_FLOATS(int n)
Aggregators.MAX_FLOATS(int)
public static final <K> CombineFn<K,Double> MAX_DOUBLES()
Aggregators.MAX_DOUBLES()
public static final <K> CombineFn<K,Double> MAX_DOUBLES(int n)
Aggregators.MAX_DOUBLES(int)
public static final <K> CombineFn<K,BigInteger> MAX_BIGINTS()
Aggregators.MAX_BIGINTS()
public static final <K> CombineFn<K,BigInteger> MAX_BIGINTS(int n)
Aggregators.MAX_BIGINTS(int)
public static final <K> CombineFn<K,Long> MIN_LONGS()
Aggregators.MIN_LONGS()
public static final <K> CombineFn<K,Long> MIN_LONGS(int n)
Aggregators.MIN_LONGS(int)
public static final <K> CombineFn<K,Integer> MIN_INTS()
Aggregators.MIN_INTS()
public static final <K> CombineFn<K,Integer> MIN_INTS(int n)
Aggregators.MIN_INTS(int)
public static final <K> CombineFn<K,Float> MIN_FLOATS()
Aggregators.MIN_FLOATS()
public static final <K> CombineFn<K,Float> MIN_FLOATS(int n)
Aggregators.MIN_FLOATS(int)
public static final <K> CombineFn<K,Double> MIN_DOUBLES()
Aggregators.MIN_DOUBLES()
public static final <K> CombineFn<K,Double> MIN_DOUBLES(int n)
Aggregators.MIN_DOUBLES(int)
public static final <K> CombineFn<K,BigInteger> MIN_BIGINTS()
Aggregators.MIN_BIGINTS()
public static final <K> CombineFn<K,BigInteger> MIN_BIGINTS(int n)
Aggregators.MIN_BIGINTS(int)
public static final <K,V> CombineFn<K,V> FIRST_N(int n)
Aggregators.FIRST_N(int)
public static final <K,V> CombineFn<K,V> LAST_N(int n)
Aggregators.LAST_N(int)
public static final <K> CombineFn<K,String> STRING_CONCAT(String separator, boolean skipNull)
Aggregators.STRING_CONCAT(String, boolean)
separator
- the separator which will be appended between each stringskipNull
- define if we should skip null values. Throw
NullPointerException if set to false and there is a null
value.
public static final <K> CombineFn<K,String> STRING_CONCAT(String separator, boolean skipNull, long maxOutputLength, long maxInputLength)
Aggregators.STRING_CONCAT(String, boolean, long, long)
separator
- the separator which will be appended between each stringskipNull
- define if we should skip null values. Throw
NullPointerException if set to false and there is a null
value.maxOutputLength
- the maximum length of the output string. If it's set <= 0,
there is no limits. The number of characters of the output
string will be < maxOutputLength.maxInputLength
- the maximum length of the input strings. If it's set <= 0,
there is no limits. The number of characters of the int string
will be < maxInputLength to be concatenated.
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |