org.apache.crunch

Class CombineFn<S,T>

java.lang.Object

org.apache.crunch.DoFn<Pair<S,Iterable<T>>,Pair<S,T>>

org.apache.crunch.CombineFn<S,T>

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

Aggregate.TopKCombineFn

public abstract class CombineFn<S,T>
extends DoFn<Pair<S,Iterable<T>>,Pair<S,T>>

A special DoFn implementation that converts an Iterable of values into a single value. If a CombineFn instance is used on a PGroupedTable, the function will be applied to the output of the map stage before the data is passed to the reducer, which can improve the runtime of certain classes of jobs.

Note that the incoming Iterable can only be used to create an Iterator once. Calling Iterable.iterator() method a second time will throw an IllegalStateException.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
CombineFn()

Method Summary

Methods inherited from class org.apache.crunch.DoFn

cleanup, configure, disableDeepCopy, initialize, process, scaleFactor, setConfiguration, setContext

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CombineFn

public CombineFn()