org.apache.crunch

Interface PGroupedTable<K,V>

All Superinterfaces:

PCollection<Pair<K,Iterable<V>>>

All Known Implementing Classes:

BaseGroupedTable, PGroupedTableImpl

public interface PGroupedTable<K,V>
extends PCollection<Pair<K,Iterable<V>>>

The Crunch representation of a grouped PTable, which corresponds to the output of the shuffle phase of a MapReduce job.

Method Summary

Methods

Modifier and Type	Method and Description
PTable<K,V>	combineValues(Aggregator<V> aggregator) Combine the values in each group using the given Aggregator.
PTable<K,V>	combineValues(Aggregator<V> combineAggregator, Aggregator<V> reduceAggregator) Combine and reduces the values in each group using the given Aggregator instances.
PTable<K,V>	combineValues(CombineFn<K,V> combineFn) Combines the values of this grouping using the given CombineFn.
PTable<K,V>	combineValues(CombineFn<K,V> combineFn, CombineFn<K,V> reduceFn) Combines and reduces the values of this grouping using the given CombineFn instances.
PGroupedTableType<K,V>	getGroupedTableType() Return the PGroupedTableType containing serialization information for this PGroupedTable.
<U> PTable<K,U>	mapValues(MapFn<Iterable<V>,U> mapFn, PType<U> ptype) Maps the Iterable<V> elements of each record to a new type.
<U> PTable<K,U>	mapValues(String name, MapFn<Iterable<V>,U> mapFn, PType<U> ptype) Maps the Iterable<V> elements of each record to a new type.
PTable<K,V>	ungroup() Convert this grouping back into a multimap.

Methods inherited from interface org.apache.crunch.PCollection

aggregate, asCollection, asReadable, by, by, cache, cache, count, filter, filter, first, getName, getPipeline, getPType, getSize, getTypeFamily, length, materialize, max, min, parallelDo, parallelDo, parallelDo, parallelDo, parallelDo, parallelDo, sequentialDo, union, union, write, write

Method Detail

combineValues

PTable<K,V> combineValues(CombineFn<K,V> combineFn)

Combines the values of this grouping using the given CombineFn.

Parameters:

combineFn - The combiner function

Returns:

A PTable where each key has a single value

combineValues

PTable<K,V> combineValues(CombineFn<K,V> combineFn,
                        CombineFn<K,V> reduceFn)

Combines and reduces the values of this grouping using the given CombineFn instances.

Parameters:

combineFn - The combiner function during the combine phase

reduceFn - The combiner function during the reduce phase

Returns:

A PTable where each key has a single value

combineValues

PTable<K,V> combineValues(Aggregator<V> aggregator)

Combine the values in each group using the given Aggregator.

Parameters:

aggregator - The function to use

Returns:

A PTable where each group key maps to an aggregated value. Group keys may be repeated if an aggregator returns more than one value.

combineValues

PTable<K,V> combineValues(Aggregator<V> combineAggregator,
                        Aggregator<V> reduceAggregator)

Combine and reduces the values in each group using the given Aggregator instances.

Parameters:

combineAggregator - The aggregator to use during the combine phase

reduceAggregator - The aggregator to use during the reduce phase

Returns:

A PTable where each group key maps to an aggregated value. Group keys may be repeated if an aggregator returns more than one value.

mapValues

<U> PTable<K,U> mapValues(MapFn<Iterable<V>,U> mapFn,
                        PType<U> ptype)

Maps the Iterable<V> elements of each record to a new type. Just like any parallelDo operation on a PGroupedTable, this may only be called once.

Parameters:

mapFn - The mapping function

ptype - The serialization information for the returned data

Returns:

A new PTable instance

mapValues

<U> PTable<K,U> mapValues(String name,
                        MapFn<Iterable<V>,U> mapFn,
                        PType<U> ptype)

Maps the Iterable<V> elements of each record to a new type. Just like any parallelDo operation on a PGroupedTable, this may only be called once.

Parameters:

name - A name for this operation

mapFn - The mapping function

ptype - The serialization information for the returned data

Returns:

A new PTable instance

ungroup

PTable<K,V> ungroup()

Convert this grouping back into a multimap.

Returns:

an ungrouped version of the data in this PGroupedTable.

getGroupedTableType

PGroupedTableType<K,V> getGroupedTableType()

Return the PGroupedTableType containing serialization information for this PGroupedTable.