org.apache.crunch.lib

Class PTables

java.lang.Object

org.apache.crunch.lib.PTables

public class PTables
extends Object

Methods for performing common operations on PTables.

Constructor Summary

Constructors

Constructor and Description
PTables()

Method Summary

Modifier and Type	Method and Description
static <K,V> PTable<K,V>	asPTable(PCollection<Pair<K,V>> pcollect) Convert the given PCollection<Pair<K, V>> to a PTable<K, V>.
static <K,V> Pair<K,V>	getDetachedValue(PTableType<K,V> tableType, Pair<K,V> value) Create a detached value for a table Pair.
static <K,V> Pair<K,Iterable<V>>	getGroupedDetachedValue(PGroupedTableType<K,V> groupedTableType, Pair<K,Iterable<V>> value) Created a detached value for a PGroupedTable value.
static <K,V> PCollection<K>	keys(PTable<K,V> ptable) Extract the keys from the given PTable<K, V> as a PCollection<K>.
static <K1,K2,V> PTable<K2,V>	mapKeys(PTable<K1,V> ptable, MapFn<K1,K2> mapFn, PType<K2> ptype) Maps a PTable<K1, V> to a PTable<K2, V> using the given MapFn<K1, K2> on the keys of the PTable.
static <K1,K2,V> PTable<K2,V>	mapKeys(String name, PTable<K1,V> ptable, MapFn<K1,K2> mapFn, PType<K2> ptype) Maps a PTable<K1, V> to a PTable<K2, V> using the given MapFn<K1, K2> on the keys of the PTable.
static <K,U,V> PTable<K,V>	mapValues(PGroupedTable<K,U> ptable, MapFn<Iterable<U>,V> mapFn, PType<V> ptype) An analogue of the mapValues function for PGroupedTable<K, U> collections.
static <K,U,V> PTable<K,V>	mapValues(PTable<K,U> ptable, MapFn<U,V> mapFn, PType<V> ptype) Maps a PTable<K, U> to a PTable<K, V> using the given MapFn<U, V> on the values of the PTable.
static <K,U,V> PTable<K,V>	mapValues(String name, PGroupedTable<K,U> ptable, MapFn<Iterable<U>,V> mapFn, PType<V> ptype) An analogue of the mapValues function for PGroupedTable<K, U> collections.
static <K,U,V> PTable<K,V>	mapValues(String name, PTable<K,U> ptable, MapFn<U,V> mapFn, PType<V> ptype) Maps a PTable<K, U> to a PTable<K, V> using the given MapFn<U, V> on the values of the PTable.
static <K,V> PTable<V,K>	swapKeyValue(PTable<K,V> table) Swap the key and value part of a table.
static <K,V> PCollection<V>	values(PTable<K,V> ptable) Extract the values from the given PTable<K, V> as a PCollection<V>.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PTables

public PTables()

Method Detail

asPTable

public static <K,V> PTable<K,V> asPTable(PCollection<Pair<K,V>> pcollect)

Convert the given PCollection<Pair<K, V>> to a PTable<K, V>.

Parameters:

pcollect - The PCollection to convert

Returns:

A PTable that contains the same data as the input PCollection

mapKeys

public static <K1,K2,V> PTable<K2,V> mapKeys(PTable<K1,V> ptable,
                                             MapFn<K1,K2> mapFn,
                                             PType<K2> ptype)

Maps a PTable<K1, V> to a PTable<K2, V> using the given MapFn<K1, K2> on the keys of the PTable.

Parameters:

ptable - The PTable to be mapped

mapFn - The mapping function

ptype - The PType for the returned keys

Returns:

A new PTable<K2, V> instance

mapKeys

public static <K1,K2,V> PTable<K2,V> mapKeys(String name,
                                             PTable<K1,V> ptable,
                                             MapFn<K1,K2> mapFn,
                                             PType<K2> ptype)

Maps a PTable<K1, V> to a PTable<K2, V> using the given MapFn<K1, K2> on the keys of the PTable.

Parameters:

name - The name of the transform

ptable - The PTable to be mapped

mapFn - The mapping function

ptype - The PType for the returned keys

Returns:

A new PTable<K2, V> instance

mapValues

public static <K,U,V> PTable<K,V> mapValues(PTable<K,U> ptable,
                                            MapFn<U,V> mapFn,
                                            PType<V> ptype)

Maps a PTable<K, U> to a PTable<K, V> using the given MapFn<U, V> on the values of the PTable.

Parameters:

ptable - The PTable to be mapped

mapFn - The mapping function

ptype - The PType for the returned values

Returns:

A new PTable<K, V> instance

mapValues

public static <K,U,V> PTable<K,V> mapValues(String name,
                                            PTable<K,U> ptable,
                                            MapFn<U,V> mapFn,
                                            PType<V> ptype)

Maps a PTable<K, U> to a PTable<K, V> using the given MapFn<U, V> on the values of the PTable.

Parameters:

name - The name of the transform

ptable - The PTable to be mapped

mapFn - The mapping function

ptype - The PType for the returned values

Returns:

A new PTable<K, V> instance

mapValues

public static <K,U,V> PTable<K,V> mapValues(PGroupedTable<K,U> ptable,
                                            MapFn<Iterable<U>,V> mapFn,
                                            PType<V> ptype)

An analogue of the mapValues function for PGroupedTable<K, U> collections.

Parameters:

ptable - The PGroupedTable to be mapped

mapFn - The mapping function

ptype - The PType for the returned values

Returns:

A new PTable<K, V> instance

mapValues

public static <K,U,V> PTable<K,V> mapValues(String name,
                                            PGroupedTable<K,U> ptable,
                                            MapFn<Iterable<U>,V> mapFn,
                                            PType<V> ptype)

An analogue of the mapValues function for PGroupedTable<K, U> collections.

Parameters:

name - The name of the operation

ptable - The PGroupedTable to be mapped

mapFn - The mapping function

ptype - The PType for the returned values

Returns:

A new PTable<K, V> instance

keys

public static <K,V> PCollection<K> keys(PTable<K,V> ptable)

Extract the keys from the given PTable<K, V> as a PCollection<K>.

Parameters:

ptable - The PTable

Returns:

A PCollection<K>

values

public static <K,V> PCollection<V> values(PTable<K,V> ptable)

Extract the values from the given PTable<K, V> as a PCollection<V>.

Parameters:

ptable - The PTable

Returns:

A PCollection<V>

getDetachedValue

public static <K,V> Pair<K,V> getDetachedValue(PTableType<K,V> tableType,
                                               Pair<K,V> value)

Create a detached value for a table Pair.

Parameters:

tableType - The table type

value - The value from which a detached value is to be created

Returns:

The detached value

See Also:

PType.getDetachedValue(Object)

getGroupedDetachedValue

public static <K,V> Pair<K,Iterable<V>> getGroupedDetachedValue(PGroupedTableType<K,V> groupedTableType,
                                                                Pair<K,Iterable<V>> value)

Created a detached value for a PGroupedTable value.

Parameters:

groupedTableType - The grouped table type

value - The value from which a detached value is to be created

Returns:

The detached value

See Also:

PType.getDetachedValue(Object)

swapKeyValue

public static <K,V> PTable<V,K> swapKeyValue(PTable<K,V> table)

Swap the key and value part of a table. The original PTypes are used in the opposite order

Type Parameters:

K - Key type (will become value type)

V - Value type (will become key type)

Parameters:

table - PTable to process

Returns:

PType<V, K> containing the same data as the original