This project has retired. For details please refer to its Attic page.

org.apache.crunch.lib
Class Channels

java.lang.Object
  org.apache.crunch.lib.Channels

public class Channels
extends Object

Utilities for splitting Pair instances emitted by DoFn into separate PCollection instances. A typical motivation for this might be to separate standard output from error output of a DoFn.

Author:

Brandon Inman

Constructor Summary

Channels()

Method Summary

static <T,U> Pair<PCollection<T>,PCollection<U>>	split(PCollection<Pair<T,U>> pCollection) Splits a PCollection of any Pair of objects into a Pair of PCollection}, to allow for the output of a DoFn to be handled using separate channels.
static <T,U> Pair<PCollection<T>,PCollection<U>>	split(PCollection<Pair<T,U>> pCollection, PType<T> firstPType, PType<U> secondPType) Splits a PCollection of any Pair of objects into a Pair of PCollection}, to allow for the output of a DoFn to be handled using separate channels.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Channels

public Channels()

Method Detail

split

public static <T,U> Pair<PCollection<T>,PCollection<U>> split(PCollection<Pair<T,U>> pCollection)

Splits a PCollection of any Pair of objects into a Pair of PCollection}, to allow for the output of a DoFn to be handled using separate channels.

Parameters:

pCollection - The PCollection to split

split

public static <T,U> Pair<PCollection<T>,PCollection<U>> split(PCollection<Pair<T,U>> pCollection,
                                                              PType<T> firstPType,
                                                              PType<U> secondPType)

Splits a PCollection of any Pair of objects into a Pair of PCollection}, to allow for the output of a DoFn to be handled using separate channels.

Parameters:

pCollection - The PCollection to split

firstPType - The PType for the first collection

secondPType - The PType for the second collection

Returns:

Pair of PCollection