Project Crunch has retired. For details please refer to its Attic page.
Channels (Apache Crunch 0.10.0 API)

org.apache.crunch.lib
Class Channels

java.lang.Object
  extended by org.apache.crunch.lib.Channels

public class Channels
extends Object

Utilities for splitting Pair instances emitted by DoFn into separate PCollection instances. A typical motivation for this might be to separate standard output from error output of a DoFn.

Author:
Brandon Inman

Constructor Summary
Channels()
           
 
Method Summary
static
<T,U> Pair<PCollection<T>,PCollection<U>>
split(PCollection<Pair<T,U>> pCollection)
          Splits a PCollection of any Pair of objects into a Pair of PCollection}, to allow for the output of a DoFn to be handled using separate channels.
static
<T,U> Pair<PCollection<T>,PCollection<U>>
split(PCollection<Pair<T,U>> pCollection, PType<T> firstPType, PType<U> secondPType)
          Splits a PCollection of any Pair of objects into a Pair of PCollection}, to allow for the output of a DoFn to be handled using separate channels.
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

Channels

public Channels()
Method Detail

split

public static <T,U> Pair<PCollection<T>,PCollection<U>> split(PCollection<Pair<T,U>> pCollection)
Splits a PCollection of any Pair of objects into a Pair of PCollection}, to allow for the output of a DoFn to be handled using separate channels.

Parameters:
pCollection - The PCollection to split

split

public static <T,U> Pair<PCollection<T>,PCollection<U>> split(PCollection<Pair<T,U>> pCollection,
                                                              PType<T> firstPType,
                                                              PType<U> secondPType)
Splits a PCollection of any Pair of objects into a Pair of PCollection}, to allow for the output of a DoFn to be handled using separate channels.

Parameters:
pCollection - The PCollection to split
firstPType - The PType for the first collection
secondPType - The PType for the second collection
Returns:
Pair of PCollection


Copyright © 2014 The Apache Software Foundation. All Rights Reserved.