org.apache.crunch.util

Class CrunchTool

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.crunch.util.CrunchTool

All Implemented Interfaces:

Serializable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

SortExample

public abstract class CrunchTool
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool, Serializable

An extension of the Tool interface that creates a Pipeline instance and provides methods for working with the Pipeline from inside of the Tool's run method.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
CrunchTool()
CrunchTool(boolean inMemory)

Method Summary

Modifier and Type	Method and Description
PipelineResult	done()
void	enableDebug()
org.apache.hadoop.conf.Configuration	getConf()
<T> Iterable<T>	materialize(PCollection<T> pcollection)
<T> PCollection<T>	read(Source<T> source)
<K,V> PTable<K,V>	read(TableSource<K,V> tableSource)
PCollection<String>	readTextFile(String pathName)
PipelineResult	run()
PipelineExecution	runAsync()
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	write(PCollection<?> pcollection, Target target)
void	writeTextFile(PCollection<?> pcollection, String pathName)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.util.Tool

run

Constructor Detail

CrunchTool

public CrunchTool()

CrunchTool

public CrunchTool(boolean inMemory)

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Overrides:

setConf in class org.apache.hadoop.conf.Configured

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

Overrides:

getConf in class org.apache.hadoop.conf.Configured

enableDebug

public void enableDebug()

read

public <T> PCollection<T> read(Source<T> source)

read

public <K,V> PTable<K,V> read(TableSource<K,V> tableSource)

readTextFile

public PCollection<String> readTextFile(String pathName)

write

public void write(PCollection<?> pcollection,
                  Target target)

writeTextFile

public void writeTextFile(PCollection<?> pcollection,
                          String pathName)

materialize

public <T> Iterable<T> materialize(PCollection<T> pcollection)

run

public PipelineResult run()

runAsync

public PipelineExecution runAsync()

done

public PipelineResult done()