Package | Description |
---|---|
org.apache.crunch |
Client-facing API and core abstractions.
|
org.apache.crunch.impl.mem |
In-memory Pipeline implementation for rapid prototyping and testing.
|
org.apache.crunch.impl.mr |
A Pipeline implementation that runs on Hadoop MapReduce.
|
org.apache.crunch.io |
Data input and output for Pipelines.
|
org.apache.crunch.util |
An assorted set of utilities.
|
Modifier and Type | Interface and Description |
---|---|
interface |
SourceTarget<T>
An interface for classes that implement both the
Source and the
Target interfaces. |
interface |
TableSourceTarget<K,V>
An interface for classes that implement both the
TableSource and the
Target interfaces. |
Modifier and Type | Method and Description |
---|---|
void |
Pipeline.write(PCollection<?> collection,
Target target)
Write the given collection to the given target on the next pipeline run.
|
void |
Pipeline.write(PCollection<?> collection,
Target target,
Target.WriteMode writeMode)
Write the contents of the
PCollection to the given Target ,
using the storage format specified by the target and the given
WriteMode for cases where the referenced Target
already exists. |
PTable<K,V> |
PTable.write(Target target)
Writes this
PTable to the given Target . |
PCollection<S> |
PCollection.write(Target target)
Write the contents of this
PCollection to the given Target ,
using the storage format specified by the target. |
PTable<K,V> |
PTable.write(Target target,
Target.WriteMode writeMode)
Writes this
PTable to the given Target , using the
given Target.WriteMode to handle existing targets. |
PCollection<S> |
PCollection.write(Target target,
Target.WriteMode writeMode)
Write the contents of this
PCollection to the given Target ,
using the given Target.WriteMode to handle existing
targets. |
Modifier and Type | Method and Description |
---|---|
void |
MemPipeline.write(PCollection<?> collection,
Target target) |
void |
MemPipeline.write(PCollection<?> collection,
Target target,
Target.WriteMode writeMode) |
Modifier and Type | Method and Description |
---|---|
void |
MRPipeline.write(PCollection<?> pcollection,
Target target) |
void |
MRPipeline.write(PCollection<?> pcollection,
Target target,
Target.WriteMode writeMode) |
Modifier and Type | Interface and Description |
---|---|
interface |
MapReduceTarget |
interface |
PathTarget
A target whose output goes to a given path on a file system.
|
interface |
ReadableSourceTarget<T>
An interface that indicates that a
SourceTarget instance can be read
into the local client. |
Modifier and Type | Class and Description |
---|---|
class |
PathTargetImpl |
Modifier and Type | Method and Description |
---|---|
static Target |
To.avroFile(org.apache.hadoop.fs.Path path)
Creates a
Target at the given Path that writes data to
Avro files. |
static Target |
To.avroFile(String pathName)
Creates a
Target at the given path name that writes data to
Avro files. |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
To.formattedFile(org.apache.hadoop.fs.Path path,
Class<? extends org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>> formatClass)
Creates a
Target at the given Path that writes data to
a custom FileOutputFormat . |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
To.formattedFile(String pathName,
Class<? extends org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>> formatClass)
Creates a
Target at the given path name that writes data to
a custom FileOutputFormat . |
static Target |
To.sequenceFile(org.apache.hadoop.fs.Path path)
Creates a
Target at the given Path that writes data to
SequenceFiles. |
static Target |
To.sequenceFile(String pathName)
Creates a
Target at the given path name that writes data to
SequenceFiles. |
static Target |
To.textFile(org.apache.hadoop.fs.Path path)
Creates a
Target at the given Path that writes data to
text files. |
static Target |
To.textFile(String pathName)
Creates a
Target at the given path name that writes data to
text files. |
Modifier and Type | Method and Description |
---|---|
boolean |
OutputHandler.configure(Target target,
PType<?> ptype) |
Modifier and Type | Method and Description |
---|---|
void |
CrunchTool.write(PCollection<?> pcollection,
Target target) |
Copyright © 2013 The Apache Software Foundation. All Rights Reserved.