| Package | Description |
|---|---|
| org.apache.crunch |
Client-facing API and core abstractions.
|
| org.apache.crunch.impl.mem |
In-memory Pipeline implementation for rapid prototyping and testing.
|
| org.apache.crunch.impl.mr |
A Pipeline implementation that runs on Hadoop MapReduce.
|
| org.apache.crunch.io |
Data input and output for Pipelines.
|
| org.apache.crunch.util |
An assorted set of utilities.
|
| Modifier and Type | Interface and Description |
|---|---|
interface |
SourceTarget<T>
An interface for classes that implement both the
Source and the
Target interfaces. |
interface |
TableSourceTarget<K,V>
An interface for classes that implement both the
TableSource and the
Target interfaces. |
| Modifier and Type | Method and Description |
|---|---|
void |
Pipeline.write(PCollection<?> collection,
Target target)
Write the given collection to the given target on the next pipeline run.
|
void |
Pipeline.write(PCollection<?> collection,
Target target,
Target.WriteMode writeMode)
Write the contents of the
PCollection to the given Target,
using the storage format specified by the target and the given
WriteMode for cases where the referenced Target
already exists. |
PTable<K,V> |
PTable.write(Target target)
Writes this
PTable to the given Target. |
PCollection<S> |
PCollection.write(Target target)
Write the contents of this
PCollection to the given Target,
using the storage format specified by the target. |
PTable<K,V> |
PTable.write(Target target,
Target.WriteMode writeMode)
Writes this
PTable to the given Target, using the
given Target.WriteMode to handle existing targets. |
PCollection<S> |
PCollection.write(Target target,
Target.WriteMode writeMode)
Write the contents of this
PCollection to the given Target,
using the given Target.WriteMode to handle existing
targets. |
| Modifier and Type | Method and Description |
|---|---|
void |
MemPipeline.write(PCollection<?> collection,
Target target) |
void |
MemPipeline.write(PCollection<?> collection,
Target target,
Target.WriteMode writeMode) |
| Modifier and Type | Method and Description |
|---|---|
void |
MRPipeline.write(PCollection<?> pcollection,
Target target) |
void |
MRPipeline.write(PCollection<?> pcollection,
Target target,
Target.WriteMode writeMode) |
| Modifier and Type | Interface and Description |
|---|---|
interface |
MapReduceTarget |
interface |
PathTarget
A target whose output goes to a given path on a file system.
|
interface |
ReadableSourceTarget<T>
An interface that indicates that a
SourceTarget instance can be read
into the local client. |
| Modifier and Type | Class and Description |
|---|---|
class |
PathTargetImpl |
| Modifier and Type | Method and Description |
|---|---|
static Target |
To.avroFile(org.apache.hadoop.fs.Path path)
Creates a
Target at the given Path that writes data to
Avro files. |
static Target |
To.avroFile(String pathName)
Creates a
Target at the given path name that writes data to
Avro files. |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
To.formattedFile(org.apache.hadoop.fs.Path path,
Class<? extends org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>> formatClass)
Creates a
Target at the given Path that writes data to
a custom FileOutputFormat. |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
To.formattedFile(String pathName,
Class<? extends org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>> formatClass)
Creates a
Target at the given path name that writes data to
a custom FileOutputFormat. |
static Target |
To.sequenceFile(org.apache.hadoop.fs.Path path)
Creates a
Target at the given Path that writes data to
SequenceFiles. |
static Target |
To.sequenceFile(String pathName)
Creates a
Target at the given path name that writes data to
SequenceFiles. |
static Target |
To.textFile(org.apache.hadoop.fs.Path path)
Creates a
Target at the given Path that writes data to
text files. |
static Target |
To.textFile(String pathName)
Creates a
Target at the given path name that writes data to
text files. |
| Modifier and Type | Method and Description |
|---|---|
boolean |
OutputHandler.configure(Target target,
PType<?> ptype) |
| Modifier and Type | Method and Description |
|---|---|
void |
CrunchTool.write(PCollection<?> pcollection,
Target target) |
Copyright © 2013 The Apache Software Foundation. All Rights Reserved.