Package | Description |
---|---|
org.apache.crunch |
Client-facing API and core abstractions.
|
org.apache.crunch.impl.mr |
A Pipeline implementation that runs on Hadoop MapReduce.
|
org.apache.crunch.io |
Data input and output for Pipelines.
|
org.apache.crunch.types |
Common functionality for business object serialization.
|
org.apache.crunch.types.avro |
Business object serialization using Apache Avro.
|
org.apache.crunch.types.writable |
Business object serialization using Hadoop's Writables framework.
|
Modifier and Type | Interface and Description |
---|---|
interface |
TableSourceTarget<K,V>
An interface for classes that implement both the
TableSource and the
Target interfaces. |
Modifier and Type | Method and Description |
---|---|
<T> SourceTarget<T> |
Target.asSourceTarget(PType<T> ptype)
Attempt to create the
SourceTarget type that corresponds to this Target
for the given PType , if possible. |
Modifier and Type | Method and Description |
---|---|
Set<SourceTarget<?>> |
ParallelDoOptions.getSourceTargets() |
Modifier and Type | Method and Description |
---|---|
ParallelDoOptions.Builder |
ParallelDoOptions.Builder.sourceTargets(SourceTarget<?>... sourceTargets) |
Modifier and Type | Method and Description |
---|---|
<T> SourceTarget<T> |
MRPipeline.createIntermediateOutput(PType<T> ptype) |
Modifier and Type | Interface and Description |
---|---|
interface |
ReadableSourceTarget<T>
An interface that indicates that a
SourceTarget instance can be read
into the local client. |
Modifier and Type | Method and Description |
---|---|
static <T> SourceTarget<T> |
At.avroFile(org.apache.hadoop.fs.Path path,
AvroType<T> avroType)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given Path . |
static <T extends org.apache.avro.specific.SpecificRecord> |
At.avroFile(org.apache.hadoop.fs.Path path,
Class<T> avroClass)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given Path . |
static <T> SourceTarget<T> |
At.avroFile(String pathName,
AvroType<T> avroType)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given path name. |
static <T extends org.apache.avro.specific.SpecificRecord> |
At.avroFile(String pathName,
Class<T> avroClass)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given path name. |
static <T extends org.apache.hadoop.io.Writable> |
At.sequenceFile(org.apache.hadoop.fs.Path path,
Class<T> valueClass)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given Path
from the value field of each key-value pair in the SequenceFile(s). |
static <T> SourceTarget<T> |
At.sequenceFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given Path
from the value field of each key-value pair in the SequenceFile(s). |
static <T extends org.apache.hadoop.io.Writable> |
At.sequenceFile(String pathName,
Class<T> valueClass)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given path name
from the value field of each key-value pair in the SequenceFile(s). |
static <T> SourceTarget<T> |
At.sequenceFile(String pathName,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given path name
from the value field of each key-value pair in the SequenceFile(s). |
static SourceTarget<String> |
At.textFile(org.apache.hadoop.fs.Path path)
Creates a
SourceTarget<String> instance for the text file(s) at the given Path . |
static <T> SourceTarget<T> |
At.textFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
SourceTarget<T> instance for the text file(s) at the given Path using
the provided PType<T> to convert the input text. |
static SourceTarget<String> |
At.textFile(String pathName)
Creates a
SourceTarget<String> instance for the text file(s) at the given path name. |
static <T> SourceTarget<T> |
At.textFile(String pathName,
PType<T> ptype)
Creates a
SourceTarget<T> instance for the text file(s) at the given path name using
the provided PType<T> to convert the input text. |
Modifier and Type | Method and Description |
---|---|
SourceTarget<T> |
PType.getDefaultFileSource(org.apache.hadoop.fs.Path path)
Returns a
SourceTarget that is able to read/write data using the serialization format
specified by this PType . |
SourceTarget<Pair<K,Iterable<V>>> |
PGroupedTableType.getDefaultFileSource(org.apache.hadoop.fs.Path path) |
Modifier and Type | Method and Description |
---|---|
SourceTarget<T> |
AvroType.getDefaultFileSource(org.apache.hadoop.fs.Path path) |
Modifier and Type | Method and Description |
---|---|
SourceTarget<T> |
WritableType.getDefaultFileSource(org.apache.hadoop.fs.Path path) |
Copyright © 2013 The Apache Software Foundation. All Rights Reserved.