Package | Description |
---|---|
org.apache.crunch |
Client-facing API and core abstractions.
|
org.apache.crunch.impl.dist | |
org.apache.crunch.impl.dist.collect | |
org.apache.crunch.io |
Data input and output for Pipelines.
|
org.apache.crunch.util |
An assorted set of utilities.
|
Modifier and Type | Interface and Description |
---|---|
interface |
TableSourceTarget<K,V>
An interface for classes that implement both the
TableSource and the
Target interfaces. |
Modifier and Type | Method and Description |
---|---|
<T> SourceTarget<T> |
Target.asSourceTarget(PType<T> ptype)
Attempt to create the
SourceTarget type that corresponds to this Target
for the given PType , if possible. |
SourceTarget<T> |
SourceTarget.conf(String key,
String value)
Adds the given key-value pair to the
Configuration instance(s) that are used to
read and write this SourceTarget<T> . |
Modifier and Type | Method and Description |
---|---|
Set<SourceTarget<?>> |
ReadableData.getSourceTargets() |
Set<SourceTarget<?>> |
ParallelDoOptions.getSourceTargets()
Deprecated.
|
Set<SourceTarget<?>> |
GroupingOptions.getSourceTargets() |
Modifier and Type | Method and Description |
---|---|
GroupingOptions.Builder |
GroupingOptions.Builder.sourceTarget(SourceTarget<?> st)
Deprecated.
|
ParallelDoOptions.Builder |
ParallelDoOptions.Builder.sourceTargets(SourceTarget<?>... sourceTargets) |
GroupingOptions.Builder |
GroupingOptions.Builder.sourceTargets(SourceTarget<?>... st) |
Modifier and Type | Method and Description |
---|---|
ParallelDoOptions.Builder |
ParallelDoOptions.Builder.sourceTargets(Collection<SourceTarget<?>> sourceTargets) |
GroupingOptions.Builder |
GroupingOptions.Builder.sourceTargets(Collection<SourceTarget<?>> st) |
Modifier and Type | Method and Description |
---|---|
<T> SourceTarget<T> |
DistributedPipeline.createIntermediateOutput(PType<T> ptype) |
Modifier and Type | Method and Description |
---|---|
SourceTarget<S> |
PCollectionImpl.getMaterializedAt() |
Modifier and Type | Method and Description |
---|---|
void |
PCollectionImpl.materializeAt(SourceTarget<S> sourceTarget) |
Modifier and Type | Interface and Description |
---|---|
interface |
ReadableSourceTarget<T>
An interface that indicates that a
SourceTarget instance can be read
into the local client. |
Modifier and Type | Method and Description |
---|---|
static SourceTarget<org.apache.avro.generic.GenericData.Record> |
At.avroFile(org.apache.hadoop.fs.Path path)
Creates a
SourceTarget<GenericData.Record> by reading the schema of the Avro file
at the given path. |
static <T extends org.apache.avro.specific.SpecificRecord> |
At.avroFile(org.apache.hadoop.fs.Path path,
Class<T> avroClass)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given Path . |
static SourceTarget<org.apache.avro.generic.GenericData.Record> |
At.avroFile(org.apache.hadoop.fs.Path path,
org.apache.hadoop.conf.Configuration conf)
Creates a
SourceTarget<GenericData.Record> by reading the schema of the Avro file
at the given path using the FileSystem information contained in the given
Configuration instance. |
static <T> SourceTarget<T> |
At.avroFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given Path . |
static SourceTarget<org.apache.avro.generic.GenericData.Record> |
At.avroFile(String pathName)
Creates a
SourceTarget<GenericData.Record> by reading the schema of the Avro file
at the given path. |
static <T extends org.apache.avro.specific.SpecificRecord> |
At.avroFile(String pathName,
Class<T> avroClass)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given path name. |
static <T> SourceTarget<T> |
At.avroFile(String pathName,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given path name. |
static <T extends org.apache.hadoop.io.Writable> |
At.sequenceFile(org.apache.hadoop.fs.Path path,
Class<T> valueClass)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given Path
from the value field of each key-value pair in the SequenceFile(s). |
static <T> SourceTarget<T> |
At.sequenceFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given Path
from the value field of each key-value pair in the SequenceFile(s). |
static <T extends org.apache.hadoop.io.Writable> |
At.sequenceFile(String pathName,
Class<T> valueClass)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given path name
from the value field of each key-value pair in the SequenceFile(s). |
static <T> SourceTarget<T> |
At.sequenceFile(String pathName,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given path name
from the value field of each key-value pair in the SequenceFile(s). |
static SourceTarget<String> |
At.textFile(org.apache.hadoop.fs.Path path)
Creates a
SourceTarget<String> instance for the text file(s) at the given Path . |
static <T> SourceTarget<T> |
At.textFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
SourceTarget<T> instance for the text file(s) at the given Path using
the provided PType<T> to convert the input text. |
static SourceTarget<String> |
At.textFile(String pathName)
Creates a
SourceTarget<String> instance for the text file(s) at the given path name. |
static <T> SourceTarget<T> |
At.textFile(String pathName,
PType<T> ptype)
Creates a
SourceTarget<T> instance for the text file(s) at the given path name using
the provided PType<T> to convert the input text. |
Modifier and Type | Method and Description |
---|---|
Set<SourceTarget<?>> |
UnionReadableData.getSourceTargets() |
Set<SourceTarget<?>> |
DelegatingReadableData.getSourceTargets() |
Copyright © 2016 The Apache Software Foundation. All rights reserved.