Package | Description |
---|---|
org.apache.crunch |
Client-facing API and core abstractions.
|
org.apache.crunch.impl.dist | |
org.apache.crunch.impl.dist.collect | |
org.apache.crunch.impl.mem |
In-memory Pipeline implementation for rapid prototyping and testing.
|
org.apache.crunch.impl.spark.collect | |
org.apache.crunch.io |
Data input and output for Pipelines.
|
org.apache.crunch.util |
An assorted set of utilities.
|
Modifier and Type | Interface and Description |
---|---|
interface |
TableSourceTarget<K,V>
An interface for classes that implement both the
TableSource and the
Target interfaces. |
Modifier and Type | Method and Description |
---|---|
<K,V> PTable<K,V> |
Pipeline.read(TableSource<K,V> tableSource)
A version of the read method for
TableSource instances that map to
PTable s. |
<K,V> PTable<K,V> |
Pipeline.read(TableSource<K,V> tableSource,
String named)
A version of the read method for
TableSource instances that map to
PTable s. |
Modifier and Type | Method and Description |
---|---|
<K,V> PTable<K,V> |
DistributedPipeline.read(TableSource<K,V> source) |
<K,V> PTable<K,V> |
DistributedPipeline.read(TableSource<K,V> source,
String named) |
Modifier and Type | Method and Description |
---|---|
TableSource<K,V> |
BaseInputTable.getSource() |
Modifier and Type | Method and Description |
---|---|
<K,V> BaseInputTable<K,V> |
PCollectionFactory.createInputTable(TableSource<K,V> source,
String named,
DistributedPipeline distributedPipeline,
ParallelDoOptions doOpts) |
Constructor and Description |
---|
BaseInputTable(TableSource<K,V> source,
DistributedPipeline pipeline) |
BaseInputTable(TableSource<K,V> source,
String name,
DistributedPipeline pipeline,
ParallelDoOptions doOpts) |
Modifier and Type | Method and Description |
---|---|
<K,V> PTable<K,V> |
MemPipeline.read(TableSource<K,V> source) |
<K,V> PTable<K,V> |
MemPipeline.read(TableSource<K,V> source,
String named) |
Modifier and Type | Method and Description |
---|---|
<K,V> BaseInputTable<K,V> |
SparkCollectFactory.createInputTable(TableSource<K,V> source,
String named,
DistributedPipeline pipeline,
ParallelDoOptions doOpts) |
Constructor and Description |
---|
InputTable(TableSource<K,V> source,
String named,
DistributedPipeline pipeline,
ParallelDoOptions doOpts) |
Modifier and Type | Method and Description |
---|---|
static <K,V> TableSource<K,V> |
From.avroTableFile(List<org.apache.hadoop.fs.Path> paths,
PTableType<K,V> tableType)
Creates a
TableSource<K,V> for reading an Avro key/value file at the given paths. |
static <K,V> TableSource<K,V> |
From.avroTableFile(org.apache.hadoop.fs.Path path,
PTableType<K,V> tableType)
Creates a
TableSource<K,V> for reading an Avro key/value file at the given path. |
static <K,V> TableSource<K,V> |
From.formattedFile(List<org.apache.hadoop.fs.Path> paths,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
From.formattedFile(List<org.apache.hadoop.fs.Path> paths,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>> formatClass,
Class<K> keyClass,
Class<V> valueClass)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat<K, V> implementations not covered by the provided TableSource
and Source factory methods. |
static <K,V> TableSource<K,V> |
From.formattedFile(org.apache.hadoop.fs.Path path,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
From.formattedFile(org.apache.hadoop.fs.Path path,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>> formatClass,
Class<K> keyClass,
Class<V> valueClass)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat<K, V> implementations not covered by the provided TableSource
and Source factory methods. |
static <K,V> TableSource<K,V> |
From.formattedFile(String pathName,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
From.formattedFile(String pathName,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>> formatClass,
Class<K> keyClass,
Class<V> valueClass)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat<K, V> implementations not covered by the provided TableSource
and Source factory methods. |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
From.sequenceFile(List<org.apache.hadoop.fs.Path> paths,
Class<K> keyClass,
Class<V> valueClass)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given Path s. |
static <K,V> TableSource<K,V> |
From.sequenceFile(List<org.apache.hadoop.fs.Path> paths,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given Path s. |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
From.sequenceFile(org.apache.hadoop.fs.Path path,
Class<K> keyClass,
Class<V> valueClass)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given Path . |
static <K,V> TableSource<K,V> |
From.sequenceFile(org.apache.hadoop.fs.Path path,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given Path . |
static <K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> |
From.sequenceFile(String pathName,
Class<K> keyClass,
Class<V> valueClass)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given path name. |
static <K,V> TableSource<K,V> |
From.sequenceFile(String pathName,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given path name. |
Modifier and Type | Method and Description |
---|---|
<K,V> PTable<K,V> |
CrunchTool.read(TableSource<K,V> tableSource) |
Copyright © 2016 The Apache Software Foundation. All rights reserved.