Package | Description |
---|---|
org.apache.crunch |
Client-facing API and core abstractions.
|
org.apache.crunch.contrib.text | |
org.apache.crunch.fn |
Commonly used functions for manipulating collections.
|
org.apache.crunch.impl.dist | |
org.apache.crunch.impl.dist.collect | |
org.apache.crunch.impl.mem |
In-memory Pipeline implementation for rapid prototyping and testing.
|
org.apache.crunch.impl.spark | |
org.apache.crunch.impl.spark.collect | |
org.apache.crunch.impl.spark.serde | |
org.apache.crunch.io |
Data input and output for Pipelines.
|
org.apache.crunch.kafka | |
org.apache.crunch.lambda |
Alternative Crunch API using Java 8 features to allow construction of pipelines using lambda functions and method
references.
|
org.apache.crunch.lib |
Joining, sorting, aggregating, and other commonly used functionality.
|
org.apache.crunch.lib.join |
Inner and outer joins on collections.
|
org.apache.crunch.lib.sort | |
org.apache.crunch.types |
Common functionality for business object serialization.
|
org.apache.crunch.types.avro |
Business object serialization using Apache Avro.
|
org.apache.crunch.types.orc | |
org.apache.crunch.types.writable |
Business object serialization using Hadoop's Writables framework.
|
Modifier and Type | Method and Description |
---|---|
static <V1,V2,V3> PType<Tuple3.Collect<V1,V2,V3>> |
Tuple3.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third) |
static <V1,V2,V3,V4> |
Tuple4.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third,
PType<V4> fourth) |
PType<K> |
PTable.getKeyType()
Returns the
PType of the key. |
PType<S> |
PCollection.getPType()
Returns the
PType of this PCollection . |
PType<T> |
Source.getType()
Returns the
PType for this source. |
PType<V> |
PTable.getValueType()
Returns the
PType of the value. |
Modifier and Type | Method and Description |
---|---|
boolean |
Target.accept(OutputHandler handler,
PType<?> ptype)
Checks to see if this
Target instance is compatible with the
given PType . |
<T> SourceTarget<T> |
Target.asSourceTarget(PType<T> ptype)
Attempt to create the
SourceTarget type that corresponds to this Target
for the given PType , if possible. |
<K> PTable<K,S> |
PCollection.by(MapFn<S,K> extractKeyFn,
PType<K> keyType)
Apply the given map function to each element of this instance in order to
create a
PTable . |
<K> PTable<K,S> |
PCollection.by(String name,
MapFn<S,K> extractKeyFn,
PType<K> keyType)
Apply the given map function to each element of this instance in order to
create a
PTable . |
<T> PCollection<T> |
Pipeline.create(Iterable<T> contents,
PType<T> ptype)
Creates a
PCollection containing the values found in the given Iterable
using an implementation-specific distribution mechanism. |
<T> PCollection<T> |
Pipeline.create(Iterable<T> contents,
PType<T> ptype,
CreateOptions options)
Creates a
PCollection containing the values found in the given Iterable
using an implementation-specific distribution mechanism. |
static <V1,V2,V3> PType<Tuple3.Collect<V1,V2,V3>> |
Tuple3.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third) |
static <V1,V2,V3> PType<Tuple3.Collect<V1,V2,V3>> |
Tuple3.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third) |
static <V1,V2,V3> PType<Tuple3.Collect<V1,V2,V3>> |
Tuple3.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third) |
static <V1,V2,V3,V4> |
Tuple4.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third,
PType<V4> fourth) |
static <V1,V2,V3,V4> |
Tuple4.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third,
PType<V4> fourth) |
static <V1,V2,V3,V4> |
Tuple4.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third,
PType<V4> fourth) |
static <V1,V2,V3,V4> |
Tuple4.Collect.derived(PType<V1> first,
PType<V2> second,
PType<V3> third,
PType<V4> fourth) |
<T> PCollection<T> |
Pipeline.emptyPCollection(PType<T> ptype)
Creates an empty
PCollection of the given PType . |
Converter<?,?,?,?> |
Target.getConverter(PType<?> ptype)
Returns the
Converter to use for mapping from the output PCollection
into the output values expected by this instance. |
<K2> PTable<K2,V> |
PTable.mapKeys(MapFn<K,K2> mapFn,
PType<K2> ptype)
Returns a
PTable that has the same values as this instance, but
uses the given function to map the keys. |
<K2> PTable<K2,V> |
PTable.mapKeys(String name,
MapFn<K,K2> mapFn,
PType<K2> ptype)
Returns a
PTable that has the same values as this instance, but
uses the given function to map the keys. |
<U> PTable<K,U> |
PGroupedTable.mapValues(MapFn<Iterable<V>,U> mapFn,
PType<U> ptype)
Maps the
Iterable<V> elements of each record to a new type. |
<U> PTable<K,U> |
PTable.mapValues(MapFn<V,U> mapFn,
PType<U> ptype)
Returns a
PTable that has the same keys as this instance, but
uses the given function to map the values. |
<U> PTable<K,U> |
PGroupedTable.mapValues(String name,
MapFn<Iterable<V>,U> mapFn,
PType<U> ptype)
Maps the
Iterable<V> elements of each record to a new type. |
<U> PTable<K,U> |
PTable.mapValues(String name,
MapFn<V,U> mapFn,
PType<U> ptype)
Returns a
PTable that has the same keys as this instance, but
uses the given function to map the values. |
<T> PCollection<T> |
PCollection.parallelDo(DoFn<S,T> doFn,
PType<T> type)
Applies the given doFn to the elements of this
PCollection and
returns a new PCollection that is the output of this processing. |
<T> PCollection<T> |
PCollection.parallelDo(String name,
DoFn<S,T> doFn,
PType<T> type)
Applies the given doFn to the elements of this
PCollection and
returns a new PCollection that is the output of this processing. |
<T> PCollection<T> |
PCollection.parallelDo(String name,
DoFn<S,T> doFn,
PType<T> type,
ParallelDoOptions options)
Applies the given doFn to the elements of this
PCollection and
returns a new PCollection that is the output of this processing. |
Modifier and Type | Method and Description |
---|---|
PType<T> |
Extractor.getPType(PTypeFamily ptf)
Returns the
PType associated with this data type for the
given PTypeFamily . |
Modifier and Type | Method and Description |
---|---|
static <V1,V2> PType<Pair<V2,V1>> |
SwapFn.ptype(PType<Pair<V1,V2>> pt) |
Modifier and Type | Method and Description |
---|---|
static <V1,V2> PType<Pair<V2,V1>> |
SwapFn.ptype(PType<Pair<V1,V2>> pt) |
static <K,V> CombineFn<K,V> |
Aggregators.toCombineFn(Aggregator<V> aggregator,
PType<V> ptype)
Wrap a
CombineFn adapter around the given aggregator. |
Modifier and Type | Method and Description |
---|---|
<S> PCollection<S> |
DistributedPipeline.create(Iterable<S> contents,
PType<S> ptype) |
<S> PCollection<S> |
DistributedPipeline.create(Iterable<S> contents,
PType<S> ptype,
CreateOptions options) |
<T> SourceTarget<T> |
DistributedPipeline.createIntermediateOutput(PType<T> ptype) |
<S> PCollection<S> |
DistributedPipeline.emptyPCollection(PType<S> ptype) |
Modifier and Type | Method and Description |
---|---|
PType<K> |
PTableBase.getKeyType() |
PType<Pair<K,V>> |
EmptyPTable.getPType() |
PType<T> |
EmptyPCollection.getPType() |
PType<Pair<K,V>> |
BaseUnionTable.getPType() |
PType<S> |
BaseUnionCollection.getPType() |
PType<Pair<K,V>> |
BaseInputTable.getPType() |
PType<S> |
BaseInputCollection.getPType() |
PType<Pair<K,Iterable<V>>> |
BaseGroupedTable.getPType() |
PType<Pair<K,V>> |
BaseDoTable.getPType() |
PType<S> |
BaseDoCollection.getPType() |
PType<V> |
PTableBase.getValueType() |
Modifier and Type | Method and Description |
---|---|
<K> PTable<K,S> |
PCollectionImpl.by(MapFn<S,K> mapFn,
PType<K> keyType) |
<K> PTable<K,S> |
PCollectionImpl.by(String name,
MapFn<S,K> mapFn,
PType<K> keyType) |
<S,T> BaseDoCollection<T> |
PCollectionFactory.createDoCollection(String name,
PCollectionImpl<S> chainingCollection,
DoFn<S,T> fn,
PType<T> type,
ParallelDoOptions options) |
<K2> PTable<K2,V> |
PTableBase.mapKeys(MapFn<K,K2> mapFn,
PType<K2> ptype) |
<K2> PTable<K2,V> |
PTableBase.mapKeys(String name,
MapFn<K,K2> mapFn,
PType<K2> ptype) |
<U> PTable<K,U> |
BaseGroupedTable.mapValues(MapFn<Iterable<V>,U> mapFn,
PType<U> ptype) |
<U> PTable<K,U> |
PTableBase.mapValues(MapFn<V,U> mapFn,
PType<U> ptype) |
<U> PTable<K,U> |
BaseGroupedTable.mapValues(String name,
MapFn<Iterable<V>,U> mapFn,
PType<U> ptype) |
<U> PTable<K,U> |
PTableBase.mapValues(String name,
MapFn<V,U> mapFn,
PType<U> ptype) |
<T> PCollection<T> |
PCollectionImpl.parallelDo(DoFn<S,T> fn,
PType<T> type) |
<T> PCollection<T> |
PCollectionImpl.parallelDo(String name,
DoFn<S,T> fn,
PType<T> type) |
<T> PCollection<T> |
PCollectionImpl.parallelDo(String name,
DoFn<S,T> fn,
PType<T> type,
ParallelDoOptions options) |
Constructor and Description |
---|
EmptyPCollection(DistributedPipeline pipeline,
PType<T> ptype) |
Modifier and Type | Method and Description |
---|---|
<T> PCollection<T> |
MemPipeline.create(Iterable<T> contents,
PType<T> ptype) |
<T> PCollection<T> |
MemPipeline.create(Iterable<T> iterable,
PType<T> ptype,
CreateOptions options) |
<T> PCollection<T> |
MemPipeline.emptyPCollection(PType<T> ptype) |
static <T> PCollection<T> |
MemPipeline.typedCollectionOf(PType<T> ptype,
Iterable<T> collect) |
static <T> PCollection<T> |
MemPipeline.typedCollectionOf(PType<T> ptype,
T... ts) |
Modifier and Type | Method and Description |
---|---|
<S> PCollection<S> |
SparkPipeline.create(Iterable<S> contents,
PType<S> ptype,
CreateOptions options) |
<S> PCollection<S> |
SparkPipeline.emptyPCollection(PType<S> ptype) |
Modifier and Type | Method and Description |
---|---|
PType<Pair<K,V>> |
CreatedTable.getPType() |
PType<T> |
CreatedCollection.getPType() |
Modifier and Type | Method and Description |
---|---|
<S,T> BaseDoCollection<T> |
SparkCollectFactory.createDoCollection(String name,
PCollectionImpl<S> parent,
DoFn<S,T> fn,
PType<T> type,
ParallelDoOptions options) |
Constructor and Description |
---|
CreatedCollection(SparkPipeline p,
Iterable<T> contents,
PType<T> ptype,
CreateOptions options) |
EmptyPCollection(DistributedPipeline pipeline,
PType<T> ptype) |
Modifier and Type | Method and Description |
---|---|
static SerDe |
SerDeFactory.create(PType<?> ptype,
org.apache.hadoop.conf.Configuration conf) |
Modifier and Type | Method and Description |
---|---|
static <T> Source<T> |
From.avroFile(List<org.apache.hadoop.fs.Path> paths,
PType<T> ptype)
Creates a
Source<T> instance from the Avro file(s) at the given Path s. |
static <T> Source<T> |
From.avroFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
Source<T> instance from the Avro file(s) at the given Path . |
static <T> SourceTarget<T> |
At.avroFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given Path . |
static <T> Source<T> |
From.avroFile(String pathName,
PType<T> ptype)
Creates a
Source<T> instance from the Avro file(s) at the given path name. |
static <T> SourceTarget<T> |
At.avroFile(String pathName,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the Avro file(s) at the given path name. |
boolean |
OutputHandler.configure(Target target,
PType<?> ptype) |
void |
MapReduceTarget.configureForMapReduce(org.apache.hadoop.mapreduce.Job job,
PType<?> ptype,
org.apache.hadoop.fs.Path outputPath,
String name) |
static <K,V> TableSource<K,V> |
From.formattedFile(List<org.apache.hadoop.fs.Path> paths,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K,V> TableSource<K,V> |
From.formattedFile(List<org.apache.hadoop.fs.Path> paths,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K,V> TableSource<K,V> |
From.formattedFile(org.apache.hadoop.fs.Path path,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K,V> TableSource<K,V> |
From.formattedFile(org.apache.hadoop.fs.Path path,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K,V> TableSource<K,V> |
From.formattedFile(String pathName,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K,V> TableSource<K,V> |
From.formattedFile(String pathName,
Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> formatClass,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> for reading data from files that have custom
FileInputFormat implementations not covered by the provided TableSource
and Source factory methods. |
static <K,V> TableSource<K,V> |
From.sequenceFile(List<org.apache.hadoop.fs.Path> paths,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given Path s. |
static <K,V> TableSource<K,V> |
From.sequenceFile(List<org.apache.hadoop.fs.Path> paths,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given Path s. |
static <T> Source<T> |
From.sequenceFile(List<org.apache.hadoop.fs.Path> paths,
PType<T> ptype)
Creates a
Source<T> instance from the SequenceFile(s) at the given Path s
from the value field of each key-value pair in the SequenceFile(s). |
static <K,V> TableSource<K,V> |
From.sequenceFile(org.apache.hadoop.fs.Path path,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given Path . |
static <K,V> TableSource<K,V> |
From.sequenceFile(org.apache.hadoop.fs.Path path,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given Path . |
static <K,V> TableSourceTarget<K,V> |
At.sequenceFile(org.apache.hadoop.fs.Path path,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSourceTarget<K, V> instance from the SequenceFile(s) at the given Path
from the key-value pairs in the SequenceFile(s). |
static <K,V> TableSourceTarget<K,V> |
At.sequenceFile(org.apache.hadoop.fs.Path path,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSourceTarget<K, V> instance from the SequenceFile(s) at the given Path
from the key-value pairs in the SequenceFile(s). |
static <T> Source<T> |
From.sequenceFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
Source<T> instance from the SequenceFile(s) at the given Path
from the value field of each key-value pair in the SequenceFile(s). |
static <T> SourceTarget<T> |
At.sequenceFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given Path
from the value field of each key-value pair in the SequenceFile(s). |
static <K,V> TableSource<K,V> |
From.sequenceFile(String pathName,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given path name. |
static <K,V> TableSource<K,V> |
From.sequenceFile(String pathName,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSource<K, V> instance for the SequenceFile(s) at the given path name. |
static <K,V> TableSourceTarget<K,V> |
At.sequenceFile(String pathName,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSourceTarget<K, V> instance from the SequenceFile(s) at the given path name
from the key-value pairs in the SequenceFile(s). |
static <K,V> TableSourceTarget<K,V> |
At.sequenceFile(String pathName,
PType<K> keyType,
PType<V> valueType)
Creates a
TableSourceTarget<K, V> instance from the SequenceFile(s) at the given path name
from the key-value pairs in the SequenceFile(s). |
static <T> Source<T> |
From.sequenceFile(String pathName,
PType<T> ptype)
Creates a
Source<T> instance from the SequenceFile(s) at the given path name
from the value field of each key-value pair in the SequenceFile(s). |
static <T> SourceTarget<T> |
At.sequenceFile(String pathName,
PType<T> ptype)
Creates a
SourceTarget<T> instance from the SequenceFile(s) at the given path name
from the value field of each key-value pair in the SequenceFile(s). |
static <T> Source<T> |
From.textFile(List<org.apache.hadoop.fs.Path> paths,
PType<T> ptype)
Creates a
Source<T> instance for the text file(s) at the given Path s using
the provided PType<T> to convert the input text. |
static <T> Source<T> |
From.textFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
Source<T> instance for the text file(s) at the given Path using
the provided PType<T> to convert the input text. |
static <T> SourceTarget<T> |
At.textFile(org.apache.hadoop.fs.Path path,
PType<T> ptype)
Creates a
SourceTarget<T> instance for the text file(s) at the given Path using
the provided PType<T> to convert the input text. |
static <T> Source<T> |
From.textFile(String pathName,
PType<T> ptype)
Creates a
Source<T> instance for the text file(s) at the given path name using
the provided PType<T> to convert the input text. |
static <T> SourceTarget<T> |
At.textFile(String pathName,
PType<T> ptype)
Creates a
SourceTarget<T> instance for the text file(s) at the given path name using
the provided PType<T> to convert the input text. |
Modifier and Type | Method and Description |
---|---|
PType<Pair<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>> |
KafkaSource.getType() |
Modifier and Type | Method and Description |
---|---|
default PType<K> |
LTable.keyType()
Get a
PType which can be used to serialize the key part of this table |
default PType<K> |
LGroupedTable.keyType()
Get a
PType which can be used to serialize the key part of this grouped table |
default PType<S> |
LCollection.pType()
Get the
PType representing how elements of this collection may be serialized. |
default PType<V> |
LTable.valueType()
Get a
PType which can be used to serialize the value part of this table |
default PType<V> |
LGroupedTable.valueType()
Get a
PType which can be used to serialize the value part of this grouped table |
Modifier and Type | Method and Description |
---|---|
default <K> LTable<K,S> |
LCollection.by(SFunction<S,K> extractFn,
PType<K> pType)
Key this LCollection by a key extracted from the element to yield a
LTable mapping the key to the whole
element. |
default <C> LTable<K,C> |
LGroupedTable.collectValues(SSupplier<C> emptySupplier,
SBiConsumer<C,V> addFn,
PType<C> pType)
Collect the values into an aggregate type.
|
default <T> LCollection<T> |
LCollection.filterMap(SFunction<S,Optional<T>> fn,
PType<T> pType)
Combination of a filter and map operation by using a function with
Optional return type. |
default <T> LCollection<T> |
LCollection.flatMap(SFunction<S,java.util.stream.Stream<T>> fn,
PType<T> pType)
Map each element to zero or more output elements using the provided stream-returning function.
|
default <T> LCollection<T> |
LCollection.map(SFunction<S,T> fn,
PType<T> pType)
Map the elements of this collection 1-1 through the supplied function.
|
default <T> LTable<T,V> |
LTable.mapKeys(SFunction<K,T> fn,
PType<T> pType)
Transform the keys of this table using the given function
|
default <T> LTable<K,T> |
LGroupedTable.mapValues(SFunction<java.util.stream.Stream<V>,T> fn,
PType<T> pType)
Map the values in this LGroupedTable using a custom function.
|
default <T> LTable<K,T> |
LTable.mapValues(SFunction<V,T> fn,
PType<T> pType)
Transform the values of this table using the given function
|
default <T> LCollection<T> |
LCollection.parallelDo(DoFn<S,T> fn,
PType<T> pType)
Transform this LCollection using a standard Crunch
DoFn |
default <T> LCollection<T> |
LCollection.parallelDo(LDoFn<S,T> fn,
PType<T> pType)
Transform this LCollection using a Lambda-friendly
LDoFn . |
Modifier and Type | Method and Description |
---|---|
static <V> PType<Quantiles.Result<V>> |
Quantiles.Result.pType(PType<V> valuePType)
Create a PType for the result type, to be stored as a derived type from Crunch primitives
|
Modifier and Type | Method and Description |
---|---|
static <K,V,T> DoFn<Pair<K,Iterable<V>>,T> |
DoFns.detach(DoFn<Pair<K,Iterable<V>>,T> reduceFn,
PType<V> valueType)
"Reduce" DoFn wrapper which detaches the values in the iterable, preventing the unexpected behaviour related to
object reuse often observed when using Avro.
|
static <K1,K2,V> PTable<K2,V> |
PTables.mapKeys(PTable<K1,V> ptable,
MapFn<K1,K2> mapFn,
PType<K2> ptype)
Maps a
PTable<K1, V> to a PTable<K2, V> using the given MapFn<K1, K2> on
the keys of the PTable . |
static <K1,K2,V> PTable<K2,V> |
PTables.mapKeys(String name,
PTable<K1,V> ptable,
MapFn<K1,K2> mapFn,
PType<K2> ptype)
Maps a
PTable<K1, V> to a PTable<K2, V> using the given MapFn<K1, K2> on
the keys of the PTable . |
static <K,U,V> PTable<K,V> |
PTables.mapValues(PGroupedTable<K,U> ptable,
MapFn<Iterable<U>,V> mapFn,
PType<V> ptype)
An analogue of the
mapValues function for PGroupedTable<K, U> collections. |
static <K,U,V> PTable<K,V> |
PTables.mapValues(PTable<K,U> ptable,
MapFn<U,V> mapFn,
PType<V> ptype)
Maps a
PTable<K, U> to a PTable<K, V> using the given MapFn<U, V> on
the values of the PTable . |
static <K,U,V> PTable<K,V> |
PTables.mapValues(String name,
PGroupedTable<K,U> ptable,
MapFn<Iterable<U>,V> mapFn,
PType<V> ptype)
An analogue of the
mapValues function for PGroupedTable<K, U> collections. |
static <K,U,V> PTable<K,V> |
PTables.mapValues(String name,
PTable<K,U> ptable,
MapFn<U,V> mapFn,
PType<V> ptype)
Maps a
PTable<K, U> to a PTable<K, V> using the given MapFn<U, V> on
the values of the PTable . |
static <V> PType<Quantiles.Result<V>> |
Quantiles.Result.pType(PType<V> valuePType)
Create a PType for the result type, to be stored as a derived type from Crunch primitives
|
static <K,V1,V2,T> |
SecondarySort.sortAndApply(PTable<K,Pair<V1,V2>> input,
DoFn<Pair<K,Iterable<Pair<V1,V2>>>,T> doFn,
PType<T> ptype)
Perform a secondary sort on the given
PTable instance and then apply a
DoFn to the resulting sorted data to yield an output PCollection<T> . |
static <K,V1,V2,T> |
SecondarySort.sortAndApply(PTable<K,Pair<V1,V2>> input,
DoFn<Pair<K,Iterable<Pair<V1,V2>>>,T> doFn,
PType<T> ptype,
int numReducers)
Perform a secondary sort on the given
PTable instance and then apply a
DoFn to the resulting sorted data to yield an output PCollection<T> , using
the given number of reducers. |
static <T,U> Pair<PCollection<T>,PCollection<U>> |
Channels.split(PCollection<Pair<T,U>> pCollection,
PType<T> firstPType,
PType<U> secondPType)
Splits a
PCollection of any Pair of objects into a Pair of
PCollection}, to allow for the output of a DoFn to be handled using
separate channels. |
static <T,U> Pair<PCollection<T>,PCollection<U>> |
Channels.split(PCollection<Pair<T,U>> pCollection,
PType<T> firstPType,
PType<U> secondPType)
Splits a
PCollection of any Pair of objects into a Pair of
PCollection}, to allow for the output of a DoFn to be handled using
separate channels. |
Constructor and Description |
---|
TopKCombineFn(int limit,
boolean maximize,
PType<Pair<K,V>> pairType) |
TopKFn(int limit,
boolean ascending,
PType<Pair<K,V>> pairType) |
Modifier and Type | Method and Description |
---|---|
static <K,U,V,T> PCollection<T> |
OneToManyJoin.oneToManyJoin(PTable<K,U> left,
PTable<K,V> right,
DoFn<Pair<U,Iterable<V>>,T> postProcessFn,
PType<T> ptype)
Performs a join on two tables, where the left table only contains a single
value per key.
|
static <K,U,V,T> PCollection<T> |
OneToManyJoin.oneToManyJoin(PTable<K,U> left,
PTable<K,V> right,
DoFn<Pair<U,Iterable<V>>,T> postProcessFn,
PType<T> ptype,
int numReducers)
Supports a user-specified number of reducers for the one-to-many join.
|
Constructor and Description |
---|
FullOuterJoinFn(PType<K> keyType,
PType<U> leftValueType) |
FullOuterJoinFn(PType<K> keyType,
PType<U> leftValueType) |
InnerJoinFn(PType<K> keyType,
PType<U> leftValueType) |
InnerJoinFn(PType<K> keyType,
PType<U> leftValueType) |
JoinFn(PType<K> keyType,
PType<U> leftValueType)
Instantiate with the PType of the value of the left side of the join (used for creating deep
copies of values).
|
JoinFn(PType<K> keyType,
PType<U> leftValueType)
Instantiate with the PType of the value of the left side of the join (used for creating deep
copies of values).
|
LeftOuterJoinFn(PType<K> keyType,
PType<U> leftValueType) |
LeftOuterJoinFn(PType<K> keyType,
PType<U> leftValueType) |
RightOuterJoinFn(PType<K> keyType,
PType<U> leftValueType) |
RightOuterJoinFn(PType<K> keyType,
PType<U> leftValueType) |
Modifier and Type | Method and Description |
---|---|
PType<Object> |
SortFns.KeyExtraction.getKeyType() |
Modifier and Type | Method and Description |
---|---|
static <S> org.apache.avro.Schema |
SortFns.createOrderedTupleSchema(PType<S> ptype,
Sort.ColumnOrder[] orders)
Constructs an Avro schema for the given
PType<S> that respects the given column
orderings. |
Constructor and Description |
---|
KeyExtraction(PType<V> ptype,
Sort.ColumnOrder[] columnOrder) |
Modifier and Type | Interface and Description |
---|---|
interface |
PTableType<K,V>
An extension of
PType specifically for PTable objects. |
Modifier and Type | Class and Description |
---|---|
class |
PGroupedTableType<K,V>
The
PType instance for PGroupedTable instances. |
Modifier and Type | Method and Description |
---|---|
<T> PType<T> |
PTypeFamily.as(PType<T> ptype)
Returns the equivalent of the given ptype for this family, if it exists.
|
static PType<BigDecimal> |
PTypes.bigDecimal(PTypeFamily typeFamily)
A PType for Java's
BigDecimal type. |
static PType<BigInteger> |
PTypes.bigInt(PTypeFamily typeFamily)
A PType for Java's
BigInteger type. |
PType<Boolean> |
PTypeFamily.booleans() |
PType<ByteBuffer> |
PTypeFamily.bytes() |
<T> PType<Collection<T>> |
PTypeFamily.collections(PType<T> ptype) |
static <T> PType<T> |
PTypeUtils.convert(PType<T> ptype,
PTypeFamily tf) |
<S,T> PType<T> |
PTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
PTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base)
A derived type whose values are immutable.
|
PType<Double> |
PTypeFamily.doubles() |
static <T extends Enum> |
PTypes.enums(Class<T> type,
PTypeFamily typeFamily)
Constructs a PType for a Java
Enum type. |
PType<Float> |
PTypeFamily.floats() |
PType<K> |
PTableType.getKeyType()
Returns the key type for the table.
|
PType<V> |
PTableType.getValueType()
Returns the value type for the table.
|
PType<Integer> |
PTypeFamily.ints() |
static <T> PType<T> |
PTypes.jsonString(Class<T> clazz,
PTypeFamily typeFamily)
Constructs a PType for reading a Java type from a JSON string using Jackson's
ObjectMapper . |
PType<Long> |
PTypeFamily.longs() |
<T> PType<Map<String,T>> |
PTypeFamily.maps(PType<T> ptype) |
PType<Void> |
PTypeFamily.nulls() |
<V1,V2> PType<Pair<V1,V2>> |
PTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
static <T extends com.google.protobuf.Message> |
PTypes.protos(Class<T> clazz,
PTypeFamily typeFamily)
Constructs a PType for the given protocol buffer.
|
static <T extends com.google.protobuf.Message> |
PTypes.protos(Class<T> clazz,
PTypeFamily typeFamily,
SerializableSupplier<com.google.protobuf.ExtensionRegistry> supplier)
Constructs a PType for a protocol buffer, using the given
SerializableSupplier to provide
an ExtensionRegistry to use in reading the given protobuf. |
<V1,V2,V3,V4> |
PTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<T> PType<T> |
PTypeFamily.records(Class<T> clazz) |
PType<String> |
PTypeFamily.strings() |
static <T extends org.apache.thrift.TBase> |
PTypes.thrifts(Class<T> clazz,
PTypeFamily typeFamily)
Constructs a PType for a Thrift record.
|
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
PTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<T extends Tuple> |
PTypeFamily.tuples(Class<T> clazz,
PType<?>... ptypes) |
PType<TupleN> |
PTypeFamily.tuples(PType<?>... ptypes) |
PType<Union> |
PTypeFamily.unionOf(PType<?>... ptypes) |
static PType<UUID> |
PTypes.uuid(PTypeFamily ptf)
A PType for Java's
UUID type. |
Modifier and Type | Method and Description |
---|---|
List<PType> |
PType.getSubTypes()
Returns the sub-types that make up this PType if it is a composite instance, such as a tuple.
|
List<PType> |
PGroupedTableType.getSubTypes() |
Modifier and Type | Method and Description |
---|---|
<T> PType<T> |
PTypeFamily.as(PType<T> ptype)
Returns the equivalent of the given ptype for this family, if it exists.
|
<T> PType<Collection<T>> |
PTypeFamily.collections(PType<T> ptype) |
static <T> PType<T> |
PTypeUtils.convert(PType<T> ptype,
PTypeFamily tf) |
<S,T> PType<T> |
PTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
PTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base)
A derived type whose values are immutable.
|
<T> PType<Map<String,T>> |
PTypeFamily.maps(PType<T> ptype) |
<V1,V2> PType<Pair<V1,V2>> |
PTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
<V1,V2> PType<Pair<V1,V2>> |
PTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
<V1,V2,V3,V4> |
PTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
PTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
PTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
PTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<K,V> PTableType<K,V> |
PTypeFamily.tableOf(PType<K> key,
PType<V> value) |
<K,V> PTableType<K,V> |
PTypeFamily.tableOf(PType<K> key,
PType<V> value) |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
PTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
PTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
PTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<T extends Tuple> |
PTypeFamily.tuples(Class<T> clazz,
PType<?>... ptypes) |
PType<TupleN> |
PTypeFamily.tuples(PType<?>... ptypes) |
PType<Union> |
PTypeFamily.unionOf(PType<?>... ptypes) |
Constructor and Description |
---|
CollectionDeepCopier(PType<T> elementType) |
MapDeepCopier(PType<T> ptype) |
TupleDeepCopier(Class<T> tupleClass,
PType... elementTypes) |
UnionDeepCopier(PType... elementTypes) |
Modifier and Type | Class and Description |
---|---|
class |
AvroType<T>
The implementation of the PType interface for Avro-based serialization.
|
Modifier and Type | Method and Description |
---|---|
<T> PType<T> |
AvroTypeFamily.as(PType<T> ptype) |
PType<Boolean> |
AvroTypeFamily.booleans() |
PType<ByteBuffer> |
AvroTypeFamily.bytes() |
<T> PType<Collection<T>> |
AvroTypeFamily.collections(PType<T> ptype) |
<T> PType<T> |
AvroTypeFamily.containers(Class<T> clazz) |
<S,T> PType<T> |
AvroTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
AvroTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
PType<Double> |
AvroTypeFamily.doubles() |
PType<Float> |
AvroTypeFamily.floats() |
PType<org.apache.avro.generic.GenericData.Record> |
AvroTypeFamily.generics(org.apache.avro.Schema schema) |
static <T> PType<T> |
Avros.getPrimitiveType(Class<T> clazz) |
PType<Integer> |
AvroTypeFamily.ints() |
static <T> PType<T> |
Avros.jsons(Class<T> clazz) |
PType<Long> |
AvroTypeFamily.longs() |
<T> PType<Map<String,T>> |
AvroTypeFamily.maps(PType<T> ptype) |
PType<Void> |
AvroTypeFamily.nulls() |
<V1,V2> PType<Pair<V1,V2>> |
AvroTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
<V1,V2,V3,V4> |
AvroTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<T> PType<T> |
AvroTypeFamily.records(Class<T> clazz) |
PType<String> |
AvroTypeFamily.strings() |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
AvroTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<T extends Tuple> |
AvroTypeFamily.tuples(Class<T> clazz,
PType<?>... ptypes) |
PType<TupleN> |
AvroTypeFamily.tuples(PType<?>... ptypes) |
PType<Union> |
AvroTypeFamily.unionOf(PType<?>... ptypes) |
static PType<Union> |
Avros.unionOf(PType<?>... ptypes) |
Modifier and Type | Method and Description |
---|---|
List<PType> |
AvroType.getSubTypes() |
Modifier and Type | Method and Description |
---|---|
<T> PType<T> |
AvroTypeFamily.as(PType<T> ptype) |
<T> PType<Collection<T>> |
AvroTypeFamily.collections(PType<T> ptype) |
static <T> AvroType<Collection<T>> |
Avros.collections(PType<T> ptype) |
<S,T> PType<T> |
AvroTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> AvroType<T> |
Avros.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
AvroTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> AvroType<T> |
Avros.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <K,V> org.apache.crunch.types.avro.AvroKeyValueTableType<K,V> |
Avros.keyValueTableOf(PType<K> key,
PType<V> value)
A table type with an Avro type as key and value.
|
static <K,V> org.apache.crunch.types.avro.AvroKeyValueTableType<K,V> |
Avros.keyValueTableOf(PType<K> key,
PType<V> value)
A table type with an Avro type as key and value.
|
<T> PType<Map<String,T>> |
AvroTypeFamily.maps(PType<T> ptype) |
static <T> AvroType<Map<String,T>> |
Avros.maps(PType<T> ptype) |
static AvroType<TupleN> |
Avros.namedTuples(String tupleName,
String[] fieldNames,
PType[] ptypes) |
<V1,V2> PType<Pair<V1,V2>> |
AvroTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
<V1,V2> PType<Pair<V1,V2>> |
AvroTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
static <V1,V2> AvroType<Pair<V1,V2>> |
Avros.pairs(PType<V1> p1,
PType<V2> p2) |
static <V1,V2> AvroType<Pair<V1,V2>> |
Avros.pairs(PType<V1> p1,
PType<V2> p2) |
<V1,V2,V3,V4> |
AvroTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
AvroTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
AvroTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
AvroTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
static <V1,V2,V3,V4> |
Avros.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
static <V1,V2,V3,V4> |
Avros.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
static <V1,V2,V3,V4> |
Avros.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
static <V1,V2,V3,V4> |
Avros.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<K,V> PTableType<K,V> |
AvroTypeFamily.tableOf(PType<K> key,
PType<V> value) |
<K,V> PTableType<K,V> |
AvroTypeFamily.tableOf(PType<K> key,
PType<V> value) |
static <K,V> org.apache.crunch.types.avro.AvroTableType<K,V> |
Avros.tableOf(PType<K> key,
PType<V> value)
A table type with an Avro type as key and as value.
|
static <K,V> org.apache.crunch.types.avro.AvroTableType<K,V> |
Avros.tableOf(PType<K> key,
PType<V> value)
A table type with an Avro type as key and as value.
|
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
AvroTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
AvroTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
AvroTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <V1,V2,V3> AvroType<Tuple3<V1,V2,V3>> |
Avros.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <V1,V2,V3> AvroType<Tuple3<V1,V2,V3>> |
Avros.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <V1,V2,V3> AvroType<Tuple3<V1,V2,V3>> |
Avros.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <T extends Tuple> |
Avros.tuples(Class<T> clazz,
PType... ptypes) |
<T extends Tuple> |
AvroTypeFamily.tuples(Class<T> clazz,
PType<?>... ptypes) |
static AvroType<TupleN> |
Avros.tuples(PType... ptypes) |
PType<TupleN> |
AvroTypeFamily.tuples(PType<?>... ptypes) |
PType<Union> |
AvroTypeFamily.unionOf(PType<?>... ptypes) |
static PType<Union> |
Avros.unionOf(PType<?>... ptypes) |
Constructor and Description |
---|
AvroType(Class<T> typeClass,
org.apache.avro.Schema schema,
DeepCopier<T> deepCopier,
PType... ptypes) |
AvroType(Class<T> typeClass,
org.apache.avro.Schema schema,
MapFn inputMapFn,
MapFn outputMapFn,
DeepCopier<T> deepCopier,
AvroType.AvroRecordType recordType,
PType... ptypes) |
Modifier and Type | Method and Description |
---|---|
static PType<org.apache.hadoop.hive.ql.io.orc.OrcStruct> |
Orcs.orcs(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo typeInfo)
Create a PType to directly use OrcStruct as the deserialized format.
|
static <T> PType<T> |
Orcs.reflects(Class<T> clazz)
Create a PType which uses reflection to serialize/deserialize java POJOs
to/from ORC.
|
static PType<TupleN> |
Orcs.tuples(PType... ptypes)
Create a tuple-based PType.
|
Modifier and Type | Method and Description |
---|---|
static PType<TupleN> |
Orcs.tuples(PType... ptypes)
Create a tuple-based PType.
|
Constructor and Description |
---|
TupleObjectInspector(TupleFactory<T> tupleFactory,
PType... ptypes) |
Modifier and Type | Class and Description |
---|---|
class |
WritableType<T,W extends org.apache.hadoop.io.Writable> |
Modifier and Type | Method and Description |
---|---|
<T> PType<T> |
WritableTypeFamily.as(PType<T> ptype) |
PType<Boolean> |
WritableTypeFamily.booleans() |
PType<ByteBuffer> |
WritableTypeFamily.bytes() |
<T> PType<Collection<T>> |
WritableTypeFamily.collections(PType<T> ptype) |
<S,T> PType<T> |
WritableTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> PType<T> |
Writables.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
WritableTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> PType<T> |
Writables.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
PType<Double> |
WritableTypeFamily.doubles() |
PType<Float> |
WritableTypeFamily.floats() |
static <T> PType<T> |
Writables.getPrimitiveType(Class<T> clazz) |
PType<Integer> |
WritableTypeFamily.ints() |
static <T> PType<T> |
Writables.jsons(Class<T> clazz) |
PType<Long> |
WritableTypeFamily.longs() |
<T> PType<Map<String,T>> |
WritableTypeFamily.maps(PType<T> ptype) |
PType<Void> |
WritableTypeFamily.nulls() |
<V1,V2> PType<Pair<V1,V2>> |
WritableTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
<V1,V2,V3,V4> |
WritableTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<T> PType<T> |
WritableTypeFamily.records(Class<T> clazz) |
PType<String> |
WritableTypeFamily.strings() |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
WritableTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <T extends Tuple> |
Writables.tuples(Class<T> clazz,
PType... ptypes) |
<T extends Tuple> |
WritableTypeFamily.tuples(Class<T> clazz,
PType<?>... ptypes) |
PType<TupleN> |
WritableTypeFamily.tuples(PType<?>... ptypes) |
PType<Union> |
WritableTypeFamily.unionOf(PType<?>... ptypes) |
static PType<Union> |
Writables.unionOf(PType<?>... ptypes) |
<W extends org.apache.hadoop.io.Writable> |
WritableTypeFamily.writables(Class<W> clazz) |
Modifier and Type | Method and Description |
---|---|
List<PType> |
WritableType.getSubTypes() |
Modifier and Type | Method and Description |
---|---|
<T> PType<T> |
WritableTypeFamily.as(PType<T> ptype) |
<T> PType<Collection<T>> |
WritableTypeFamily.collections(PType<T> ptype) |
static <T> WritableType<Collection<T>,org.apache.crunch.types.writable.GenericArrayWritable> |
Writables.collections(PType<T> ptype) |
<S,T> PType<T> |
WritableTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> PType<T> |
Writables.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
WritableTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> PType<T> |
Writables.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <T,W extends org.apache.hadoop.io.Writable> |
WritableType.immutableType(Class<T> typeClass,
Class<W> writableClass,
MapFn<W,T> inputDoFn,
MapFn<T,W> outputDoFn,
PType... subTypes)
Factory method for a new WritableType instance whose type class is immutable.
|
<T> PType<Map<String,T>> |
WritableTypeFamily.maps(PType<T> ptype) |
static <T> WritableType<Map<String,T>,org.apache.hadoop.io.MapWritable> |
Writables.maps(PType<T> ptype) |
<V1,V2> PType<Pair<V1,V2>> |
WritableTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
<V1,V2> PType<Pair<V1,V2>> |
WritableTypeFamily.pairs(PType<V1> p1,
PType<V2> p2) |
static <V1,V2> WritableType<Pair<V1,V2>,TupleWritable> |
Writables.pairs(PType<V1> p1,
PType<V2> p2) |
static <V1,V2> WritableType<Pair<V1,V2>,TupleWritable> |
Writables.pairs(PType<V1> p1,
PType<V2> p2) |
<V1,V2,V3,V4> |
WritableTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
WritableTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
WritableTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<V1,V2,V3,V4> |
WritableTypeFamily.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
static <V1,V2,V3,V4> |
Writables.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
static <V1,V2,V3,V4> |
Writables.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
static <V1,V2,V3,V4> |
Writables.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
static <V1,V2,V3,V4> |
Writables.quads(PType<V1> p1,
PType<V2> p2,
PType<V3> p3,
PType<V4> p4) |
<K,V> PTableType<K,V> |
WritableTypeFamily.tableOf(PType<K> key,
PType<V> value) |
<K,V> PTableType<K,V> |
WritableTypeFamily.tableOf(PType<K> key,
PType<V> value) |
static <K,V> org.apache.crunch.types.writable.WritableTableType<K,V> |
Writables.tableOf(PType<K> key,
PType<V> value) |
static <K,V> org.apache.crunch.types.writable.WritableTableType<K,V> |
Writables.tableOf(PType<K> key,
PType<V> value) |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
WritableTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
WritableTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
<V1,V2,V3> PType<Tuple3<V1,V2,V3>> |
WritableTypeFamily.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <V1,V2,V3> WritableType<Tuple3<V1,V2,V3>,TupleWritable> |
Writables.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <V1,V2,V3> WritableType<Tuple3<V1,V2,V3>,TupleWritable> |
Writables.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <V1,V2,V3> WritableType<Tuple3<V1,V2,V3>,TupleWritable> |
Writables.triples(PType<V1> p1,
PType<V2> p2,
PType<V3> p3) |
static <T extends Tuple> |
Writables.tuples(Class<T> clazz,
PType... ptypes) |
<T extends Tuple> |
WritableTypeFamily.tuples(Class<T> clazz,
PType<?>... ptypes) |
static WritableType<TupleN,TupleWritable> |
Writables.tuples(PType... ptypes) |
PType<TupleN> |
WritableTypeFamily.tuples(PType<?>... ptypes) |
PType<Union> |
WritableTypeFamily.unionOf(PType<?>... ptypes) |
static PType<Union> |
Writables.unionOf(PType<?>... ptypes) |
Constructor and Description |
---|
WritableType(Class<T> typeClass,
Class<W> writableClass,
MapFn<W,T> inputDoFn,
MapFn<T,W> outputDoFn,
PType... subTypes) |
Copyright © 2017 The Apache Software Foundation. All rights reserved.