| Package | Description |
|---|---|
| org.apache.crunch |
Client-facing API and core abstractions.
|
| org.apache.crunch.fn |
Commonly used functions for manipulating collections.
|
| org.apache.crunch.impl.dist.collect | |
| org.apache.crunch.impl.spark.fn | |
| org.apache.crunch.lib |
Joining, sorting, aggregating, and other commonly used functionality.
|
| org.apache.crunch.lib.sort | |
| org.apache.crunch.types |
Common functionality for business object serialization.
|
| org.apache.crunch.types.avro |
Business object serialization using Apache Avro.
|
| org.apache.crunch.types.writable |
Business object serialization using Hadoop's Writables framework.
|
| Modifier and Type | Method and Description |
|---|---|
<K> PTable<K,S> |
PCollection.by(MapFn<S,K> extractKeyFn,
PType<K> keyType)
Apply the given map function to each element of this instance in order to
create a
PTable. |
<K> PTable<K,S> |
PCollection.by(String name,
MapFn<S,K> extractKeyFn,
PType<K> keyType)
Apply the given map function to each element of this instance in order to
create a
PTable. |
<K2> PTable<K2,V> |
PTable.mapKeys(MapFn<K,K2> mapFn,
PType<K2> ptype)
Returns a
PTable that has the same values as this instance, but
uses the given function to map the keys. |
<K2> PTable<K2,V> |
PTable.mapKeys(String name,
MapFn<K,K2> mapFn,
PType<K2> ptype)
Returns a
PTable that has the same values as this instance, but
uses the given function to map the keys. |
<U> PTable<K,U> |
PGroupedTable.mapValues(MapFn<Iterable<V>,U> mapFn,
PType<U> ptype)
Maps the
Iterable<V> elements of each record to a new type. |
<U> PTable<K,U> |
PTable.mapValues(MapFn<V,U> mapFn,
PType<U> ptype)
Returns a
PTable that has the same keys as this instance, but
uses the given function to map the values. |
<U> PTable<K,U> |
PGroupedTable.mapValues(String name,
MapFn<Iterable<V>,U> mapFn,
PType<U> ptype)
Maps the
Iterable<V> elements of each record to a new type. |
<U> PTable<K,U> |
PTable.mapValues(String name,
MapFn<V,U> mapFn,
PType<U> ptype)
Returns a
PTable that has the same keys as this instance, but
uses the given function to map the values. |
| Modifier and Type | Class and Description |
|---|---|
class |
CompositeMapFn<R,S,T> |
class |
ExtractKeyFn<K,V>
Wrapper function for converting a key-from-value extractor
MapFn<V, K> into a
key-value pair extractor that is used to convert from a PCollection<V> to a
PTable<K, V>. |
class |
IdentityFn<T> |
class |
PairMapFn<K,V,S,T> |
class |
SDoubleFunction<T>
A Crunch-compatible abstract base class for Spark's
DoubleFunction. |
class |
SFunction<T,R>
A Crunch-compatible abstract base class for Spark's
Function. |
class |
SFunction2<K,V,R>
A Crunch-compatible abstract base class for Spark's
Function2. |
class |
SPairFunction<T,K,V>
A Crunch-compatible abstract base class for Spark's
PairFunction. |
class |
SwapFn<V1,V2>
Swap the elements of a
Pair type. |
| Modifier and Type | Method and Description |
|---|---|
MapFn<R,S> |
CompositeMapFn.getFirst() |
MapFn<S,T> |
CompositeMapFn.getSecond() |
| Constructor and Description |
|---|
CompositeMapFn(MapFn<R,S> first,
MapFn<S,T> second) |
CompositeMapFn(MapFn<R,S> first,
MapFn<S,T> second) |
ExtractKeyFn(MapFn<V,K> mapFn) |
PairMapFn(MapFn<K,S> keys,
MapFn<V,T> values) |
PairMapFn(MapFn<K,S> keys,
MapFn<V,T> values) |
| Modifier and Type | Method and Description |
|---|---|
<K> PTable<K,S> |
PCollectionImpl.by(MapFn<S,K> mapFn,
PType<K> keyType) |
<K> PTable<K,S> |
PCollectionImpl.by(String name,
MapFn<S,K> mapFn,
PType<K> keyType) |
<K2> PTable<K2,V> |
PTableBase.mapKeys(MapFn<K,K2> mapFn,
PType<K2> ptype) |
<K2> PTable<K2,V> |
PTableBase.mapKeys(String name,
MapFn<K,K2> mapFn,
PType<K2> ptype) |
<U> PTable<K,U> |
BaseGroupedTable.mapValues(MapFn<Iterable<V>,U> mapFn,
PType<U> ptype) |
<U> PTable<K,U> |
PTableBase.mapValues(MapFn<V,U> mapFn,
PType<U> ptype) |
<U> PTable<K,U> |
BaseGroupedTable.mapValues(String name,
MapFn<Iterable<V>,U> mapFn,
PType<U> ptype) |
<U> PTable<K,U> |
PTableBase.mapValues(String name,
MapFn<V,U> mapFn,
PType<U> ptype) |
| Constructor and Description |
|---|
MapFunction(MapFn fn,
SparkRuntimeContext ctxt) |
PairMapFunction(MapFn<Pair<K,V>,S> fn,
SparkRuntimeContext ctxt) |
PairMapIterableFunction(MapFn<Pair<K,List<V>>,Pair<S,Iterable<T>>> fn,
SparkRuntimeContext runtimeContext) |
Tuple2MapFunction(MapFn<Pair<K,V>,Pair<K,V>> fn,
SparkRuntimeContext ctxt) |
| Modifier and Type | Method and Description |
|---|---|
static <K1,K2,V> PTable<K2,V> |
PTables.mapKeys(PTable<K1,V> ptable,
MapFn<K1,K2> mapFn,
PType<K2> ptype)
Maps a
PTable<K1, V> to a PTable<K2, V> using the given MapFn<K1, K2> on
the keys of the PTable. |
static <K1,K2,V> PTable<K2,V> |
PTables.mapKeys(String name,
PTable<K1,V> ptable,
MapFn<K1,K2> mapFn,
PType<K2> ptype)
Maps a
PTable<K1, V> to a PTable<K2, V> using the given MapFn<K1, K2> on
the keys of the PTable. |
static <K,U,V> PTable<K,V> |
PTables.mapValues(PGroupedTable<K,U> ptable,
MapFn<Iterable<U>,V> mapFn,
PType<V> ptype)
An analogue of the
mapValues function for PGroupedTable<K, U> collections. |
static <K,U,V> PTable<K,V> |
PTables.mapValues(PTable<K,U> ptable,
MapFn<U,V> mapFn,
PType<V> ptype)
Maps a
PTable<K, U> to a PTable<K, V> using the given MapFn<U, V> on
the values of the PTable. |
static <K,U,V> PTable<K,V> |
PTables.mapValues(String name,
PGroupedTable<K,U> ptable,
MapFn<Iterable<U>,V> mapFn,
PType<V> ptype)
An analogue of the
mapValues function for PGroupedTable<K, U> collections. |
static <K,U,V> PTable<K,V> |
PTables.mapValues(String name,
PTable<K,U> ptable,
MapFn<U,V> mapFn,
PType<V> ptype)
Maps a
PTable<K, U> to a PTable<K, V> using the given MapFn<U, V> on
the values of the PTable. |
| Modifier and Type | Class and Description |
|---|---|
static class |
SortFns.AvroGenericFn<V extends Tuple>
Pulls a composite set of keys from an Avro
GenericRecord instance. |
static class |
SortFns.SingleKeyFn<V extends Tuple,K>
Extracts a single indexed key from a
Tuple instance. |
static class |
SortFns.TupleKeyFn<V extends Tuple,K extends Tuple>
Extracts a composite key from a
Tuple instance. |
| Modifier and Type | Method and Description |
|---|---|
MapFn<V,Object> |
SortFns.KeyExtraction.getByFn() |
| Modifier and Type | Class and Description |
|---|---|
static class |
PGroupedTableType.PairIterableMapFn<K,V> |
| Modifier and Type | Field and Description |
|---|---|
static MapFn<BigDecimal,ByteBuffer> |
PTypes.BIGDECIMAL_TO_BYTE |
static MapFn<BigInteger,ByteBuffer> |
PTypes.BIGINT_TO_BYTE |
static MapFn<ByteBuffer,BigDecimal> |
PTypes.BYTE_TO_BIGDECIMAL |
static MapFn<ByteBuffer,BigInteger> |
PTypes.BYTE_TO_BIGINT |
| Modifier and Type | Method and Description |
|---|---|
static <M extends com.google.protobuf.Message,K> |
Protos.extractKey(String fieldName) |
MapFn<Object,T> |
PType.getInputMapFn() |
MapFn<T,Object> |
PType.getOutputMapFn() |
| Modifier and Type | Method and Description |
|---|---|
<S,T> PType<T> |
PTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
PTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
PTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base)
A derived type whose values are immutable.
|
<S,T> PType<T> |
PTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base)
A derived type whose values are immutable.
|
| Constructor and Description |
|---|
PairIterableMapFn(MapFn<Object,K> keys,
MapFn<Object,V> values) |
PairIterableMapFn(MapFn<Object,K> keys,
MapFn<Object,V> values) |
| Modifier and Type | Field and Description |
|---|---|
static MapFn<Object,ByteBuffer> |
Avros.BYTES_IN |
static MapFn<String,org.apache.avro.util.Utf8> |
Avros.STRING_TO_UTF8 |
static MapFn<CharSequence,String> |
Avros.UTF8_TO_STRING |
| Modifier and Type | Method and Description |
|---|---|
MapFn<Object,T> |
AvroType.getInputMapFn() |
MapFn<T,Object> |
AvroType.getOutputMapFn() |
| Modifier and Type | Method and Description |
|---|---|
<S,T> PType<T> |
AvroTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
AvroTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> AvroType<T> |
Avros.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> AvroType<T> |
Avros.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
AvroTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
AvroTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> AvroType<T> |
Avros.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> AvroType<T> |
Avros.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
| Constructor and Description |
|---|
AvroDerivedValueDeepCopier(MapFn<T,S> derivedToAvroFn,
MapFn<S,T> avroToDerivedFn,
AvroType<S> avroBaseType) |
AvroDerivedValueDeepCopier(MapFn<T,S> derivedToAvroFn,
MapFn<S,T> avroToDerivedFn,
AvroType<S> avroBaseType) |
AvroType(Class<T> typeClass,
org.apache.avro.Schema schema,
MapFn inputMapFn,
MapFn outputMapFn,
DeepCopier<T> deepCopier,
AvroType.AvroRecordType recordType,
PType... ptypes) |
AvroType(Class<T> typeClass,
org.apache.avro.Schema schema,
MapFn inputMapFn,
MapFn outputMapFn,
DeepCopier<T> deepCopier,
AvroType.AvroRecordType recordType,
PType... ptypes) |
| Modifier and Type | Method and Description |
|---|---|
MapFn |
WritableType.getInputMapFn() |
MapFn |
WritableType.getOutputMapFn() |
| Modifier and Type | Method and Description |
|---|---|
<S,T> PType<T> |
WritableTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
WritableTypeFamily.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> PType<T> |
Writables.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> PType<T> |
Writables.derived(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
WritableTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
<S,T> PType<T> |
WritableTypeFamily.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> PType<T> |
Writables.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <S,T> PType<T> |
Writables.derivedImmutable(Class<T> clazz,
MapFn<S,T> inputFn,
MapFn<T,S> outputFn,
PType<S> base) |
static <T,W extends org.apache.hadoop.io.Writable> |
WritableType.immutableType(Class<T> typeClass,
Class<W> writableClass,
MapFn<W,T> inputDoFn,
MapFn<T,W> outputDoFn,
PType... subTypes)
Factory method for a new WritableType instance whose type class is immutable.
|
static <T,W extends org.apache.hadoop.io.Writable> |
WritableType.immutableType(Class<T> typeClass,
Class<W> writableClass,
MapFn<W,T> inputDoFn,
MapFn<T,W> outputDoFn,
PType... subTypes)
Factory method for a new WritableType instance whose type class is immutable.
|
| Constructor and Description |
|---|
WritableType(Class<T> typeClass,
Class<W> writableClass,
MapFn<W,T> inputDoFn,
MapFn<T,W> outputDoFn,
PType... subTypes) |
WritableType(Class<T> typeClass,
Class<W> writableClass,
MapFn<W,T> inputDoFn,
MapFn<T,W> outputDoFn,
PType... subTypes) |
Copyright © 2017 The Apache Software Foundation. All rights reserved.