| Package | Description |
|---|---|
| org.apache.crunch |
Client-facing API and core abstractions.
|
| org.apache.crunch.fn |
Commonly used functions for manipulating collections.
|
| org.apache.crunch.impl.dist.collect | |
| org.apache.crunch.lambda |
Alternative Crunch API using Java 8 features to allow construction of pipelines using lambda functions and method
references.
|
| org.apache.crunch.lib |
Joining, sorting, aggregating, and other commonly used functionality.
|
| Modifier and Type | Method and Description |
|---|---|
PCollection<S> |
PCollection.aggregate(Aggregator<S> aggregator)
Returns a
PCollection that contains the result of aggregating all values in this instance. |
PTable<K,V> |
PGroupedTable.combineValues(Aggregator<V> aggregator)
Combine the values in each group using the given
Aggregator. |
PTable<K,V> |
PGroupedTable.combineValues(Aggregator<V> combineAggregator,
Aggregator<V> reduceAggregator)
Combine and reduces the values in each group using the given
Aggregator instances. |
PTable<K,V> |
PGroupedTable.combineValues(Aggregator<V> combineAggregator,
Aggregator<V> reduceAggregator)
Combine and reduces the values in each group using the given
Aggregator instances. |
| Modifier and Type | Class and Description |
|---|---|
static class |
Aggregators.SimpleAggregator<T>
Base class for aggregators that do not require any initialization.
|
| Modifier and Type | Method and Description |
|---|---|
static <V> Aggregator<V> |
Aggregators.FIRST_N(int n)
Return the first
n values (or fewer if there are fewer values than n). |
static <V> Aggregator<V> |
Aggregators.LAST_N(int n)
Return the last
n values (or fewer if there are fewer values than n). |
static Aggregator<BigDecimal> |
Aggregators.MAX_BIGDECIMALS()
Return the maximum of all given
BigDecimal values. |
static Aggregator<BigDecimal> |
Aggregators.MAX_BIGDECIMALS(int n)
|
static Aggregator<BigInteger> |
Aggregators.MAX_BIGINTS()
Return the maximum of all given
BigInteger values. |
static Aggregator<BigInteger> |
Aggregators.MAX_BIGINTS(int n)
|
static <C extends Comparable<C>> |
Aggregators.MAX_COMPARABLES()
Return the maximum of all given
Comparable values. |
static Aggregator<Double> |
Aggregators.MAX_DOUBLES()
Return the maximum of all given
double values. |
static Aggregator<Double> |
Aggregators.MAX_DOUBLES(int n)
Return the
n largest double values (or fewer if there are fewer
values than n). |
static Aggregator<Float> |
Aggregators.MAX_FLOATS()
Return the maximum of all given
float values. |
static Aggregator<Float> |
Aggregators.MAX_FLOATS(int n)
Return the
n largest float values (or fewer if there are fewer
values than n). |
static Aggregator<Integer> |
Aggregators.MAX_INTS()
Return the maximum of all given
int values. |
static Aggregator<Integer> |
Aggregators.MAX_INTS(int n)
Return the
n largest int values (or fewer if there are fewer
values than n). |
static Aggregator<Long> |
Aggregators.MAX_LONGS()
Return the maximum of all given
long values. |
static Aggregator<Long> |
Aggregators.MAX_LONGS(int n)
Return the
n largest long values (or fewer if there are fewer
values than n). |
static <V extends Comparable<V>> |
Aggregators.MAX_N(int n,
Class<V> cls)
Return the
n largest values (or fewer if there are fewer
values than n). |
static <V extends Comparable<V>> |
Aggregators.MAX_UNIQUE_N(int n,
Class<V> cls)
Return the
n largest unique values (or fewer if there are fewer
values than n). |
static Aggregator<BigDecimal> |
Aggregators.MIN_BIGDECIMALS()
Return the minimum of all given
BigDecimal values. |
static Aggregator<BigDecimal> |
Aggregators.MIN_BIGDECIMALS(int n)
|
static Aggregator<BigInteger> |
Aggregators.MIN_BIGINTS()
Return the minimum of all given
BigInteger values. |
static Aggregator<BigInteger> |
Aggregators.MIN_BIGINTS(int n)
|
static <C extends Comparable<C>> |
Aggregators.MIN_COMPARABLES()
Return the minimum of all given
Comparable values. |
static Aggregator<Double> |
Aggregators.MIN_DOUBLES()
Return the minimum of all given
double values. |
static Aggregator<Double> |
Aggregators.MIN_DOUBLES(int n)
Return the
n smallest double values (or fewer if there are fewer
values than n). |
static Aggregator<Float> |
Aggregators.MIN_FLOATS()
Return the minimum of all given
float values. |
static Aggregator<Float> |
Aggregators.MIN_FLOATS(int n)
Return the
n smallest float values (or fewer if there are fewer
values than n). |
static Aggregator<Integer> |
Aggregators.MIN_INTS()
Return the minimum of all given
int values. |
static Aggregator<Integer> |
Aggregators.MIN_INTS(int n)
Return the
n smallest int values (or fewer if there are fewer
values than n). |
static Aggregator<Long> |
Aggregators.MIN_LONGS()
Return the minimum of all given
long values. |
static Aggregator<Long> |
Aggregators.MIN_LONGS(int n)
Return the
n smallest long values (or fewer if there are fewer
values than n). |
static <V extends Comparable<V>> |
Aggregators.MIN_N(int n,
Class<V> cls)
Return the
n smallest values (or fewer if there are fewer
values than n). |
static <V extends Comparable<V>> |
Aggregators.MIN_UNIQUE_N(int n,
Class<V> cls)
Returns the
n smallest unique values (or fewer if there are fewer unique values than n). |
static <V1,V2> Aggregator<Pair<V1,V2>> |
Aggregators.pairAggregator(Aggregator<V1> a1,
Aggregator<V2> a2)
Apply separate aggregators to each component of a
Pair. |
static <V1,V2,V3,V4> |
Aggregators.quadAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3,
Aggregator<V4> a4)
Apply separate aggregators to each component of a
Tuple4. |
static <V> Aggregator<V> |
Aggregators.SAMPLE_UNIQUE_ELEMENTS(int maximumSampleSize)
Collect a sample of unique elements from the input, where 'unique' is defined by
the
equals method for the input objects. |
static Aggregator<String> |
Aggregators.STRING_CONCAT(String separator,
boolean skipNull)
Concatenate strings, with a separator between strings.
|
static Aggregator<String> |
Aggregators.STRING_CONCAT(String separator,
boolean skipNull,
long maxOutputLength,
long maxInputLength)
Concatenate strings, with a separator between strings.
|
static Aggregator<BigDecimal> |
Aggregators.SUM_BIGDECIMALS()
Sum up all
BigDecimal values. |
static Aggregator<BigInteger> |
Aggregators.SUM_BIGINTS()
Sum up all
BigInteger values. |
static Aggregator<Double> |
Aggregators.SUM_DOUBLES()
Sum up all
double values. |
static Aggregator<Float> |
Aggregators.SUM_FLOATS()
Sum up all
float values. |
static Aggregator<Integer> |
Aggregators.SUM_INTS()
Sum up all
int values. |
static Aggregator<Long> |
Aggregators.SUM_LONGS()
Sum up all
long values. |
static <V1,V2,V3> Aggregator<Tuple3<V1,V2,V3>> |
Aggregators.tripAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3)
Apply separate aggregators to each component of a
Tuple3. |
static Aggregator<TupleN> |
Aggregators.tupleAggregator(Aggregator<?>... aggregators)
Apply separate aggregators to each component of a
Tuple. |
static <V> Aggregator<V> |
Aggregators.UNIQUE_ELEMENTS()
Collect the unique elements of the input, as defined by the
equals method for
the input objects. |
| Modifier and Type | Method and Description |
|---|---|
static <V1,V2> Aggregator<Pair<V1,V2>> |
Aggregators.pairAggregator(Aggregator<V1> a1,
Aggregator<V2> a2)
Apply separate aggregators to each component of a
Pair. |
static <V1,V2> Aggregator<Pair<V1,V2>> |
Aggregators.pairAggregator(Aggregator<V1> a1,
Aggregator<V2> a2)
Apply separate aggregators to each component of a
Pair. |
static <V1,V2,V3,V4> |
Aggregators.quadAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3,
Aggregator<V4> a4)
Apply separate aggregators to each component of a
Tuple4. |
static <V1,V2,V3,V4> |
Aggregators.quadAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3,
Aggregator<V4> a4)
Apply separate aggregators to each component of a
Tuple4. |
static <V1,V2,V3,V4> |
Aggregators.quadAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3,
Aggregator<V4> a4)
Apply separate aggregators to each component of a
Tuple4. |
static <V1,V2,V3,V4> |
Aggregators.quadAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3,
Aggregator<V4> a4)
Apply separate aggregators to each component of a
Tuple4. |
static <K,V> CombineFn<K,V> |
Aggregators.toCombineFn(Aggregator<V> aggregator)
Deprecated.
use the safer
Aggregators.toCombineFn(Aggregator, PType) instead. |
static <K,V> CombineFn<K,V> |
Aggregators.toCombineFn(Aggregator<V> aggregator,
PType<V> ptype)
Wrap a
CombineFn adapter around the given aggregator. |
static <V1,V2,V3> Aggregator<Tuple3<V1,V2,V3>> |
Aggregators.tripAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3)
Apply separate aggregators to each component of a
Tuple3. |
static <V1,V2,V3> Aggregator<Tuple3<V1,V2,V3>> |
Aggregators.tripAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3)
Apply separate aggregators to each component of a
Tuple3. |
static <V1,V2,V3> Aggregator<Tuple3<V1,V2,V3>> |
Aggregators.tripAggregator(Aggregator<V1> a1,
Aggregator<V2> a2,
Aggregator<V3> a3)
Apply separate aggregators to each component of a
Tuple3. |
static Aggregator<TupleN> |
Aggregators.tupleAggregator(Aggregator<?>... aggregators)
Apply separate aggregators to each component of a
Tuple. |
| Modifier and Type | Method and Description |
|---|---|
PCollection<S> |
PCollectionImpl.aggregate(Aggregator<S> aggregator) |
PTable<K,V> |
BaseGroupedTable.combineValues(Aggregator<V> agg) |
PTable<K,V> |
BaseGroupedTable.combineValues(Aggregator<V> combineAgg,
Aggregator<V> reduceAgg) |
PTable<K,V> |
BaseGroupedTable.combineValues(Aggregator<V> combineAgg,
Aggregator<V> reduceAgg) |
| Modifier and Type | Class and Description |
|---|---|
class |
LAggregator<V,A>
Crunch Aggregator expressed as a composition of functional interface implementations
|
| Modifier and Type | Method and Description |
|---|---|
default LTable<K,V> |
LGroupedTable.combineValues(Aggregator<V> aggregator)
Combine the value part of the table using the provided Crunch
Aggregator. |
| Modifier and Type | Method and Description |
|---|---|
static <S> PCollection<S> |
Aggregate.aggregate(PCollection<S> collect,
Aggregator<S> aggregator) |
Copyright © 2017 The Apache Software Foundation. All rights reserved.