LCollection (Apache Crunch 0.15.0 API)

Type Parameters:

S - The type of the elements in this collection

All Known Subinterfaces:

LGroupedTable<K,V>, LTable<K,V>
```
public interface LCollection<S>
```
Java 8 friendly version of the PCollection interface, allowing distributed operations to be expressed in terms of lambda expressions and method references, instead of creating a new class implementation for each operation.

Method Summary

All Methods Instance Methods Abstract Methods Default Methods
Modifier and Type	Method and Description
`default <K> LTable<K,S>`	`by(SFunction<S,K> extractFn, PType<K> pType)` Key this LCollection by a key extracted from the element to yield a `LTable` mapping the key to the whole element.
`default LCollection<S>`	`cache()` Cache the underlying `PCollection`
`default LCollection<S>`	`cache(CachingOptions options)` Cache the underlying `PCollection`
`default LTable<S,Long>`	`count()` Count distict values in this LCollection, yielding an `LTable` mapping each value to the number of occurrences in the collection.
`LCollectionFactory`	`factory()` Get the `LCollectionFactory` which can be used to create new Ltype instances
`default LCollection<S>`	`filter(SPredicate<S> predicate)` Filter the collection using the supplied predicate.
`default <K,V> LTable<K,V>`	`filterMap(SFunction<S,Optional<Pair<K,V>>> fn, PTableType<K,V> pType)` Combination of a filter and map operation by using a function with `Optional` return type.
`default <T> LCollection<T>`	`filterMap(SFunction<S,Optional<T>> fn, PType<T> pType)` Combination of a filter and map operation by using a function with `Optional` return type.
`default <K,V> LTable<K,V>`	`flatMap(SFunction<S,java.util.stream.Stream<Pair<K,V>>> fn, PTableType<K,V> pType)` Map each element to zero or more output elements using the provided stream-returning function to yield an `LTable`
`default <T> LCollection<T>`	`flatMap(SFunction<S,java.util.stream.Stream<T>> fn, PType<T> pType)` Map each element to zero or more output elements using the provided stream-returning function.
`default LCollection<S>`	`increment(Enum<?> counter)` Increment a counter for every element in the collection
`default LCollection<S>`	`increment(String counterGroup, String counterName)` Increment a counter for every element in the collection
`default LCollection<S>`	`incrementIf(Enum<?> counter, SPredicate<S> condition)` Increment a counter for every element satisfying the conditional predicate supplied.
`default LCollection<S>`	`incrementIf(String counterGroup, String counterName, SPredicate<S> condition)` Increment a counter for every element satisfying the conditional predicate supplied.
`default <K,V> LTable<K,V>`	`map(SFunction<S,Pair<K,V>> fn, PTableType<K,V> pType)` Map the elements of this collection 1-1 through the supplied function to yield an `LTable`
`default <T> LCollection<T>`	`map(SFunction<S,T> fn, PType<T> pType)` Map the elements of this collection 1-1 through the supplied function.
`default java.util.stream.Stream<S>`	`materialize()` Obtain the contents of this LCollection as a `Stream` that can be processed locally.
`default <K,V> LTable<K,V>`	`parallelDo(DoFn<S,Pair<K,V>> fn, PTableType<K,V> pType)` Transform this LCollection to an `LTable` using a standard Crunch `DoFn`
`default <T> LCollection<T>`	`parallelDo(DoFn<S,T> fn, PType<T> pType)` Transform this LCollection using a standard Crunch `DoFn`
`default <K,V> LTable<K,V>`	`parallelDo(LDoFn<S,Pair<K,V>> fn, PTableType<K,V> pType)` Transform this LCollection using a Lambda-friendly `LDoFn`.
`default <T> LCollection<T>`	`parallelDo(LDoFn<S,T> fn, PType<T> pType)` Transform this LCollection using a Lambda-friendly `LDoFn`.
`default PTypeFamily`	`ptf()` Get the `PTypeFamily` representing how elements of this collection may be serialized.
`default PType<S>`	`pType()` Get the `PType` representing how elements of this collection may be serialized.
`PCollection<S>`	`underlying()` Get the underlying `PCollection` for this LCollection
`default LCollection<S>`	`union(LCollection<S> other)` Union this LCollection with another LCollection of the same type
`default LCollection<S>`	`union(PCollection<S> other)` Union this LCollection with a `PCollection` of the same type
`default LCollection<S>`	`write(Target target)` Write this collection to the specified `Target`
`default LCollection<S>`	`write(Target target, Target.WriteMode writeMode)` Write this collection to the specified `Target` with the given `Target.WriteMode`

Method Detail

underlying
```
PCollection<S> underlying()
```
Get the underlying PCollection for this LCollection

factory
```
LCollectionFactory factory()
```
Get the LCollectionFactory which can be used to create new Ltype instances

parallelDo

default <T> LCollection<T> parallelDo(DoFn<S,T> fn,
                                      PType<T> pType)

Transform this LCollection using a standard Crunch DoFn

parallelDo

default <K,V> LTable<K,V> parallelDo(DoFn<S,Pair<K,V>> fn,
                                     PTableType<K,V> pType)

Transform this LCollection to an LTable using a standard Crunch DoFn

parallelDo

default <T> LCollection<T> parallelDo(LDoFn<S,T> fn,
                                      PType<T> pType)

Transform this LCollection using a Lambda-friendly LDoFn.

parallelDo

default <K,V> LTable<K,V> parallelDo(LDoFn<S,Pair<K,V>> fn,
                                     PTableType<K,V> pType)

Transform this LCollection using a Lambda-friendly LDoFn.

map

default <T> LCollection<T> map(SFunction<S,T> fn,
                               PType<T> pType)

Map the elements of this collection 1-1 through the supplied function.

map

default <K,V> LTable<K,V> map(SFunction<S,Pair<K,V>> fn,
                              PTableType<K,V> pType)

Map the elements of this collection 1-1 through the supplied function to yield an LTable

flatMap

default <T> LCollection<T> flatMap(SFunction<S,java.util.stream.Stream<T>> fn,
                                   PType<T> pType)

Map each element to zero or more output elements using the provided stream-returning function.

flatMap

default <K,V> LTable<K,V> flatMap(SFunction<S,java.util.stream.Stream<Pair<K,V>>> fn,
                                  PTableType<K,V> pType)

Map each element to zero or more output elements using the provided stream-returning function to yield an LTable

filterMap

default <T> LCollection<T> filterMap(SFunction<S,Optional<T>> fn,
                                     PType<T> pType)

Combination of a filter and map operation by using a function with Optional return type.

filterMap

default <K,V> LTable<K,V> filterMap(SFunction<S,Optional<Pair<K,V>>> fn,
                                    PTableType<K,V> pType)

Combination of a filter and map operation by using a function with Optional return type.

filter

default LCollection<S> filter(SPredicate<S> predicate)

Filter the collection using the supplied predicate.

union
```
default LCollection<S> union(LCollection<S> other)
```
Union this LCollection with another LCollection of the same type

union
```
default LCollection<S> union(PCollection<S> other)
```
Union this LCollection with a PCollection of the same type

increment
```
default LCollection<S> increment(Enum<?> counter)
```
Increment a counter for every element in the collection

increment

default LCollection<S> increment(String counterGroup,
                                 String counterName)

Increment a counter for every element in the collection

incrementIf

default LCollection<S> incrementIf(Enum<?> counter,
                                   SPredicate<S> condition)

Increment a counter for every element satisfying the conditional predicate supplied.

incrementIf

default LCollection<S> incrementIf(String counterGroup,
                                   String counterName,
                                   SPredicate<S> condition)

Increment a counter for every element satisfying the conditional predicate supplied.

cache
```
default LCollection<S> cache()
```
Cache the underlying PCollection

cache

default LCollection<S> cache(CachingOptions options)

Cache the underlying PCollection

by

default <K> LTable<K,S> by(SFunction<S,K> extractFn,
                           PType<K> pType)

Key this LCollection by a key extracted from the element to yield a LTable mapping the key to the whole element.

count
```
default LTable<S,Long> count()
```
Count distict values in this LCollection, yielding an LTable mapping each value to the number of occurrences in the collection.

materialize
```
default java.util.stream.Stream<S> materialize()
```
Obtain the contents of this LCollection as a Stream that can be processed locally. Note, this may trigger your job to execute in a distributed environment if the pipeline has not yet been run.

ptf
```
default PTypeFamily ptf()
```
Get the PTypeFamily representing how elements of this collection may be serialized.

pType
```
default PType<S> pType()
```
Get the PType representing how elements of this collection may be serialized.

write

default LCollection<S> write(Target target)

Write this collection to the specified Target

write

default LCollection<S> write(Target target,
                             Target.WriteMode writeMode)

Write this collection to the specified Target with the given Target.WriteMode

Interface LCollection<S>

Method Summary

Method Detail

underlying

factory

parallelDo

parallelDo

parallelDo

parallelDo

map

map

flatMap

flatMap

filterMap

filterMap

filter

union

union

increment

increment

incrementIf

incrementIf

cache

cache

by

count

materialize

ptf

pType

write

write