Package | Description |
---|---|
org.apache.crunch.impl.dist.collect | |
org.apache.crunch.impl.spark | |
org.apache.crunch.impl.spark.collect |
Modifier and Type | Class and Description |
---|---|
class |
BaseDoCollection<S> |
class |
BaseDoTable<K,V> |
class |
BaseGroupedTable<K,V> |
class |
BaseInputCollection<S> |
class |
BaseInputTable<K,V> |
class |
BaseUnionCollection<S> |
class |
BaseUnionTable<K,V> |
class |
EmptyPCollection<T> |
class |
EmptyPTable<K,V> |
class |
PTableBase<K,V> |
Modifier and Type | Method and Description |
---|---|
PCollectionImpl<?> |
PCollectionImpl.getOnlyParent() |
Modifier and Type | Method and Description |
---|---|
abstract List<PCollectionImpl<?>> |
PCollectionImpl.getParents() |
List<PCollectionImpl<?>> |
EmptyPTable.getParents() |
List<PCollectionImpl<?>> |
EmptyPCollection.getParents() |
List<PCollectionImpl<?>> |
BaseUnionTable.getParents() |
List<PCollectionImpl<?>> |
BaseUnionCollection.getParents() |
List<PCollectionImpl<?>> |
BaseInputTable.getParents() |
List<PCollectionImpl<?>> |
BaseInputCollection.getParents() |
List<PCollectionImpl<?>> |
BaseGroupedTable.getParents() |
List<PCollectionImpl<?>> |
BaseDoTable.getParents() |
List<PCollectionImpl<?>> |
BaseDoCollection.getParents() |
Modifier and Type | Method and Description |
---|---|
<S,T> BaseDoCollection<T> |
PCollectionFactory.createDoCollection(String name,
PCollectionImpl<S> chainingCollection,
DoFn<S,T> fn,
PType<T> type,
ParallelDoOptions options) |
<S,K,V> BaseDoTable<K,V> |
PCollectionFactory.createDoTable(String name,
PCollectionImpl<S> chainingCollection,
CombineFn<K,V> combineFn,
DoFn<S,Pair<K,V>> fn,
PTableType<K,V> type) |
<S,K,V> BaseDoTable<K,V> |
PCollectionFactory.createDoTable(String name,
PCollectionImpl<S> chainingCollection,
DoFn<S,Pair<K,V>> fn,
PTableType<K,V> type,
ParallelDoOptions options) |
Modifier and Type | Method and Description |
---|---|
<S> BaseUnionCollection<S> |
PCollectionFactory.createUnionCollection(List<? extends PCollectionImpl<S>> internal) |
Constructor and Description |
---|
SparkRuntime(SparkPipeline pipeline,
org.apache.spark.api.java.JavaSparkContext sparkContext,
org.apache.hadoop.conf.Configuration conf,
Map<PCollectionImpl<?>,Set<Target>> outputTargets,
Map<PCollectionImpl<?>,org.apache.crunch.materialize.MaterializableIterable> toMaterialize,
Map<PCollection<?>,org.apache.spark.storage.StorageLevel> toCache,
Map<PipelineCallable<?>,Set<Target>> allPipelineCallables) |
SparkRuntime(SparkPipeline pipeline,
org.apache.spark.api.java.JavaSparkContext sparkContext,
org.apache.hadoop.conf.Configuration conf,
Map<PCollectionImpl<?>,Set<Target>> outputTargets,
Map<PCollectionImpl<?>,org.apache.crunch.materialize.MaterializableIterable> toMaterialize,
Map<PCollection<?>,org.apache.spark.storage.StorageLevel> toCache,
Map<PipelineCallable<?>,Set<Target>> allPipelineCallables) |
Modifier and Type | Class and Description |
---|---|
class |
CreatedCollection<T>
Represents a Spark-based PCollection that was created from a Java
Iterable of
values. |
class |
CreatedTable<K,V>
Represents a Spark-based PTable that was created from a Java
Iterable of
key-value pairs. |
class |
DoCollection<S> |
class |
DoTable<K,V> |
class |
InputCollection<S> |
class |
InputTable<K,V> |
class |
PGroupedTableImpl<K,V> |
class |
UnionCollection<S> |
class |
UnionTable<K,V> |
Modifier and Type | Method and Description |
---|---|
List<PCollectionImpl<?>> |
CreatedTable.getParents() |
List<PCollectionImpl<?>> |
CreatedCollection.getParents() |
Modifier and Type | Method and Description |
---|---|
<S,T> BaseDoCollection<T> |
SparkCollectFactory.createDoCollection(String name,
PCollectionImpl<S> parent,
DoFn<S,T> fn,
PType<T> type,
ParallelDoOptions options) |
<S,K,V> BaseDoTable<K,V> |
SparkCollectFactory.createDoTable(String name,
PCollectionImpl<S> parent,
CombineFn<K,V> combineFn,
DoFn<S,Pair<K,V>> fn,
PTableType<K,V> type) |
<S,K,V> BaseDoTable<K,V> |
SparkCollectFactory.createDoTable(String name,
PCollectionImpl<S> parent,
DoFn<S,Pair<K,V>> fn,
PTableType<K,V> type,
ParallelDoOptions options) |
Modifier and Type | Method and Description |
---|---|
<S> BaseUnionCollection<S> |
SparkCollectFactory.createUnionCollection(List<? extends PCollectionImpl<S>> internal) |
Copyright © 2016 The Apache Software Foundation. All rights reserved.