org.apache.crunch.impl.spark.collect

Class CreatedTable<K,V>

java.lang.Object

org.apache.crunch.impl.dist.collect.PCollectionImpl<Pair<K,V>>

org.apache.crunch.impl.dist.collect.PTableBase<K,V>

org.apache.crunch.impl.spark.collect.CreatedTable<K,V>

All Implemented Interfaces:

SparkCollection, PCollection<Pair<K,V>>, PTable<K,V>

public class CreatedTable<K,V>
extends PTableBase<K,V>
implements SparkCollection

Represents a Spark-based PTable that was created from a Java Iterable of key-value pairs.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.crunch.impl.dist.collect.PCollectionImpl

PCollectionImpl.Visitor

Constructor Summary

Constructors

Constructor and Description
CreatedTable(SparkPipeline pipeline, Iterable<Pair<K,V>> contents, PTableType<K,V> ptype, CreateOptions options)

Method Summary

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaRDDLike<?,?>	getJavaRDDLike(SparkRuntime runtime)
long	getLastModifiedAt() The time of the most recent modification to one of the input sources to the collection.
List<PCollectionImpl<?>>	getParents()
PTableType<K,V>	getPTableType() Returns the PTableType of this PTable.
PType<Pair<K,V>>	getPType() Returns the PType of this PCollection.

Methods inherited from class org.apache.crunch.impl.dist.collect.PTableBase

asMap, bottom, cache, cache, cogroup, collectValues, filter, filter, getKeyType, getValueType, groupByKey, groupByKey, groupByKey, join, keys, mapKeys, mapKeys, mapValues, mapValues, materializeToMap, top, union, union, values, write, write

Methods inherited from class org.apache.crunch.impl.dist.collect.PCollectionImpl

accept, aggregate, asCollection, asReadable, by, by, count, first, getDepth, getMaterializedAt, getName, getOnlyParent, getParallelDoOptions, getPipeline, getSize, getTargetDependencies, getTypeFamily, isBreakpoint, length, materialize, materializeAt, max, min, parallelDo, parallelDo, parallelDo, parallelDo, parallelDo, parallelDo, sequentialDo, setBreakpoint, toString, union, union

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.crunch.PCollection

aggregate, asCollection, asReadable, by, by, count, first, getName, getPipeline, getSize, getTypeFamily, length, materialize, max, min, parallelDo, parallelDo, parallelDo, parallelDo, parallelDo, parallelDo, sequentialDo, union, union

Constructor Detail

CreatedTable

public CreatedTable(SparkPipeline pipeline,
                    Iterable<Pair<K,V>> contents,
                    PTableType<K,V> ptype,
                    CreateOptions options)

Method Detail

getParents

public List<PCollectionImpl<?>> getParents()

Specified by:

getParents in class PCollectionImpl<Pair<K,V>>

getLastModifiedAt

public long getLastModifiedAt()

Description copied from class: PCollectionImpl

The time of the most recent modification to one of the input sources to the collection. If the time can not be determined then -1 should be returned.

Specified by:

getLastModifiedAt in class PCollectionImpl<Pair<K,V>>

Returns:

time of the most recent modification to one of the input sources to the collection.

getPTableType

public PTableType<K,V> getPTableType()

Description copied from interface: PTable

Returns the PTableType of this PTable.

Specified by:

getPTableType in interface PTable<K,V>

getPType

public PType<Pair<K,V>> getPType()

Description copied from interface: PCollection

Returns the PType of this PCollection.

Specified by:

getPType in interface PCollection<Pair<K,V>>

getJavaRDDLike

public org.apache.spark.api.java.JavaRDDLike<?,?> getJavaRDDLike(SparkRuntime runtime)

Specified by:

getJavaRDDLike in interface SparkCollection