org.apache.crunch.impl.spark.collect

Class CreatedCollection<T>

java.lang.Object

org.apache.crunch.impl.dist.collect.PCollectionImpl<T>

org.apache.crunch.impl.spark.collect.CreatedCollection<T>

All Implemented Interfaces:

SparkCollection, PCollection<T>

public class CreatedCollection<T>
extends PCollectionImpl<T>
implements SparkCollection

Represents a Spark-based PCollection that was created from a Java Iterable of values.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.crunch.impl.dist.collect.PCollectionImpl

PCollectionImpl.Visitor

Constructor Summary

Constructors

Constructor and Description
CreatedCollection(SparkPipeline p, Iterable<T> contents, PType<T> ptype, CreateOptions options)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaRDDLike<?,?>	getJavaRDDLike(SparkRuntime runtime)
long	getLastModifiedAt() The time of the most recent modification to one of the input sources to the collection.
List<PCollectionImpl<?>>	getParents()
PType<T>	getPType() Returns the PType of this PCollection.

Methods inherited from class org.apache.crunch.impl.dist.collect.PCollectionImpl

accept, aggregate, asCollection, asReadable, by, by, cache, cache, count, filter, filter, first, getDepth, getMaterializedAt, getName, getOnlyParent, getParallelDoOptions, getPipeline, getSize, getTargetDependencies, getTypeFamily, isBreakpoint, length, materialize, materializeAt, max, min, parallelDo, parallelDo, parallelDo, parallelDo, parallelDo, parallelDo, sequentialDo, setBreakpoint, toString, union, union, write, write

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CreatedCollection

public CreatedCollection(SparkPipeline p,
                 Iterable<T> contents,
                 PType<T> ptype,
                 CreateOptions options)

Method Detail

getParents

public List<PCollectionImpl<?>> getParents()

Specified by:

getParents in class PCollectionImpl<T>

getLastModifiedAt

public long getLastModifiedAt()

Description copied from class: PCollectionImpl

The time of the most recent modification to one of the input sources to the collection. If the time can not be determined then -1 should be returned.

Specified by:

getLastModifiedAt in class PCollectionImpl<T>

Returns:

time of the most recent modification to one of the input sources to the collection.

getPType

public PType<T> getPType()

Description copied from interface: PCollection

Returns the PType of this PCollection.

Specified by:

getPType in interface PCollection<T>

getJavaRDDLike

public org.apache.spark.api.java.JavaRDDLike<?,?> getJavaRDDLike(SparkRuntime runtime)

Specified by:

getJavaRDDLike in interface SparkCollection