Uses of Interface
org.apache.crunch.Source

Packages that use Source

Package	Description
org.apache.crunch	Client-facing API and core abstractions.
org.apache.crunch.contrib.io.jdbc	Support for reading data from RDBMS using JDBC
org.apache.crunch.impl.dist
org.apache.crunch.impl.dist.collect
org.apache.crunch.impl.mem	In-memory Pipeline implementation for rapid prototyping and testing.
org.apache.crunch.impl.spark.collect
org.apache.crunch.io	Data input and output for Pipelines.
org.apache.crunch.io.impl
org.apache.crunch.kafka
org.apache.crunch.util	An assorted set of utilities.

Uses of Source in org.apache.crunch

Subinterfaces of Source in org.apache.crunch

Modifier and Type	Interface and Description
interface	SourceTarget<T> An interface for classes that implement both the Source and the Target interfaces.
interface	TableSource<K,V> The interface Source implementations that return a PTable.
interface	TableSourceTarget<K,V> An interface for classes that implement both the TableSource and the Target interfaces.

Methods in org.apache.crunch that return Source

Modifier and Type	Method and Description
Source<T>	Source.inputConf(String key, String value) Adds the given key-value pair to the Configuration instance that is used to read this Source<T></T>.

Methods in org.apache.crunch with parameters of type Source

Modifier and Type	Method and Description
<T> PCollection<T>	Pipeline.read(Source<T> source) Converts the given Source into a PCollection that is available to jobs run using this Pipeline instance.
<T> PCollection<T>	Pipeline.read(Source<T> source, String named) Converts the given Source into a PCollection that is available to jobs run using this Pipeline instance.
ParallelDoOptions.Builder	ParallelDoOptions.Builder.sources(Source<?>... sources)

Method parameters in org.apache.crunch with type arguments of type Source

Modifier and Type	Method and Description
ParallelDoOptions.Builder	ParallelDoOptions.Builder.sources(Collection<Source<?>> sources)

Uses of Source in org.apache.crunch.contrib.io.jdbc

Classes in org.apache.crunch.contrib.io.jdbc that implement Source

Modifier and Type	Class and Description
class	DataBaseSource<T extends org.apache.hadoop.mapreduce.lib.db.DBWritable & org.apache.hadoop.io.Writable> Source from reading from a database via a JDBC connection.

Uses of Source in org.apache.crunch.impl.dist

Methods in org.apache.crunch.impl.dist with parameters of type Source

Modifier and Type	Method and Description
<S> PCollection<S>	DistributedPipeline.read(Source<S> source)
<S> PCollection<S>	DistributedPipeline.read(Source<S> source, String named)

Uses of Source in org.apache.crunch.impl.dist.collect

Methods in org.apache.crunch.impl.dist.collect that return Source

Modifier and Type	Method and Description
Source<S>	BaseInputCollection.getSource()

Methods in org.apache.crunch.impl.dist.collect with parameters of type Source

Modifier and Type	Method and Description
<S> BaseInputCollection<S>	PCollectionFactory.createInputCollection(Source<S> source, String named, DistributedPipeline distributedPipeline, ParallelDoOptions doOpts)

Constructors in org.apache.crunch.impl.dist.collect with parameters of type Source

Constructor and Description
BaseInputCollection(Source<S> source, DistributedPipeline pipeline)
BaseInputCollection(Source<S> source, String name, DistributedPipeline pipeline, ParallelDoOptions doOpts)

Uses of Source in org.apache.crunch.impl.mem

Methods in org.apache.crunch.impl.mem with parameters of type Source

Modifier and Type	Method and Description
<T> PCollection<T>	MemPipeline.read(Source<T> source)
<T> PCollection<T>	MemPipeline.read(Source<T> source, String named)

Uses of Source in org.apache.crunch.impl.spark.collect

Methods in org.apache.crunch.impl.spark.collect with parameters of type Source

Modifier and Type	Method and Description
<S> BaseInputCollection<S>	SparkCollectFactory.createInputCollection(Source<S> source, String named, DistributedPipeline pipeline, ParallelDoOptions doOpts)

Uses of Source in org.apache.crunch.io

Subinterfaces of Source in org.apache.crunch.io

Modifier and Type	Interface and Description
interface	ReadableSource<T> An extension of the Source interface that indicates that a Source instance may be read as a series of records by the client code.
interface	ReadableSourceTarget<T> An interface that indicates that a SourceTarget instance can be read into the local client.

Methods in org.apache.crunch.io that return Source

Modifier and Type	Method and Description
static Source<org.apache.avro.generic.GenericData.Record>	From.avroFile(List<org.apache.hadoop.fs.Path> paths) Creates a Source<GenericData.Record> by reading the schema of the Avro file at the given paths.
static <T extends org.apache.avro.specific.SpecificRecord> Source<T>	From.avroFile(List<org.apache.hadoop.fs.Path> paths, Class<T> avroClass) Creates a Source<T> instance from the Avro file(s) at the given Paths.
static Source<org.apache.avro.generic.GenericData.Record>	From.avroFile(List<org.apache.hadoop.fs.Path> paths, org.apache.hadoop.conf.Configuration conf) Creates a Source<GenericData.Record> by reading the schema of the Avro file at the given paths using the FileSystem information contained in the given Configuration instance.
static <T> Source<T>	From.avroFile(List<org.apache.hadoop.fs.Path> paths, PType<T> ptype) Creates a Source<T> instance from the Avro file(s) at the given Paths.
static Source<org.apache.avro.generic.GenericData.Record>	From.avroFile(org.apache.hadoop.fs.Path path) Creates a Source<GenericData.Record> by reading the schema of the Avro file at the given path.
static <T extends org.apache.avro.specific.SpecificRecord> Source<T>	From.avroFile(org.apache.hadoop.fs.Path path, Class<T> avroClass) Creates a Source<T> instance from the Avro file(s) at the given Path.
static Source<org.apache.avro.generic.GenericData.Record>	From.avroFile(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf) Creates a Source<GenericData.Record> by reading the schema of the Avro file at the given path using the FileSystem information contained in the given Configuration instance.
static <T> Source<T>	From.avroFile(org.apache.hadoop.fs.Path path, PType<T> ptype) Creates a Source<T> instance from the Avro file(s) at the given Path.
static Source<org.apache.avro.generic.GenericData.Record>	From.avroFile(String pathName) Creates a Source<GenericData.Record> by reading the schema of the Avro file at the given path.
static <T extends org.apache.avro.specific.SpecificRecord> Source<T>	From.avroFile(String pathName, Class<T> avroClass) Creates a Source<T> instance from the Avro file(s) at the given path name.
static <T> Source<T>	From.avroFile(String pathName, PType<T> ptype) Creates a Source<T> instance from the Avro file(s) at the given path name.
static <T extends org.apache.hadoop.io.Writable> Source<T>	From.sequenceFile(List<org.apache.hadoop.fs.Path> paths, Class<T> valueClass) Creates a Source<T> instance from the SequenceFile(s) at the given Paths from the value field of each key-value pair in the SequenceFile(s).
static <T> Source<T>	From.sequenceFile(List<org.apache.hadoop.fs.Path> paths, PType<T> ptype) Creates a Source<T> instance from the SequenceFile(s) at the given Paths from the value field of each key-value pair in the SequenceFile(s).
static <T extends org.apache.hadoop.io.Writable> Source<T>	From.sequenceFile(org.apache.hadoop.fs.Path path, Class<T> valueClass) Creates a Source<T> instance from the SequenceFile(s) at the given Path from the value field of each key-value pair in the SequenceFile(s).
static <T> Source<T>	From.sequenceFile(org.apache.hadoop.fs.Path path, PType<T> ptype) Creates a Source<T> instance from the SequenceFile(s) at the given Path from the value field of each key-value pair in the SequenceFile(s).
static <T extends org.apache.hadoop.io.Writable> Source<T>	From.sequenceFile(String pathName, Class<T> valueClass) Creates a Source<T> instance from the SequenceFile(s) at the given path name from the value field of each key-value pair in the SequenceFile(s).
static <T> Source<T>	From.sequenceFile(String pathName, PType<T> ptype) Creates a Source<T> instance from the SequenceFile(s) at the given path name from the value field of each key-value pair in the SequenceFile(s).
static Source<String>	From.textFile(List<org.apache.hadoop.fs.Path> paths) Creates a Source<String> instance for the text file(s) at the given Paths.
static <T> Source<T>	From.textFile(List<org.apache.hadoop.fs.Path> paths, PType<T> ptype) Creates a Source<T> instance for the text file(s) at the given Paths using the provided PType<T> to convert the input text.
static Source<String>	From.textFile(org.apache.hadoop.fs.Path path) Creates a Source<String> instance for the text file(s) at the given Path.
static <T> Source<T>	From.textFile(org.apache.hadoop.fs.Path path, PType<T> ptype) Creates a Source<T> instance for the text file(s) at the given Path using the provided PType<T> to convert the input text.
static Source<String>	From.textFile(String pathName) Creates a Source<String> instance for the text file(s) at the given path name.
static <T> Source<T>	From.textFile(String pathName, PType<T> ptype) Creates a Source<T> instance for the text file(s) at the given path name using the provided PType<T> to convert the input text.

Uses of Source in org.apache.crunch.io.impl

Classes in org.apache.crunch.io.impl with annotations of type with type parameters of type that implement declared as with annotations of type with type parameters of type with annotations of type with annotations of type with type parameters of type that return that return types with arguments of type with parameters of type with type arguments of type that throw with annotations of type with annotations of type with parameters of type with type arguments of type that throw Source

Modifier and Type	Class and Description
class	org.apache.crunch.io.impl.FileSourceImpl<T>

Uses of Source in org.apache.crunch.kafka

Classes in org.apache.crunch.kafka that implement Source

Modifier and Type	Class and Description
class	KafkaSource A Crunch Source that will retrieve events from Kafka given start and end offsets.

Methods in org.apache.crunch.kafka that return Source

Modifier and Type	Method and Description
Source<Pair<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>>	KafkaSource.inputConf(String key, String value)

Uses of Source in org.apache.crunch.util

Methods in org.apache.crunch.util with parameters of type Source

Modifier and Type	Method and Description
<T> PCollection<T>	CrunchTool.read(Source<T> source)