This project has retired. For details please refer to its Attic page.

org.apache.crunch.io.avro
Class AvroFileSource<T>

java.lang.Object
  org.apache.crunch.io.impl.FileSourceImpl<T>
      org.apache.crunch.io.avro.AvroFileSource<T>

All Implemented Interfaces:

ReadableSource<T>, Source<T>

public class AvroFileSource<T>
extends FileSourceImpl<T>
implements ReadableSource<T>

Field Summary

Fields inherited from class org.apache.crunch.io.impl.FileSourceImpl

inputBundle, path, paths, ptype

Constructor Summary

AvroFileSource(List<org.apache.hadoop.fs.Path> paths, AvroType<T> ptype)

AvroFileSource(List<org.apache.hadoop.fs.Path> paths, AvroType<T> ptype, org.apache.avro.io.DatumReader<T> reader)

AvroFileSource(org.apache.hadoop.fs.Path path, AvroType<T> ptype)

AvroFileSource(org.apache.hadoop.fs.Path path, AvroType<T> ptype, org.apache.avro.io.DatumReader<T> reader)

Method Summary

ReadableData<T>	asReadable()
protected AvroFileReaderFactory<T>	getFileReaderFactory(AvroType<T> ptype)
Iterable<T>	read(org.apache.hadoop.conf.Configuration conf) Returns an Iterable that contains the contents of this source.
String	toString()

Methods inherited from class org.apache.crunch.io.impl.FileSourceImpl

configureSource, equals, getBundle, getConverter, getLastModifiedAt, getPath, getPaths, getSize, getType, hashCode, inputConf, pathsAsString, read

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.crunch.Source

configureSource, getConverter, getLastModifiedAt, getSize, getType, inputConf

Constructor Detail

AvroFileSource

public AvroFileSource(org.apache.hadoop.fs.Path path,
                      AvroType<T> ptype)

AvroFileSource

public AvroFileSource(org.apache.hadoop.fs.Path path,
                      AvroType<T> ptype,
                      org.apache.avro.io.DatumReader<T> reader)

AvroFileSource

public AvroFileSource(List<org.apache.hadoop.fs.Path> paths,
                      AvroType<T> ptype)

AvroFileSource

public AvroFileSource(List<org.apache.hadoop.fs.Path> paths,
                      AvroType<T> ptype,
                      org.apache.avro.io.DatumReader<T> reader)

Method Detail

toString

public String toString()

Overrides:

toString in class FileSourceImpl<T>

read

public Iterable<T> read(org.apache.hadoop.conf.Configuration conf)
                 throws IOException

Description copied from interface: ReadableSource

Returns an Iterable that contains the contents of this source.

Specified by:

read in interface ReadableSource<T>

Parameters:

conf - The current Configuration instance

Returns:

the contents of this Source as an Iterable instance

Throws:

IOException

asReadable

public ReadableData<T> asReadable()

Specified by:

asReadable in interface ReadableSource<T>

Returns:

a ReadableData instance containing the data referenced by this ReadableSource.

getFileReaderFactory

protected AvroFileReaderFactory<T> getFileReaderFactory(AvroType<T> ptype)