This project has retired. For details please refer to its Attic page.

org.apache.crunch.io.parquet
Class AvroParquetFileTarget

java.lang.Object
  org.apache.crunch.io.impl.FileTargetImpl
      org.apache.crunch.io.parquet.AvroParquetFileTarget

All Implemented Interfaces:

MapReduceTarget, PathTarget, Target

public class AvroParquetFileTarget
extends FileTargetImpl

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.crunch.Target

Target.WriteMode

Field Summary

Fields inherited from class org.apache.crunch.io.impl.FileTargetImpl

path

Constructor Summary

AvroParquetFileTarget(org.apache.hadoop.fs.Path path)

AvroParquetFileTarget(org.apache.hadoop.fs.Path path, FileNamingScheme fileNamingScheme)

AvroParquetFileTarget(String path)

Method Summary

boolean	accept(OutputHandler handler, PType<?> ptype) Checks to see if this Target instance is compatible with the given PType.
<T> SourceTarget<T>	asSourceTarget(PType<T> ptype) Attempt to create the SourceTarget type that corresponds to this Target for the given PType, if possible.
void	configureForMapReduce(org.apache.hadoop.mapreduce.Job job, PType<?> ptype, org.apache.hadoop.fs.Path outputPath, String name)
Converter<?,?,?,?>	getConverter(PType<?> ptype) Returns the Converter to use for mapping from the output PCollection into the output values expected by this instance.
String	toString()

Methods inherited from class org.apache.crunch.io.impl.FileTargetImpl

configureForMapReduce, configureForMapReduce, equals, extractPartitionNumber, getDestFile, getFileNamingScheme, getPath, getSourcePattern, getSuccessIndicator, handleExisting, handleOutputs, hashCode, isCompatible, outputConf

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

AvroParquetFileTarget

public AvroParquetFileTarget(String path)

AvroParquetFileTarget

public AvroParquetFileTarget(org.apache.hadoop.fs.Path path)

AvroParquetFileTarget

public AvroParquetFileTarget(org.apache.hadoop.fs.Path path,
                             FileNamingScheme fileNamingScheme)

Method Detail

toString

public String toString()

Overrides:

toString in class FileTargetImpl

accept

public boolean accept(OutputHandler handler,
                      PType<?> ptype)

Description copied from interface: Target

Checks to see if this Target instance is compatible with the given PType.

Specified by:

accept in interface Target

Overrides:

accept in class FileTargetImpl

Parameters:

handler - The OutputHandler that is managing the output for the job

ptype - The PType to check

Returns:

True if this Target can write data in the form of the given PType, false otherwise

getConverter

public Converter<?,?,?,?> getConverter(PType<?> ptype)

Description copied from interface: Target

Returns the Converter to use for mapping from the output PCollection into the output values expected by this instance.

Specified by:

getConverter in interface Target

Overrides:

getConverter in class FileTargetImpl

Parameters:

ptype - The PType of the data that is being written to this instance

Returns:

A valid Converter for the output represented by this instance

configureForMapReduce

public void configureForMapReduce(org.apache.hadoop.mapreduce.Job job,
                                  PType<?> ptype,
                                  org.apache.hadoop.fs.Path outputPath,
                                  String name)

Specified by:

configureForMapReduce in interface MapReduceTarget

Overrides:

configureForMapReduce in class FileTargetImpl

asSourceTarget

public <T> SourceTarget<T> asSourceTarget(PType<T> ptype)

Description copied from interface: Target

Attempt to create the SourceTarget type that corresponds to this Target for the given PType, if possible. If it is not possible, return null.

Specified by:

asSourceTarget in interface Target

Overrides:

asSourceTarget in class FileTargetImpl

Parameters:

ptype - The PType to use in constructing the SourceTarget

Returns:

A new SourceTarget or null if such a SourceTarget does not exist