This project has retired. For details please refer to its Attic page.

org.apache.crunch.io.avro
Class AvroPathPerKeyTarget

java.lang.Object
  org.apache.crunch.io.impl.FileTargetImpl
      org.apache.crunch.io.avro.AvroPathPerKeyTarget

All Implemented Interfaces:

MapReduceTarget, PathTarget, Target

public class AvroPathPerKeyTarget
extends FileTargetImpl

A Target that wraps AvroPathPerKeyOutputFormat to allow one file per key to be written as the output of a PTable<String, T>.

Note the restrictions that apply to the AvroPathPerKeyOutputFormat; in particular, it's a good idea to write out all of the records for the same key together within each partition of the data.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.crunch.Target

Target.WriteMode

Field Summary

Fields inherited from class org.apache.crunch.io.impl.FileTargetImpl

path

Constructor Summary

AvroPathPerKeyTarget(org.apache.hadoop.fs.Path path)

AvroPathPerKeyTarget(org.apache.hadoop.fs.Path path, FileNamingScheme fileNamingScheme)

AvroPathPerKeyTarget(String path)

Method Summary

boolean	accept(OutputHandler handler, PType<?> ptype) Checks to see if this Target instance is compatible with the given PType.
void	configureForMapReduce(org.apache.hadoop.mapreduce.Job job, PType<?> ptype, org.apache.hadoop.fs.Path outputPath, String name)
void	handleOutputs(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path workingPath, int index) Handles moving the output data for this target from a temporary location on the filesystem to its target path at the end of a MapReduce job.
String	toString()

Methods inherited from class org.apache.crunch.io.impl.FileTargetImpl

asSourceTarget, configureForMapReduce, configureForMapReduce, equals, extractPartitionNumber, getConverter, getDestFile, getFileNamingScheme, getPath, getSourcePattern, getSuccessIndicator, handleExisting, hashCode, isCompatible, outputConf

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

AvroPathPerKeyTarget

public AvroPathPerKeyTarget(String path)

AvroPathPerKeyTarget

public AvroPathPerKeyTarget(org.apache.hadoop.fs.Path path)

AvroPathPerKeyTarget

public AvroPathPerKeyTarget(org.apache.hadoop.fs.Path path,
                            FileNamingScheme fileNamingScheme)

Method Detail

accept

public boolean accept(OutputHandler handler,
                      PType<?> ptype)

Description copied from interface: Target

Checks to see if this Target instance is compatible with the given PType.

Specified by:

accept in interface Target

Overrides:

accept in class FileTargetImpl

Parameters:

handler - The OutputHandler that is managing the output for the job

ptype - The PType to check

Returns:

True if this Target can write data in the form of the given PType, false otherwise

configureForMapReduce

public void configureForMapReduce(org.apache.hadoop.mapreduce.Job job,
                                  PType<?> ptype,
                                  org.apache.hadoop.fs.Path outputPath,
                                  String name)

Specified by:

configureForMapReduce in interface MapReduceTarget

Overrides:

configureForMapReduce in class FileTargetImpl

handleOutputs

public void handleOutputs(org.apache.hadoop.conf.Configuration conf,
                          org.apache.hadoop.fs.Path workingPath,
                          int index)
                   throws IOException

Description copied from interface: PathTarget

Handles moving the output data for this target from a temporary location on the filesystem to its target path at the end of a MapReduce job.

Specified by:

handleOutputs in interface PathTarget

Overrides:

handleOutputs in class FileTargetImpl

Parameters:

conf - The job Configuration

workingPath - The temp directory that contains the output of the job

index - The index of this target for jobs that write multiple output files to a single directory

Throws:

IOException

toString

public String toString()

Overrides:

toString in class FileTargetImpl