This project has retired. For details please refer to its
Attic page .
AvroPathPerKeyTarget (Apache Crunch 0.9.0 API)
org.apache.crunch.io.avro
Class AvroPathPerKeyTarget
java.lang.Object
org.apache.crunch.io.impl.FileTargetImpl
org.apache.crunch.io.avro.AvroPathPerKeyTarget
All Implemented Interfaces: MapReduceTarget , PathTarget , Target
public class AvroPathPerKeyTarget extends FileTargetImpl
A Target
that wraps AvroPathPerKeyOutputFormat
to allow one file
per key to be written as the output of a PTable<String, T>
.
Note the restrictions that apply to the AvroPathPerKeyOutputFormat
; in particular, it's a good
idea to write out all of the records for the same key together within each partition of the data.
Method Summary
boolean
accept (OutputHandler handler,
PType <?> ptype)
Checks to see if this Target
instance is compatible with the
given PType
.
void
configureForMapReduce (org.apache.hadoop.mapreduce.Job job,
PType <?> ptype,
org.apache.hadoop.fs.Path outputPath,
String name)
void
handleOutputs (org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path workingPath,
int index)
Handles moving the output data for this target from a temporary location on the
filesystem to its target path at the end of a MapReduce job.
String
toString ()
Methods inherited from class org.apache.crunch.io.impl.FileTargetImpl
asSourceTarget , configureForMapReduce , configureForMapReduce , equals , extractPartitionNumber , getConverter , getDestFile , getFileNamingScheme , getPath , getSourcePattern , getSuccessIndicator , handleExisting , hashCode , isCompatible , outputConf
AvroPathPerKeyTarget
public AvroPathPerKeyTarget (String path)
AvroPathPerKeyTarget
public AvroPathPerKeyTarget (org.apache.hadoop.fs.Path path)
AvroPathPerKeyTarget
public AvroPathPerKeyTarget (org.apache.hadoop.fs.Path path,
FileNamingScheme fileNamingScheme)
accept
public boolean accept (OutputHandler handler,
PType <?> ptype)
Description copied from interface: Target
Checks to see if this Target
instance is compatible with the
given PType
.
Specified by: accept
in interface Target
Overrides: accept
in class FileTargetImpl
Parameters: handler
- The OutputHandler
that is managing the output for the jobptype
- The PType
to check
Returns: True if this Target can write data in the form of the given PType
,
false otherwise
configureForMapReduce
public void configureForMapReduce (org.apache.hadoop.mapreduce.Job job,
PType <?> ptype,
org.apache.hadoop.fs.Path outputPath,
String name)
Specified by: configureForMapReduce
in interface MapReduceTarget
Overrides: configureForMapReduce
in class FileTargetImpl
handleOutputs
public void handleOutputs (org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path workingPath,
int index)
throws IOException
Description copied from interface: PathTarget
Handles moving the output data for this target from a temporary location on the
filesystem to its target path at the end of a MapReduce job.
Specified by: handleOutputs
in interface PathTarget
Overrides: handleOutputs
in class FileTargetImpl
Parameters: conf
- The job Configuration
workingPath
- The temp directory that contains the output of the jobindex
- The index of this target for jobs that write multiple output files to a single directory
Throws:
IOException
toString
public String toString ()
Overrides: toString
in class FileTargetImpl
Copyright © 2014 The Apache Software Foundation . All Rights Reserved.