This project has retired. For details please refer to its Attic page.
FileNamingScheme (Apache Crunch 0.10.0 API)

org.apache.crunch.io
Interface FileNamingScheme

All Known Implementing Classes:
SequentialFileNamingScheme

public interface FileNamingScheme

Encapsulates rules for naming output files. It is the responsibility of implementors to avoid file name collisions.


Method Summary
 String getMapOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory)
          Get the output file name for a map task.
 String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory, int partitionId)
          Get the output file name for a reduce task.
 

Method Detail

getMapOutputName

String getMapOutputName(org.apache.hadoop.conf.Configuration configuration,
                        org.apache.hadoop.fs.Path outputDirectory)
                        throws IOException
Get the output file name for a map task. Note that the implementation is responsible for avoiding naming collisions.

Parameters:
configuration - The configuration of the job for which the map output is being written
outputDirectory - The directory where the output will be written
Returns:
The filename for the output of the map task
Throws:
IOException - if an exception occurs while accessing the output file system

getReduceOutputName

String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration,
                           org.apache.hadoop.fs.Path outputDirectory,
                           int partitionId)
                           throws IOException
Get the output file name for a reduce task. Note that the implementation is responsible for avoiding naming collisions.

Parameters:
configuration - The configuration of the job for which output is being written
outputDirectory - The directory where the file will be written
partitionId - The partition of the reduce task being output
Returns:
The filename for the output of the reduce task
Throws:
IOException - if an exception occurs while accessing output file system


Copyright © 2014 The Apache Software Foundation. All Rights Reserved.