This project has retired. For details please refer to its Attic page.

org.apache.crunch.io
Interface FileNamingScheme

All Known Implementing Classes:

SequentialFileNamingScheme

public interface FileNamingScheme

Encapsulates rules for naming output files. It is the responsibility of implementors to avoid file name collisions.

Method Summary

String	getMapOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory) Get the output file name for a map task.
String	getReduceOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory, int partitionId) Get the output file name for a reduce task.

Method Detail

getMapOutputName

String getMapOutputName(org.apache.hadoop.conf.Configuration configuration,
                        org.apache.hadoop.fs.Path outputDirectory)
                        throws IOException

Get the output file name for a map task. Note that the implementation is responsible for avoiding naming collisions.

Parameters:

configuration - The configuration of the job for which the map output is being written

outputDirectory - The directory where the output will be written

Returns:

The filename for the output of the map task

Throws:

IOException - if an exception occurs while accessing the output file system

getReduceOutputName

String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration,
                           org.apache.hadoop.fs.Path outputDirectory,
                           int partitionId)
                           throws IOException

Get the output file name for a reduce task. Note that the implementation is responsible for avoiding naming collisions.

Parameters:

configuration - The configuration of the job for which output is being written

outputDirectory - The directory where the file will be written

partitionId - The partition of the reduce task being output

Returns:

The filename for the output of the reduce task

Throws:

IOException - if an exception occurs while accessing output file system