org.apache.crunch.io

Interface FileNamingScheme

All Known Implementing Classes:

SequentialFileNamingScheme

public interface FileNamingScheme

Encapsulates rules for naming output files. It is the responsibility of implementors to avoid file name collisions.

Method Summary

Methods

Modifier and Type	Method and Description
String	getMapOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory) Get the output file name for a map task.
String	getReduceOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory, int partitionId) Get the output file name for a reduce task.

Method Detail

getMapOutputName

String getMapOutputName(org.apache.hadoop.conf.Configuration configuration,
                      org.apache.hadoop.fs.Path outputDirectory)
                        throws IOException

Get the output file name for a map task. Note that the implementation is responsible for avoiding naming collisions.

Parameters:

configuration - The configuration of the job for which the map output is being written

outputDirectory - The directory where the output will be written

Returns:

The filename for the output of the map task

Throws:

IOException - if an exception occurs while accessing the output file system

getReduceOutputName

String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration,
                         org.apache.hadoop.fs.Path outputDirectory,
                         int partitionId)
                           throws IOException

Get the output file name for a reduce task. Note that the implementation is responsible for avoiding naming collisions.

Parameters:

configuration - The configuration of the job for which output is being written

outputDirectory - The directory where the file will be written

partitionId - The partition of the reduce task being output

Returns:

The filename for the output of the reduce task

Throws:

IOException - if an exception occurs while accessing output file system