org.apache.crunch.io

Class SequentialFileNamingScheme

java.lang.Object

org.apache.crunch.io.SequentialFileNamingScheme

All Implemented Interfaces:

FileNamingScheme

public class SequentialFileNamingScheme
extends Object
implements FileNamingScheme

Default FileNamingScheme that uses an incrementing sequence number in order to generate unique file names.

Method Summary

Methods

Modifier and Type	Method and Description
static SequentialFileNamingScheme	getInstance()
String	getMapOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory) Get the output file name for a map task.
String	getReduceOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory, int partitionId) Get the output file name for a reduce task.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getInstance

public static SequentialFileNamingScheme getInstance()

getMapOutputName

public String getMapOutputName(org.apache.hadoop.conf.Configuration configuration,
                      org.apache.hadoop.fs.Path outputDirectory)
                        throws IOException

Description copied from interface: FileNamingScheme

Get the output file name for a map task. Note that the implementation is responsible for avoiding naming collisions.

Specified by:

getMapOutputName in interface FileNamingScheme

Parameters:

configuration - The configuration of the job for which the map output is being written

outputDirectory - The directory where the output will be written

Returns:

The filename for the output of the map task

Throws:

IOException - if an exception occurs while accessing the output file system

getReduceOutputName

public String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration,
                         org.apache.hadoop.fs.Path outputDirectory,
                         int partitionId)
                           throws IOException

Description copied from interface: FileNamingScheme

Get the output file name for a reduce task. Note that the implementation is responsible for avoiding naming collisions.

Specified by:

getReduceOutputName in interface FileNamingScheme

Parameters:

configuration - The configuration of the job for which output is being written

outputDirectory - The directory where the file will be written

partitionId - The partition of the reduce task being output

Returns:

The filename for the output of the reduce task

Throws:

IOException - if an exception occurs while accessing output file system