This project has retired. For details please refer to its Attic page.

org.apache.crunch.io
Class SequentialFileNamingScheme

java.lang.Object
  org.apache.crunch.io.SequentialFileNamingScheme

All Implemented Interfaces:

FileNamingScheme

public class SequentialFileNamingScheme
extends Object
implements FileNamingScheme

Default FileNamingScheme that uses an incrementing sequence number in order to generate unique file names.

Method Summary

static SequentialFileNamingScheme	getInstance()
String	getMapOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory) Get the output file name for a map task.
String	getReduceOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory, int partitionId) Get the output file name for a reduce task.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getInstance

public static SequentialFileNamingScheme getInstance()

getMapOutputName

public String getMapOutputName(org.apache.hadoop.conf.Configuration configuration,
                               org.apache.hadoop.fs.Path outputDirectory)
                        throws IOException

Description copied from interface: FileNamingScheme

Get the output file name for a map task. Note that the implementation is responsible for avoiding naming collisions.

Specified by:

getMapOutputName in interface FileNamingScheme

Parameters:

configuration - The configuration of the job for which the map output is being written

outputDirectory - The directory where the output will be written

Returns:

The filename for the output of the map task

Throws:

IOException - if an exception occurs while accessing the output file system

getReduceOutputName

public String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration,
                                  org.apache.hadoop.fs.Path outputDirectory,
                                  int partitionId)
                           throws IOException

Description copied from interface: FileNamingScheme

Get the output file name for a reduce task. Note that the implementation is responsible for avoiding naming collisions.

Specified by:

getReduceOutputName in interface FileNamingScheme

Parameters:

configuration - The configuration of the job for which output is being written

outputDirectory - The directory where the file will be written

partitionId - The partition of the reduce task being output

Returns:

The filename for the output of the reduce task

Throws:

IOException - if an exception occurs while accessing output file system