This project has retired. For details please refer to its Attic page.
SequentialFileNamingScheme (Apache Crunch 0.11.0 API)

org.apache.crunch.io
Class SequentialFileNamingScheme

java.lang.Object
  extended by org.apache.crunch.io.SequentialFileNamingScheme
All Implemented Interfaces:
FileNamingScheme

public class SequentialFileNamingScheme
extends Object
implements FileNamingScheme

Default FileNamingScheme that uses an incrementing sequence number in order to generate unique file names.


Method Summary
static SequentialFileNamingScheme getInstance()
           
 String getMapOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory)
          Get the output file name for a map task.
 String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputDirectory, int partitionId)
          Get the output file name for a reduce task.
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

getInstance

public static SequentialFileNamingScheme getInstance()

getMapOutputName

public String getMapOutputName(org.apache.hadoop.conf.Configuration configuration,
                               org.apache.hadoop.fs.Path outputDirectory)
                        throws IOException
Description copied from interface: FileNamingScheme
Get the output file name for a map task. Note that the implementation is responsible for avoiding naming collisions.

Specified by:
getMapOutputName in interface FileNamingScheme
Parameters:
configuration - The configuration of the job for which the map output is being written
outputDirectory - The directory where the output will be written
Returns:
The filename for the output of the map task
Throws:
IOException - if an exception occurs while accessing the output file system

getReduceOutputName

public String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration,
                                  org.apache.hadoop.fs.Path outputDirectory,
                                  int partitionId)
                           throws IOException
Description copied from interface: FileNamingScheme
Get the output file name for a reduce task. Note that the implementation is responsible for avoiding naming collisions.

Specified by:
getReduceOutputName in interface FileNamingScheme
Parameters:
configuration - The configuration of the job for which output is being written
outputDirectory - The directory where the file will be written
partitionId - The partition of the reduce task being output
Returns:
The filename for the output of the reduce task
Throws:
IOException - if an exception occurs while accessing output file system


Copyright © 2014 The Apache Software Foundation. All Rights Reserved.