org.apache.crunch
Interface Target

All Known Subinterfaces:
MapReduceTarget, PathTarget, ReadableSourceTarget<T>, SourceTarget<T>, TableSourceTarget<K,V>

public interface Target

A Target represents the output destination of a Crunch PCollection in the context of a Crunch job.


Nested Class Summary
static class Target.WriteMode
          An enum to represent different options the client may specify for handling the case where the output path, table, etc.
 
Method Summary
 boolean accept(OutputHandler handler, PType<?> ptype)
          Checks to see if this Target instance is compatible with the given PType.
<T> SourceTarget<T>
asSourceTarget(PType<T> ptype)
          Attempt to create the SourceTarget type that corresponds to this Target for the given PType, if possible.
 Converter<?,?,?,?> getConverter(PType<?> ptype)
          Returns the Converter to use for mapping from the output PCollection into the output values expected by this instance.
 boolean handleExisting(Target.WriteMode writeMode, long lastModifiedAt, org.apache.hadoop.conf.Configuration conf)
          Apply the given WriteMode to this Target instance.
 Target outputConf(String key, String value)
          Adds the given key-value pair to the Configuration instance that is used to write this Target.
 

Method Detail

outputConf

Target outputConf(String key,
                  String value)
Adds the given key-value pair to the Configuration instance that is used to write this Target. Allows for multiple target outputs to re-use the same config keys with different values when necessary.


handleExisting

boolean handleExisting(Target.WriteMode writeMode,
                       long lastModifiedAt,
                       org.apache.hadoop.conf.Configuration conf)
Apply the given WriteMode to this Target instance.

Parameters:
writeMode - The strategy for handling existing outputs
lastModifiedAt - the time of the most recent modification to one of the source inputs for handling based on the provided writeMode.
conf - The ever-useful Configuration instance
Returns:
true if the target did exist

accept

boolean accept(OutputHandler handler,
               PType<?> ptype)
Checks to see if this Target instance is compatible with the given PType.

Parameters:
handler - The OutputHandler that is managing the output for the job
ptype - The PType to check
Returns:
True if this Target can write data in the form of the given PType, false otherwise

getConverter

Converter<?,?,?,?> getConverter(PType<?> ptype)
Returns the Converter to use for mapping from the output PCollection into the output values expected by this instance.

Parameters:
ptype - The PType of the data that is being written to this instance
Returns:
A valid Converter for the output represented by this instance

asSourceTarget

<T> SourceTarget<T> asSourceTarget(PType<T> ptype)
Attempt to create the SourceTarget type that corresponds to this Target for the given PType, if possible. If it is not possible, return null.

Parameters:
ptype - The PType to use in constructing the SourceTarget
Returns:
A new SourceTarget or null if such a SourceTarget does not exist


Copyright © 2014 The Apache Software Foundation. All Rights Reserved.