org.apache.crunch

Interface Target

All Known Subinterfaces:

MapReduceTarget, PathTarget, ReadableSourceTarget<T>, SourceTarget<T>, TableSourceTarget<K,V>

public interface Target

A Target represents the output destination of a Crunch PCollection in the context of a Crunch job.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	Target.WriteMode An enum to represent different options the client may specify for handling the case where the output path, table, etc.

Method Summary

Modifier and Type	Method and Description
boolean	accept(OutputHandler handler, PType<?> ptype) Checks to see if this Target instance is compatible with the given PType.
<T> SourceTarget<T>	asSourceTarget(PType<T> ptype) Attempt to create the SourceTarget type that corresponds to this Target for the given PType, if possible.
Converter<?,?,?,?>	getConverter(PType<?> ptype) Returns the Converter to use for mapping from the output PCollection into the output values expected by this instance.
boolean	handleExisting(Target.WriteMode writeMode, long lastModifiedAt, org.apache.hadoop.conf.Configuration conf) Apply the given WriteMode to this Target instance.
Target	outputConf(String key, String value) Adds the given key-value pair to the Configuration instance that is used to write this Target.

Method Detail

outputConf

Target outputConf(String key,
                  String value)

Adds the given key-value pair to the Configuration instance that is used to write this Target. Allows for multiple target outputs to re-use the same config keys with different values when necessary.

handleExisting

boolean handleExisting(Target.WriteMode writeMode,
                       long lastModifiedAt,
                       org.apache.hadoop.conf.Configuration conf)

Apply the given WriteMode to this Target instance.

Parameters:

writeMode - The strategy for handling existing outputs

lastModifiedAt - the time of the most recent modification to one of the source inputs for handling based on the provided writeMode.

conf - The ever-useful Configuration instance

Returns:

true if the target did exist

accept

boolean accept(OutputHandler handler,
               PType<?> ptype)

Checks to see if this Target instance is compatible with the given PType.

Parameters:

handler - The OutputHandler that is managing the output for the job

ptype - The PType to check

Returns:

True if this Target can write data in the form of the given PType, false otherwise

getConverter

Converter<?,?,?,?> getConverter(PType<?> ptype)

Returns the Converter to use for mapping from the output PCollection into the output values expected by this instance.

Parameters:

ptype - The PType of the data that is being written to this instance

Returns:

A valid Converter for the output represented by this instance

asSourceTarget

<T> SourceTarget<T> asSourceTarget(PType<T> ptype)

Attempt to create the SourceTarget type that corresponds to this Target for the given PType, if possible. If it is not possible, return null.

Parameters:

ptype - The PType to use in constructing the SourceTarget

Returns:

A new SourceTarget or null if such a SourceTarget does not exist