public class WritableType<T,W extends org.apache.hadoop.io.Writable> extends Object implements PType<T>
Constructor and Description |
---|
WritableType(Class<T> typeClass,
Class<W> writableClass,
MapFn<W,T> inputDoFn,
MapFn<T,W> outputDoFn,
PType... subTypes) |
Modifier and Type | Method and Description |
---|---|
ReadableSource<T> |
createSourceTarget(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path path,
Iterable<T> contents,
int parallelism)
Returns a
ReadableSource that contains the data in the given Iterable . |
boolean |
equals(Object obj) |
Converter |
getConverter() |
ReadableSourceTarget<T> |
getDefaultFileSource(org.apache.hadoop.fs.Path path)
Returns a
SourceTarget that is able to read/write data using the serialization format
specified by this PType . |
T |
getDetachedValue(T value)
Returns a copy of a value (or the value itself) that can safely be retained.
|
PTypeFamily |
getFamily()
Returns the
PTypeFamily that this PType belongs to. |
MapFn |
getInputMapFn() |
MapFn |
getOutputMapFn() |
Class<W> |
getSerializationClass() |
List<PType> |
getSubTypes()
Returns the sub-types that make up this PType if it is a composite instance, such as a tuple.
|
Class<T> |
getTypeClass()
Returns the Java type represented by this
PType . |
int |
hashCode() |
static <T,W extends org.apache.hadoop.io.Writable> |
immutableType(Class<T> typeClass,
Class<W> writableClass,
MapFn<W,T> inputDoFn,
MapFn<T,W> outputDoFn,
PType... subTypes)
Factory method for a new WritableType instance whose type class is immutable.
|
void |
initialize(org.apache.hadoop.conf.Configuration conf)
Initialize this PType for use within a DoFn.
|
public static <T,W extends org.apache.hadoop.io.Writable> WritableType<T,W> immutableType(Class<T> typeClass, Class<W> writableClass, MapFn<W,T> inputDoFn, MapFn<T,W> outputDoFn, PType... subTypes)
public PTypeFamily getFamily()
PType
PTypeFamily
that this PType
belongs to.public Class<T> getTypeClass()
PType
PType
.getTypeClass
in interface PType<T>
public Converter getConverter()
getConverter
in interface PType<T>
public MapFn getInputMapFn()
getInputMapFn
in interface PType<T>
public MapFn getOutputMapFn()
getOutputMapFn
in interface PType<T>
public List<PType> getSubTypes()
PType
getSubTypes
in interface PType<T>
public ReadableSourceTarget<T> getDefaultFileSource(org.apache.hadoop.fs.Path path)
PType
SourceTarget
that is able to read/write data using the serialization format
specified by this PType
.getDefaultFileSource
in interface PType<T>
public ReadableSource<T> createSourceTarget(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path path, Iterable<T> contents, int parallelism) throws IOException
PType
ReadableSource
that contains the data in the given Iterable
.createSourceTarget
in interface PType<T>
conf
- The Configuration to usepath
- The path to write the data tocontents
- The contents to writeparallelism
- The desired parallelismIOException
public void initialize(org.apache.hadoop.conf.Configuration conf)
PType
PType.getDetachedValue(Object)
.initialize
in interface PType<T>
conf
- Configuration objectPType.getDetachedValue(Object)
public T getDetachedValue(T value)
PType
This is useful when iterable values being processed in a DoFn (via a reducer) need to be held
on to for more than the scope of a single iteration, as a reducer (and therefore also a DoFn
that has an Iterable as input) re-use deserialized values. More information on object reuse is
available in the DoFn
class documentation.
getDetachedValue
in interface PType<T>
value
- The value to be deep-copiedCopyright © 2017 The Apache Software Foundation. All rights reserved.