This project has retired. For details please refer to its Attic page.
AvroMode (Apache Crunch 0.11.0 API)

org.apache.crunch.types.avro
Class AvroMode

java.lang.Object
  extended by org.apache.crunch.types.avro.AvroMode
All Implemented Interfaces:
ReaderWriterFactory

public class AvroMode
extends Object
implements ReaderWriterFactory

AvroMode is an immutable object used for configuring the reading and writing of Avro types. The mode will not be used or honored unless it has been appropriately configured using one of the supported methods. Certain sources might also support specifying a specific mode to use.


Nested Class Summary
static class AvroMode.ModeType
          Internal enum which represents the various Avro data types.
 
Field Summary
static String AVRO_MODE_PROPERTY
           
static String AVRO_SHUFFLE_MODE_PROPERTY
           
static AvroMode GENERIC
          Default mode to use for reading and writing Generic types.
static AvroMode REFLECT
          Default mode to use for reading and writing Reflect types.
static AvroMode SPECIFIC
          Default mode to use for reading and writing Specific types.
 
Method Summary
 void configure(org.apache.hadoop.conf.Configuration conf)
          Populates the conf with mode specific settings.
 void configure(FormatBundle bundle)
          Populates the bundle with mode specific settings for the specific FormatBundle.
 void configureFactory(org.apache.hadoop.conf.Configuration conf)
          Deprecated. use configure(org.apache.hadoop.conf.Configuration)
 void configureShuffle(org.apache.hadoop.conf.Configuration conf)
          Populates the conf with mode specific settings for use during the shuffle phase.
 boolean equals(Object o)
           
static AvroMode fromConfiguration(org.apache.hadoop.conf.Configuration conf)
          Creates an AvroMode based on the AVRO_MODE_PROPERTY property in the conf.
static AvroMode fromShuffleConfiguration(org.apache.hadoop.conf.Configuration conf)
          Creates an AvroMode based on the AVRO_SHUFFLE_MODE_PROPERTY property in the conf.
static AvroMode fromType(AvroType<?> type)
          Creates an AvroMode based upon the specified type.
 org.apache.avro.generic.GenericData getData()
          Returns a GenericData instance based on the mode type.
 ReaderWriterFactory getFactory()
          Returns the factory that will be used for the mode.
 Map<String,String> getModeProperties()
          Returns the entries that a Configuration instance needs to enable this AvroMode as a serializable map of key-value pairs.
<T> org.apache.avro.io.DatumReader<T>
getReader(org.apache.avro.Schema schema)
          Creates a DatumReader based on the schema.
static ClassLoader getSpecificClassLoader()
          Get the configured ClassLoader to be used for loading Avro org.apache.specific.SpecificRecord and reflection implementation classes.
<T> org.apache.avro.io.DatumWriter<T>
getWriter(org.apache.avro.Schema schema)
          Creates a DatumWriter based on the schema.
 int hashCode()
           
 AvroMode override(ReaderWriterFactory factory)
          Deprecated. use withFactory(ReaderWriterFactory) instead.
static void setSpecificClassLoader(ClassLoader loader)
          Set the ClassLoader that will be used for loading Avro org.apache.avro.specific.SpecificRecord and reflection implementation classes.
 AvroMode withFactory(ReaderWriterFactory factory)
          Creates a new AvroMode instance which will utilize the factory instance for creating Avro readers and writers.
 AvroMode withFactoryFromConfiguration(org.apache.hadoop.conf.Configuration conf)
           
 
Methods inherited from class java.lang.Object
getClass, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

REFLECT

public static final AvroMode REFLECT
Default mode to use for reading and writing Reflect types.


SPECIFIC

public static final AvroMode SPECIFIC
Default mode to use for reading and writing Specific types.


GENERIC

public static final AvroMode GENERIC
Default mode to use for reading and writing Generic types.


AVRO_MODE_PROPERTY

public static final String AVRO_MODE_PROPERTY
See Also:
Constant Field Values

AVRO_SHUFFLE_MODE_PROPERTY

public static final String AVRO_SHUFFLE_MODE_PROPERTY
See Also:
Constant Field Values
Method Detail

fromConfiguration

public static AvroMode fromConfiguration(org.apache.hadoop.conf.Configuration conf)
Creates an AvroMode based on the AVRO_MODE_PROPERTY property in the conf.

Parameters:
conf - The configuration holding the properties for mode to be created.
Returns:
an AvroMode based on the AVRO_MODE_PROPERTY property in the conf.

fromShuffleConfiguration

public static AvroMode fromShuffleConfiguration(org.apache.hadoop.conf.Configuration conf)
Creates an AvroMode based on the AVRO_SHUFFLE_MODE_PROPERTY property in the conf.

Parameters:
conf - The configuration holding the properties for mode to be created.
Returns:
an AvroMode based on the AVRO_SHUFFLE_MODE_PROPERTY property in the conf.

fromType

public static AvroMode fromType(AvroType<?> type)
Creates an AvroMode based upon the specified type.

Parameters:
type - the Avro type which indicates a specific mode.
Returns:
an AvroMode based upon the specified type.

setSpecificClassLoader

public static void setSpecificClassLoader(ClassLoader loader)
Set the ClassLoader that will be used for loading Avro org.apache.avro.specific.SpecificRecord and reflection implementation classes. It is typically not necessary to call this method -- it should only be used if a specific class loader is needed in order to load the specific datum classes.

Parameters:
loader - the ClassLoader to be used for loading specific datum classes

getSpecificClassLoader

public static ClassLoader getSpecificClassLoader()
Get the configured ClassLoader to be used for loading Avro org.apache.specific.SpecificRecord and reflection implementation classes. The return value may be null.

Returns:
the configured ClassLoader for loading specific or reflection datum classes, may be null

getData

public org.apache.avro.generic.GenericData getData()
Returns a GenericData instance based on the mode type.

Specified by:
getData in interface ReaderWriterFactory
Returns:
a GenericData instance based on the mode type.

getReader

public <T> org.apache.avro.io.DatumReader<T> getReader(org.apache.avro.Schema schema)
Creates a DatumReader based on the schema.

Specified by:
getReader in interface ReaderWriterFactory
Type Parameters:
T - the record type created by the reader.
Parameters:
schema - the schema to be read
Returns:
a DatumReader based on the schema.

getWriter

public <T> org.apache.avro.io.DatumWriter<T> getWriter(org.apache.avro.Schema schema)
Creates a DatumWriter based on the schema.

Specified by:
getWriter in interface ReaderWriterFactory
Type Parameters:
T - the record type created by the writer.
Parameters:
schema - the schema to be read
Returns:
a DatumWriter based on the schema.

override

@Deprecated
public AvroMode override(ReaderWriterFactory factory)
Deprecated. use withFactory(ReaderWriterFactory) instead.

Creates a new AvroMode instance which will utilize the factory instance for creating Avro readers and writers.

Parameters:
factory - factory implementation for the mode to use
Returns:
a new AvroMode instance which will utilize the factory instance for creating Avro readers and writers.

withFactory

public AvroMode withFactory(ReaderWriterFactory factory)
Creates a new AvroMode instance which will utilize the factory instance for creating Avro readers and writers. If null the default factory for the mode will be used.

Parameters:
factory - factory implementation for the mode to use
Returns:
a new AvroMode instance which will utilize the factory instance for creating Avro readers and writers.

configureShuffle

public void configureShuffle(org.apache.hadoop.conf.Configuration conf)
Populates the conf with mode specific settings for use during the shuffle phase.

Parameters:
conf - the configuration to populate.

configure

public void configure(FormatBundle bundle)
Populates the bundle with mode specific settings for the specific FormatBundle.

Parameters:
bundle - the bundle to populate.

configure

public void configure(org.apache.hadoop.conf.Configuration conf)
Populates the conf with mode specific settings.

Parameters:
conf - the configuration to populate.

getModeProperties

public Map<String,String> getModeProperties()
Returns the entries that a Configuration instance needs to enable this AvroMode as a serializable map of key-value pairs.


configureFactory

@Deprecated
public void configureFactory(org.apache.hadoop.conf.Configuration conf)
Deprecated. use configure(org.apache.hadoop.conf.Configuration)

Populates the conf with mode specific settings.

Parameters:
conf - the configuration to populate.

getFactory

public ReaderWriterFactory getFactory()
Returns the factory that will be used for the mode.

Returns:
the factory that will be used for the mode.

equals

public boolean equals(Object o)
Overrides:
equals in class Object

hashCode

public int hashCode()
Overrides:
hashCode in class Object

withFactoryFromConfiguration

public AvroMode withFactoryFromConfiguration(org.apache.hadoop.conf.Configuration conf)


Copyright © 2014 The Apache Software Foundation. All Rights Reserved.