This project has retired. For details please refer to its Attic page.
Avros (Apache Crunch 0.11.0 API)

org.apache.crunch.types.avro
Class Avros

java.lang.Object
  extended by org.apache.crunch.types.avro.Avros

public class Avros
extends Object

Defines static methods that are analogous to the methods defined in AvroTypeFamily for convenient static importing.


Field Summary
static MapFn<Object,ByteBuffer> BYTES_IN
           
static boolean CAN_COMBINE_SPECIFIC_AND_REFLECT_SCHEMAS
          Older versions of Avro (i.e., before 1.7.0) do not support schemas that are composed of a mix of specific and reflection-based schemas.
static ReflectDataFactory REFLECT_DATA_FACTORY
          Deprecated. as of 0.9.0; use AvroMode.REFLECT.override(ReaderWriterFactory)
static String REFLECT_DATA_FACTORY_CLASS
          The name of the configuration parameter that tracks which reflection factory to use.
static MapFn<String,org.apache.avro.util.Utf8> STRING_TO_UTF8
           
static MapFn<CharSequence,String> UTF8_TO_STRING
           
 
Method Summary
static AvroType<Boolean> booleans()
           
static AvroType<ByteBuffer> bytes()
           
static void checkCombiningSpecificAndReflectionSchemas()
           
static
<T> AvroType<Collection<T>>
collections(PType<T> ptype)
           
static void configureReflectDataFactory(org.apache.hadoop.conf.Configuration conf)
          Deprecated. as of 0.9.0; use AvroMode.REFLECT.configure(Configuration)
static
<T> AvroType<T>
containers(Class<T> clazz)
           
static
<S,T> AvroType<T>
derived(Class<T> clazz, MapFn<S,T> inputFn, MapFn<T,S> outputFn, PType<S> base)
           
static
<S,T> AvroType<T>
derivedImmutable(Class<T> clazz, MapFn<S,T> inputFn, MapFn<T,S> outputFn, PType<S> base)
           
static AvroType<Double> doubles()
           
static AvroType<Float> floats()
           
static AvroType<org.apache.avro.generic.GenericData.Record> generics(org.apache.avro.Schema schema)
           
static
<T> PType<T>
getPrimitiveType(Class<T> clazz)
           
static ReflectDataFactory getReflectDataFactory(org.apache.hadoop.conf.Configuration conf)
          Deprecated. as of 0.9.0; use AvroMode.fromConfiguration(conf)
static AvroType<Integer> ints()
           
static
<T> PType<T>
jsons(Class<T> clazz)
           
static
<K,V> org.apache.crunch.types.avro.AvroKeyValueTableType<K,V>
keyValueTableOf(PType<K> key, PType<V> value)
          A table type with an Avro type as key and value.
static AvroType<Long> longs()
           
static
<T> AvroType<Map<String,T>>
maps(PType<T> ptype)
           
static AvroType<TupleN> namedTuples(String tupleName, String[] fieldNames, PType[] ptypes)
           
static
<T> org.apache.avro.io.DatumReader<T>
newReader(AvroType<T> type)
           
static
<T> org.apache.avro.io.DatumReader<T>
newReader(org.apache.avro.Schema schema)
           
static
<T> org.apache.avro.io.DatumWriter<T>
newWriter(AvroType<T> type)
           
static
<T> org.apache.avro.io.DatumWriter<T>
newWriter(org.apache.avro.Schema schema)
           
static AvroType<Void> nulls()
           
static
<V1,V2> AvroType<Pair<V1,V2>>
pairs(PType<V1> p1, PType<V2> p2)
           
static
<V1,V2,V3,V4>
AvroType<Tuple4<V1,V2,V3,V4>>
quads(PType<V1> p1, PType<V2> p2, PType<V3> p3, PType<V4> p4)
           
static
<T> AvroType<T>
records(Class<T> clazz)
           
static
<T> AvroType<T>
reflects(Class<T> clazz)
           
static
<T> AvroType<T>
reflects(Class<T> clazz, org.apache.avro.Schema schema)
           
static
<T> void
register(Class<T> clazz, AvroType<T> ptype)
           
static
<T extends org.apache.avro.specific.SpecificRecord>
AvroType<T>
specifics(Class<T> clazz)
           
static AvroType<String> strings()
           
static
<K,V> org.apache.crunch.types.avro.AvroTableType<K,V>
tableOf(PType<K> key, PType<V> value)
          A table type with an Avro type as key and as value.
static
<V1,V2,V3> AvroType<Tuple3<V1,V2,V3>>
triples(PType<V1> p1, PType<V2> p2, PType<V3> p3)
           
static
<T extends Tuple>
AvroType<T>
tuples(Class<T> clazz, PType... ptypes)
           
static AvroType<TupleN> tuples(PType... ptypes)
           
static PType<Union> unionOf(PType<?>... ptypes)
           
static
<T extends org.apache.hadoop.io.Writable>
AvroType<T>
writables(Class<T> clazz)
           
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

CAN_COMBINE_SPECIFIC_AND_REFLECT_SCHEMAS

public static final boolean CAN_COMBINE_SPECIFIC_AND_REFLECT_SCHEMAS
Older versions of Avro (i.e., before 1.7.0) do not support schemas that are composed of a mix of specific and reflection-based schemas. This bit controls whether or not we allow Crunch jobs to be created that involve mixing specific and reflection-based schemas and can be overridden by the client developer.


REFLECT_DATA_FACTORY

public static ReflectDataFactory REFLECT_DATA_FACTORY
Deprecated. as of 0.9.0; use AvroMode.REFLECT.override(ReaderWriterFactory)
The instance we use for generating reflected schemas. In releases up to 0.8.0, this may be modified by clients (e.g., Scrunch.) to override the reader, writer, and data instances used. Configuring the ReaderWriterFactory by setting this field is deprecated. Instead, use AvroMode.override(ReaderWriterFactory).


REFLECT_DATA_FACTORY_CLASS

public static final String REFLECT_DATA_FACTORY_CLASS
The name of the configuration parameter that tracks which reflection factory to use.

See Also:
Constant Field Values

UTF8_TO_STRING

public static MapFn<CharSequence,String> UTF8_TO_STRING

STRING_TO_UTF8

public static MapFn<String,org.apache.avro.util.Utf8> STRING_TO_UTF8

BYTES_IN

public static MapFn<Object,ByteBuffer> BYTES_IN
Method Detail

configureReflectDataFactory

@Deprecated
public static void configureReflectDataFactory(org.apache.hadoop.conf.Configuration conf)
Deprecated. as of 0.9.0; use AvroMode.REFLECT.configure(Configuration)


getReflectDataFactory

public static ReflectDataFactory getReflectDataFactory(org.apache.hadoop.conf.Configuration conf)
Deprecated. as of 0.9.0; use AvroMode.fromConfiguration(conf)


checkCombiningSpecificAndReflectionSchemas

public static void checkCombiningSpecificAndReflectionSchemas()

newReader

public static <T> org.apache.avro.io.DatumReader<T> newReader(org.apache.avro.Schema schema)

newReader

public static <T> org.apache.avro.io.DatumReader<T> newReader(AvroType<T> type)

newWriter

public static <T> org.apache.avro.io.DatumWriter<T> newWriter(org.apache.avro.Schema schema)

newWriter

public static <T> org.apache.avro.io.DatumWriter<T> newWriter(AvroType<T> type)

register

public static <T> void register(Class<T> clazz,
                                AvroType<T> ptype)

getPrimitiveType

public static <T> PType<T> getPrimitiveType(Class<T> clazz)

nulls

public static final AvroType<Void> nulls()

strings

public static final AvroType<String> strings()

longs

public static final AvroType<Long> longs()

ints

public static final AvroType<Integer> ints()

floats

public static final AvroType<Float> floats()

doubles

public static final AvroType<Double> doubles()

booleans

public static final AvroType<Boolean> booleans()

bytes

public static final AvroType<ByteBuffer> bytes()

records

public static final <T> AvroType<T> records(Class<T> clazz)

generics

public static final AvroType<org.apache.avro.generic.GenericData.Record> generics(org.apache.avro.Schema schema)

containers

public static final <T> AvroType<T> containers(Class<T> clazz)

specifics

public static final <T extends org.apache.avro.specific.SpecificRecord> AvroType<T> specifics(Class<T> clazz)

reflects

public static final <T> AvroType<T> reflects(Class<T> clazz)

reflects

public static final <T> AvroType<T> reflects(Class<T> clazz,
                                             org.apache.avro.Schema schema)

writables

public static final <T extends org.apache.hadoop.io.Writable> AvroType<T> writables(Class<T> clazz)

collections

public static final <T> AvroType<Collection<T>> collections(PType<T> ptype)

maps

public static final <T> AvroType<Map<String,T>> maps(PType<T> ptype)

pairs

public static final <V1,V2> AvroType<Pair<V1,V2>> pairs(PType<V1> p1,
                                                        PType<V2> p2)

triples

public static final <V1,V2,V3> AvroType<Tuple3<V1,V2,V3>> triples(PType<V1> p1,
                                                                  PType<V2> p2,
                                                                  PType<V3> p3)

quads

public static final <V1,V2,V3,V4> AvroType<Tuple4<V1,V2,V3,V4>> quads(PType<V1> p1,
                                                                      PType<V2> p2,
                                                                      PType<V3> p3,
                                                                      PType<V4> p4)

tuples

public static final AvroType<TupleN> tuples(PType... ptypes)

namedTuples

public static final AvroType<TupleN> namedTuples(String tupleName,
                                                 String[] fieldNames,
                                                 PType[] ptypes)

tuples

public static <T extends Tuple> AvroType<T> tuples(Class<T> clazz,
                                                   PType... ptypes)

unionOf

public static PType<Union> unionOf(PType<?>... ptypes)

derived

public static final <S,T> AvroType<T> derived(Class<T> clazz,
                                              MapFn<S,T> inputFn,
                                              MapFn<T,S> outputFn,
                                              PType<S> base)

derivedImmutable

public static final <S,T> AvroType<T> derivedImmutable(Class<T> clazz,
                                                       MapFn<S,T> inputFn,
                                                       MapFn<T,S> outputFn,
                                                       PType<S> base)

jsons

public static <T> PType<T> jsons(Class<T> clazz)

tableOf

public static final <K,V> org.apache.crunch.types.avro.AvroTableType<K,V> tableOf(PType<K> key,
                                                                                  PType<V> value)
A table type with an Avro type as key and as value.

The {code PTableType} returned by this method is also compatible with files containing Avro Pairs that are created using the org.apache.avro.mapred.AvroJob class.

Parameters:
key - the PType of the key in the table
value - the PType of the value in the table
Returns:
PTableType for reading and writing avro tables

keyValueTableOf

public static final <K,V> org.apache.crunch.types.avro.AvroKeyValueTableType<K,V> keyValueTableOf(PType<K> key,
                                                                                                  PType<V> value)
A table type with an Avro type as key and value. The PTableType returned by this method is specifically for reading and writing files that are compatible with those created via the org.apache.avro.mapreduce.AvroJob class. For all other Avro table purposes, the tableOf(org.apache.crunch.types.PType, org.apache.crunch.types.PType) method should be used.

Parameters:
key - the PType of the key in the table
value - the PType of the value in the table
Returns:
PTableType for reading and writing files compatible with those created via the org.apache.avro.mapreduce.AvroJob class


Copyright © 2014 The Apache Software Foundation. All Rights Reserved.