This project has retired. For details please refer to its Attic page.

org.apache.crunch.io.avro.trevni
Class TrevniRecordWriter<T>

java.lang.Object
  org.apache.hadoop.mapreduce.RecordWriter<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable>
      org.apache.crunch.io.avro.trevni.TrevniRecordWriter<T>

public class TrevniRecordWriter<T>
extends org.apache.hadoop.mapreduce.RecordWriter<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable>

Field Summary

static String	EXT trevni file extension
protected org.apache.trevni.ColumnFileMetaData	meta meta data to be stored in the output file.
static String	META_PREFIX prefix of job configs that we care about
protected int	part Counter that increments as new trevni files are create because the current file has exceeded the block size
protected org.apache.avro.Schema	schema Provided avro schema from the context
protected org.apache.trevni.avro.AvroColumnWriter<T>	writer Trevni file writer

Constructor Summary

TrevniRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Method Summary

void	close(org.apache.hadoop.mapreduce.TaskAttemptContext arg0)
void	flush() A Trevni flush will close the current file and prep a new writer
protected org.apache.avro.Schema	initSchema(org.apache.hadoop.mapreduce.TaskAttemptContext context)
void	write(org.apache.avro.mapred.AvroKey<T> key, org.apache.hadoop.io.NullWritable value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EXT

public static final String EXT

trevni file extension

See Also:

Constant Field Values

META_PREFIX

public static final String META_PREFIX

prefix of job configs that we care about

See Also:

Constant Field Values

part

protected int part

Counter that increments as new trevni files are create because the current file has exceeded the block size

writer

protected org.apache.trevni.avro.AvroColumnWriter<T> writer

Trevni file writer

schema

protected org.apache.avro.Schema schema

Provided avro schema from the context

meta

protected org.apache.trevni.ColumnFileMetaData meta

meta data to be stored in the output file.

Constructor Detail

TrevniRecordWriter

public TrevniRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                   throws IOException

Throws:

IOException

Method Detail

write

public void write(org.apache.avro.mapred.AvroKey<T> key,
                  org.apache.hadoop.io.NullWritable value)
           throws IOException,
                  InterruptedException

Specified by:

write in class org.apache.hadoop.mapreduce.RecordWriter<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable>

Throws:

IOException

InterruptedException

initSchema

protected org.apache.avro.Schema initSchema(org.apache.hadoop.mapreduce.TaskAttemptContext context)

flush

public void flush()
           throws IOException

A Trevni flush will close the current file and prep a new writer

Throws:

IOException

close

public void close(org.apache.hadoop.mapreduce.TaskAttemptContext arg0)
           throws IOException,
                  InterruptedException

Specified by:

close in class org.apache.hadoop.mapreduce.RecordWriter<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable>

Throws:

IOException

InterruptedException