org.apache.crunch.kafka.inputformat

Class KafkaInputSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.crunch.kafka.inputformat.KafkaInputSplit

All Implemented Interfaces:

org.apache.hadoop.io.Writable

public class KafkaInputSplit
extends org.apache.hadoop.mapreduce.InputSplit
implements org.apache.hadoop.io.Writable

InputSplit that represent retrieving data from a single TopicPartition between the specified start and end offsets.

Constructor Summary

Constructors

Constructor and Description
KafkaInputSplit() Nullary Constructor for creating the instance inside the Mapper instance.
KafkaInputSplit(String topic, int partition, long startingOffset, long endingOffset) Constructs an input split for the provided topic and partition restricting data to be between the startingOffset and endingOffset

Method Summary

Modifier and Type	Method and Description
long	getEndingOffset() Returns the ending offset for the split
long	getLength()
String[]	getLocations()
long	getStartingOffset() Returns the starting offset for the split
org.apache.kafka.common.TopicPartition	getTopicPartition() Returns the topic and partition for the split
void	readFields(DataInput dataInput)
String	toString()
void	write(DataOutput dataOutput)

Methods inherited from class org.apache.hadoop.mapreduce.InputSplit

getLocationInfo

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

KafkaInputSplit

public KafkaInputSplit()

Nullary Constructor for creating the instance inside the Mapper instance.

KafkaInputSplit

public KafkaInputSplit(String topic,
                       int partition,
                       long startingOffset,
                       long endingOffset)

Constructs an input split for the provided topic and partition restricting data to be between the startingOffset and endingOffset

Parameters:

topic - the topic for the split

partition - the partition for the topic

startingOffset - the start of the split

endingOffset - the end of the split

Method Detail

getLength

public long getLength()
               throws IOException,
                      InterruptedException

Specified by:

getLength in class org.apache.hadoop.mapreduce.InputSplit

Throws:

IOException

InterruptedException

getLocations

public String[] getLocations()
                      throws IOException,
                             InterruptedException

Specified by:

getLocations in class org.apache.hadoop.mapreduce.InputSplit

Throws:

IOException

InterruptedException

getTopicPartition

public org.apache.kafka.common.TopicPartition getTopicPartition()

Returns the topic and partition for the split

Returns:

the topic and partition for the split

getStartingOffset

public long getStartingOffset()

Returns the starting offset for the split

Returns:

the starting offset for the split

getEndingOffset

public long getEndingOffset()

Returns the ending offset for the split

Returns:

the ending offset for the split

write

public void write(DataOutput dataOutput)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

IOException

readFields

public void readFields(DataInput dataInput)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object