This project has retired. For details please refer to its Attic page.

org.apache.crunch.io.hbase
Class HBaseSourceTarget

java.lang.Object
  org.apache.crunch.io.hbase.HBaseTarget
      org.apache.crunch.io.hbase.HBaseSourceTarget

All Implemented Interfaces:

MapReduceTarget, ReadableSource<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>, ReadableSourceTarget<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>, Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>, SourceTarget<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>, TableSource<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>, Target

public class HBaseSourceTarget
extends HBaseTarget
implements ReadableSourceTarget<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>, TableSource<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.crunch.Target

Target.WriteMode

Field Summary

protected org.apache.hadoop.hbase.client.Scan

scan

Fields inherited from class org.apache.crunch.io.hbase.HBaseTarget

table

Constructor Summary

HBaseSourceTarget(String table, org.apache.hadoop.hbase.client.Scan scan)

Method Summary

ReadableData<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>	asReadable()
SourceTarget<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>	conf(String key, String value) Adds the given key-value pair to the Configuration instance(s) that are used to read and write this SourceTarget<T>.
void	configureSource(org.apache.hadoop.mapreduce.Job job, int inputId) Configure the given job to use this source as an input.
static org.apache.hadoop.hbase.client.Scan	convertStringToScan(String string)
boolean	equals(Object other)
Converter<?,?,?,?>	getConverter() Returns the Converter used for mapping the inputs from this instance into PCollection or PTable values.
long	getLastModifiedAt(org.apache.hadoop.conf.Configuration configuration) Returns the time (in milliseconds) that this Source was most recently modified (e.g., because an input file was edited or new files were added to a directory.)
long	getSize(org.apache.hadoop.conf.Configuration conf) Returns the number of bytes in this Source.
PTableType<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>	getTableType()
PType<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>	getType() Returns the PType for this source.
int	hashCode()
Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>	inputConf(String key, String value) Adds the given key-value pair to the Configuration instance that is used to read this Source<T></T>.
Iterable<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>	read(org.apache.hadoop.conf.Configuration conf) Returns an Iterable that contains the contents of this source.
String	toString()

Methods inherited from class org.apache.crunch.io.hbase.HBaseTarget

accept, asSourceTarget, configureForMapReduce, getConverter, handleExisting, outputConf

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.crunch.Target

accept, asSourceTarget, getConverter, handleExisting, outputConf

Field Detail

scan

protected org.apache.hadoop.hbase.client.Scan scan

Constructor Detail

HBaseSourceTarget

public HBaseSourceTarget(String table,
                         org.apache.hadoop.hbase.client.Scan scan)

Method Detail

inputConf

public Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>> inputConf(String key,
                                                                                                                       String value)

Description copied from interface: Source

Adds the given key-value pair to the Configuration instance that is used to read this Source<T></T>. Allows for multiple inputs to re-use the same config keys with different values when necessary.

Specified by:

inputConf in interface Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>

getType

public PType<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>> getType()

Description copied from interface: Source

Returns the PType for this source.

Specified by:

getType in interface Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>

getTableType

public PTableType<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result> getTableType()

Specified by:

getTableType in interface TableSource<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>

equals

public boolean equals(Object other)

Overrides:

equals in class HBaseTarget

hashCode

public int hashCode()

Overrides:

hashCode in class HBaseTarget

toString

public String toString()

Overrides:

toString in class HBaseTarget

configureSource

public void configureSource(org.apache.hadoop.mapreduce.Job job,
                            int inputId)
                     throws IOException

Description copied from interface: Source

Configure the given job to use this source as an input.

Specified by:

configureSource in interface Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>

Parameters:

job - The job to configure

inputId - For a multi-input job, an identifier for this input to the job

Throws:

IOException

convertStringToScan

public static org.apache.hadoop.hbase.client.Scan convertStringToScan(String string)
                                                               throws IOException

Throws:

IOException

getSize

public long getSize(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: Source

Returns the number of bytes in this Source.

Specified by:

getSize in interface Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>

getLastModifiedAt

public long getLastModifiedAt(org.apache.hadoop.conf.Configuration configuration)

Description copied from interface: Source

Returns the time (in milliseconds) that this Source was most recently modified (e.g., because an input file was edited or new files were added to a directory.)

Specified by:

getLastModifiedAt in interface Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>

getConverter

public Converter<?,?,?,?> getConverter()

Description copied from interface: Source

Returns the Converter used for mapping the inputs from this instance into PCollection or PTable values.

Specified by:

getConverter in interface Source<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>

read

public Iterable<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>> read(org.apache.hadoop.conf.Configuration conf)
                                                                                                             throws IOException

Description copied from interface: ReadableSource

Returns an Iterable that contains the contents of this source.

Specified by:

read in interface ReadableSource<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>

Parameters:

conf - The current Configuration instance

Returns:

the contents of this Source as an Iterable instance

Throws:

IOException

asReadable

public ReadableData<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>> asReadable()

Specified by:

asReadable in interface ReadableSource<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>

Returns:

a ReadableData instance containing the data referenced by this ReadableSource.

conf

public SourceTarget<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>> conf(String key,
                                                                                                                        String value)

Description copied from interface: SourceTarget

Adds the given key-value pair to the Configuration instance(s) that are used to read and write this SourceTarget<T>. Allows for multiple inputs and outputs to re-use the same config keys with different values when necessary.

Specified by:

conf in interface SourceTarget<Pair<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>>