This project has retired. For details please refer to its Attic page.

org.apache.crunch.contrib.text
Class Tokenizer

java.lang.Object
  org.apache.crunch.contrib.text.Tokenizer

public class Tokenizer
extends Object

Manages a Scanner instance and provides support for returning only a subset of the fields returned by the underlying Scanner.

Constructor Summary

Tokenizer(Scanner scanner, Set<Integer> indices, boolean keep)
Create a new Tokenizer instance.

Method Summary

boolean	hasNext() Returns true if the underlying Scanner has any tokens remaining.
String	next() Advance this Tokenizer and return the next String from the Scanner.
Boolean	nextBoolean() Advance this Tokenizer and return the next Boolean from the Scanner.
Double	nextDouble() Advance this Tokenizer and return the next Double from the Scanner.
Float	nextFloat() Advance this Tokenizer and return the next Float from the Scanner.
Integer	nextInt() Advance this Tokenizer and return the next Integer from the Scanner.
Long	nextLong() Advance this Tokenizer and return the next Long from the Scanner.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tokenizer

public Tokenizer(Scanner scanner,
                 Set<Integer> indices,
                 boolean keep)

Create a new Tokenizer instance.

Parameters:

scanner - The scanner to manage

indices - The indices to keep/drop

keep - Whether the indices should be kept (true) or dropped (false)

Method Detail

hasNext

public boolean hasNext()

Returns true if the underlying Scanner has any tokens remaining.

next

public String next()

Advance this Tokenizer and return the next String from the Scanner.

Returns:

The next String from the Scanner

nextLong

public Long nextLong()

Advance this Tokenizer and return the next Long from the Scanner.

Returns:

The next Long from the Scanner

nextBoolean

public Boolean nextBoolean()

Advance this Tokenizer and return the next Boolean from the Scanner.

Returns:

The next Boolean from the Scanner

nextDouble

public Double nextDouble()

Advance this Tokenizer and return the next Double from the Scanner.

Returns:

The next Double from the Scanner

nextFloat

public Float nextFloat()

Advance this Tokenizer and return the next Float from the Scanner.

Returns:

The next Float from the Scanner

nextInt

public Integer nextInt()

Advance this Tokenizer and return the next Integer from the Scanner.

Returns:

The next Integer from the Scanner