This project has retired. For details please refer to its Attic page.
Tokenizer (Apache Crunch 0.11.0 API)

org.apache.crunch.contrib.text
Class Tokenizer

java.lang.Object
  extended by org.apache.crunch.contrib.text.Tokenizer

public class Tokenizer
extends Object

Manages a Scanner instance and provides support for returning only a subset of the fields returned by the underlying Scanner.


Constructor Summary
Tokenizer(Scanner scanner, Set<Integer> indices, boolean keep)
          Create a new Tokenizer instance.
 
Method Summary
 boolean hasNext()
          Returns true if the underlying Scanner has any tokens remaining.
 String next()
          Advance this Tokenizer and return the next String from the Scanner.
 Boolean nextBoolean()
          Advance this Tokenizer and return the next Boolean from the Scanner.
 Double nextDouble()
          Advance this Tokenizer and return the next Double from the Scanner.
 Float nextFloat()
          Advance this Tokenizer and return the next Float from the Scanner.
 Integer nextInt()
          Advance this Tokenizer and return the next Integer from the Scanner.
 Long nextLong()
          Advance this Tokenizer and return the next Long from the Scanner.
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

Tokenizer

public Tokenizer(Scanner scanner,
                 Set<Integer> indices,
                 boolean keep)
Create a new Tokenizer instance.

Parameters:
scanner - The scanner to manage
indices - The indices to keep/drop
keep - Whether the indices should be kept (true) or dropped (false)
Method Detail

hasNext

public boolean hasNext()
Returns true if the underlying Scanner has any tokens remaining.


next

public String next()
Advance this Tokenizer and return the next String from the Scanner.

Returns:
The next String from the Scanner

nextLong

public Long nextLong()
Advance this Tokenizer and return the next Long from the Scanner.

Returns:
The next Long from the Scanner

nextBoolean

public Boolean nextBoolean()
Advance this Tokenizer and return the next Boolean from the Scanner.

Returns:
The next Boolean from the Scanner

nextDouble

public Double nextDouble()
Advance this Tokenizer and return the next Double from the Scanner.

Returns:
The next Double from the Scanner

nextFloat

public Float nextFloat()
Advance this Tokenizer and return the next Float from the Scanner.

Returns:
The next Float from the Scanner

nextInt

public Integer nextInt()
Advance this Tokenizer and return the next Integer from the Scanner.

Returns:
The next Integer from the Scanner


Copyright © 2014 The Apache Software Foundation. All Rights Reserved.