org.apache.crunch.contrib.bloomfilter

Class BloomFilterFactory

java.lang.Object

org.apache.crunch.contrib.bloomfilter.BloomFilterFactory

public class BloomFilterFactory
extends Object

Factory Class for creating BloomFilters. The APIs require a BloomFilterFn which is responsible for generating keys of the filter.

Constructor Summary

Constructors

Constructor and Description
BloomFilterFactory()

Method Summary

Methods

Modifier and Type	Method and Description
static PObject<Map<String,org.apache.hadoop.util.bloom.BloomFilter>>	createFilter(org.apache.hadoop.fs.Path inputPath, BloomFilterFn<String> filterFn) The method will take an input path and generates BloomFilters for all text files in that path.
static <T> PObject<org.apache.hadoop.util.bloom.BloomFilter>	createFilter(PCollection<T> collection, BloomFilterFn<T> filterFn)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BloomFilterFactory

public BloomFilterFactory()

Method Detail

createFilter

public static PObject<Map<String,org.apache.hadoop.util.bloom.BloomFilter>> createFilter(org.apache.hadoop.fs.Path inputPath,
                                                                         BloomFilterFn<String> filterFn)
                                                                                  throws IOException

The method will take an input path and generates BloomFilters for all text files in that path. The method return back a PObject containing a Map having file names as keys and filters as values

Throws:

IOException

createFilter

public static <T> PObject<org.apache.hadoop.util.bloom.BloomFilter> createFilter(PCollection<T> collection,
                                                                 BloomFilterFn<T> filterFn)