This project has retired. For details please refer to its
Attic page .
SparkRuntimeContext (Apache Crunch 0.9.0 API)
org.apache.crunch.impl.spark
Class SparkRuntimeContext
java.lang.Object
org.apache.crunch.impl.spark.SparkRuntimeContext
All Implemented Interfaces: Serializable
public class SparkRuntimeContext extends Object implements Serializable
See Also: Serialized Form
Constructor Summary
SparkRuntimeContext (org.apache.spark.broadcast.Broadcast<org.apache.hadoop.conf.Configuration> broadConf,
org.apache.spark.Accumulator<Map <String ,Long >> counters)
Method Summary
org.apache.hadoop.conf.Configuration
getConfiguration ()
static org.apache.hadoop.mapreduce.TaskInputOutputContext
getTaskIOContext (org.apache.spark.broadcast.Broadcast<org.apache.hadoop.conf.Configuration> conf,
org.apache.spark.Accumulator<Map <String ,Long >> counters)
void
initialize (DoFn <?,?> fn)
Methods inherited from class java.lang.Object
clone , equals , finalize , getClass , hashCode , notify , notifyAll , toString , wait , wait , wait
SparkRuntimeContext
public SparkRuntimeContext (org.apache.spark.broadcast.Broadcast<org.apache.hadoop.conf.Configuration> broadConf,
org.apache.spark.Accumulator<Map <String ,Long >> counters)
initialize
public void initialize (DoFn <?,?> fn)
getConfiguration
public org.apache.hadoop.conf.Configuration getConfiguration ()
getTaskIOContext
public static org.apache.hadoop.mapreduce.TaskInputOutputContext getTaskIOContext (org.apache.spark.broadcast.Broadcast<org.apache.hadoop.conf.Configuration> conf,
org.apache.spark.Accumulator<Map <String ,Long >> counters)
Copyright © 2014 The Apache Software Foundation . All Rights Reserved.