org.apache.crunch.impl.spark

Class SparkRuntimeContext

java.lang.Object

org.apache.crunch.impl.spark.SparkRuntimeContext

All Implemented Interfaces:

Serializable

public class SparkRuntimeContext
extends Object
implements Serializable

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SparkRuntimeContext(String jobName, org.apache.spark.Accumulator<Map<String,Map<String,Long>>> counters, org.apache.spark.broadcast.Broadcast<byte[]> broadConf)

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.conf.Configuration	getConfiguration()
void	initialize(DoFn<?,?> fn, Integer tid)
void	setConf(org.apache.spark.broadcast.Broadcast<byte[]> broadConf)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkRuntimeContext

public SparkRuntimeContext(String jobName,
                           org.apache.spark.Accumulator<Map<String,Map<String,Long>>> counters,
                           org.apache.spark.broadcast.Broadcast<byte[]> broadConf)

Method Detail

setConf

public void setConf(org.apache.spark.broadcast.Broadcast<byte[]> broadConf)

initialize

public void initialize(DoFn<?,?> fn,
                       Integer tid)

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()