This project has retired. For details please refer to its Attic page.

org.apache.crunch.hadoop.mapreduce.lib.jobcontrol
Class CrunchJobControl

java.lang.Object
  org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl

All Implemented Interfaces:

Runnable

public class CrunchJobControl
extends Object
implements Runnable

This class encapsulates a set of MapReduce jobs and its dependency. It tracks the states of the jobs by placing them into different tables according to their states. This class provides APIs for the client app to add a job to the group and to get the jobs in the group in different states. When a job is added, an ID unique to the group is assigned to the job. This class has a thread that submits jobs when they become ready, monitors the states of the running jobs, and updates the states of jobs based on the state changes of their depending jobs states. The class provides APIs for suspending/resuming the thread, and for stopping the thread. TODO This is mostly a copy of the JobControl class in Hadoop MapReduce core. Once the location and interface of the class are more stable in CDH, this class should be removed completely and be based on the hadoop-core class.

Nested Class Summary

static class

CrunchJobControl.ThreadState

Constructor Summary

CrunchJobControl(String groupName)
Construct a job control for a group of jobs.

Method Summary

String	addJob(CrunchControlledJob aJob) Add a new job.
void	addJobCollection(Collection<CrunchControlledJob> jobs) Add a collection of jobs
boolean	allFinished()
List<CrunchControlledJob>	getFailedJobList()
List<CrunchControlledJob>	getReadyJobsList()
List<CrunchControlledJob>	getRunningJobList()
List<CrunchControlledJob>	getSuccessfulJobList()
CrunchJobControl.ThreadState	getThreadState()
List<CrunchControlledJob>	getWaitingJobList()
void	resume() resume the suspended thread
void	run() The main loop for the thread.
void	stop() set the thread state to STOPPING so that the thread will stop when it wakes up.
void	suspend() suspend the running thread

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CrunchJobControl

public CrunchJobControl(String groupName)

Construct a job control for a group of jobs.

Parameters:

groupName - a name identifying this group

Method Detail

getWaitingJobList

public List<CrunchControlledJob> getWaitingJobList()

Returns:

the jobs in the waiting state

getRunningJobList

public List<CrunchControlledJob> getRunningJobList()

Returns:

the jobs in the running state

getReadyJobsList

public List<CrunchControlledJob> getReadyJobsList()

Returns:

the jobs in the ready state

getSuccessfulJobList

public List<CrunchControlledJob> getSuccessfulJobList()

Returns:

the jobs in the success state

getFailedJobList

public List<CrunchControlledJob> getFailedJobList()

addJob

public String addJob(CrunchControlledJob aJob)

Add a new job.

Parameters:

aJob - the new job

addJobCollection

public void addJobCollection(Collection<CrunchControlledJob> jobs)

Add a collection of jobs

Parameters:

jobs -

getThreadState

public CrunchJobControl.ThreadState getThreadState()

Returns:

the thread state

stop

public void stop()

set the thread state to STOPPING so that the thread will stop when it wakes up.

suspend

public void suspend()

suspend the running thread

resume

public void resume()

resume the suspended thread

allFinished

public boolean allFinished()

run

public void run()

The main loop for the thread. The loop does the following: Check the states of the running jobs Update the states of waiting jobs Submit the jobs in ready state

Specified by:

run in interface Runnable