org.apache.crunch.hadoop.mapreduce.lib.jobcontrol
Class CrunchJobControl

java.lang.Object
  extended by org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl
All Implemented Interfaces:
Runnable

public class CrunchJobControl
extends Object
implements Runnable

This class encapsulates a set of MapReduce jobs and its dependency. It tracks the states of the jobs by placing them into different tables according to their states. This class provides APIs for the client app to add a job to the group and to get the jobs in the group in different states. When a job is added, an ID unique to the group is assigned to the job. This class has a thread that submits jobs when they become ready, monitors the states of the running jobs, and updates the states of jobs based on the state changes of their depending jobs states. The class provides APIs for suspending/resuming the thread, and for stopping the thread. TODO This is mostly a copy of the JobControl class in Hadoop MapReduce core. Once the location and interface of the class are more stable in CDH, this class should be removed completely and be based on the hadoop-core class.


Nested Class Summary
static class CrunchJobControl.ThreadState
           
 
Constructor Summary
CrunchJobControl(String groupName)
          Construct a job control for a group of jobs.
 
Method Summary
 String addJob(CrunchControlledJob aJob)
          Add a new job.
 void addJobCollection(Collection<CrunchControlledJob> jobs)
          Add a collection of jobs
 boolean allFinished()
           
 List<CrunchControlledJob> getFailedJobList()
           
 List<CrunchControlledJob> getReadyJobsList()
           
 List<CrunchControlledJob> getRunningJobList()
           
 List<CrunchControlledJob> getSuccessfulJobList()
           
 CrunchJobControl.ThreadState getThreadState()
           
 List<CrunchControlledJob> getWaitingJobList()
           
 void resume()
          resume the suspended thread
 void run()
          The main loop for the thread.
 void stop()
          set the thread state to STOPPING so that the thread will stop when it wakes up.
 void suspend()
          suspend the running thread
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

CrunchJobControl

public CrunchJobControl(String groupName)
Construct a job control for a group of jobs.

Parameters:
groupName - a name identifying this group
Method Detail

getWaitingJobList

public List<CrunchControlledJob> getWaitingJobList()
Returns:
the jobs in the waiting state

getRunningJobList

public List<CrunchControlledJob> getRunningJobList()
Returns:
the jobs in the running state

getReadyJobsList

public List<CrunchControlledJob> getReadyJobsList()
Returns:
the jobs in the ready state

getSuccessfulJobList

public List<CrunchControlledJob> getSuccessfulJobList()
Returns:
the jobs in the success state

getFailedJobList

public List<CrunchControlledJob> getFailedJobList()

addJob

public String addJob(CrunchControlledJob aJob)
Add a new job.

Parameters:
aJob - the new job

addJobCollection

public void addJobCollection(Collection<CrunchControlledJob> jobs)
Add a collection of jobs

Parameters:
jobs -

getThreadState

public CrunchJobControl.ThreadState getThreadState()
Returns:
the thread state

stop

public void stop()
set the thread state to STOPPING so that the thread will stop when it wakes up.


suspend

public void suspend()
suspend the running thread


resume

public void resume()
resume the suspended thread


allFinished

public boolean allFinished()

run

public void run()
The main loop for the thread. The loop does the following: Check the states of the running jobs Update the states of waiting jobs Submit the jobs in ready state

Specified by:
run in interface Runnable


Copyright © 2012 The Apache Software Foundation. All Rights Reserved.