Page History

...

Fields

A job, derived from JobInProgress, in general holds some basic information, including:

...

• jobId: BSPJobID. The id of this job.
• jobName: String. The name of this job.
• user: String. See BSPJobContext.getUser() wherein configures via conf.get("user.name")
• jobFile: Path. job.xml. See BSPJobClient.submitJobInternal()
• localJobFile: Path. A local copied jobFile.
• localJarFile: Path. A local copied jar file.
• numBSPTasks: int. The number of BSP tasks to be
• masterjobName: String. The name of master server that is responsible for this job.
• maxTaskAttempts: int. default to 2.
• MAX_TASK_EXEC: int. bound to 1.
• inputPath: Path. Dir where split files are stored. i/o.
• State: RUNNING(1), SUCCEEDED(2), FAILED(3), PREP(4), KILLED(5)
• progress: long. % of bsp(). job status.
• setupProgress: JobStatus. Mainly JobStatus.Statelong. % of setup() job status.
• cleanupProgresssuperstepCount: long. This job's superstep count% of cleanup() job status.
• startTime: long. The time this job is started. job status.
• finishTime: long. The time this job is finished.
• maxTaskAttempts: int. default to 2.
• MAX_TASK_EXEC: int. bound to 1.
• numBSPTasks: int. The number of BSP tasks to be launched.
• job status.
• superstepCount: long. This job's superstep count. job satus.
• conf: Configuration. Specific setting master: String. The name of master server that is responsible for this job.
• inputPath: Path. Dir where split files are stored.
• Tasks: Actual launched TaskInProgress depends on numBSPTasks.

TODO: An object that holds multiple tasks execution including successful/ failed/ current task

• Tasks: TaskTable. See TaskInProgress and Job Task Relation section.
  • attemptId: TaskAttemptID. A task attempt id : TaskID object.in which a TaskID, containing BSPJobID, is embedded.
  • failed: Boolean. If a task fails or not. default to false.
  • NUM_ATTEMPTS_PER_RESTART: int. Set to 1000.startTime: System.currentTimeMillis()
  • partition: The _i_th numBSPTasks.
  • RawSplit: The _i_th RawSplit if RawSplit not null
  • SuccessfulTaskId: Set when TaskInProgress.completed() is called. TODO: isSuccessful
  • firstTaskId: The first task id.
  • currentTaskId:
  • TaskStatus:
    • Phase: Denote at which phase current task is. See Task Phase below.
    • State: Denote the current state the task is. See Task State below.
      

Job Status

• • Phase: SETUP, COMPUTE, BARRIER_SYNC, CLEANUP
  • State: RUNNING, SUCCEEDED, FAILED, KILLED
  • completed: Boolean. if a task is completed/ succeed. default to false.
  • groom: String. On which groom a task runs.
  • startTime: System.currentTimeMillis(). default -1.
  • RawSplit: The _i_th RawSplit if RawSplit is not null
    

State

A job status contains:

...

• PREP: This status is started when a job is created until all tasks are dispatched to groom servers.
  • SetUp a job
  • Scheding
• RUNNING: When tasks are executed in GroomServers.
• SUCCEEDED: When all tasks are in the SUCCEEDED state.
• FAILED: When one or more tasks exceed its max attempt
• KILLEDCANCELLED: User/ Admin issues a kill command.
  

Image Removed

Task Phase

A single task phase.

• STARTING: Initial phase
• COMPUTE: Within bsp()
• BARRIER_SYNC: Reaching sync().
• CLEANUP: Finish entire supersteps execution.

Image Removed

Task State

This only indicate a task state.

• UNASSIGNED: Initial state of a task.
• RUNNING: Only valid when a task is either in COMPUTE or in BARRIER_SYNC phase.
• SUCCEEDED: A task completes its execution at the end of CLEANUP phase.
• FAILED: An exception or an error occurrs.
• KILLED

Image Removed

• cancel command.

Image Modified