From e9090ffc74f687cf887cd99123fb1006499fa2b6 Mon Sep 17 00:00:00 2001 From: zeekling Date: Thu, 7 Dec 2023 22:54:36 +0800 Subject: [PATCH] =?UTF-8?q?=E5=AD=A6=E4=B9=A0=E4=BD=9C=E4=B8=9A=E5=90=AF?= =?UTF-8?q?=E5=8A=A8?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- yarn/job_start.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/yarn/job_start.md b/yarn/job_start.md index 4bdd466..baf61b0 100644 --- a/yarn/job_start.md +++ b/yarn/job_start.md @@ -288,6 +288,22 @@ protected void handleJobSetup(CommitterJobSetupEvent event) { } ``` +SetupCompletedTransition的处理逻辑如下,可以看到会定时启动MapTask和ReduceTask。 + +```java +public void transition(JobImpl job, JobEvent event) { + job.setupProgress = 1.0f; + job.scheduleTasks(job.mapTasks, job.numReduceTasks == 0); + job.scheduleTasks(job.reduceTasks, true); + + // If we have no tasks, just transition to job completed + if (job.numReduceTasks == 0 && job.numMapTasks == 0) { + job.eventHandler.handle(new JobEvent(job.jobId, + JobEventType.JOB_COMPLETED)); + } +} +``` + checkReadyForCommit函数的实现如下,可以看到在触发了CommitterJobCommitEvent事件,在CommitterJobCommitEvent里面会触发JOB_COMMIT事件。主要处理逻辑在handleJobCommit里面。 ```java @@ -344,3 +360,14 @@ protected void handleJobCommit(CommitterJobCommitEvent event) { } } ``` + + + +##### CommitSucceededTransition + +提交成功的事件处理handler为CommitSucceededTransition,核心处理逻辑如下: + +```java +job.logJobHistoryFinishedEvent(); +job.finished(JobStateInternal.SUCCEEDED); +```