学习作业启动
This commit is contained in:
parent
ccc687cf68
commit
1d7b0176a9
@ -59,7 +59,86 @@ Yarn资源调度过程待完善,后面会单独章节学习。
|
||||
|
||||
### initAndStartAppMaster:启动AppMaster
|
||||
|
||||
MRAppMaster是
|
||||
MRAppMaster在yarn内部是一个服务,最终启动的时候会调用到serviceStart函数里面,所以我们主要看这个函数里面做了什么。
|
||||
|
||||
|
||||
|
||||
#### 1、创建并且初始化Job
|
||||
|
||||
创建Job对象并且将其初始化掉。但是不会启动当前作业。
|
||||
|
||||
- 初始化JobImpl对象。在JobImpl初始化的时候做了下面几件事:
|
||||
|
||||
- 初始化线程池。
|
||||
|
||||
- 初始化作业状态机的核心代码如下:
|
||||
|
||||
```java
|
||||
protected static final
|
||||
StateMachineFactory<JobImpl, JobStateInternal, JobEventType, JobEvent>
|
||||
stateMachineFactory
|
||||
= new StateMachineFactory<JobImpl, JobStateInternal, JobEventType, JobEvent>
|
||||
(JobStateInternal.NEW)
|
||||
// Transitions from NEW state
|
||||
.addTransition(JobStateInternal.NEW, JobStateInternal.NEW,
|
||||
JobEventType.JOB_DIAGNOSTIC_UPDATE,
|
||||
DIAGNOSTIC_UPDATE_TRANSITION)
|
||||
.addTransition(JobStateInternal.NEW, JobStateInternal.NEW,
|
||||
JobEventType.JOB_COUNTER_UPDATE, COUNTER_UPDATE_TRANSITION)
|
||||
// ....省略...
|
||||
.addTransition(JobStateInternal.REBOOT, JobStateInternal.REBOOT,
|
||||
JobEventType.JOB_COUNTER_UPDATE, COUNTER_UPDATE_TRANSITION)
|
||||
// create the topology tables
|
||||
.installTopology();
|
||||
|
||||
- 初始化其他配置。
|
||||
|
||||
- 在中央处理器里面注册JobFinishEvent类型事件以及事件处理的handler。
|
||||
|
||||
```java
|
||||
protected Job createJob(Configuration conf, JobStateInternal forcedState,
|
||||
String diagnostic) {
|
||||
// create single job
|
||||
Job newJob =
|
||||
new JobImpl(jobId, appAttemptID, conf, dispatcher.getEventHandler(),
|
||||
taskAttemptListener, jobTokenSecretManager, jobCredentials, clock,
|
||||
completedTasksFromPreviousRun, metrics,
|
||||
committer, newApiCommitter,
|
||||
currentUser.getUserName(), appSubmitTime, amInfos, context,
|
||||
forcedState, diagnostic);
|
||||
((RunningAppContext) context).jobs.put(newJob.getID(), newJob);
|
||||
dispatcher.register(JobFinishEvent.Type.class,
|
||||
createJobFinishEventHandler());
|
||||
return newJob;
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
|
||||
#### 2、发送inited事件
|
||||
|
||||
发送inited事件的对象主要是下面两个:
|
||||
|
||||
- 通过dispatcher给历史AM发送。
|
||||
- 当前AM。代码如下:
|
||||
|
||||
```java
|
||||
// Send out an MR AM inited event for this AM.
|
||||
dispatcher.getEventHandler().handle(
|
||||
new JobHistoryEvent(job.getID(), new AMStartedEvent(amInfo
|
||||
.getAppAttemptId(), amInfo.getStartTime(), amInfo.getContainerId(),
|
||||
amInfo.getNodeManagerHost(), amInfo.getNodeManagerPort(), amInfo
|
||||
.getNodeManagerHttpPort(), this.forcedState == null ? null
|
||||
: this.forcedState.toString(), appSubmitTime)));
|
||||
```
|
||||
|
||||
|
||||
|
||||
#### 3、创建job init事件,并且处理
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user