big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong

Browse Source
This commit is contained in:
Jason Lowe 2016-02-02 19:30:45 +00:00
parent dd9ebf6eed
commit 43e669b22d
2 changed files with 14 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
hadoop-mapreduce-project/CHANGES.txt
View File

@ -739,6 +739,9 @@ Release 2.7.3 - UNRELEASED
MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread.
(Xuan Gong via jlowe) (Xuan Gong via jlowe)
MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal() (Xuan Gong
via jlowe)
Release 2.7.2 - 2016-01-25 Release 2.7.2 - 2016-01-25
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES
@ -1047,6 +1050,9 @@ Release 2.6.4 - UNRELEASED
MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread.
(Xuan Gong via jlowe) (Xuan Gong via jlowe)
MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal() (Xuan Gong
via jlowe)
Release 2.6.3 - 2015-12-17 Release 2.6.3 - 2015-12-17
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

8
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java
View File

@ -577,10 +577,18 @@ public Job run() throws IOException, ClassNotFoundException,
return job; return job;
} }
}); });
Cluster prev = cluster;
// update our Cluster instance with the one created by Job for submission // update our Cluster instance with the one created by Job for submission
// (we can't pass our Cluster instance to Job, since Job wraps the config // (we can't pass our Cluster instance to Job, since Job wraps the config
// instance, and the two configs would then diverge) // instance, and the two configs would then diverge)
cluster = job.getCluster(); cluster = job.getCluster();
// It is important to close the previous cluster instance
// to cleanup resources.
if (prev != null) {
prev.close();
}
return new NetworkedJob(job); return new NetworkedJob(job);
} catch (InterruptedException ie) { } catch (InterruptedException ie) {
throw new IOException("interrupted", ie); throw new IOException("interrupted", ie);