big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MAPREDUCE-3154. Fix JobSubmitter to check for output specs before copying job submission files to fail fast. Contributed by Abhijit Suresh Shingate.

Browse Source 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180774 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Arun Murthy 2011-10-10 03:29:38 +00:00
parent 811a6a9e30
commit d0016c6120
2 changed files with 6 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-mapreduce-project/CHANGES.txt
View File

@ -1555,6 +1555,9 @@ Release 0.23.0 - Unreleased
MAPREDUCE-3123. Fix NM to quote symlink names to escape special MAPREDUCE-3123. Fix NM to quote symlink names to escape special
characters. (Hitesh Shah via acmurthy) characters. (Hitesh Shah via acmurthy)
MAPREDUCE-3154. Fix JobSubmitter to check for output specs before copying
job submission files to fail fast. (Abhijit Suresh Shingate via acmurthy)
Release 0.22.0 - Unreleased Release 0.22.0 - Unreleased
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

5
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
View File

@ -322,6 +322,9 @@ private void copyAndConfigureFiles(Job job, Path jobSubmitDir)
JobStatus submitJobInternal(Job job, Cluster cluster) JobStatus submitJobInternal(Job job, Cluster cluster)
throws ClassNotFoundException, InterruptedException, IOException { throws ClassNotFoundException, InterruptedException, IOException {
//validate the jobs output specs
checkSpecs(job);
Path jobStagingArea = JobSubmissionFiles.getStagingDir(cluster, Path jobStagingArea = JobSubmissionFiles.getStagingDir(cluster,
job.getConfiguration()); job.getConfiguration());
//configure the command line options correctly on the submitting dfs //configure the command line options correctly on the submitting dfs
@ -350,8 +353,6 @@ JobStatus submitJobInternal(Job job, Cluster cluster)
copyAndConfigureFiles(job, submitJobDir); copyAndConfigureFiles(job, submitJobDir);
Path submitJobFile = JobSubmissionFiles.getJobConfPath(submitJobDir); Path submitJobFile = JobSubmissionFiles.getJobConfPath(submitJobDir);
checkSpecs(job);
// Create the splits for the job // Create the splits for the job
LOG.debug("Creating splits at " + jtFs.makeQualified(submitJobDir)); LOG.debug("Creating splits at " + jtFs.makeQualified(submitJobDir));
int maps = writeSplits(job, submitJobDir); int maps = writeSplits(job, submitJobDir);