big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MAPREDUCE-5912. Task.calculateOutputSize does not handle Windows files after MAPREDUCE-5196. Contributed by Remus Rusanu.

Browse Source 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602282 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Chris Nauroth 2014-06-12 20:02:45 +00:00
parent 2368332138
commit 180ac8dc1d
2 changed files with 5 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-mapreduce-project/CHANGES.txt
View File

@ -77,6 +77,9 @@ Trunk (Unreleased)
MAPREDUCE-5196. Add bookkeeping for managing checkpoints of task state. MAPREDUCE-5196. Add bookkeeping for managing checkpoints of task state.
(Carlo Curino via cdouglas) (Carlo Curino via cdouglas)
MAPREDUCE-5912. Task.calculateOutputSize does not handle Windows files after
MAPREDUCE-5196. (Remus Rusanu via cnauroth)
BUG FIXES BUG FIXES
MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob. MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob.

4
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java
View File

@ -1120,8 +1120,8 @@ private long calculateOutputSize() throws IOException {
if (isMapTask() && conf.getNumReduceTasks() > 0) { if (isMapTask() && conf.getNumReduceTasks() > 0) {
try { try {
Path mapOutput = mapOutputFile.getOutputFile(); Path mapOutput = mapOutputFile.getOutputFile();
FileSystem fs = mapOutput.getFileSystem(conf); FileSystem localFS = FileSystem.getLocal(conf);
return fs.getFileStatus(mapOutput).getLen(); return localFS.getFileStatus(mapOutput).getLen();
} catch (IOException e) { } catch (IOException e) {
LOG.warn ("Could not find output size " , e); LOG.warn ("Could not find output size " , e);
} }