big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MAPREDUCE-6357. MultipleOutputs.write() API should document that output committing is not utilized when input path is absolute. Contributed by Dustin Cote.

Browse Source
This commit is contained in:
Akira Ajisaka 2015-08-21 10:41:54 +09:00
parent 33474939a0
commit 2ba90c93d7
2 changed files with 17 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
hadoop-mapreduce-project/CHANGES.txt
View File

@ -555,6 +555,10 @@ Release 2.8.0 - UNRELEASED
MAPREDUCE-5817. Mappers get rescheduled on node transition even after all MAPREDUCE-5817. Mappers get rescheduled on node transition even after all
reducers are completed. (Sangjin Lee via kasha) reducers are completed. (Sangjin Lee via kasha)
MAPREDUCE-6357. MultipleOutputs.write() API should document that output
committing is not utilized when input path is absolute.
(Dustin Cote via aajisaka)
Release 2.7.2 - UNRELEASED Release 2.7.2 - UNRELEASED
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

14
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.java
View File

@ -120,7 +120,11 @@
* *
* <p> * <p>
* Use <code>MultipleOutputs.write(KEYOUT key, VALUEOUT value, String baseOutputPath)</code> to write key and * Use <code>MultipleOutputs.write(KEYOUT key, VALUEOUT value, String baseOutputPath)</code> to write key and
* value to a path specified by <code>baseOutputPath</code>, with no need to specify a named output: * value to a path specified by <code>baseOutputPath</code>, with no need to specify a named output.
* <b>Warning</b>: when the baseOutputPath passed to MultipleOutputs.write
* is a path that resolves outside of the final job output directory, the
* directory is created immediately and then persists through subsequent
* task retries, breaking the concept of output committing:
* </p> * </p>
* *
* <pre> * <pre>
@ -418,6 +422,10 @@ public <K, V> void write(String namedOutput, K key, V value)
* @param value the value * @param value the value
* @param baseOutputPath base-output path to write the record to. * @param baseOutputPath base-output path to write the record to.
* Note: Framework will generate unique filename for the baseOutputPath * Note: Framework will generate unique filename for the baseOutputPath
* <b>Warning</b>: when the baseOutputPath is a path that resolves
* outside of the final job output directory, the directory is created
* immediately and then persists through subsequent task retries, breaking
* the concept of output committing.
*/ */
@SuppressWarnings("unchecked") @SuppressWarnings("unchecked")
public <K, V> void write(String namedOutput, K key, V value, public <K, V> void write(String namedOutput, K key, V value,
@ -442,6 +450,10 @@ public <K, V> void write(String namedOutput, K key, V value,
* @param value the value * @param value the value
* @param baseOutputPath base-output path to write the record to. * @param baseOutputPath base-output path to write the record to.
* Note: Framework will generate unique filename for the baseOutputPath * Note: Framework will generate unique filename for the baseOutputPath
* <b>Warning</b>: when the baseOutputPath is a path that resolves
* outside of the final job output directory, the directory is created
* immediately and then persists through subsequent task retries, breaking
* the concept of output committing.
*/ */
@SuppressWarnings("unchecked") @SuppressWarnings("unchecked")
public void write(KEYOUT key, VALUEOUT value, String baseOutputPath) public void write(KEYOUT key, VALUEOUT value, String baseOutputPath)