big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin.

Browse Source
This commit is contained in:
Akira Ajisaka 2016-11-01 20:51:35 +09:00
parent 9d0c2ad5a5
commit 235f39b7c4
1 changed files with 8 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
View File

@ -1347,9 +1347,14 @@ public void setReduceSpeculativeExecution(boolean speculativeExecution) {
* <b id="NoOfReduces">How many reduces?</b>
*
* <p>The right number of reduces seems to be <code>0.95</code> or
* <code>1.75</code> multiplied by (&lt;<i>no. of nodes</i>&gt; *
* <a href="{@docRoot}/../mapred-default.html#mapreduce.tasktracker.reduce.tasks.maximum">
* mapreduce.tasktracker.reduce.tasks.maximum</a>).
* <code>1.75</code> multiplied by (
* <i>available memory for reduce tasks</i>
* (The value of this should be smaller than
* numNodes * yarn.nodemanager.resource.memory-mb
* since the resource of memory is shared by map tasks and other
* applications) /
* <a href="{@docRoot}/../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml#mapreduce.reduce.memory.mb">
* mapreduce.reduce.memory.mb</a>).
* </p>
*
* <p>With <code>0.95</code> all of the reduces can launch immediately and