big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MAPREDUCE-6366. mapreduce.terasort.final.sync configuration in TeraSort doesn't work. Contributed by Takuya Fukudome.

Browse Source
This commit is contained in:
Tsuyoshi Ozawa 2015-05-13 16:44:37 +09:00
parent fcd0702c10
commit e82067bfe6
3 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-mapreduce-project/CHANGES.txt
View File

@ -419,6 +419,9 @@ Release 2.8.0 - UNRELEASED
copySucceeded() in one thread and copyFailed() in another thread on the copySucceeded() in one thread and copyFailed() in another thread on the
same host. (Junping Du via ozawa) same host. (Junping Du via ozawa)
MAPREDUCE-6366. mapreduce.terasort.final.sync configuration in TeraSort
doesn't work. (Takuya Fukudome via ozawa)
Release 2.7.1 - UNRELEASED Release 2.7.1 - UNRELEASED
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

1
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
View File

@ -328,7 +328,6 @@ public int run(String[] args) throws Exception {
} }
job.getConfiguration().setInt("dfs.replication", getOutputReplication(job)); job.getConfiguration().setInt("dfs.replication", getOutputReplication(job));
TeraOutputFormat.setFinalSync(job, true);
int ret = job.waitForCompletion(true) ? 0 : 1; int ret = job.waitForCompletion(true) ? 0 : 1;
LOG.info("done"); LOG.info("done");
return ret; return ret;

2
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
View File

@ -70,7 +70,7 @@ public String toString() {
public static final long DEFAULT_NUM_ROWS = 0L; public static final long DEFAULT_NUM_ROWS = 0L;
public static final int DEFAULT_NUM_PARTITIONS = 10; public static final int DEFAULT_NUM_PARTITIONS = 10;
public static final long DEFAULT_SAMPLE_SIZE = 100000L; public static final long DEFAULT_SAMPLE_SIZE = 100000L;
public static final boolean DEFAULT_FINAL_SYNC_ATTRIBUTE = false; public static final boolean DEFAULT_FINAL_SYNC_ATTRIBUTE = true;
public static final boolean DEFAULT_USE_TERA_SCHEDULER = true; public static final boolean DEFAULT_USE_TERA_SCHEDULER = true;
public static final boolean DEFAULT_USE_SIMPLE_PARTITIONER = false; public static final boolean DEFAULT_USE_SIMPLE_PARTITIONER = false;
public static final int DEFAULT_OUTPUT_REPLICATION = 1; public static final int DEFAULT_OUTPUT_REPLICATION = 1;