big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MAPREDUCE-5529. Fix compat with hadoop-1 in mapred.TotalOrderPartitioner by re-introducing (get,set)PartitionFile which takes in JobConf. Contributed by Robert Kanter.

Browse Source 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527048 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Arun Murthy 2013-09-27 19:42:34 +00:00
parent 711abc3827
commit a40810d267
2 changed files with 31 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
hadoop-mapreduce-project/CHANGES.txt
View File

@ -241,6 +241,10 @@ Release 2.1.2 - UNRELEASED
MAPREDUCE-5545. org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow MAPREDUCE-5545. org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow
times out (Robert Kanter via jlowe) times out (Robert Kanter via jlowe)
MAPREDUCE-5529. Fix compat with hadoop-1 in mapred.TotalOrderPartitioner
by re-introducing (get,set)PartitionFile which takes in JobConf. (Robert
Kanter via acmurthy)
Release 2.1.1-beta - 2013-09-23 Release 2.1.1-beta - 2013-09-23
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

28
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/TotalOrderPartitioner.java
View File

@ -21,7 +21,7 @@
import org.apache.hadoop.classification.InterfaceAudience; import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability; import org.apache.hadoop.classification.InterfaceStability;
import org.apache.hadoop.io.WritableComparable; import org.apache.hadoop.fs.Path;
import org.apache.hadoop.mapred.JobConf; import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.Partitioner; import org.apache.hadoop.mapred.Partitioner;
@ -41,4 +41,30 @@ public void configure(JobConf job) {
super.setConf(job); super.setConf(job);
} }
/**
* Set the path to the SequenceFile storing the sorted partition keyset.
* It must be the case that for <tt>R</tt> reduces, there are <tt>R-1</tt>
* keys in the SequenceFile.
* @deprecated Use
* {@link #setPartitionFile(Configuration, Path)}
* instead
*/
@Deprecated
public static void setPartitionFile(JobConf job, Path p) {
org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.
setPartitionFile(job, p);
}
/**
* Get the path to the SequenceFile storing the sorted partition keyset.
* @see #setPartitionFile(JobConf,Path)
* @deprecated Use
* {@link #getPartitionFile(Configuration)}
* instead
*/
@Deprecated
public static String getPartitionFile(JobConf job) {
return org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.
getPartitionFile(job);
}
} }