big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,174 Commits 2 Branches 2 Tags 264 MiB
0a452a30ab
Commit Graph

71 Commits

Author SHA1 Message Date
Christopher Douglas
0a452a30ab MAPREDUCE-5176. Add annotation for tagging tasks as responsive to 
preemption. Contributed by Carlo Curino, cdouglas


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488103 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 06:07:18 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Vinod Kumar Vavilapalli
0edae7a96a MAPREDUCE-5237. Add methods that were removed from ClusterStatus back into 2.x so as to be compatible with 1.x. Contributed by Zhijie Shen. 
Incompatible as ClusterStatus.UNINITIALIZED_MEMORY_VALUE is a long now and so breaks 0.23.x but it shouldn't be a big deal in reality.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487175 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 00:43:13 +00:00
Vinod Kumar Vavilapalli
6377bfc505 MAPREDUCE-5235. Bring back old fields and exceptions in Counters for binary compatibility with mapred in 1.x. Contributed by Mayank Bansal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484992 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-21 22:00:08 +00:00
Hitesh Shah
6517c3f714 MAPREDUCE-5191. TestQueue#testQueue fails with timeout on Windows. (Contributed by Ivan Mitic) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484575 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-20 20:02:07 +00:00
Vinod Kumar Vavilapalli
0b668c21b2 MAPREDUCE-5234. Change mapred.TaskReport and mapreduce.TaskReport for binary compatibility with mapred in 1.x but incompatible with 0.23.x. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483940 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 18:16:33 +00:00
Arun Murthy
794f9bb3e4 MAPREDUCE-5066. Added a timeout for the job.end.notification.url. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470216 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 19:18:24 +00:00
Alejandro Abdelnur
d12c463121 MAPREDUCE-4898. FileOutputFormat.checkOutputSpecs and FileOutputFormat.setOutputPath incompatible with MR1. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469523 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 18:51:45 +00:00
Siddharth Seth
d41e67b966 MAPREDUCE-5077. Remove mapreduce.util.ResourceCalculatorPlugin and related code. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461251 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 18:15:40 +00:00
Robert Joseph Evans
a0a844222d MAPREDUCE-4875. coverage fixing for org.apache.hadoop.mapred (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461146 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 14:16:13 +00:00
Jonathan Turner Eagles
523d3daac0 MAPREDUCE-5053. java.lang.InternalError from decompression codec cause reducer to fail (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458350 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-19 15:45:35 +00:00
Jason Darrell Lowe
57803245ec MAPREDUCE-5060. Fetch failures that time out only count against the first map task. Contributed by Robert Joseph Evans 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1455740 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-12 22:51:28 +00:00
Arun Murthy
df68c56267 MAPREDUCE-3685. Fix bugs in MergeManager to ensure compression codec is appropriately used and that on-disk segments are correctly sorted on file-size. Contributed by Anty Rao and Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453365 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 15:02:45 +00:00
Alejandro Abdelnur
0f430e53fd MAPREDUCE-2264. Job status exceeds 100% in some cases. (devaraj.k and sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440076 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-29 19:38:40 +00:00
Alejandro Abdelnur
da4cab1099 Revering MAPREDUCE-2264 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439561 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-28 18:58:34 +00:00
Alejandro Abdelnur
539153a679 MAPREDUCE-2264. Job status exceeds 100% in some cases. (devaraj.k and sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1438277 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-25 00:25:59 +00:00
Alejandro Abdelnur
73fd247c76 MAPREDUCE-4808. Refactor MapOutput and MergeManager to facilitate reuse by Shuffle implementations. (masokan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1436936 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 14:10:42 +00:00
Alejandro Abdelnur
d7c48409e3 MAPREDUCE-4907. Ammendment, forgot to svn add testcase in original commit 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431510 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-10 17:27:58 +00:00
Jason Darrell Lowe
92774331cc MAPREDUCE-4842. Shuffle race can hang reducer. Contributed by Mariappan Asokan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425071 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 18:24:25 +00:00
Doug Cutting
aa30056d09 MAPREDUCE-4887. Add RehashPartitioner, to smooth distributions with poor implementations of Object#hashCode(). Contributed by Radim Kolar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424158 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-19 21:50:17 +00:00
Arun Murthy
b096f61fe2 MAPREDUCE-4049. Experimental api to allow for alternate shuffle plugins. Contributed by Anver BenHanoch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1418173 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-07 02:36:33 +00:00
Robert Joseph Evans
6ee6eb8430 MAPREDUCE-4772. Fetch failures can take way too long for a map to be restarted (bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407118 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-08 15:17:34 +00:00
Vinod Kumar Vavilapalli
895029b2f2 MAPREDUCE-4730. Fix Reducer's EventFetcher to scale the map-completion requests slowly to avoid HADOOP-8942. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401941 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-25 01:26:30 +00:00
Arun Murthy
036530fc12 Reverted MAPREDUCE-4253. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390652 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 18:41:08 +00:00
Robert Joseph Evans
6f6e170325 HADOOP-8726. The Secrets in Credentials are not available to MR tasks (daryn and Benoy Antony via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379100 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 19:58:07 +00:00
Robert Joseph Evans
aeb364d8b5 HADOOP-8225. DistCp fails when invoked by Oozie (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376618 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-23 18:06:56 +00:00
Daryn Sharp
8fa10b184e HADOOP-7967. Need generalized multi-token filesystem support (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374271 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 14:05:11 +00:00
Thomas White
3204b1f450 MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name. Contributed by Jarek Jarcec Cecho. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373823 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-16 13:26:12 +00:00
Thomas Graves
dfad16fed2 MAPREDUCE-4053. Counters group names deprecation is wrong, iterating over group names deprecated names don't show up (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372636 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-13 21:48:33 +00:00
Thomas Graves
d87b545165 MAPREDUCE-4423. Potential infinite fetching of map output (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366258 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-27 01:48:09 +00:00
Thomas Graves
d45922de2c svn merge --change -1363454 for reverting MAPREDUCE-4423 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363935 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-20 20:18:48 +00:00
Thomas Graves
8e69f883a0 MAPREDUCE-4423. Potential infinite fetching of map output (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363454 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-19 18:19:59 +00:00
Robert Joseph Evans
26910fd058 MAPREDUCE-4416. Some tests fail if Clover is enabled (Kihwal Lee via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360735 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-12 15:52:12 +00:00
Robert Joseph Evans
c1bf8a3f03 MAPREDUCE-4392. Counters.makeCompactString() changed behavior from 0.20 (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1357776 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-05 18:09:09 +00:00
Robert Joseph Evans
eff5abb4f7 MAPREDUCE-4371. Check for cyclic dependencies in Jobcontrol job DAG (madhukara phatak via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356414 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-02 19:02:51 +00:00
Harsh J
e967e98f74 MAPREDUCE-4253. Tests for mapreduce-client-core are lying under mapreduce-client-jobclient. Contributed by Tsuyoshi Ozawa. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355766 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-30 16:49:24 +00:00
Robert Joseph Evans
d1f05ce29f Reverting MAPREDUCE-4371 to address test failures. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355382 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-29 13:41:46 +00:00
Robert Joseph Evans
78c27ca2e3 MAPREDUCE-4371. Check for cyclic dependencies in Jobcontrol job DAG (madhukara phatak via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355139 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-28 19:59:19 +00:00
Thomas Graves
27d1c74a0c MAPREDUCE-3927. Shuffle hang when set map.failures.percent (Bhallamudi Venkata Siva Kamesh via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348846 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-11 13:54:46 +00:00
Robert Joseph Evans
67a514b647 MAPREDUCE-3850. Avoid redundant calls for tokens in TokenCache (Daryn Sharp via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335585 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-08 15:25:08 +00:00
Thomas White
55cd161a35 MAPREDUCE-4226. ConcurrentModificationException in FileSystemCounterGroup. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335197 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-07 19:05:59 +00:00
Robert Joseph Evans
48ab08f1c6 MAPREDUCE-4117. mapred job -status throws NullPointerException (Devaraj K via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311479 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-09 21:52:50 +00:00
Siddharth Seth
3cbb9d20d0 MAPREDUCE-3903. Add support for mapreduce admin users. (Contributed by Thomas Graves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295262 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-29 20:48:09 +00:00
Thomas Graves
547ded7bfe MAPREDUCE-3920. Revise yarn default port number selection (Dave Thompson via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295162 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-29 15:49:27 +00:00
Robert Joseph Evans
cfc4ad76a3 MAPREDUCE-3849. Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1245099 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-16 17:49:20 +00:00
Arun Murthy
35f12b9556 MAPREDUCE-3697. Support binary compatibility for Counters after MAPREDUCE-901. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241319 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 01:41:28 +00:00
Mahadev Konar
59d1f07d6c MAPREDUCE-3794. Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility (Tom White via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241299 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 00:53:36 +00:00
Vinod Kumar Vavilapalli
9f2b77aee4 MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241217 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 22:16:26 +00:00
Mahadev Konar
f2280f612c MAPREDUCE-3795. "job -status" command line output is malformed. (vinodkv via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240593 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 19:32:10 +00:00
Vinod Kumar Vavilapalli
94242c9385 MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240413 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 00:04:53 +00:00