big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,505 Commits 2 Branches 2 Tags 264 MiB
02e6b72ae1
Commit Graph

601 Commits

Author SHA1 Message Date
Siddharth Seth
35832053bf MAPREDUCE-4671. AM does not tell the RM about container requests which are no longer needed. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443323 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-07 07:07:11 +00:00
Jason Darrell Lowe
ea94b7b9dd MAPREDUCE-4822. Unnecessary conversions in History Events. Contributed by Chu Tong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443257 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 22:47:10 +00:00
Alejandro Abdelnur
17e72be6d8 MAPREDUCE-4977. Documentation for pluggable shuffle and pluggable sort. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443168 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 19:52:25 +00:00
Robert Joseph Evans
79b12f6f4c MAPREDUCE-4905. test org.apache.hadoop.mapred.pipes (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443027 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 15:50:23 +00:00
Siddharth Seth
a04bee4567 MAPREDUCE-4971. Minor extensibility enhancements to Counters & FileOutputFormat. Contributed by Arun C Murthy 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441670 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-02 00:47:58 +00:00
Hitesh Shah
b1baf11fb2 YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441668 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-02 00:43:54 +00:00
Suresh Srinivas
ac21e15a39 MAPREDUCE-4969. TestKeyValueTextInputFormat test fails with Open JDK 7. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441069 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-31 17:11:36 +00:00
Vinod Kumar Vavilapalli
28c308d5e8 MAPREDUCE-4893. Fixed MR ApplicationMaster to do optimal assignment of containers to get maximum locality. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440749 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-31 00:28:10 +00:00
Alejandro Abdelnur
0f430e53fd MAPREDUCE-2264. Job status exceeds 100% in some cases. (devaraj.k and sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440076 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-29 19:38:40 +00:00
Siddharth Seth
c163dc2fce MAPREDUCE-4838. Add additional fields like Locality, Avataar to the JobHistory logs. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439714 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-29 00:21:58 +00:00
Siddharth Seth
8a5fadc5a0 MAPREDUCE-4803. Remove duplicate copy of TestIndexCache. Contributed by Mariappan Asokan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439590 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-28 19:18:35 +00:00
Alejandro Abdelnur
da4cab1099 Revering MAPREDUCE-2264 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439561 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-28 18:58:34 +00:00
Alejandro Abdelnur
539153a679 MAPREDUCE-2264. Job status exceeds 100% in some cases. (devaraj.k and sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1438277 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-25 00:25:59 +00:00
Siddharth Seth
74ffc7a74d MAPREDUCE-4946. Fix a performance problem for large jobs by reducing the number of map completion event type conversions. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437103 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 19:07:08 +00:00
Alejandro Abdelnur
73fd247c76 MAPREDUCE-4808. Refactor MapOutput and MergeManager to facilitate reuse by Shuffle implementations. (masokan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1436936 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 14:10:42 +00:00
Jonathan Turner Eagles
88c7fdd065 MAPREDUCE-4458. Warn if java.library.path is used for AM or Task (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435386 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 22:41:19 +00:00
Siddharth Seth
76f9af4ccf MAPREDUCE-4948. Fix a failing unit test - TestYARNRunner.testHistoryServerToken. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435312 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 19:30:00 +00:00
Alejandro Abdelnur
1c308188bb MAPREDUCE-4923. Add toString method to TaggedInputSplit. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434993 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 00:40:58 +00:00
Alejandro Abdelnur
26275e5323 MAPREDUCE-4924. flakey test: org.apache.hadoop.mapred.TestClusterMRNotification.testMR. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433753 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-16 00:08:25 +00:00
Siddharth Seth
6a2f2551fd YARN-135. Client tokens should be per app-attempt, and should be unregistered on App-finish. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433570 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 18:33:38 +00:00
Jason Darrell Lowe
dd6aa0cf12 MAPREDUCE-4936. JobImpl uber checks for cpu are wrong. Contributed by Arun C Murthy 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433445 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 14:45:08 +00:00
Suresh Srinivas
86825d8390 MAPREDUCE-4938. Use token request messages defined in hadoop common. Contributed by Suresh Srinvias. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433227 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 00:11:16 +00:00
Thomas Graves
e7aeb9d129 MAPREDUCE-4934. Maven RAT plugin is not checking all source files (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432932 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-14 14:35:45 +00:00
Robert Joseph Evans
12293f8a13 MAPREDUCE-4921. JobClient should acquire HS token with RM principal (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432230 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-11 19:00:51 +00:00
Alejandro Abdelnur
d7c48409e3 MAPREDUCE-4907. Ammendment, forgot to svn add testcase in original commit 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431510 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-10 17:27:58 +00:00
Alejandro Abdelnur
59d9d8bca9 MAPREDUCE-4907. TrackerDistributedCacheManager issues too many getFileStatus calls. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431166 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-10 00:50:03 +00:00
Jason Darrell Lowe
b16dfc125d MAPREDUCE-4848. TaskAttemptContext cast error during AM recovery. Contributed by Jerry Chen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431131 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 22:56:09 +00:00
Thomas White
0ba7078ef4 MAPREDUCE-1700. User supplied dependencies may conflict with MapReduce system JARs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430929 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 16:12:58 +00:00
Vinod Kumar Vavilapalli
5f7d4d2b45 MAPREDUCE-4810. Added new admin command options for MR AM. Contributed by Jerry Chen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430707 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 06:31:36 +00:00
Arun Murthy
2c5c8fdb80 MAPREDUCE-4520. Added support for MapReduce applications to request for CPU cores along-with memory post YARN-2. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430688 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 05:28:40 +00:00
Arun Murthy
4539263971 YARN-2. Enhanced CapacityScheduler to account for CPU alongwith memory for multi-dimensional resource scheduling. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430682 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 05:08:23 +00:00
Thomas White
0f1f5491bc MAPREDUCE-4278. Cannot run two local jobs in parallel from the same gateway. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430363 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-08 16:26:33 +00:00
Vinod Kumar Vavilapalli
b8a7a17191 MAPREDUCE-4920. Use security token protobuf definition from hadoop common. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429751 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-07 11:13:26 +00:00
Thomas Graves
bbc426f53a MAPREDUCE-4913. TestMRAppMaster#testMRAppMasterMissingStaging occasionally exits (Jason Lowe via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429371 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-05 20:07:24 +00:00
Robert Joseph Evans
64e4fb983e MAPREDUCE-4819. AM can rerun job after reporting final job status to the client (bobby and Bikas Saha via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429114 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:35:56 +00:00
Thomas Graves
e17cecf550 MAPREDUCE-4894. Renewal / cancellation of JobHistory tokens (Siddharth Seth via tgraves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429086 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:15:43 +00:00
Jason Darrell Lowe
78ab699fe9 MAPREDUCE-4832. MR AM can get in a split brain situation. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429040 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 19:15:21 +00:00
Robert Joseph Evans
e229e25cc3 MAPREDUCE-4279. getClusterStatus() fails with null pointer exception when running jobs in local mode (Devaraj K via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428482 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 17:09:18 +00:00
Robert Joseph Evans
402eb18513 MAPREDUCE-4813. AM timing out during job commit (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1426536 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-28 15:01:46 +00:00
Jason Darrell Lowe
a7d444d002 MAPREDUCE-4902. Fix typo "receievd" should be "received" in log output. Contributed by Albert Chu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1426018 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-26 20:47:39 +00:00
Vinod Kumar Vavilapalli
dae6f0b7d9 MAPREDUCE-4899. Implemented a MR specific plugin for tracking finished applications that YARN's ResourceManager doesn't keep track off anymore. Contributed by Derek Dagit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425225 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-22 01:47:51 +00:00
Jason Darrell Lowe
04db3ce177 MAPREDUCE-4890. Invalid TaskImpl state transitions when task fails while speculating. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425223 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-22 01:41:26 +00:00
Luke Lu
02e940d6a4 MAPREDUCE-4272. SortedRanges.Range#compareTo was not spec compliant. (Yu Gao via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425180 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 23:10:52 +00:00
Jason Darrell Lowe
92692c863c MAPREDUCE-4833. Task can get stuck in FAIL_CONTAINER_CLEANUP. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425167 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 22:27:54 +00:00
Jason Darrell Lowe
92774331cc MAPREDUCE-4842. Shuffle race can hang reducer. Contributed by Mariappan Asokan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425071 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 18:24:25 +00:00
Luke Lu
1a49c85438 YARN-223. Update process tree instead of getting new process trees. (Radim Kolar via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424244 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-20 00:20:53 +00:00
Doug Cutting
aa30056d09 MAPREDUCE-4887. Add RehashPartitioner, to smooth distributions with poor implementations of Object#hashCode(). Contributed by Radim Kolar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424158 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-19 21:50:17 +00:00
Thomas White
9189917be5 MAPREDUCE-4845. ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1423470 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-18 15:23:49 +00:00
Alejandro Abdelnur
8329fae686 MAPREDUCE-4807. Allow MapOutputBuffer to be pluggable. (masokan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1422345 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-15 20:23:08 +00:00
Alejandro Abdelnur
803e5155d1 MAPREDUCE-4809. Change visibility of classes for pluggable sort changes. (masokan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1422341 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-15 20:18:23 +00:00