big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,383 Commits 2 Branches 2 Tags 264 MiB
d28be56239
Commit Graph

422 Commits

Author SHA1 Message Date
Robert Joseph Evans
1b6f0582fb YARN-237. Refreshing the RM page forgets how many rows I had in my Datatables (jian he via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456536 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-14 16:56:10 +00:00
Thomas Graves
7122cb5bf4 MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1454156 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-07 23:02:28 +00:00
Suresh Srinivas
638801cce1 HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 19:15:18 +00:00
Robert Joseph Evans
ec13f1eb3a MAPREDUCE-5043. Fetch failure processing can cause AM event queue to backup and eventually OOM (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452372 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-04 16:32:18 +00:00
Jonathan Turner Eagles
38c4e17ec1 MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451826 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-02 03:30:58 +00:00
Siddharth Seth
979fb054f8 MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450956 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-27 21:03:19 +00:00
Thomas White
0b73dde6ce MAPREDUCE-4951. Container preemption interpreted as task failure. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448615 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-21 11:56:08 +00:00
Jonathan Turner Eagles
c8f35bc3d2 MAPREDUCE-5009. Killing the Task Attempt slated for commit does not clear the value from the Task commitAttempt member (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1447965 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-19 23:35:56 +00:00
Siddharth Seth
43204f0c83 MAPREDUCE-5000. Fixes getCounters when speculating by fixing the selection of the best attempt for a task. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445871 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 19:20:31 +00:00
Jason Darrell Lowe
390d1fbef8 MAPREDUCE-4992. AM hangs in RecoveryService when recovering tasks with speculative attempts. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445456 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 02:11:36 +00:00
Jason Darrell Lowe
c2d6407070 MAPREDUCE-4989. JSONify DataTables input data for Attempts page. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445448 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 01:15:48 +00:00
Siddharth Seth
35832053bf MAPREDUCE-4671. AM does not tell the RM about container requests which are no longer needed. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443323 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-07 07:07:11 +00:00
Hitesh Shah
b1baf11fb2 YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441668 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-02 00:43:54 +00:00
Vinod Kumar Vavilapalli
28c308d5e8 MAPREDUCE-4893. Fixed MR ApplicationMaster to do optimal assignment of containers to get maximum locality. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440749 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-31 00:28:10 +00:00
Siddharth Seth
c163dc2fce MAPREDUCE-4838. Add additional fields like Locality, Avataar to the JobHistory logs. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439714 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-29 00:21:58 +00:00
Siddharth Seth
74ffc7a74d MAPREDUCE-4946. Fix a performance problem for large jobs by reducing the number of map completion event type conversions. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437103 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 19:07:08 +00:00
Siddharth Seth
6a2f2551fd YARN-135. Client tokens should be per app-attempt, and should be unregistered on App-finish. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433570 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 18:33:38 +00:00
Jason Darrell Lowe
dd6aa0cf12 MAPREDUCE-4936. JobImpl uber checks for cpu are wrong. Contributed by Arun C Murthy 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433445 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 14:45:08 +00:00
Thomas Graves
e7aeb9d129 MAPREDUCE-4934. Maven RAT plugin is not checking all source files (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432932 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-14 14:35:45 +00:00
Jason Darrell Lowe
b16dfc125d MAPREDUCE-4848. TaskAttemptContext cast error during AM recovery. Contributed by Jerry Chen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431131 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 22:56:09 +00:00
Thomas White
0ba7078ef4 MAPREDUCE-1700. User supplied dependencies may conflict with MapReduce system JARs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430929 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 16:12:58 +00:00
Arun Murthy
2c5c8fdb80 MAPREDUCE-4520. Added support for MapReduce applications to request for CPU cores along-with memory post YARN-2. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430688 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 05:28:40 +00:00
Arun Murthy
4539263971 YARN-2. Enhanced CapacityScheduler to account for CPU alongwith memory for multi-dimensional resource scheduling. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430682 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 05:08:23 +00:00
Thomas Graves
bbc426f53a MAPREDUCE-4913. TestMRAppMaster#testMRAppMasterMissingStaging occasionally exits (Jason Lowe via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429371 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-05 20:07:24 +00:00
Robert Joseph Evans
64e4fb983e MAPREDUCE-4819. AM can rerun job after reporting final job status to the client (bobby and Bikas Saha via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429114 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:35:56 +00:00
Thomas Graves
e17cecf550 MAPREDUCE-4894. Renewal / cancellation of JobHistory tokens (Siddharth Seth via tgraves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429086 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:15:43 +00:00
Jason Darrell Lowe
78ab699fe9 MAPREDUCE-4832. MR AM can get in a split brain situation. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429040 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 19:15:21 +00:00
Robert Joseph Evans
402eb18513 MAPREDUCE-4813. AM timing out during job commit (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1426536 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-28 15:01:46 +00:00
Jason Darrell Lowe
04db3ce177 MAPREDUCE-4890. Invalid TaskImpl state transitions when task fails while speculating. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425223 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-22 01:41:26 +00:00
Jason Darrell Lowe
92692c863c MAPREDUCE-4833. Task can get stuck in FAIL_CONTAINER_CLEANUP. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425167 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 22:27:54 +00:00
Jonathan Turner Eagles
fdcfef0a26 MAPREDUCE-4836. Elapsed time for running tasks on AM web UI tasks page is 0. (Ravi Prakash via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1415860 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-30 21:43:43 +00:00
Thomas Graves
5caef48947 MAPREDUCE-4817. Hardcoded task ping timeout kills tasks localizing large amounts of data (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414873 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-28 19:19:16 +00:00
Robert Joseph Evans
b43deb9af8 MAPREDUCE-4825. JobImpl.finished doesn't expect ERROR as a final job state (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414840 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-28 17:52:45 +00:00
Siddharth Seth
c271f3cded MAPREDUCE-4723 amendment. Fix JobHistory Event handling. (Contributed by Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411292 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-19 16:43:29 +00:00
Thomas White
e464607b7d YARN-129. Simplify classpath construction for mini YARN tests. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411235 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-19 15:12:22 +00:00
Robert Joseph Evans
d2b9a604bd MAPREDUCE-4802. Takes a long time to load the task list on the AM for large jobs (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410498 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-16 17:30:22 +00:00
Eli Collins
905b17876c MAPREDUCE-4723. Fix warnings found by findbugs 2. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409601 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-15 00:16:35 +00:00
Robert Joseph Evans
316c23dd7e MAPREDUCE-4797. LocalContainerAllocator can loop forever trying to contact the RM (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409525 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-14 23:02:29 +00:00
Jason Darrell Lowe
397c261433 YARN-216. Remove jquery theming support. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409313 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-14 19:22:11 +00:00
Thomas Graves
b2f7a66e50 MAPREDUCE-4517. Too many INFO messages written out during AM to RM heartbeat (Jason Lowe via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409032 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-13 23:47:57 +00:00
Robert Joseph Evans
cf6ab9b2e7 MAPREDUCE-4786. Job End Notification retry interval is 5 milliseconds by default (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408444 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-12 20:32:04 +00:00
Robert Joseph Evans
c493d06b1b MAPREDUCE-4425. Speculation + Fetch failures can lead to a hung job (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408360 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-12 17:15:45 +00:00
Robert Joseph Evans
dd72ca3536 MAPREDUCE-4751. AM stuck in KILL_WAIT for days (vinodkv via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408314 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-12 15:15:52 +00:00
Robert Joseph Evans
7c9778e2c3 MAPREDUCE-4774. JobImpl does not handle asynchronous task events in FAILED state (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407679 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 22:55:57 +00:00
Robert Joseph Evans
6ee6eb8430 MAPREDUCE-4772. Fetch failures can take way too long for a map to be restarted (bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407118 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-08 15:17:34 +00:00
Jason Darrell Lowe
42d1eaf237 MAPREDUCE-4729. job history UI not showing all job attempts. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404817 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-01 22:59:43 +00:00
Thomas Graves
aac5c149c7 MAPREDUCE-4752. Reduce MR AM memory usage through String Interning (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404177 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-31 14:57:12 +00:00
Jason Darrell Lowe
286c171987 MAPREDUCE-4748. Invalid event: T_ATTEMPT_SUCCEEDED at SUCCEEDED. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1402658 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-26 21:02:52 +00:00
Jason Darrell Lowe
1e45b1f1fd MAPREDUCE-4741. WARN and ERROR messages logged during normal AM shutdown. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401738 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-24 15:45:35 +00:00
Jason Darrell Lowe
614a743fea YARN-139. Interrupted Exception within AsyncDispatcher leads to user confusion. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401726 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-24 15:20:58 +00:00
Siddharth Seth
382b565e8c MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401474 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-23 21:02:22 +00:00
Vinod Kumar Vavilapalli
1195f844a9 MAPREDUCE-4733. Reducer can fail to make progress during shuffle if too many reducers complete consecutively. Contributed by Jason Lowe via. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400264 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 20:18:46 +00:00
Vinod Kumar Vavilapalli
022f7b4a25 MAPREDUCE-4596. Split StateMachine state from states seen by MRClientProtocol for Job, Task and TaskAttempt. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399976 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 05:57:57 +00:00
Siddharth Seth
c074cfd6f0 YARN-134. Fixes ClientToAMSecretManager creates keys without checking for validity of the appID. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395841 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 01:56:05 +00:00
Robert Joseph Evans
49b20c2ed1 MAPREDUCE-4554. Job Credentials are not transmitted if security is turned off (Benoy Antony via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395769 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-08 20:50:40 +00:00
Thomas Graves
050fd3a117 MAPREDUCE-4647. We should only unjar jobjar if there is a lib directory in it. (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390557 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 15:22:21 +00:00
Vinod Kumar Vavilapalli
229a79bbc3 MAPREDUCE-4646. Fixed MR framework to send diagnostic information correctly to clients in case of failed jobs also. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383709 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-12 00:59:15 +00:00
Thomas White
3b46295c28 MAPREDUCE-4607. Race condition in ReduceTask completion can result in Task being incorrectly failed. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383422 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-11 14:04:03 +00:00
Robert Joseph Evans
09038b741d MAPREDUCE-4641. Exception in commitJob marks job as successful in job history (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381784 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 22:20:15 +00:00
Thomas Graves
25e96e455b MAPREDUCE-4611. MR AM dies badly when Node is decommissioned (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379599 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:43:46 +00:00
Robert Joseph Evans
63f941d2ad MAPREDUCE-4614. Simplify debugging a job's tokens (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379595 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:37:41 +00:00
Robert Joseph Evans
015256524c MAPREDUCE-4612. job summary file permissions not set when its created (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379584 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:25:27 +00:00
Robert Joseph Evans
6f6e170325 HADOOP-8726. The Secrets in Credentials are not available to MR tasks (daryn and Benoy Antony via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379100 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 19:58:07 +00:00
Vinod Kumar Vavilapalli
cfe25b8828 MAPREDUCE-4579. Split TestTaskAttempt into two so as to pass tests on jdk7. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377943 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-28 02:03:26 +00:00
Siddharth Seth
ffd2e01604 YARN-39. RM-NM secret-keys should be randomly generated and rolled every so often. (Contributed by Vinod Kumar Vavilapalli and Siddharth Seth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377180 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-25 02:18:49 +00:00
Robert Joseph Evans
aeb364d8b5 HADOOP-8225. DistCp fails when invoked by Oozie (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376618 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-23 18:06:56 +00:00
Alejandro Abdelnur
dc33a0765c MAPREDUCE-4068. Jars in lib subdirectory of the submittable JAR are not added to the classpath (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376253 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-22 21:18:34 +00:00
Alejandro Abdelnur
4bca22005f HADOOP-8681. add support for HTTPS to the web UIs. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371525 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-09 22:52:56 +00:00
Thomas Graves
9d42fb2e8e MAPREDUCE-4457. mr job invalid transition TA_TOO_MANY_FETCH_FAILURE at FAILED (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367771 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 20:52:41 +00:00
Robert Joseph Evans
a37272e0e3 MAPREDUCE-4496. AM logs link is missing user name (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367601 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 15:08:47 +00:00
Thomas Graves
a3e695ab2c MAPREDUCE-4375. Show Configuration Tracability in MR UI (bobby via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367539 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 13:39:41 +00:00
Alejandro Abdelnur
9d16c9354b MAPREDUCE-4417. add support for encrypted shuffle (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365979 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-26 13:23:05 +00:00
Robert Joseph Evans
e1c5e7dd2b MAPREDUCE-4437. Race in MR ApplicationMaster can cause reducers to never be scheduled (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362209 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-16 19:11:08 +00:00
Vinod Kumar Vavilapalli
3bfb26ad3b MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359910 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-10 21:26:48 +00:00
Robert Joseph Evans
6804ef32fc MAPREDUCE-4252. MR2 job never completes with 1 pending task (Tom White via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359747 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-10 16:10:14 +00:00
Thomas Graves
11782dd3a5 MAPREDUCE-4300. OOM in AM can turn it into a zombie. (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359399 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-09 21:08:40 +00:00
Robert Joseph Evans
aad2a6f0fc MAPREDUCE-4376. TestClusterMRNotification times out (Kihwal Lee via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355124 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-28 19:28:42 +00:00
Robert Joseph Evans
03f2f9b580 MAPREDUCE-4228. mapreduce.job.reduce.slowstart.completedmaps is not working properly (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354181 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-26 19:17:33 +00:00
Siddharth Seth
eff9fa1aad MAPREDUCE-3921. MR AM should act on node health status changes. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349065 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-11 23:14:06 +00:00
Thomas Graves
27d1c74a0c MAPREDUCE-3927. Shuffle hang when set map.failures.percent (Bhallamudi Venkata Siva Kamesh via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348846 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-11 13:54:46 +00:00
Thomas Graves
9368be0e73 MAPREDUCE-3842. Stop webpages from automatic refreshing (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348123 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-08 15:47:50 +00:00
Robert Joseph Evans
0a80f82a30 MAPREDUCE-4152. map task left hanging after AM dies trying to connect to RM (Tom Graves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344283 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-30 14:48:53 +00:00
Harsh J
1fab229f30 HADOOP-8268. A few pom.xml across Hadoop project may fail XML validation. Contributed by Radim Kolar. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1343272 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-28 14:51:05 +00:00
Thomas Graves
7e3d016845 MAPREDUCE-4102. job counters not available in Jobhistory webui for killed jobs (Bhallamudi Venkata Siva Kamesh via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339174 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-16 14:08:08 +00:00
Robert Joseph Evans
aa60da6c2e MAPREDUCE-4162. Correctly set token service (Daryn Sharp via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335567 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-08 15:07:40 +00:00
Robert Joseph Evans
48f95779c1 MAPREDUCE-4048. NullPointerException exception while accessing the Application Master UI (Devaraj K via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1334013 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-04 15:02:54 +00:00
Alejandro Abdelnur
0fca4fbaf3 MAPREDUCE-4205. retrofit all JVM shutdown hooks to use ShutdownHookManager (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333748 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-04 03:20:16 +00:00
Robert Joseph Evans
48414b0827 MAPREDUCE-4163. consistently set the bind address (Daryn Sharp via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333579 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-03 18:35:21 +00:00
Thomas Graves
e7583d816a MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330560 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-25 21:13:06 +00:00
Thomas Graves
f340d6c894 MAPREDUCE-4079. Allow MR AppMaster to limit ephemeral port range.(bobby via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329694 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-24 13:03:15 +00:00
Robert Joseph Evans
80ab78c217 MAPREDUCE-4133. MR over viewfs is broken (John George via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329403 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-23 19:44:30 +00:00
Robert Joseph Evans
796e1a48ac MAPREDUCE-4190. Improve web UI for task attempts userlog link (Tom Graves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329398 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-23 19:38:28 +00:00
Robert Joseph Evans
1eb1d0ae4f MAPREDUCE-3812. Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329282 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-23 15:26:35 +00:00
Robert Joseph Evans
e3806060ce MAPREDUCE-4159. Job is running in Uber mode after setting "mapreduce.job.ubertask.maxreduces" to zero (Devaraj K via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328031 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-19 16:22:22 +00:00
Robert Joseph Evans
32d511065a MAPREDUCE-4161. create sockets consistently (Daryn Sharp via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327621 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-18 18:46:34 +00:00
Siddharth Seth
7d04a96027 MAPREDUCE-3972. Fix locking and exception issues in JobHistory server. (Contributed by Robert Joseph Evans) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327354 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-18 01:59:16 +00:00
Robert Joseph Evans
27ea3ab6ba MAPREDUCE-4128. AM Recovery expects all attempts of a completed task to also be completed. (Bikas Saha via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325765 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-13 13:45:03 +00:00
Siddharth Seth
ecb30a48c4 MAPREDUCE-4050. For tasks without assigned containers, changes the node text on the UI to N/A instead of a link to null. (Contributed by Bhallamudi Venkata Siva Kamesh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325435 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 18:39:49 +00:00
Siddharth Seth
7809acd67b MAPREDUCE-3932. Fix the TaskAttempt state machine to handle CONTIANER_LAUNCHED and CONTIANER_LAUNCH_FAILED events in additional states. (Contributed by Robert Joseph Evans) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1324902 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-11 18:18:40 +00:00
Siddharth Seth
cfafd8c29d MAPREDUCE-4099 amendment. ApplicationMaster will remove staging directory after the history service is stopped. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1324866 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-11 17:09:54 +00:00
Robert Joseph Evans
2accda38a1 MAPREDUCE-4017. Add jobname to jobsummary log (tgraves and Koji Noguchi via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311972 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-10 20:36:09 +00:00
Robert Joseph Evans
793746870b MAPREDUCE-4099. ApplicationMaster may fail to remove staging directory (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311926 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-10 18:57:34 +00:00
Alejandro Abdelnur
1d6038ecfb MAPREDUCE-4097. tools testcases fail because missing mrapp-generated-classpath file in classpath (rvs via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309583 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-04 20:24:01 +00:00
Thomas Graves
4e44259cbd MAPREDUCE-3682 Tracker URL says AM tasks run on localhost. (Ravi Prakash via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309440 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-04 15:08:39 +00:00
Thomas Graves
d99c2958dd MAPREDUCE-3672. Killed maps shouldn't be counted towards JobCounter.NUM_FAILED_MAPS. (Anupam Seth via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309401 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-04 14:17:22 +00:00
Robert Joseph Evans
b45c450026 MAPREDUCE-3988. mapreduce.job.local.dir doesn't point to a single directory on a node. (Eric Payne via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309086 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-03 18:29:40 +00:00
Robert Joseph Evans
2b5c36b695 MAPREDUCE-4062. AM Launcher thread can hang forever (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309037 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-03 16:59:26 +00:00
Thomas Graves
bb74427da2 MAPREDUCE-4089. Hung Tasks never time out. (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308531 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-02 20:20:41 +00:00
Robert Joseph Evans
3a8d123ccb MAPREDUCE-4092. commitJob Exception does not fail job (Jon Eagles via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308507 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-02 19:51:06 +00:00
Alejandro Abdelnur
c4c7ee5b8c MAPREDUCE-4091. tools testcases failing because of MAPREDUCE-4082 (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307623 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-30 21:30:11 +00:00
Alejandro Abdelnur
57e175fddf hadoop-mapreduce-client-app's mrapp-generated-classpath file should not be in the module JAR (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307124 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-29 21:43:50 +00:00
Siddharth Seth
db3e592df0 REDUCE-3955. Change MR to use ProtobufRpcEngine from hadoop-common instead of ProtoOverHadoopRpcEngine. (Contributed by Jitendra Nath Pandey) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1306689 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-29 02:02:17 +00:00
Arun Murthy
67a74198b1 Changed version in trunk to 3.0.0-SNAPSHOT. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1306200 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-28 07:37:06 +00:00
Robert Joseph Evans
1650a49993 MAPREDUCE-4061. RM only has 1 AM launcher thread (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305607 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-26 21:23:31 +00:00
Robert Joseph Evans
f67c2d1bd0 MAPREDUCE-4043. Secret keys set in Credentials are not seen by tasks (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1304587 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-23 20:46:18 +00:00
Robert Joseph Evans
9d8d02b68b MAPREDUCE-4025. AM can crash if task attempt reports bogus progress value (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1302645 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-19 20:31:55 +00:00
Alejandro Abdelnur
7609243e58 MAPREDUCE-4007. JobClient getJob(JobID) should return NULL if the job does not exist (for backwards compatibility) (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1300750 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-14 22:02:53 +00:00
Robert Joseph Evans
13ea1d575c MAPREDUCE-3975. Default value not set for Configuration parameter mapreduce.job.local.dir (Eric Payne via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1297825 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-07 01:27:02 +00:00
Robert Joseph Evans
c3a4de0ec0 MAPREDUCE-3961. Map/ReduceSlotMillis computation incorrect (Siddharth Seth via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1297788 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-06 23:21:13 +00:00
Robert Joseph Evans
a98b756c73 MAPREDUCE-3976. TestRMContainerAllocator failing (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1297557 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-06 16:17:41 +00:00
Vinod Kumar Vavilapalli
5f52156aa2 MAPREDUCE-3614. Fixed MR AM to close history file quickly and send a correct final state to the RM when it is killed. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1296747 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-04 05:23:16 +00:00
Thomas Graves
547ded7bfe MAPREDUCE-3920. Revise yarn default port number selection (Dave Thompson via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295162 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-29 15:49:27 +00:00
Vinod Kumar Vavilapalli
7a082ec2bd MAPREDUCE-2793. Corrected AppIDs, JobIDs, TaskAttemptIDs to be of correct format on the web pages. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293517 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-25 02:03:59 +00:00
Thomas Graves
582b97c3e7 MAPREDUCE-3904. Job history produced with mapreduce.cluster.acls.enabled false can not be viewed with mapreduce.cluster.acls.enabled true (Jonathon Eagles via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293456 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-24 22:30:47 +00:00
Thomas Graves
81f5c5933d MAPREDUCE-3878. Null user on filtered jobhistory job page (Jonathon Eagles via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1292831 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-23 15:50:53 +00:00
Vinod Kumar Vavilapalli
b2172c394e MAPREDUCE-3798. Fixed failing TestJobCleanup.testCusomCleanup() and moved it to the maven build. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1291606 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-21 05:30:23 +00:00
Vinod Kumar Vavilapalli
5ee495e6f3 MAPREDUCE-3634. Fixed all daemons to crash instead of hanging around when their EventHandlers get exceptions. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1291598 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-21 05:08:25 +00:00
Mahadev Konar
83ab8f087b MAPREDUCE-3858. Task attempt failure during commit results in task never completing. (Tom White via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1244254 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-14 22:49:00 +00:00
Vinod Kumar Vavilapalli
1903e544a4 MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1244178 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-14 19:24:00 +00:00
Vinod Kumar Vavilapalli
0515b3322f MAPREDUCE-3846. Addressed MR AM hanging issues during AM restart and then the recovery. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1243752 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-14 00:06:37 +00:00
Robert Joseph Evans
b2d49acd08 MAPREDUCE-3840 JobEndNotifier doesn't use the proxyToUse during connecting (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241999 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-08 17:31:13 +00:00
Vinod Kumar Vavilapalli
7ae94c6678 MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241734 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-08 01:56:02 +00:00
Siddharth Seth
ccc2807d86 MAPREDUCE-3823. Ensure counters are calculated only once after a job finishes. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241692 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 23:07:03 +00:00
Vinod Kumar Vavilapalli
90bc439b66 MAPREDUCE-3815. Fixed MR AM to always use hostnames and never IPs when requesting containers so that scheduler can give off data local containers correctly. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241654 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 21:58:54 +00:00
Arun Murthy
28a2eb9d72 MAPREDUCE-3810. Performance tweaks - reduced logging in AM and defined hascode/equals for ResourceRequest & Priority. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241205 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 22:01:22 +00:00
Mahadev Konar
caed1cfaf9 MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240590 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 18:55:03 +00:00
Vinod Kumar Vavilapalli
fe1cf3b0ac MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240428 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 01:21:12 +00:00
Vinod Kumar Vavilapalli
94242c9385 MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240413 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 00:04:53 +00:00
Siddharth Seth
b4929bcf14 MAPREDUCE-3640. Allow AMRecovery to work with partial JobHistory files. (Contributed by Arun C Murthy) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1239402 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-01 23:33:58 +00:00
Mahadev Konar
443d8f6c0b MAPREDUCE-3679. AM logs and others should not automatically refresh after every 1 second. (Vinod KV via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238146 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-31 04:53:49 +00:00
Vinod Kumar Vavilapalli
76b653a367 MAPREDUCE-3716. Fixing YARN+MR to allow MR jobs to be able to use java.io.File.createTempFile to create temporary files as part of their tasks. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238136 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-31 02:23:38 +00:00
Alejandro Abdelnur
f73daf6af1 MAPREDUCE-3505. yarn APPLICATION_CLASSPATH needs to be overridable. (ahmed via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235391 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 18:21:27 +00:00
Mahadev Konar
8a1719e999 MAPREDUCE-3549. write api documentation for web service apis for RM, NM, mapreduce app master, and job history server (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1234222 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-21 00:53:02 +00:00
Mahadev Konar
c6923061d0 MAPREDUCE-3698. Client cannot talk to the history server in secure mode. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1234120 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-20 20:44:17 +00:00
Eli Collins
6861560098 MAPREDUCE-3692. yarn-resourcemanager out and log files can get big. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1233605 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-19 22:23:53 +00:00
Mahadev Konar
a24339e50d MAPREDUCE-3649. Job End notification gives an error on calling back. (Ravi Prakash via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1232126 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-16 19:35:06 +00:00
Vinod Kumar Vavilapalli
b62c1b8563 MAPREDUCE-3404. Corrected MR AM to honor speculative configuration and enable speculating either maps or reduces. Contributed by Eric Payne. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1231395 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-13 23:41:23 +00:00
Vinod Kumar Vavilapalli
0c278b0f63 MAPREDUCE-3656. Fixed a race condition in MR AM which is failing the sort benchmark consistently. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1231314 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-13 21:31:40 +00:00
Vinod Kumar Vavilapalli
74697f2317 MAPREDUCE-3512. Batching JobHistory flushing to DFS so that we don't flush for every event slowing down AM. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1230353 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-12 01:42:10 +00:00
Mahadev Konar
5f79f180f6 MAPREDUCE-3553. Add support for data returned when exceptions thrown from web service apis to be in either xml or in JSON. (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1230330 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 23:59:23 +00:00
Vinod Kumar Vavilapalli
39b8bbe663 MAPREDUCE-3618. Fixed TaskHeartbeatHandler to not hold a global lock for all task-updates. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229906 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 06:53:10 +00:00
Mahadev Konar
bc4b1f48d3 MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229855 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 01:50:10 +00:00
Vinod Kumar Vavilapalli
bc374626be MAPREDUCE-3299. Added AMInfo table to the MR AM job pages to list all the job-attempts when AM restarts and recovers. (Jonathan Eagles via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229766 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 22:26:45 +00:00
Vinod Kumar Vavilapalli
44e0bb831b MAPREDUCE-3382. Enhanced MR AM to use a proxy to ping the job-end notification URL. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229736 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 21:08:34 +00:00
Vinod Kumar Vavilapalli
849c68c7b5 MAPREDUCE-3312. Modified MR AM to not send a stop-container request for a container that isn't launched at all. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229451 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 02:15:48 +00:00
Vinod Kumar Vavilapalli
428529b58f MAPREDUCE-3528. Fixed TaskHeartBeatHandler to use a new configuration for the thread loop interval separate from task-timeout configuration property. (Siddharth Seth via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229403 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-09 22:37:00 +00:00
Siddharth Seth
239a5549ea MAPREDUCE-3616. Thread pool for launching containers in MR AM not expanding as expected. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229394 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-09 22:20:23 +00:00
Vinod Kumar Vavilapalli
e8645636ce MAPREDUCE-3511. Removed a multitude of cloned/duplicate counters in the AM thereby reducing the AM heap size and preventing full GCs. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229347 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-09 21:04:58 +00:00
Arun Murthy
7440a8aa37 MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227801 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 20:01:20 +00:00
Alejandro Abdelnur
6d45b9e31d HADOOP-7934. Normalize dependencies versions across all modules. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227787 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 19:46:42 +00:00
Siddharth Seth
03d46dc571 MAPREDUCE-3569. TaskAttemptListener holds a global lock for all task-updates. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227485 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 05:19:44 +00:00
Arun Murthy
08f8abf563 MAPREDUCE-3572. Moved AM event dispatcher to a separate thread for performance reasons. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227426 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 01:37:13 +00:00
Arun Murthy
0870734787 MAPREDUCE-3566. Fixed MR AM to construct CLC only once across all tasks. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227422 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 01:29:52 +00:00
Vinod Kumar Vavilapalli
55e94dc5ef MAPREDUCE-3490. Fixed MapReduce AM to count failed maps also towards Reduce ramp up. Contributed by Sharad Agarwal and Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227226 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-04 17:10:31 +00:00
Harsh J
2f19c59f1d MAPREDUCE-3610. Remove use of the 'dfs.block.size' config for default block size fetching. Use FS#getDefaultBlocksize instead. (Sho Shimauchi via harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227091 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-04 09:39:14 +00:00
Vinod Kumar Vavilapalli
fcbad14a3d MAPREDUCE-3568. Optimized Job's progress calculations in MR AM. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1224995 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-27 19:54:51 +00:00
Siddharth Seth
8fa0a3c737 MAPREDUCE-3567. Extraneous JobConf objects in AM heap. Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1222498 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-22 22:34:40 +00:00
Amar Kamat
264d3b7dd0 MAPREDUCE-3349. Log rack-name in JobHistory for unsuccessful tasks. (Devaraj K and Amar Kamat via amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221578 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 02:58:00 +00:00
Vinod Kumar Vavilapalli
e7543b944c MAPREDUCE-3339. Fixed MR AM to stop considering node blacklisting after the number of nodes blacklisted crosses a threshold. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221523 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-20 23:27:02 +00:00
Vinod Kumar Vavilapalli
f73bd5402e MAPREDUCE-3564. Fixed failures in TestStagingCleanup and TestJobEndNotifier tests. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1215022 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-16 02:09:00 +00:00
Vinod Kumar Vavilapalli
c2fe4a7e83 MAPREDUCE-3487. Fixed JobHistory web-UI to display links to single task's counters' page. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1215016 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-16 01:51:15 +00:00
Vinod Kumar Vavilapalli
cfbde6ac0f MAPREDUCE-3484. Fixed JobEndNotifier to not get interrupted before completing all its retries. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214563 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 23:58:35 +00:00
Suresh Srinivas
ec4971a280 MAPREDUCE-3545. Remove Avro RPC. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214066 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 06:57:00 +00:00
Vinod Kumar Vavilapalli
b7ae5a6cb7 MAPREDUCE-3426. Fixed MR AM in uber mode to write map intermediate outputs in the correct directory to work properly in secure mode. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213987 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:35:11 +00:00
Vinod Kumar Vavilapalli
0ea8570be5 MAPREDUCE-2863. Support web services for YARN and MR components. (Thomas Graves via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213975 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:05:56 +00:00
Siddharth Seth
a3f37e15f7 MAPREDUCE-3460. MR AM can hang if containers are allocated on a node blacklisted by the AM. (Contributed by Hitesh Shah and Robert Joseph Evans) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1209737 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-02 22:18:38 +00:00
Mahadev Konar
08da8ea5db MAPREDUCE-3463. Second AM fails to recover properly when first AM is killed with java.lang.IllegalArgumentException causing lost job. (Siddharth Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1208994 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 08:35:20 +00:00
Mahadev Konar
94bf0dacba MAPREDUCE-3450. NM port info no longer available in JobHistory. (Siddharth Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1208327 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-30 08:43:00 +00:00
Mahadev Konar
cc4a468113 MAPREDUCE-3413. RM web ui applications not sorted in any order by default. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1206926 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-27 23:56:46 +00:00
Vinod Kumar Vavilapalli
00b50a5c94 MAPREDUCE-3355. Fixed MR AM's ContainerLauncher to handle node-command timeouts correctly. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1202744 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-16 15:37:27 +00:00
Vinod Kumar Vavilapalli
15150cd997 MAPREDUCE-3280. Removed the unnecessary job user-name configuration in mapred-site.xml. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200441 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 17:28:16 +00:00
Vinod Kumar Vavilapalli
2df83e5b4d MAPREDUCE-3333. Fixed bugs in ContainerLauncher of MR AppMaster due to which per-container connections to NodeManager were lingering long enough to hit the ulimits on number of processes. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1199751 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-09 13:41:19 +00:00
Vinod Kumar Vavilapalli
9fe9f42c8f MAPREDUCE-3342. Fixed JobHistoryServer to also show the job's queue name. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1199133 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-08 07:28:56 +00:00
Arun Murthy
0df4878033 MAPREDUCE-3337. Added missing license headers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196977 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 07:06:53 +00:00
Thomas White
8e04fa8b84 HADOOP-7782. Aggregate project javadocs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195817 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 04:47:10 +00:00
Mahadev Konar
68fa208b1c MAPREDUCE-3103. Implement Job ACLs for MRAppMaster. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195761 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 01:51:56 +00:00
Arun Murthy
9db078212f MAPREDUCE-3035. Fixed MR JobHistory to ensure rack information is present. Contributed by chakravarthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195575 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 17:27:42 +00:00
Vinod Kumar Vavilapalli
47a381e306 MAPREDUCE-3146. Added a MR specific command line to dump logs for a given TaskAttemptID. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195349 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 06:42:06 +00:00
Vinod Kumar Vavilapalli
1c8d64f38a MAPREDUCE-3274. Fixed a race condition in MRAppMaster that was causing a task-scheduling deadlock. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195145 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-30 11:29:41 +00:00
Arun Murthy
7f4dc27757 MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1194850 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 09:35:36 +00:00
Arun Murthy
b93abdcd66 MAPREDUCE-3304. Fixed intermittent test failure due to a race in TestRMContainerAllocator#testBlackListedNodes. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190495 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 18:45:48 +00:00
Vinod Kumar Vavilapalli
a39992ea63 MAPREDUCE-3296. Fixed the remaining nine FindBugs warnings. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190187 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 07:44:35 +00:00
Vinod Kumar Vavilapalli
670fa24b48 MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the JobHistoryServer. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:45:04 +00:00
Arun Murthy
34e6de8f85 MAPREDUCE-3258. Fixed AM & JobHistory web-ui to display counters properly. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190160 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 05:08:04 +00:00
Mahadev Konar
b304062f1f MAPREDUCE-3186. User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed. (Eric Payne via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190122 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 01:40:24 +00:00
Arun Murthy
724f217343 MAPREDUCE-3228. Fixed MR AM to timeout RPCs to bad NodeManagers. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189879 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 17:32:02 +00:00
Vinod Kumar Vavilapalli
2fe343f963 MAPREDUCE-3240. Fixed NodeManager to be able to forcefully cleanup its containers (process-trees) irrespective of whether the container succeeded, or killed. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189711 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 12:03:17 +00:00
Arun Murthy
db8ac0ec3c MAPREDUCE-3257. Added authorization checks for the protocol between ResourceManager and ApplicatoinMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189630 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 06:24:22 +00:00
Arun Murthy
8aabd3d4e6 MAPREDUCE-2858. Added a WebApp Proxy for applications. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189036 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 06:29:21 +00:00
Arun Murthy
29c6c3ed32 MAPREDUCE-2977. Fix ResourceManager to renew HDFS delegation tokens for applications. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189012 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 02:53:22 +00:00