big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,172 Commits 2 Branches 2 Tags 264 MiB
a815cc157c
Commit Graph

1381 Commits

Author SHA1 Message Date
Christopher Douglas
9ca394d54d MAPREDUCE-5189. Add policies and wiring to respond to preemption requests 
from YARN. Contributed by Carlo Curino.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551748 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 22:54:31 +00:00
Christopher Douglas
9218b4aec1 MAPREDUCE-5197. Add a service for checkpointing task state. 
Contributed by Carlo Curino


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551726 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 21:37:25 +00:00
Jason Darrell Lowe
44429c4bb3 MAPREDUCE-5679. TestJobHistoryParsing has race condition. Contributed by Liyin Liang 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551616 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 16:45:23 +00:00
Jason Darrell Lowe
fc77ed153e MAPREDUCE-5623. TestJobCleanup fails because of RejectedExecutionException and NPE. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551285 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 17:44:49 +00:00
Chris Nauroth
ee0530ee0e MAPREDUCE-5674. Missing start and finish time in mapred.JobStatus. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550472 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 18:30:12 +00:00
Jonathan Turner Eagles
ee5351bf22 MAPREDUCE-5052. Job History UI and web services confusing job start time and job submit time (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1549896 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-10 17:17:47 +00:00
Jason Darrell Lowe
18d99c12c3 MAPREDUCE-5656. bzip2 codec can drop records when reading data in splits. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1549705 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-09 23:31:30 +00:00
Jason Darrell Lowe
0f2ce45aeb MAPREDUCE-5409. MRAppMaster throws InvalidStateTransitonException: Invalid event: TA_TOO_MANY_FETCH_FAILURE at KILLED for TaskAttemptImpl. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548197 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 16:30:12 +00:00
Jonathan Turner Eagles
9ba98e6af1 MAPREDUCE-5632. TestRMContainerAllocator#testUpdatedNodes fails (jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547929 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-04 22:11:28 +00:00
Jonathan Turner Eagles
5afc1242ea MAPREDUCE-5645. TestFixedLengthInputFormat fails with native libs (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547624 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-03 22:44:07 +00:00
Arun Murthy
eec0ec2d3f Add ACL check to JHS web-services. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547159 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 19:32:17 +00:00
Jonathan Turner Eagles
74851537de MAPREDUCE-5640. Rename TestLineRecordReader in jobclient module (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547149 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 19:04:51 +00:00
Jing Zhao
2214871d91 HDFS-5545. Allow specifying endpoints for listeners in HttpServer. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546151 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 18:20:14 +00:00
Jason Darrell Lowe
0a5b0499f3 MAPREDUCE-5631. TestJobEndNotifier.testNotifyRetries fails with Should have taken more than 5 seconds in jdk7. Contributed by Jonathan Eagles 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544266 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 17:53:39 +00:00
Colin McCabe
efea68dc35 HDFS-5511. improve CacheManipulator interface to allow better unit testing (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543676 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 00:48:05 +00:00
Jonathan Turner Eagles
fdad179980 MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542733 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-17 14:29:33 +00:00
Jonathan Turner Eagles
a02e60fefc MAPREDUCE-5373. TestFetchFailure.testFetchFailureMultipleReduces could fail intermittently (jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542456 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-16 03:29:51 +00:00
Chris Nauroth
88c4d4a714 MAPREDUCE-5616. MR Client-AppMaster RPC max retries on socket timeout is too high. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542001 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 18:09:14 +00:00
Sanford Ryza
3d95049f79 MAPREDUCE-5481. Enable uber jobs to have multiple reducers (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541844 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 07:56:11 +00:00
Steve Loughran
01ccd4d3ab MAPREDUCE-5624 Move grizzly-test and junit dependencies to test scope 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541702 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 20:46:54 +00:00
Steve Loughran
4a0f804d5e MAPREDUCE-5431 Missing pom dependency in MR-client 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541682 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 19:52:43 +00:00
Sanford Ryza
e94ed91114 MAPREDUCE-5613. DefaultSpeculator holds and checks hashmap that is alway empty (Gera Shegalov via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541433 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 09:23:18 +00:00
Sanford Ryza
eeda370249 MAPREDUCE-1176. FixedLengthInputFormat and FixedLengthRecordReader (Mariappan Asokan and BitsOfInfo via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540931 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-12 03:06:10 +00:00
Jason Darrell Lowe
38a3b925e9 MAPREDUCE-5186. mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail. Contributed by Robert Parker and Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540813 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-11 19:22:38 +00:00
Chris Nauroth
bee6822441 MAPREDUCE-5451. MR uses LD_LIBRARY_PATH which doesn't mean anything in Windows. Contributed by Yingda Chen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539413 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-06 18:07:39 +00:00
Colin McCabe
7abf7e8151 HADOOP-9478. Fix race conditions during the initialization of Configuration related to deprecatedKeyMap (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538248 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-02 18:42:27 +00:00
Chris Nauroth
1606a76a54 MAPREDUCE-5604. TestMRAMWithNonNormalizedCapabilities fails on Windows due to exceeding max path length. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538035 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-01 19:08:43 +00:00
Sanford Ryza
86dc50c1eb MAPREDUCE-5601. ShuffleHandler fadvises file regions as DONTNEED even when fetch fails (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537855 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-01 08:51:54 +00:00
Bikas Saha
7dd201c541 Fix inadvertent file changes made via r1536888 (bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536894 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 21:11:32 +00:00
Bikas Saha
03510d00f4 YARN-1068. Add admin support for HA operations (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536888 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 21:03:54 +00:00
Jason Darrell Lowe
c9643795c1 MAPREDUCE-5598. TestUserDefinedCounters.testMapReduceJob is flakey. Contributed by Robert Kanter 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536724 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 14:06:39 +00:00
Jason Darrell Lowe
e0c99b80d4 MAPREDUCE-5596. Allow configuring the number of threads used to serve shuffle connections. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536711 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 13:41:50 +00:00
Sanford Ryza
84cec3c805 MAPREDUCE-4680. Job history cleaner should only check timestamps of files in old enough directories (Robert Kanter via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536558 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-28 23:48:38 +00:00
Jason Darrell Lowe
298bddf00b MAPREDUCE-5561. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534832 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 22:42:43 +00:00
Jonathan Turner Eagles
2baa42dd01 MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533629 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-18 20:57:48 +00:00
Suresh Srinivas
dea1b2e84a HADOOP-10005. No need to check INFO severity level is enabled or not. Contributed by Jackie Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532907 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 21:00:07 +00:00
Sanford Ryza
6a1929fe31 MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531822 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-14 09:11:23 +00:00
Siddharth Seth
f2745babf3 MAPREDUCE-5329. Allow MR applications to use additional AuxServices, which are compatible with the default MapReduce shuffle. Contributed by Avner BenHanoch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531741 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-13 20:17:53 +00:00
Vinod Kumar Vavilapalli
9b9ddf29e2 YARN-1283. Fixed RM to give a fully-qualified proxy URL for an application so that clients don't need to do scheme-mangling. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530819 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 22:49:06 +00:00
Jason Darrell Lowe
00395a0064 MAPREDUCE-5102. fix coverage org.apache.hadoop.mapreduce.lib.db and org.apache.hadoop.mapred.lib.db. Contributed by Aleksey Gorshkov, Andrey Klochkov, and Nathan Roberts 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530816 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 22:39:39 +00:00
Jason Darrell Lowe
7b7caa1e56 MAPREDUCE-5569. FloatSplitter is not generating correct splits. Contributed by Nathan Roberts 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530683 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 15:43:36 +00:00
Daryn Sharp
b858b812b7 HADOOP-9470. eliminate duplicate FQN tests in different Hadoop modules (Ivan A. Veselovsky via daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530667 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 15:09:37 +00:00
Vinod Kumar Vavilapalli
f0799c5536 MAPREDUCE-5562. Fixed MR App Master to perform pending tasks like staging-dir cleanup, sending job-end notification correctly when unregister with RM fails. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529682 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 20:53:28 +00:00
Vinod Kumar Vavilapalli
21181b6553 YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related 
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529662 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 18:43:36 +00:00
Vinod Kumar Vavilapalli
d4324eef14 MAPREDUCE-5533. Fixed MR speculation code to track any TaskAttempts that aren't heart-beating for a while, so that we can aggressively speculate instead of waiting for task-timeout. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529229 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 17:26:56 +00:00
Chris Nauroth
65cd7bf6b1 MAPREDUCE-5442. $HADOOP_MAPRED_HOME/$HADOOP_CONF_DIR setting not working on Windows. Contributed by Yingda Chen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529077 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 05:37:54 +00:00
Hitesh Shah
6ff600d9e3 YARN-1131. logs command should return an appropriate error message if YARN application is still running. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529068 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 02:36:17 +00:00
Bikas Saha
fb3f338c65 MAPREDUCE-5489. MR jobs hangs as it does not use the node-blacklisting feature in RM requests (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529005 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-03 21:14:35 +00:00
Jonathan Turner Eagles
7b687dda09 YARN-1199. Make NM/RM Versions Available (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529003 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-03 21:04:12 +00:00
Sanford Ryza
90a5dc34f7 MAPREDUCE-5554. hdfs-site.xml included in hadoop-mapreduce-client-jobclient tests jar is breaking tests for downstream components (Robert Kanter via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528643 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 21:53:57 +00:00
Arun Murthy
3c7813bbad MAPREDUCE-5459. Update documentation on how to run MRv1 examples on YARN. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528626 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 21:08:27 +00:00
Arun Murthy
07ee205038 MAPREDUCE-5530. Fix compat with hadoop-1 in mapred.lib.CombinFileInputFormat by re-introducing isSplittable(FileSystem, Path) api and ensuring semantic compatibility. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528533 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 16:08:45 +00:00
Vinod Kumar Vavilapalli
82f4348f27 MAPREDUCE-5536. Fixed MR AM and JHS to respect mapreduce.jobhistory.webapp.https.address. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528251 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 22:52:25 +00:00
Jason Darrell Lowe
db06f1bcb9 MAPREDUCE-4421. Run MapReduce framework via the distributed cache. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528237 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 22:34:31 +00:00
Sanford Ryza
cc8a0cab59 MAPREDUCE-5544. JobClient#getJob loads job conf twice. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528196 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 20:01:20 +00:00
Arun Murthy
9c24615ae6 MAPREDUCE-5551. Fix compat with hadoop-1 in SequenceFileAsBinaryOutputFormat.WritableValueBytes by re-introducing missing constructors. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527848 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 01:24:27 +00:00
Vinod Kumar Vavilapalli
aa12a7914f MAPREDUCE-5538. Fixed MR AppMaster to send job-notification URL only after the job is really done - a bug caused by MAPREDUCE-5505. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527219 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-28 19:14:16 +00:00
Arun Murthy
a40810d267 MAPREDUCE-5529. Fix compat with hadoop-1 in mapred.TotalOrderPartitioner by re-introducing (get,set)PartitionFile which takes in JobConf. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527048 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-27 19:42:34 +00:00
Jason Darrell Lowe
711abc3827 MAPREDUCE-5545. org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow times out. Contributed by Robert Kanter 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527037 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-27 19:14:07 +00:00
Arun Murthy
58f40d3bf4 MAPREDUCE-5531. Fix compat with hadoop-1 in mapreduce.(TaskID, TaskAttemptID) by re-introducing missing constructors. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527033 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-27 19:08:28 +00:00
Jason Darrell Lowe
2627e352d6 MAPREDUCE-5332. Support token-preserving restart of history server. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527015 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-27 18:19:41 +00:00
Arun Murthy
96c11fe602 MAPREDUCE-5514. Fix TestRMContainerAllocator. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526735 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 23:02:51 +00:00
Jason Darrell Lowe
cf3d8b2286 MAPREDUCE-5513. ConcurrentModificationException in JobControl. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526688 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 21:10:05 +00:00
Chris Nauroth
7e73b9cde6 MAPREDUCE-5525. Increase timeout of TestDFSIO.testAppend and TestMRJobsWithHistoryService.testJobHistoryData. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526624 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 17:45:14 +00:00
Colin McCabe
8a66e493ba HADOOP-9998. Provide methods to clear only part of the DNSToSwitchMapping. (Junping Du via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526567 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 15:24:30 +00:00
Vinod Kumar Vavilapalli
476eca47b5 MAPREDUCE-5170. Fixed a wrong log message in CombineFileInputFormat class. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526377 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 04:30:08 +00:00
Vinod Kumar Vavilapalli
fb48b6cdc9 MAPREDUCE-5503. Fixed a test issue in TestMRJobClient. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526362 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 03:00:28 +00:00
Bikas Saha
524dad1109 MAPREDUCE-5505. Clients should be notified job finished only after job successfully unregistered (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526071 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-25 00:46:48 +00:00
Siddharth Seth
10a4289ebf YARN-1229. Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526065 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-25 00:35:03 +00:00
Vinod Kumar Vavilapalli
78e995a37d YARN-1204. Added separate configuration properties for https for RM and NM without which servers enabled with https will also start on http ports. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5523. Added separate configuration properties for https for JHS without which even when https is enabled, it starts on http port itself. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525947 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-24 17:20:18 +00:00
Robert Joseph Evans
743d8ae055 MAPREDUCE-5522. Incorrect oreder expected from JobQueueInfo (Jinghui Wang via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525670 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-23 18:33:12 +00:00
Vinod Kumar Vavilapalli
13420d01f2 YARN-1203. Changed YARN web-app proxy to handle http and https URLs from AM registration and finish correctly. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config mapreduce.ssl.enabled to enable https and disabling it by default as MR AM needs
to set up its own certificates etc and not depend on clusters'. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524864 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-19 23:35:58 +00:00
Vinod Kumar Vavilapalli
af78fd729c MAPREDUCE-5488. Changed MR client to keep trying to reach the application when it sees that on attempt's AM is down. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524856 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-19 22:35:12 +00:00
Thomas Graves
6b1f5073a7 MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524841 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-19 20:58:18 +00:00
Sanford Ryza
c09f00771e MAPREDUCE-5487. In task processes, JobConf is unnecessarily loaded again in Limits (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524408 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-18 13:44:33 +00:00
Arun Murthy
59587d9fad MAPREDUCE-5493. Cleanup in-memory & on-disk segments to prevent leak on shuffle completion. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523660 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 14:18:42 +00:00
Arun Murthy
60a1a70197 YARN-1170. YARN & MapReduce proto definitions fixed to specify protobuf package as hadoop.yarn and hadoop.mapreduce respectively. Contibuted by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523652 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 14:04:04 +00:00
Alejandro Abdelnur
e60b057c0a MAPREDUCE-5379. Include token tracking ids in jobconf. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523605 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 12:17:17 +00:00
Bikas Saha
ec010a2936 YARN-540. Race condition causing RM to potentially relaunch already unregistered AMs on RM restart (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523376 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-15 01:01:06 +00:00
Bikas Saha
f4951e0708 YARN-1116. Populate AMRMTokens back to AMRMTokenSecretManager after RM restarts (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523146 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-13 23:51:50 +00:00
Vinod Kumar Vavilapalli
1c1ebc1553 MAPREDUCE-5497. Changed MRAppMaster to sleep only after doing everything else but just before ClientService to avoid race conditions during RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521699 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-11 00:38:49 +00:00
Thomas Graves
75c31df9dd MAPREDUCE-5020. Compile failure with JDK8 (Trevor Robinson via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521576 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-10 18:31:50 +00:00
Devarajulu K
b2af6c7024 MAPREDUCE-5414. TestTaskAttempt fails in JDK7 with NPE. Contributed by Nemon Lou. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520964 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-09 06:42:42 +00:00
Jason Darrell Lowe
5540d77e2f MAPREDUCE-5475. MRClientService does not verify ACLs properly. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520156 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-04 22:23:40 +00:00
Sanford Ryza
236b8530bd HADOOP-9918. Add addIfService to CompositeService (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519129 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-31 02:09:45 +00:00
Sanford Ryza
2cc851a66e MAPREDUCE-5484. YarnChild unnecessarily loads job conf twice (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518857 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-29 23:45:16 +00:00
Vinod Kumar Vavilapalli
5d4b684c02 MAPREDUCE-5441. Changed MR AM to return RUNNING state if exiting when RM commands to reboot, so that client can continue to track the overall job. Contributed by xJian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518821 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-29 21:07:57 +00:00
Alejandro Abdelnur
a29714f077 MAPREDUCE-5483. revert MAPREDUCE-5357. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518772 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-29 19:24:41 +00:00
Vinod Kumar Vavilapalli
5adba5597c YARN-1085. Addendum patch to address issues with the earlier patch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517721 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-26 22:59:09 +00:00
Vinod Kumar Vavilapalli
962da4dcc7 YARN-1074. Cleaned up YARN CLI application list to only display running applications by default. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517196 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-24 21:16:40 +00:00
Vinod Kumar Vavilapalli
e86036662c YARN-1085. Modified YARN and MR2 web-apps to do HTTP authentication in secure setup with kerberos. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517101 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-24 02:47:47 +00:00
Jason Darrell Lowe
c660339c09 Revert MAPREDUCE-5475 and YARN-707 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517097 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-24 01:15:37 +00:00
Jason Darrell Lowe
6f93f20515 MAPREDUCE-5475. MRClientService does not verify ACLs properly. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517085 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-23 22:47:37 +00:00
Vinod Kumar Vavilapalli
740f4cb97a MAPREDUCE-5476. Changed MR AM recovery code to cleanup staging-directory only after unregistering from the RM. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516660 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-22 23:17:12 +00:00
Chris Nauroth
200220e8f3 MAPREDUCE-5470. LocalJobRunner does not work on Windows. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516623 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-22 21:39:30 +00:00
Jason Darrell Lowe
d2c70e3e94 Revert MAPREDUCE-5475 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516594 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-22 20:34:46 +00:00
Arun Murthy
148bf3ea4e MAPREDUCE-5475. Ensure MRClientService verifies ACLs for users. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516361 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-22 01:59:25 +00:00
Arun Murthy
ded91b4cfa MAPREDUCE-5468. Fix MR AM recovery for map-only jobs. Contributed by Vinod K. V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516358 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-22 01:52:24 +00:00
Vinod Kumar Vavilapalli
2d614a916c MAPREDUCE-5466. Changed MR AM to not promote history files of intermediate AMs in case they are exiting because of errors and thus help history-server pick up the right history file for the last successful AM. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516238 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-21 18:16:47 +00:00
Luke Lu
782191f1ba HADOOP-9784. Add a builder for HttpServer. (Junping Du via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516128 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-21 10:12:13 +00:00
Jason Darrell Lowe
f7ca7ec4c9 MAPREDUCE-5001. LocalJobRunner has race condition resulting in job failures. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1515863 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-20 16:41:16 +00:00
Sanford Ryza
cae55de2cd MAPREDUCE-5462. In map-side sort, swap entire meta entries instead of indexes for better cache performance. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514608 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-16 08:11:04 +00:00
Alejandro Abdelnur
8451ab5c01 HADOOP-9872. Improve protoc version handling and detection. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514068 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-14 22:15:04 +00:00
Sanford Ryza
a6ff2270c5 MAPREDUCE-5454. TestDFSIO fails intermittently on JDK7 (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1513368 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-13 07:03:43 +00:00
Alejandro Abdelnur
19b36f0900 HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1513281 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-12 22:15:10 +00:00
Jason Darrell Lowe
c5c90ff8f5 MAPREDUCE-5425. Junit in TestJobHistoryServer failing in jdk 7. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1511464 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-07 20:02:10 +00:00
Sanford Ryza
0cb2fdc3b4 MAPREDUCE-434. LocalJobRunner limited to single reducer (Sandy Ryza and Aaron Kimball via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1510866 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-06 06:36:21 +00:00
Sanford Ryza
a00a729729 MAPREDUCE-5399. Unnecessary Configuration instantiation in IFileInputStream slows down merge. (Stanislav Barton via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1510811 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-05 23:32:33 +00:00
Sanford Ryza
6ea797d0d0 MAPREDUCE-5367. Local jobs all use same local working directory (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1510610 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-05 17:07:58 +00:00
Kihwal Lee
34ae96fd8c MAPREDUCE-5446. TestJobHistoryEvents and TestJobHistoryParsing have race conditions. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1510581 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-05 16:03:02 +00:00
Jason Darrell Lowe
bfe5a528d8 MAPREDUCE-5428. HistoryFileManager doesn't stop threads when service is stopped. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509401 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-01 19:57:31 +00:00
Siddharth Seth
381a4c4213 MAPREDUCE-5352. Optimize node local splits generated by CombineFileInputFormat. (sseth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509345 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-01 17:42:24 +00:00
Vinod Kumar Vavilapalli
1d915238a6 MAPREDUCE-5385. Fixed a bug with JobContext getCacheFiles API. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508595 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-30 20:04:37 +00:00
Jason Darrell Lowe
8bb035509e MAPREDUCE-5411. Refresh size of loaded job cache on history server. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508220 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-29 22:33:45 +00:00
Alejandro Abdelnur
f179afc68d YARN-937. Fix unmanaged AM in non-secure/secure setup post YARN-701. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507706 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-27 20:04:38 +00:00
Jason Darrell Lowe
7b5aa72d0a MAPREDUCE-5419. TestSlive is getting FileNotFound Exception. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507486 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-26 22:01:25 +00:00
Vinod Kumar Vavilapalli
e55d0ef515 MAPREDUCE-5421. Fixed TestNonExistentJob failure after YARN-873. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507401 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-26 19:15:36 +00:00
Jason Darrell Lowe
ec18984252 MAPREDUCE-1981. Improve getSplits performance by using listLocatedStatus. Contributed by Hairong Kuang and Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507385 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-26 18:16:19 +00:00
Tsz-wo Sze
a690a215db HADOOP-9756. Remove the deprecated getServer(..) methods from RPC. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507259 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-26 10:59:20 +00:00
Jason Darrell Lowe
8fa3ebd134 MAPREDUCE-5386. Ability to refresh history server job retention and job cleaner settings. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507135 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-25 21:27:56 +00:00
Jason Darrell Lowe
b9b2647ebc MAPREDUCE-5251. Reducer should not implicate map attempt if it has insufficient space to fetch map output. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507104 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-25 19:38:49 +00:00
Alejandro Abdelnur
c43c763151 MAPREDUCE-5213. Re-assess TokenCache methods marked @Private. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506758 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-24 22:31:31 +00:00
Vinod Kumar Vavilapalli
3ca3137179 YARN-926. Modified ContainerManagerProtcol APIs to take in requests for multiple containers. Contributed by Jian He. 
MAPREDUCE-5412. Update MR app to use multiple containers API of ContainerManager after YARN-926. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506391 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-24 03:41:03 +00:00
Jason Darrell Lowe
a42e459b9e MAPREDUCE-5356. Ability to refresh aggregated log retention period and check interval. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506226 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-23 19:20:57 +00:00
Jason Darrell Lowe
0939c49368 MAPREDUCE-5317. Stale files left behind for failed jobs. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506154 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-23 16:52:24 +00:00
Jason Darrell Lowe
77278064a7 MAPREDUCE-5404. HSAdminServer does not use ephemeral ports in minicluster mode. Contributed by Ted Yu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1505771 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-22 19:05:12 +00:00
Colin McCabe
c1314eb2a3 HDFS-4817. Make HDFS advisory caching configurable on a per-file basis. (Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1505753 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-22 18:15:18 +00:00
Arun Murthy
ac914f79bc YARN-918. Remove ApplicationAttemptId from RegisterApplicationMasterRequestProto. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504735 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-19 00:57:40 +00:00
Jason Darrell Lowe
cc536fe4da MAPREDUCE-5265. History server admin service to refresh user and superuser group mappings. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504645 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-18 20:41:14 +00:00
Arun Murthy
32bc200d54 YARN-701. Use application tokens irrespective of secure or non-secure mode. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504604 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-18 19:03:48 +00:00
Bikas Saha
bf876f70fb MAPREDUCE-5398. MR changes for YARN-513 (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503934 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 22:51:12 +00:00
Jason Darrell Lowe
44ea6e3ad5 Back out revision 1503499 for MAPREDUCE-5317. Stale files left behind for failed jobs 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503744 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 15:05:16 +00:00
Jason Darrell Lowe
8a037edbfb MAPREDUCE-5317. Stale files left behind for failed jobs. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503499 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-15 22:24:37 +00:00
Chris Nauroth
413bddf596 MAPREDUCE-4374. Fix child task environment variable config and add support for Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502046 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-10 22:59:08 +00:00
Hitesh Shah
e2cce7bd26 YARN-727, MAPREDUCE-5325. ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501599 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-09 23:01:01 +00:00
Vinod Kumar Vavilapalli
5e4f6ad1d9 YARN-791. Changed RM APIs and web-services related to nodes to ensure that both are consistent with each other. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500994 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-08 22:30:52 +00:00
Chris Nauroth
4175e41548 MAPREDUCE-5366. TestMRAsyncDiskService fails on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500842 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-08 17:07:38 +00:00
Chris Nauroth
e01cbdac25 MAPREDUCE-5360. TestMRJobClient fails on Windows due to path format. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500194 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-06 03:34:23 +00:00
Jason Darrell Lowe
cf4d0cf189 MAPREDUCE-5358. MRAppMaster throws invalid transitions for JobImpl. Contributed by Devaraj K 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499425 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-03 14:41:18 +00:00
Chris Nauroth
5b64e3c2be MAPREDUCE-5357. Job staging directory owner checking could fail on Windows. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499210 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-03 05:15:21 +00:00
Chris Nauroth
06ea8ffcf2 MAPREDUCE-5359. JobHistory should not use File.separator to match timestamp in path. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499153 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-02 23:57:30 +00:00
Chris Nauroth
f047f9cac6 MAPREDUCE-5355. MiniMRYarnCluster with localFs does not work on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499148 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-02 23:19:36 +00:00
Jason Darrell Lowe
e846c98397 MAPREDUCE-3193. FileInputFormat doesn't read files recursively in the input path dir. Contributed by Devaraj K 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499125 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-02 21:47:17 +00:00
Alejandro Abdelnur
27a50a2bef MAPREDUCE-5333. Add test that verifies MRAM works correctly when sending requests with non-normalized capabilities. (ywskycn via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1497877 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-28 18:53:19 +00:00
Arun Murthy
11bcd2ed12 MAPREDUCE-5326. Added version to shuffle header. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496741 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-26 05:49:31 +00:00
Vinod Kumar Vavilapalli
1a06175440 YARN-851. Share NMTokens using NMTokenCache (api-based) between AMRMClient and NMClient instead of memory based approach which is used currently. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495247 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-21 00:08:13 +00:00
Siddharth Seth
84862b54aa MAPREDUCE-5334. Fix failing unit tests - TestContainerLauncher, TestContainerLauncherImpl. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494820 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-20 00:35:51 +00:00
Jonathan Turner Eagles
1ab4659b6b MAPREDUCE-5084. fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and org.apache.hadoop.mapreduce.v2.hs.webapp (Aleksey Gorshkov via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494676 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-19 16:22:12 +00:00
Arun Murthy
d3198dddc8 YARN-553. Replaced YarnClient.getNewApplication with YarnClient.createApplication which provides a directly usable ApplicationSubmissionContext to simplify the api. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494476 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-19 07:06:54 +00:00
Christopher Douglas
16616f9e5b MAPREDUCE-5194. Heed interrupts during Fetcher shutdown. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494416 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-19 02:12:12 +00:00
Vinod Kumar Vavilapalli
243bcd367f YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 23:19:49 +00:00
Jason Darrell Lowe
52fe4e6be9 MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494285 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 20:25:00 +00:00
Vinod Kumar Vavilapalli
f4d80e91ae YARN-841. Move Auxiliary service to yarn-api, annotate and document it. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494031 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 06:20:37 +00:00
Arun Murthy
acc0d3eb52 YARN-840. Moved ProtoUtils to yarn.api.records.pb.impl. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494027 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 05:47:54 +00:00
Vinod Kumar Vavilapalli
7ef54faad4 YARN-834. Fixed annotations for yarn-client module, reorganized packages and clearly differentiated *Async apis. Contributed by Arun C Murthy and Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494017 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 04:02:47 +00:00
Siddharth Seth
2b14656ab5 YARN-805. Fix javadoc and annotations on classes in the yarn-api package. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493992 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 03:10:11 +00:00
Vinod Kumar Vavilapalli
5d1b453b85 YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 23:13:17 +00:00
Vinod Kumar Vavilapalli
7d0c43d7cf YARN-822. Renamed ApplicationToken to be AMRMToken, and similarly the corresponding TokenSelector and SecretManager. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493889 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 19:11:29 +00:00
Vinod Kumar Vavilapalli
cd30058193 YARN-837. Moved yarn.ClusterInfo into MapReduce project as it doesn't belong to YARN. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493676 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 08:13:48 +00:00
Vinod Kumar Vavilapalli
4662b8b443 YARN-833. Moved Graph and VisualizeStateMachine into yarn.state package. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493654 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 06:51:51 +00:00
Vinod Kumar Vavilapalli
b9efe6bd4a HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common for usage across all Hadoop projects. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493650 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 06:39:33 +00:00
Vinod Kumar Vavilapalli
d4a811edb2 YARN-826. Moved Clock and SystemClock into yarn.util package. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493639 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 04:49:13 +00:00
Arun Murthy
77e134d13f YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 02:27:18 +00:00
Alejandro Abdelnur
789c03438c MAPREDUCE-5171. Expose blacklisted nodes from the MR AM REST API. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493600 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 22:15:37 +00:00
Vinod Kumar Vavilapalli
e3769ea1d2 YARN-829. Renamed RMTokenSelector to be RMDelegationTokenSelector. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493577 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 20:33:30 +00:00
Arun Murthy
7f96128e43 MAPREDUCE-5184. Document compatibility for MapReduce applications in hadoop-2 vis-a-vis hadoop-1. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493570 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 19:09:48 +00:00
Arun Murthy
98d97d316c YARN-787. Removed minimum resource from RegisterApplicationMasterResponse. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493509 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 13:59:41 +00:00
Arun Murthy
1a389305b2 MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents. Contributed by Chris Douglas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493445 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 03:07:12 +00:00
Vinod Kumar Vavilapalli
c2d0cf1b20 YARN-821. Renamed setFinishApplicationStatus to setFinalApplicationStatus in FinishApplicationMasterRequest for consistency. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493315 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-15 04:57:04 +00:00
Alejandro Abdelnur
079ed1c9e1 MAPREDUCE-5310. MRAM should not normalize allocation request capabilities. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493207 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 19:06:50 +00:00
Vinod Kumar Vavilapalli
fdc9412a81 YARN-806. Moved ContainerExitStatus from yarn.api to yarn.api.records. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493138 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 16:10:37 +00:00
Arun Murthy
2166fa201e MAPREDUCE-5319. Set user.name in job.xml. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492962 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 06:26:55 +00:00
Vinod Kumar Vavilapalli
c0cd68d8fb YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 23:25:14 +00:00
Jason Darrell Lowe
8dc0d5af43 MAPREDUCE-4019. -list-attempt-ids is not working. Contributed by Ashwin Shankar, Devaraj K, and B Anil Kumar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492868 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 21:01:37 +00:00
Vinod Kumar Vavilapalli
b64572b06b MAPREDUCE-5199. Removing ApplicationTokens file as it is no longer needed. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492848 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 20:20:33 +00:00
Vinod Kumar Vavilapalli
0928502029 YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran. 
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 15:54:38 +00:00
Chris Nauroth
b090bc1c21 MAPREDUCE-5259. TestTaskLog fails on Windows because of path separators missmatch. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492364 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-12 18:54:56 +00:00
Alejandro Abdelnur
c9fce677b3 MAPREDUCE-5304. mapreduce.Job killTask/failTask/getTaskCompletionEvents methods have incompatible signature changes. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492360 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-12 18:47:33 +00:00
Jason Darrell Lowe
c0737b0bdb MAPREDUCE-5308. Shuffling to memory can get out-of-sync when fetching multiple compressed map outputs. Contributed by Nathan Roberts 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491611 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-10 21:39:34 +00:00
Alejandro Abdelnur
d00d968c49 MAPREDUCE-5312. TestRMNMInfo is failing. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491043 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-08 19:40:59 +00:00
Bikas Saha
9fcfbf5f51 YARN-759. Create Command enum in AllocateResponse (bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490470 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-06 23:33:48 +00:00
Bikas Saha
2051fd5ee2 YARN-750. Allow for black-listing resources in YARN API and Impl in CS (acmurthy via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490392 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-06 18:46:59 +00:00
Vinod Kumar Vavilapalli
66de4ccf60 YARN-724. Moved ProtoBase from api.records to api.records.impl.pb. Contributed by Jian He. 
MAPREDUCE-5303. Changed MR app after moving ProtoBase to package impl.pb via YARN-724. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489658 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 22:43:19 +00:00
Vinod Kumar Vavilapalli
978012b9b6 YARN-755. Renamed AllocateResponse.reboot to AllocateResponse.resync. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489295 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 05:53:47 +00:00
Vinod Kumar Vavilapalli
5c097a134e YARN-635 and MAPREDUCE-5301. Bug fix as the committed patch ended up being stale. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489287 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:41:55 +00:00
Vinod Kumar Vavilapalli
a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth. 
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:05:50 +00:00
Jason Darrell Lowe
78daceaea4 MAPREDUCE-5283. Over 10 different tests have near identical implementations of AppContext. Committed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489271 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 01:50:15 +00:00
Vinod Kumar Vavilapalli
d33534c4fb YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He. 
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 00:34:18 +00:00
Arun Murthy
72e1568be3 MAPREDUCE-5245. Added back constants to JobConf to fix incompatibilities. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489236 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 23:56:33 +00:00
Arun Murthy
78331cfeca MAPREDUCE-5300. Fix backward incompatibility for o.a.h.mapreduce.filecache.DistributedCache. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489227 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 23:42:28 +00:00
Arun Murthy
1caf3e363f MAPREDUCE-5296. Fix backward incompatibility for JobControl by adding the omitted addJob. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489219 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 22:54:48 +00:00
Arun Murthy
0add6b4edc MAPREDUCE-5220. Fix backward incompatibility for TaskCompletionEvent by adding back setTaskID. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489187 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 21:35:04 +00:00
Thomas White
9a3379f6b1 MAPREDUCE-5036. Default shuffle handler port should not be 8080. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489060 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 16:53:13 +00:00
Jason Darrell Lowe
ac44e0a0d0 MAPREDUCE-5268. Improve history server startup performance. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489012 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 14:46:30 +00:00
Siddharth Seth
a2205a3b72 YARN-720 and MAPREDUCE-5291. container-log4j.properties should not refer to mapreduce properties. Update MRApp to use YARN properties for log setup. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488829 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 02:35:34 +00:00
Arun Murthy
f27e484636 YARN-749. Rename ResourceRequest.(get,set)HostName to ResourceRequest.(get,set)ResourceName. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488806 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-02 22:53:58 +00:00
Vinod Kumar Vavilapalli
05cdca070e MAPREDUCE-5274. Bring back SecureShuffleUtils.toHex in mapreduce for binary compatibility with 1.x APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488625 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 22:21:43 +00:00
Vinod Kumar Vavilapalli
8267117e10 MAPREDUCE-5229. Put back FileOutputCommitter.TEMP_DIR_NAME in mapreduce for binary compatibility with 1.x APIs. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488621 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 22:10:42 +00:00
Vinod Kumar Vavilapalli
a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He. 
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 21:43:17 +00:00
Vinod Kumar Vavilapalli
a77030abdc MAPREDUCE-5280. Bring back removed constructor and a method in mapreduce ClusterMetrics for binary compatibility with 1.x APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488458 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 03:57:40 +00:00
Vinod Kumar Vavilapalli
b8e9f06774 MAPREDUCE-5273. Bring back a couple of protected variables in mapred and mapreduce CombineFileRecordReader for binary compatibility with 1.x APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488442 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:33:49 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Vinod Kumar Vavilapalli
b65e7ed920 MAPREDUCE-5231. Bring back a constructor in mapred's DBInputFormat.DBRecordReader for binary compatibility with 1.x mapred APIs. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488436 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 23:56:28 +00:00
Vinod Kumar Vavilapalli
73b88956f5 MAPREDUCE-5275. Bring back a couple of APIs in mapreduce.security.TokenCache for binary compatibility with 1.x mapreduce APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488369 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 19:48:03 +00:00
Christopher Douglas
0a452a30ab MAPREDUCE-5176. Add annotation for tagging tasks as responsive to 
preemption. Contributed by Carlo Curino, cdouglas


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488103 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 06:07:18 +00:00
Siddharth Seth
f3f7a11336 MAPREDUCE-5286. Change MapReduce to use ContainerTokenIdentifier instead of the entire Container in the startContainer call - YARN-684. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488087 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 04:17:33 +00:00
Vinod Kumar Vavilapalli
4394e5edb0 MAPREDUCE-5228. Bring back FileInputFormat.Counter and FileOuputFormat.Counter for binary compatibility with 1.x mapred APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488060 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 23:53:21 +00:00
Vinod Kumar Vavilapalli
131bfc91a6 MAPREDUCE-5261. Reverting the patch as it is no longer needed. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488032 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 21:53:34 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Siddharth Seth
b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli
a720a524e9 MAPREDUCE-5263. Bring back old methods and fields in filecache.DistributedCache for binary compatibility with mapred in 1.x. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487695 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 01:23:16 +00:00
Vinod Kumar Vavilapalli
0edae7a96a MAPREDUCE-5237. Add methods that were removed from ClusterStatus back into 2.x so as to be compatible with 1.x. Contributed by Zhijie Shen. 
Incompatible as ClusterStatus.UNINITIALIZED_MEMORY_VALUE is a long now and so breaks 0.23.x but it shouldn't be a big deal in reality.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487175 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 00:43:13 +00:00
Vinod Kumar Vavilapalli
643155cbee MAPREDUCE-5270. Migrated MR app from using BuilderUtil factory methods to individual record factory methods. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1486271 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-25 01:46:14 +00:00
Vinod Kumar Vavilapalli
259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 03:22:54 +00:00
Vinod Kumar Vavilapalli
a791527dd1 MAPREDUCE-5230. Bring back NLineInputFormat.createFileSplit for binary compatibility with mapred in 1.x Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485906 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 00:42:22 +00:00
Vinod Kumar Vavilapalli
43876770d9 YARN-563. Add the concept of an application-type for each application. Contributed by Mayank Bansal. 
MAPREDUCE-5246. Specify application-type at the time of job submission after YARN-563. Contributed by Mayank Bansal.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485790 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 18:02:04 +00:00
Vinod Kumar Vavilapalli
9b284150f1 MAPREDUCE-5233. Add methods that are changed or removed from JobControl.Job when compared to 1.x. This breaks 0.23.x users of one API in Job. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485491 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 23:12:45 +00:00
Vinod Kumar Vavilapalli
27e6673ec3 MAPREDUCE-5156. Change hadoop examples ProgramDriver to be able to run 1.x examples jar on top of YARN. This change breaks 0.23.x direct usages of ProgramDriver. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485085 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 06:36:24 +00:00
Vinod Kumar Vavilapalli
4a4e42d919 MAPREDUCE-5261. Fix issues in TestRMContainerAllocator after YARN-617. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485079 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 06:24:17 +00:00
Vinod Kumar Vavilapalli
6377bfc505 MAPREDUCE-5235. Bring back old fields and exceptions in Counters for binary compatibility with mapred in 1.x. Contributed by Mayank Bansal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484992 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-21 22:00:08 +00:00
Hitesh Shah
6517c3f714 MAPREDUCE-5191. TestQueue#testQueue fails with timeout on Windows. (Contributed by Ivan Mitic) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484575 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-20 20:02:07 +00:00
Vinod Kumar Vavilapalli
9cf436aef5 MAPREDUCE-5257. Fix issues in TestContainerLauncherImpl after YARN-617. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484349 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-19 21:15:11 +00:00
Jason Darrell Lowe
ca2265b581 MAPREDUCE-4927. Historyserver 500 error due to NPE when accessing specific counters page for failed job. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483974 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 20:19:48 +00:00
Vinod Kumar Vavilapalli
0b668c21b2 MAPREDUCE-5234. Change mapred.TaskReport and mapreduce.TaskReport for binary compatibility with mapred in 1.x but incompatible with 0.23.x. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483940 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 18:16:33 +00:00
Vinod Kumar Vavilapalli
ca80246731 YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 06:36:33 +00:00
Vinod Kumar Vavilapalli
4d8e350750 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 23:43:53 +00:00
Alejandro Abdelnur
3472800eb6 MAPREDUCE-5244. Two functions changed their visibility in JobStatus. (zjshen via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482622 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 22:12:40 +00:00
Vinod Kumar Vavilapalli
febf951220 MAPREDUCE-5222. Bring back some methods and constants in Jobclient for binary compatibility with mapred in 1.x. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482208 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 05:52:11 +00:00
Siddharth Seth
1a119f87b4 YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 04:11:55 +00:00
Vinod Kumar Vavilapalli
14d856be57 MAPREDUCE-5240. Fix a bug in MRAppMaster because of which OutputCommitter could not access credentials set by the user. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482175 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 02:48:53 +00:00
Alejandro Abdelnur
2103c80b77 MAPREDUCE-5220. Setter methods in TaskCompletionEvent are public in MR1 and protected in MR2. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482142 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-13 23:08:10 +00:00
Vinod Kumar Vavilapalli
7359dc32d3 MAPREDUCE-4067. Changed MRClientProtocol api to throw IOException only (Xuan Gong via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481695 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-13 03:34:25 +00:00
Vinod Kumar Vavilapalli
47d1ca402f MAPREDUCE-5208. Modified ShuffleHandler to use SecureIOUtils for reading local files. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481657 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-12 21:59:16 +00:00
Vinod Kumar Vavilapalli
2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Alejandro Abdelnur
d303f01297 MAPREDUCE-4942. mapreduce.Job has a bunch of methods that throw InterruptedException so its incompatible with MR1. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480748 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 20:04:32 +00:00
Vinod Kumar Vavilapalli
9c4f86879c YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli
0727ecaf50 YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli
f6167ef6c6 MAPREDUCE-5157. Bring back old sampler related code so that we can support binary compatibility with hadoop-1 sorter example. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480474 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 21:42:47 +00:00
Christopher Douglas
67d7d8f4d3 MAPREDUCE-5209. Fix units in a ShuffleScheduler log message. 
Contributed by Tsuyoshi OZAWA


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480464 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 21:12:27 +00:00
Vinod Kumar Vavilapalli
8888d3fc49 MAPREDUCE-5159. Change ValueAggregatorJob to add APIs which can support binary compatibility with hadoop-1 examples. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480394 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 18:28:22 +00:00
Vinod Kumar Vavilapalli
92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli
6e37a7e5e1 MAPREDUCE-5205. Fixed MR App to load tokens correctly. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479016 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-04 00:27:47 +00:00
Vinod Kumar Vavilapalli
fc275fed66 YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 22:44:12 +00:00
Thomas White
9b97df9abe MAPREDUCE-5036. Default shuffle handler port should not be 8080. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478422 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 15:58:53 +00:00
Aaron Myers
ca2c38fba1 MAPREDUCE-5193. A few MR tests use block sizes which are smaller than the default minimum block size. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478250 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 01:59:36 +00:00
Vinod Kumar Vavilapalli
71be8ad325 MAPREDUCE-5145. Changed default max-attempts to be more than one for MR jobs inline with YARN. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477889 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-01 00:42:32 +00:00
Suresh Srinivas
8f7ce62085 MAPREDUCE-5177. Use common utils FileUtil#setReadable/Writable/Executable & FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477403 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 23:00:39 +00:00
Vinod Kumar Vavilapalli
ae393ab26e MAPREDUCE-5179. Fix unit test in TestHSWebServices which fails when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476835 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 21:02:25 +00:00
Suresh Srinivas
fe3853484e HADOOP-9290. Some tests cannot load native library on windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476586 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:24:02 +00:00
Vinod Kumar Vavilapalli
fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Vinod Kumar Vavilapalli
448df18646 YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah. 
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 05:42:22 +00:00
Siddharth Seth
e96077aaf4 MAPREDUCE-5181. RMCommunicator should not use AMToken from the env. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471816 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 02:50:16 +00:00
Arun Murthy
daaa73b657 MAPREDUCE-5166. Fix ConcurrentModificationException due to insufficient synchronization on updates to task Counters. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471796 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 00:54:04 +00:00
Arun Murthy
40e78c2ca2 MAPREDUCE-4737. Ensure that mapreduce APIs are semantically consistent with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that cleanup is now called even if there is an error. The old mapred API already ensures that Mapper.close and Reducer.close are invoked during error handling. Note that it is an incompatible change, however end-users can override Mapper.run and Reducer.run to get the old (inconsistent) behaviour. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471556 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 17:38:43 +00:00
Robert Joseph Evans
0777474cc2 MAPREDUCE-5069. add concrete common implementations of CombineFileInputFormat (Sangjin Lee via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471424 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 14:11:50 +00:00
Vinod Kumar Vavilapalli
27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00
Thomas White
009af54d51 MAPREDUCE-5146. application classloader may be used too early to load classes. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470694 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-22 20:07:06 +00:00
Arun Murthy
794f9bb3e4 MAPREDUCE-5066. Added a timeout for the job.end.notification.url. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470216 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 19:18:24 +00:00
Thomas Graves
febc52be37 MAPREDUCE-5147. Maven build should create hadoop-mapreduce-client-app-VERSION.jar directly (Robert Parker via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470035 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 20:50:48 +00:00
Bikas Saha
c1ce3c5ad2 MAPREDUCE-4987. TestMRJobs#testDistributedCache fails on Windows due to classpath problems and unexpected behavior of symlinks (Chris Nauroth via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470003 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 19:32:33 +00:00
Vinod Kumar Vavilapalli
8e1c2823fc YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong. 
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 01:33:13 +00:00
Alejandro Abdelnur
e222d1e783 MAPREDUCE-4932. mapreduce.job#getTaskCompletionEvents incompatible with Hadoop 1. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469587 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 21:42:03 +00:00
Vinod Kumar Vavilapalli
cb78a65a15 MAPREDUCE-5152. Make MR App to simply pass through the container from RM instead of extracting and populating information itself to start any container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469544 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 20:13:40 +00:00
Alejandro Abdelnur
d12c463121 MAPREDUCE-4898. FileOutputFormat.checkOutputSpecs and FileOutputFormat.setOutputPath incompatible with MR1. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469523 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 18:51:45 +00:00
Alejandro Abdelnur
cd27afa71a MAPREDUCE-5128. mapred-default.xml is missing a bunch of history server configs. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469095 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 23:14:34 +00:00
Bikas Saha
9e650d36b7 MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469060 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 20:25:21 +00:00
Thomas Graves
9ea20fc537 MAPREDUCE-5015. Coverage fix for org.apache.hadoop.mapreduce.tools.CLI (Aleksey Gorshkov via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468483 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-16 16:11:35 +00:00
Vinod Kumar Vavilapalli
0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:54:09 +00:00
Robert Joseph Evans
947e97f354 MAPREDUCE-4974. Optimising the LineRecordReader initialize() method (Gelesh via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468232 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 21:33:47 +00:00
Konstantin Shvachko
8e838bd03c MAPREDUCE-4985. Add compression option to TestDFSIO usage. Contributed by Plamen Jeliazkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467145 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 23:49:16 +00:00
Vinod Kumar Vavilapalli
3c8bd60616 MAPREDUCE-5094. Disabled memory monitoring by default in MiniMRYarnCluster to avoid some downstream tests failing. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467124 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 22:54:02 +00:00
Vinod Kumar Vavilapalli
5fd460e688 MAPREUDUCE-5059. Change average merge time on Job overview page to be the time delta between the end of the shuffle and the start of the reduce. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467120 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 22:27:45 +00:00
Vinod Kumar Vavilapalli
e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong. 
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:28:51 +00:00
Jason Darrell Lowe
8e86ffd91e MAPREDUCE-5136. TestJobImpl->testJobNoTasks fails with IBM JAVA. Contributed by Amir Sanjar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467061 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:12:59 +00:00
Jason Darrell Lowe
d04e65b797 MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466948 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 16:27:24 +00:00
Siddharth Seth
6a1c41111e MAPREDUCE-5079. Changes job recovery to restore state directly from job history, instaed of simulating state machine events. Contributed by Jason Lowe and Robert Parker. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466767 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 04:52:38 +00:00
Vinod Kumar Vavilapalli
3a54a5653b YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466196 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 19:56:10 +00:00
Arun Murthy
7f13207ed1 MAPREDUCE-5129. Allow tags to JobHistory for deeper analytics. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465188 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-06 05:30:56 +00:00
Thomas Graves
a5734cd38a MAPREDUCE-5007. fix coverage org.apache.hadoop.mapreduce.v2.hs (Aleksey Gorshkov via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465017 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 15:43:53 +00:00
Jason Darrell Lowe
fc75d3f3dc MAPREDUCE-5086. MR app master deletes staging dir when sent a reboot command from the RM. Contributed by Jian He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464255 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 01:56:49 +00:00
Konstantin Boudnik
d18cc69d4e MAPREDUCE-5088. MR Client gets an renewer token exception while Oozie is submitting a job (daryn) 
merge -c1463804 from branch-2.0.4-alpha



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464153 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 18:45:15 +00:00
Vinod Kumar Vavilapalli
38678cc3d9 MAPREDUCE-5117. Changed MRClientProtocolPBClientImpl to be closeable and thus fix failures in renewal of HistoryServer's delegations tokens. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463828 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 05:37:10 +00:00
Robert Joseph Evans
6f0c4dca74 Reverted MAPREDUCE-4974 because of test failures. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463359 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 02:13:18 +00:00
Robert Joseph Evans
b55756dd03 MAPREDUCE-4974. Optimising the LineRecordReader initialize() method (Gelesh via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463221 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-01 17:45:07 +00:00
Bikas Saha
520033b1cd YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462271 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-28 19:44:28 +00:00
Thomas White
375584ae0f MAPREDUCE-5006. Fix failing streaming tests due to MAPREDUCE-4994. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461541 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-27 11:47:06 +00:00
Siddharth Seth
d41e67b966 MAPREDUCE-5077. Remove mapreduce.util.ResourceCalculatorPlugin and related code. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461251 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 18:15:40 +00:00
Robert Joseph Evans
a0a844222d MAPREDUCE-4875. coverage fixing for org.apache.hadoop.mapred (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461146 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 14:16:13 +00:00
Vinod Kumar Vavilapalli
46315a2d91 MAPREDUCE-5062. Fix MR AM to read max-retries from the RM. Contributed by *Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460923 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 22:33:43 +00:00
Hitesh Shah
b8c74a8a25 MAPREDUCE-5083. MiniMRCluster should use a random component when creating an actual cluster. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459932 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-22 19:11:01 +00:00
Siddharth Seth
cd2c5fab8b MAPREDUCE-3872. Fix an event handling races in ContainerLauncherImpl. (Contributed by Robert Kanter) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459547 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 21:55:31 +00:00
Siddharth Seth
6610212dd3 MAPREDUCE-5078. TestMRAppMaster fails on Windows due to mismatched path separators. (Contributed by Chris Nauroth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459544 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 21:44:13 +00:00
Hitesh Shah
1bd345d6e3 YARN-396. Rationalize AllocateResponse in RM Scheduler API. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459040 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-20 20:44:35 +00:00
Robert Joseph Evans
f7df0cb7df MAPREDUCE-4972. Coverage fixing for org.apache.hadoop.mapreduce.jobhistory (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458906 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-20 16:05:33 +00:00
Alejandro Abdelnur
c19633da5b Reverting MAPREDUCE-5028 (commit 1457918) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458433 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-19 17:56:29 +00:00
Jonathan Turner Eagles
523d3daac0 MAPREDUCE-5053. java.lang.InternalError from decompression codec cause reducer to fail (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458350 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-19 15:45:35 +00:00
Alejandro Abdelnur
53e7aaa6dd MAPREDUCE-5028. Maps fail when io.sort.mb is set to high value. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457918 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-18 18:56:40 +00:00
Alejandro Abdelnur
5ac6abe107 MAPREDUCE-4990. Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput(). (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457914 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-18 18:45:55 +00:00
Aaron Myers
64741f4635 HDFS-4521. Invalid network toploogies should not be cached. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457878 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-18 17:20:13 +00:00
Robert Joseph Evans
7d7553c4eb MAPREDUCE-5042. Reducer unable to fetch for a map task that was recovered (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457119 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-15 21:09:25 +00:00
Alejandro Abdelnur
8c8ff3f5e0 MAPREDUCE-4716. TestHsWebServicesJobsQuery.testJobsQueryStateInvalid fails with jdk7. (tgraves via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457065 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-15 18:55:22 +00:00
Alejandro Abdelnur
d412a1a1a8 MAPREDUCE-4571. TestHsWebServicesJobs fails on jdk7. (tgraves via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457061 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-15 18:50:51 +00:00
Robert Joseph Evans
1b6f0582fb YARN-237. Refreshing the RM page forgets how many rows I had in my Datatables (jian he via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456536 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-14 16:56:10 +00:00
Jason Darrell Lowe
57803245ec MAPREDUCE-5060. Fetch failures that time out only count against the first map task. Contributed by Robert Joseph Evans 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1455740 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-12 22:51:28 +00:00
Thomas Graves
7122cb5bf4 MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1454156 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-07 23:02:28 +00:00
Suresh Srinivas
6942fd1db5 HADOOP-9372. Fix bad timeout annotations on tests. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453637 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-07 01:09:53 +00:00
Suresh Srinivas
638801cce1 HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 19:15:18 +00:00
Arun Murthy
df68c56267 MAPREDUCE-3685. Fix bugs in MergeManager to ensure compression codec is appropriately used and that on-disk segments are correctly sorted on file-size. Contributed by Anty Rao and Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453365 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 15:02:45 +00:00
Jonathan Turner Eagles
e1062b8b78 MAPREDUCE-5027. Shuffle does not limit number of outstanding connections (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453098 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 00:03:41 +00:00
Robert Joseph Evans
ec13f1eb3a MAPREDUCE-5043. Fetch failure processing can cause AM event queue to backup and eventually OOM (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452372 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-04 16:32:18 +00:00
Jonathan Turner Eagles
38c4e17ec1 MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451826 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-02 03:30:58 +00:00
Alejandro Abdelnur
5d6eca08bd MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does not exist. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451452 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-01 01:28:18 +00:00
Alejandro Abdelnur
892e088773 HADOOP-9349. Confusing output when running hadoop version from one hadoop installation when HADOOP_HOME points to another. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451448 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-01 01:20:53 +00:00
Jonathan Turner Eagles
892846dc04 MAPREDUCE-4871. AM uses mapreduce.jobtracker.split.metainfo.maxsize but mapred-default has mapreduce.job.split.metainfo.maxsize (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451318 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-28 19:55:10 +00:00
Siddharth Seth
979fb054f8 MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450956 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-27 21:03:19 +00:00
Vinod Kumar Vavilapalli
0b9ed2364a MAPREDUCE-4892. Modify CombineFileInputFormat to not skew input slits' allocation on small clusters. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450912 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-27 18:49:37 +00:00
Thomas White
14089f1e57 MAPREDUCE-5008. Merger progress miscounts with respect to EOF_MARKER. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450723 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-27 10:40:00 +00:00
Suresh Srinivas
70b1c5ca75 HADOOP-9334. Upgrade netty version. Contributed by Nicolas Liochon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450463 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-26 21:54:25 +00:00
Thomas White
0b73dde6ce MAPREDUCE-4951. Container preemption interpreted as task failure. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448615 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-21 11:56:08 +00:00
Thomas White
87035a004c MAPREDUCE-5013. mapred.JobStatus compatibility: MR2 missing constructors from MR1. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448602 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-21 11:42:48 +00:00
Thomas White
7555b4df74 MAPREDUCE-4846. Some JobQueueInfo methods are public in MR1 but protected in MR2. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448597 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-21 11:34:09 +00:00
Jonathan Turner Eagles
c8f35bc3d2 MAPREDUCE-5009. Killing the Task Attempt slated for commit does not clear the value from the Task commitAttempt member (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1447965 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-19 23:35:56 +00:00
Suresh Srinivas
1c49e77998 MAPREDUCE-5012. Typo in javadoc for IdentityMapper class. Contributed by Adam Monsen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1447865 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-19 18:45:22 +00:00
Alejandro Abdelnur
0a8cfa9016 HADOOP-9304. remove addition of avro genreated-sources dirs to build. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1446296 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-14 18:48:14 +00:00
Alejandro Abdelnur
ab330b33bb MAPREDUCE-4994. Addendum fixing testcases failures. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1446042 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-14 04:22:04 +00:00
Alejandro Abdelnur
6bcbf492a2 HADOOP-9117. replace protoc ant plugin exec with a maven plugin. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445956 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 22:13:44 +00:00
Siddharth Seth
43204f0c83 MAPREDUCE-5000. Fixes getCounters when speculating by fixing the selection of the best attempt for a task. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445871 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 19:20:31 +00:00
Jason Darrell Lowe
390d1fbef8 MAPREDUCE-4992. AM hangs in RecoveryService when recovering tasks with speculative attempts. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445456 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 02:11:36 +00:00
Jason Darrell Lowe
c2d6407070 MAPREDUCE-4989. JSONify DataTables input data for Attempts page. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445448 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 01:15:48 +00:00
Alejandro Abdelnur
7e615c7f75 MAPREDUCE-4994. -jt generic command line option does not work. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445330 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-12 20:16:08 +00:00
Siddharth Seth
35832053bf MAPREDUCE-4671. AM does not tell the RM about container requests which are no longer needed. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443323 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-07 07:07:11 +00:00
Jason Darrell Lowe
ea94b7b9dd MAPREDUCE-4822. Unnecessary conversions in History Events. Contributed by Chu Tong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443257 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 22:47:10 +00:00
Alejandro Abdelnur
17e72be6d8 MAPREDUCE-4977. Documentation for pluggable shuffle and pluggable sort. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443168 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 19:52:25 +00:00
Robert Joseph Evans
79b12f6f4c MAPREDUCE-4905. test org.apache.hadoop.mapred.pipes (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443027 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 15:50:23 +00:00
Siddharth Seth
a04bee4567 MAPREDUCE-4971. Minor extensibility enhancements to Counters & FileOutputFormat. Contributed by Arun C Murthy 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441670 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-02 00:47:58 +00:00
Hitesh Shah
b1baf11fb2 YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441668 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-02 00:43:54 +00:00
Suresh Srinivas
ac21e15a39 MAPREDUCE-4969. TestKeyValueTextInputFormat test fails with Open JDK 7. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441069 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-31 17:11:36 +00:00
Vinod Kumar Vavilapalli
28c308d5e8 MAPREDUCE-4893. Fixed MR ApplicationMaster to do optimal assignment of containers to get maximum locality. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440749 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-31 00:28:10 +00:00
Alejandro Abdelnur
0f430e53fd MAPREDUCE-2264. Job status exceeds 100% in some cases. (devaraj.k and sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440076 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-29 19:38:40 +00:00
Siddharth Seth
c163dc2fce MAPREDUCE-4838. Add additional fields like Locality, Avataar to the JobHistory logs. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439714 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-29 00:21:58 +00:00
Siddharth Seth
8a5fadc5a0 MAPREDUCE-4803. Remove duplicate copy of TestIndexCache. Contributed by Mariappan Asokan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439590 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-28 19:18:35 +00:00
Alejandro Abdelnur
da4cab1099 Revering MAPREDUCE-2264 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439561 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-28 18:58:34 +00:00
Alejandro Abdelnur
539153a679 MAPREDUCE-2264. Job status exceeds 100% in some cases. (devaraj.k and sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1438277 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-25 00:25:59 +00:00
Siddharth Seth
74ffc7a74d MAPREDUCE-4946. Fix a performance problem for large jobs by reducing the number of map completion event type conversions. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437103 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 19:07:08 +00:00
Alejandro Abdelnur
73fd247c76 MAPREDUCE-4808. Refactor MapOutput and MergeManager to facilitate reuse by Shuffle implementations. (masokan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1436936 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 14:10:42 +00:00
Jonathan Turner Eagles
88c7fdd065 MAPREDUCE-4458. Warn if java.library.path is used for AM or Task (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435386 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 22:41:19 +00:00
Siddharth Seth
76f9af4ccf MAPREDUCE-4948. Fix a failing unit test - TestYARNRunner.testHistoryServerToken. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435312 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 19:30:00 +00:00
Alejandro Abdelnur
1c308188bb MAPREDUCE-4923. Add toString method to TaggedInputSplit. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434993 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 00:40:58 +00:00
Alejandro Abdelnur
26275e5323 MAPREDUCE-4924. flakey test: org.apache.hadoop.mapred.TestClusterMRNotification.testMR. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433753 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-16 00:08:25 +00:00
Siddharth Seth
6a2f2551fd YARN-135. Client tokens should be per app-attempt, and should be unregistered on App-finish. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433570 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 18:33:38 +00:00
Jason Darrell Lowe
dd6aa0cf12 MAPREDUCE-4936. JobImpl uber checks for cpu are wrong. Contributed by Arun C Murthy 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433445 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 14:45:08 +00:00
Suresh Srinivas
86825d8390 MAPREDUCE-4938. Use token request messages defined in hadoop common. Contributed by Suresh Srinvias. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433227 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 00:11:16 +00:00
Thomas Graves
e7aeb9d129 MAPREDUCE-4934. Maven RAT plugin is not checking all source files (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432932 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-14 14:35:45 +00:00
Robert Joseph Evans
12293f8a13 MAPREDUCE-4921. JobClient should acquire HS token with RM principal (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432230 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-11 19:00:51 +00:00
Alejandro Abdelnur
d7c48409e3 MAPREDUCE-4907. Ammendment, forgot to svn add testcase in original commit 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431510 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-10 17:27:58 +00:00
Alejandro Abdelnur
59d9d8bca9 MAPREDUCE-4907. TrackerDistributedCacheManager issues too many getFileStatus calls. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431166 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-10 00:50:03 +00:00
Jason Darrell Lowe
b16dfc125d MAPREDUCE-4848. TaskAttemptContext cast error during AM recovery. Contributed by Jerry Chen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431131 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 22:56:09 +00:00
Thomas White
0ba7078ef4 MAPREDUCE-1700. User supplied dependencies may conflict with MapReduce system JARs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430929 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 16:12:58 +00:00
Vinod Kumar Vavilapalli
5f7d4d2b45 MAPREDUCE-4810. Added new admin command options for MR AM. Contributed by Jerry Chen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430707 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 06:31:36 +00:00
Arun Murthy
2c5c8fdb80 MAPREDUCE-4520. Added support for MapReduce applications to request for CPU cores along-with memory post YARN-2. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430688 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 05:28:40 +00:00
Arun Murthy
4539263971 YARN-2. Enhanced CapacityScheduler to account for CPU alongwith memory for multi-dimensional resource scheduling. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430682 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 05:08:23 +00:00
Thomas White
0f1f5491bc MAPREDUCE-4278. Cannot run two local jobs in parallel from the same gateway. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430363 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-08 16:26:33 +00:00
Vinod Kumar Vavilapalli
b8a7a17191 MAPREDUCE-4920. Use security token protobuf definition from hadoop common. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429751 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-07 11:13:26 +00:00
Thomas Graves
bbc426f53a MAPREDUCE-4913. TestMRAppMaster#testMRAppMasterMissingStaging occasionally exits (Jason Lowe via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429371 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-05 20:07:24 +00:00
Robert Joseph Evans
64e4fb983e MAPREDUCE-4819. AM can rerun job after reporting final job status to the client (bobby and Bikas Saha via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429114 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:35:56 +00:00
Thomas Graves
e17cecf550 MAPREDUCE-4894. Renewal / cancellation of JobHistory tokens (Siddharth Seth via tgraves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429086 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:15:43 +00:00
Jason Darrell Lowe
78ab699fe9 MAPREDUCE-4832. MR AM can get in a split brain situation. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429040 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 19:15:21 +00:00
Robert Joseph Evans
e229e25cc3 MAPREDUCE-4279. getClusterStatus() fails with null pointer exception when running jobs in local mode (Devaraj K via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428482 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 17:09:18 +00:00
Robert Joseph Evans
402eb18513 MAPREDUCE-4813. AM timing out during job commit (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1426536 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-28 15:01:46 +00:00
Jason Darrell Lowe
a7d444d002 MAPREDUCE-4902. Fix typo "receievd" should be "received" in log output. Contributed by Albert Chu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1426018 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-26 20:47:39 +00:00
Vinod Kumar Vavilapalli
dae6f0b7d9 MAPREDUCE-4899. Implemented a MR specific plugin for tracking finished applications that YARN's ResourceManager doesn't keep track off anymore. Contributed by Derek Dagit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425225 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-22 01:47:51 +00:00
Jason Darrell Lowe
04db3ce177 MAPREDUCE-4890. Invalid TaskImpl state transitions when task fails while speculating. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425223 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-22 01:41:26 +00:00
Luke Lu
02e940d6a4 MAPREDUCE-4272. SortedRanges.Range#compareTo was not spec compliant. (Yu Gao via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425180 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 23:10:52 +00:00
Jason Darrell Lowe
92692c863c MAPREDUCE-4833. Task can get stuck in FAIL_CONTAINER_CLEANUP. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425167 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 22:27:54 +00:00
Jason Darrell Lowe
92774331cc MAPREDUCE-4842. Shuffle race can hang reducer. Contributed by Mariappan Asokan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425071 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 18:24:25 +00:00
Luke Lu
1a49c85438 YARN-223. Update process tree instead of getting new process trees. (Radim Kolar via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424244 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-20 00:20:53 +00:00
Doug Cutting
aa30056d09 MAPREDUCE-4887. Add RehashPartitioner, to smooth distributions with poor implementations of Object#hashCode(). Contributed by Radim Kolar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424158 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-19 21:50:17 +00:00
Thomas White
9189917be5 MAPREDUCE-4845. ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1423470 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-18 15:23:49 +00:00
Alejandro Abdelnur
8329fae686 MAPREDUCE-4807. Allow MapOutputBuffer to be pluggable. (masokan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1422345 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-15 20:23:08 +00:00
Alejandro Abdelnur
803e5155d1 MAPREDUCE-4809. Change visibility of classes for pluggable sort changes. (masokan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1422341 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-15 20:18:23 +00:00
Thomas White
4993314895 MAPREDUCE-4856. TestJobOutputCommitter uses same directory as TestJobCleanup. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420634 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-12 12:12:10 +00:00
Alejandro Abdelnur
c5080ef336 MAPREDUCE-4861. Cleanup: Remove unused mapreduce.security.token.DelegationTokenRenewal. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420345 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-11 19:26:57 +00:00
Jason Darrell Lowe
b72b36009a YARN-266. RM and JHS Web UIs are blank because AppsBlock is not escaping string properly. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420232 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-11 15:45:10 +00:00
Alejandro Abdelnur
054270cc28 MAPREDUCE-4703. Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1419618 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-10 18:07:20 +00:00
Arun Murthy
b096f61fe2 MAPREDUCE-4049. Experimental api to allow for alternate shuffle plugins. Contributed by Anver BenHanoch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1418173 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-07 02:36:33 +00:00
Jonathan Turner Eagles
fdcfef0a26 MAPREDUCE-4836. Elapsed time for running tasks on AM web UI tasks page is 0. (Ravi Prakash via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1415860 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-30 21:43:43 +00:00
Thomas Graves
5caef48947 MAPREDUCE-4817. Hardcoded task ping timeout kills tasks localizing large amounts of data (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414873 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-28 19:19:16 +00:00
Robert Joseph Evans
b43deb9af8 MAPREDUCE-4825. JobImpl.finished doesn't expect ERROR as a final job state (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414840 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-28 17:52:45 +00:00
Robert Joseph Evans
481f451ca2 MAPREDUCE-4764. repair TestBinaryTokenFile (Ivan A. Veselovsky via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1413739 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-26 17:37:05 +00:00
Jason Darrell Lowe
6c5f37e46d MAPREDUCE-4811. JobHistoryServer should show when it was started in WebUI About page. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1412377 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-22 00:05:26 +00:00
Siddharth Seth
c271f3cded MAPREDUCE-4723 amendment. Fix JobHistory Event handling. (Contributed by Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411292 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-19 16:43:29 +00:00
Thomas White
e464607b7d YARN-129. Simplify classpath construction for mini YARN tests. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411235 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-19 15:12:22 +00:00
Alejandro Abdelnur
0f0caede29 MAPREDUCE-4800. Cleanup o.a.h.mapred.MapTaskStatus - remove unused code. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410700 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-17 06:08:06 +00:00
Robert Joseph Evans
d2b9a604bd MAPREDUCE-4802. Takes a long time to load the task list on the AM for large jobs (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410498 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-16 17:30:22 +00:00
Robert Joseph Evans
f13da263bf MAPREDUCE-4801. ShuffleHandler can generate large logs due to prematurely closed channels (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410131 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-16 00:52:43 +00:00
Eli Collins
905b17876c MAPREDUCE-4723. Fix warnings found by findbugs 2. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409601 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-15 00:16:35 +00:00
Robert Joseph Evans
7e2ecffd88 MAPREDUCE-4720. Browser thinks History Server main page JS is taking too long (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409580 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-15 00:01:07 +00:00
Robert Joseph Evans
316c23dd7e MAPREDUCE-4797. LocalContainerAllocator can loop forever trying to contact the RM (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409525 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-14 23:02:29 +00:00
Jason Darrell Lowe
397c261433 YARN-216. Remove jquery theming support. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409313 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-14 19:22:11 +00:00
Thomas Graves
b2f7a66e50 MAPREDUCE-4517. Too many INFO messages written out during AM to RM heartbeat (Jason Lowe via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409032 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-13 23:47:57 +00:00
Robert Joseph Evans
cf6ab9b2e7 MAPREDUCE-4786. Job End Notification retry interval is 5 milliseconds by default (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408444 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-12 20:32:04 +00:00
Robert Joseph Evans
c493d06b1b MAPREDUCE-4425. Speculation + Fetch failures can lead to a hung job (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408360 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-12 17:15:45 +00:00
Robert Joseph Evans
1a45b7e357 MAPREDUCE-4787. TestJobMonitorAndPrint is broken (Rob Parker via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408355 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-12 17:06:07 +00:00
Robert Joseph Evans
dd72ca3536 MAPREDUCE-4751. AM stuck in KILL_WAIT for days (vinodkv via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408314 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-12 15:15:52 +00:00
Thomas White
42f4691ead Add new files missed from HADOOP-8860 commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408288 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-12 13:54:27 +00:00
Robert Joseph Evans
7c9778e2c3 MAPREDUCE-4774. JobImpl does not handle asynchronous task events in FAILED state (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407679 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 22:55:57 +00:00
Jonathan Turner Eagles
3ccc76c29b MAPREDUCE-4666. JVM metrics for history server. (jlowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407669 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 22:14:06 +00:00
Alejandro Abdelnur
9d5205aadd HADOOP-8860. Split MapReduce and YARN sections in documentation navigation. (tomwhite via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407658 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 21:44:00 +00:00
Robert Joseph Evans
1f40b8b4e8 MAPREDUCE-4782. NLineInputFormat skips first line of last InputSplit (Mark Fuhs via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407505 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 15:52:33 +00:00
Robert Joseph Evans
6ee6eb8430 MAPREDUCE-4772. Fetch failures can take way too long for a map to be restarted (bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407118 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-08 15:17:34 +00:00
Thomas White
8d483fff2c MAPREDUCE-4777. In TestIFile, testIFileReaderWithCodec relies on testIFileWriterWithCodec. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1406645 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-07 14:11:59 +00:00
Robert Joseph Evans
0eadfcfcf5 MAPREDUCE-4771. KeyFieldBasedPartitioner not partitioning properly when configured (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1405975 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-05 21:52:06 +00:00
Robert Joseph Evans
1783f209f0 MAPREDUCE-4763 repair test TestUmbilicalProtocolWithJobToken (Ivan A. Veselovsky via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404988 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-02 15:00:56 +00:00
Jason Darrell Lowe
42d1eaf237 MAPREDUCE-4729. job history UI not showing all job attempts. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404817 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-01 22:59:43 +00:00
Robert Joseph Evans
b50a3c5de3 MAPREDUCE-4746. The MR Application Master does not have a config to set environment variables (Rob Parker via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404674 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-01 17:02:52 +00:00
Robert Joseph Evans
50993e9d48 MAPREDUCE-4724. job history web ui applications page should be sorted to display last app first (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404305 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-31 19:13:03 +00:00
Thomas Graves
aac5c149c7 MAPREDUCE-4752. Reduce MR AM memory usage through String Interning (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404177 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-31 14:57:12 +00:00
Alejandro Abdelnur
16e21dfe92 MAPREDUCE-1806. CombineFileInputFormat does not work with paths not on default FS. (Gera Shegalov via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1403614 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-30 05:41:59 +00:00
Jason Darrell Lowe
286c171987 MAPREDUCE-4748. Invalid event: T_ATTEMPT_SUCCEEDED at SUCCEEDED. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1402658 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-26 21:02:52 +00:00
Vinod Kumar Vavilapalli
895029b2f2 MAPREDUCE-4730. Fix Reducer's EventFetcher to scale the map-completion requests slowly to avoid HADOOP-8942. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401941 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-25 01:26:30 +00:00
Jason Darrell Lowe
1e45b1f1fd MAPREDUCE-4741. WARN and ERROR messages logged during normal AM shutdown. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401738 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-24 15:45:35 +00:00
Jason Darrell Lowe
614a743fea YARN-139. Interrupted Exception within AsyncDispatcher leads to user confusion. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401726 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-24 15:20:58 +00:00
Siddharth Seth
382b565e8c MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401474 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-23 21:02:22 +00:00
Daryn Sharp
def0b949d4 MAPREDUCE-4229. Intern counter names in the JT (bobby via daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401467 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-23 20:57:36 +00:00
Jason Darrell Lowe
49beea562a MAPREDUCE-4740. only .jars can be added to the Distributed Cache classpath. Contributed by Robert Joseph Evans 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401054 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-22 19:50:23 +00:00
Suresh Srinivas
c195ce4d67 MAPREDUCE-4736. Remove obsolete option [-rootDir] from TestDFSIO. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400313 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 22:05:30 +00:00
Vinod Kumar Vavilapalli
1195f844a9 MAPREDUCE-4733. Reducer can fail to make progress during shuffle if too many reducers complete consecutively. Contributed by Jason Lowe via. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400264 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 20:18:46 +00:00
Robert Joseph Evans
a234472d1c MAPREDUCE-4479. Fix parameter order in assertEquals() in TestCombineInputFileFormat.java (Mariappan Asokan via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400192 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 17:51:33 +00:00
Vinod Kumar Vavilapalli
022f7b4a25 MAPREDUCE-4596. Split StateMachine state from states seen by MRClientProtocol for Job, Task and TaskAttempt. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399976 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 05:57:57 +00:00
Suresh Srinivas
adcf7d10b9 MAPREDUCE-4735. Make arguments in TestDFSIO case insensitive. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399952 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 02:37:59 +00:00
Robert Joseph Evans
c63811790a MAPREDUCE-4721. Task startup time in JHS is same as job startup time. (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399289 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-17 14:58:08 +00:00
Robert Joseph Evans
0e9f40df97 MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399011 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-16 22:00:03 +00:00
Suresh Srinivas
94ed14c48c HADOOP-8911. CRLF characters in source and text files. Contributed Raja Aluri. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397432 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-12 04:35:42 +00:00
Arun Murthy
f174adfc15 MAPREDUCE-4616. Improve javadoc for MultipleOutputs. Contributed by Tony Burton. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397182 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-11 17:21:45 +00:00
Harsh J
0aa8188d18 MAPREDUCE-3678. The Map tasks logs should have the value of input split it processed. Contributed by Harsh J. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396032 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 13:39:26 +00:00
Harsh J
5c3a331040 MAPREDUCE-4574. Fix TotalOrderParitioner to work with non-WritableComparable key types. Contributed by Harsh J. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395936 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 09:58:21 +00:00
Siddharth Seth
8ac3910ae0 MAPREDUCE-4705. Fix a bug in job history lookup, which makes older jobs inaccessible despite the presence of a valid history file. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395850 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 03:23:07 +00:00
Siddharth Seth
c074cfd6f0 YARN-134. Fixes ClientToAMSecretManager creates keys without checking for validity of the appID. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395841 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 01:56:05 +00:00
Robert Joseph Evans
49b20c2ed1 MAPREDUCE-4554. Job Credentials are not transmitted if security is turned off (Benoy Antony via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395769 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-08 20:50:40 +00:00
Harsh J
7137503eb7 MAPREDUCE-4695. Fix LocalRunner on trunk after MAPREDUCE-3223 broke it. Contributed by Harsh J. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1392446 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-01 17:10:11 +00:00
Robert Joseph Evans
b897d6c35a MAPREDUCE-4689. JobClient.getMapTaskReports on failed job results in NPE (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391679 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-28 22:17:32 +00:00
Jason Darrell Lowe
03b7ad04fa MAPREDUCE-4691. Historyserver can report "Unknown job" after RM says job has completed. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391671 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-28 21:55:45 +00:00
Harsh J
6d77365a83 MAPREDUCE-3223. Remove MR1 configs from mapred-default.xml. Contributed by Todd Lipcon (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391498 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-28 15:08:48 +00:00
Arun Murthy
036530fc12 Reverted MAPREDUCE-4253. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390652 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 18:41:08 +00:00
Thomas Graves
050fd3a117 MAPREDUCE-4647. We should only unjar jobjar if there is a lib directory in it. (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390557 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 15:22:21 +00:00
Harsh J
462ba6a3de MAPREDUCE-4686. hadoop-mapreduce-client-core fails compilation in Eclipse due to missing Avro-generated classes. Contributed by Chris Nauroth. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390446 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 12:52:41 +00:00
Arun Murthy
1eedee177e MAPREDUCE-4649. Ensure MapReduce JobHistory Daemon doens't assume HADOOP_YARN_HOME and HADOOP_MAPRED_HOME are the same. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390224 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-25 23:49:09 +00:00
Konstantin Shvachko
33a3efcd18 MAPREDUCE-4651. Benchmarking random reads with DFSIO. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390159 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-25 21:32:29 +00:00
Konstantin Shvachko
b26ed9d6f8 MAPREDUCE-4645. Provide a random seed to Slive to make the sequence of file names deterministic. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389568 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-24 20:54:20 +00:00
Thomas White
d89828573d HADOOP-8780. Update DeprecatedProperties apt file. Contributed by Ahmed Radwan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384833 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-14 16:05:30 +00:00
Aaron Myers
d8e386f01a HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384627 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-14 01:44:23 +00:00
Vinod Kumar Vavilapalli
229a79bbc3 MAPREDUCE-4646. Fixed MR framework to send diagnostic information correctly to clients in case of failed jobs also. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383709 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-12 00:59:15 +00:00
Eli Collins
dc1fd84bd1 HDFS-3910. DFSTestUtil#waitReplication should timeout. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383618 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-11 21:24:12 +00:00
Thomas White
3b46295c28 MAPREDUCE-4607. Race condition in ReduceTask completion can result in Task being incorrectly failed. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383422 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-11 14:04:03 +00:00
Alejandro Abdelnur
a7998921a7 MAPREDUCE-4642. MiniMRClientClusterFactory should not use job.setJar() (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382093 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-07 17:05:11 +00:00
Robert Joseph Evans
77805c7ae4 MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382088 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-07 16:45:03 +00:00
Arun Murthy
0056914773 MAPREDUCE-4638. MR AM supplies MapReduce jars in classpath rather than rely on YARN. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381868 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-07 04:56:32 +00:00
Robert Joseph Evans
09038b741d MAPREDUCE-4641. Exception in commitJob marks job as successful in job history (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381784 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 22:20:15 +00:00
Robert Joseph Evans
5dcb594cb8 MAPREDUCE-4633. history server doesn't set permissions on all subdirs (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381614 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 14:20:43 +00:00
Eli Collins
aa049397f1 Fix MAPREDUCE-4580 build breakage. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381315 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 19:34:03 +00:00
Suresh Srinivas
60af999c73 YARN-84. Use Builder to build RPC server. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380921 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-04 22:24:04 +00:00
Konstantin Shvachko
ab986d7cf6 MAPREDUCE-2786. Add compression option for TestDFSIO. Contributed by Plamen Jeliazkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380310 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-03 18:54:27 +00:00
Jonathan Turner Eagles
94129df354 MAPREDUCE-4604. In mapred-default, mapreduce.map.maxattempts & mapreduce.reduce.maxattempts defaults are set to 4 as well as mapreduce.job.maxtaskfailures.per.tracker. (Ravi Prakash via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379617 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 21:25:25 +00:00
Thomas Graves
25e96e455b MAPREDUCE-4611. MR AM dies badly when Node is decommissioned (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379599 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:43:46 +00:00
Robert Joseph Evans
63f941d2ad MAPREDUCE-4614. Simplify debugging a job's tokens (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379595 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:37:41 +00:00
Robert Joseph Evans
015256524c MAPREDUCE-4612. job summary file permissions not set when its created (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379584 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:25:27 +00:00
Vinod Kumar Vavilapalli
65e447cd8b MAPREDUCE-4569. Fixed TestHsWebServicesJobsQuery to pass on JDK7 by not depending on test order. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379146 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 21:02:55 +00:00
Robert Joseph Evans
6f6e170325 HADOOP-8726. The Secrets in Credentials are not available to MR tasks (daryn and Benoy Antony via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379100 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 19:58:07 +00:00
Thomas White
2f1a133dcf MAPREDUCE-4610. Support deprecated mapreduce.job.counters.limit property in MR2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379022 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 17:12:14 +00:00
Thomas White
95af351615 MAPREDUCE-4608. hadoop-mapreduce-client is missing some dependencies. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378873 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 09:57:45 +00:00
Vinod Kumar Vavilapalli
cfe25b8828 MAPREDUCE-4579. Split TestTaskAttempt into two so as to pass tests on jdk7. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377943 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-28 02:03:26 +00:00
Siddharth Seth
24e47ebc18 MAPREDUCE-4580. Change MapReduce to use the yarn-client module. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377922 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-28 00:40:02 +00:00
Siddharth Seth
ffd2e01604 YARN-39. RM-NM secret-keys should be randomly generated and rolled every so often. (Contributed by Vinod Kumar Vavilapalli and Siddharth Seth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377180 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-25 02:18:49 +00:00
Alejandro Abdelnur
7b541d619f MAPREDUCE-4408. allow jobs to set a JAR that is in the distributed cached (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377149 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-24 23:36:55 +00:00
Robert Joseph Evans
aeb364d8b5 HADOOP-8225. DistCp fails when invoked by Oozie (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376618 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-23 18:06:56 +00:00
Alejandro Abdelnur
f909a1d4d7 MAPREDUCE-4470. Fix TestCombineFileInputFormat.testForEmptyFile (ikatsov via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376325 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-23 00:22:14 +00:00
Aaron Myers
0555a2145d MAPREDUCE-4577. HDFS-3672 broke TestCombineFileInputFormat.testMissingBlocks() test. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376297 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-22 22:43:50 +00:00
Alejandro Abdelnur
dc33a0765c MAPREDUCE-4068. Jars in lib subdirectory of the submittable JAR are not added to the classpath (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376253 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-22 21:18:34 +00:00
Robert Joseph Evans
ca2dc3e785 MAPREDUCE-4570. ProcfsBasedProcessTree#constructProcessInfo() prints a warning if procfsDir/<pid>/stat is not found. (Ahmed Radwan via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375687 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-21 17:46:07 +00:00
Robert Joseph Evans
0c2887b617 MAPREDUCE-3506. Calling getPriority on JobInfo after parsing a history log with JobHistoryParser throws a NullPointerException (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375602 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-21 15:20:21 +00:00
Thomas Graves
7fc6ad661d YARN-25. remove old aggregated logs (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374424 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 20:29:38 +00:00
Daryn Sharp
8fa10b184e HADOOP-7967. Need generalized multi-token filesystem support (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374271 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 14:05:11 +00:00
Thomas White
3204b1f450 MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name. Contributed by Jarek Jarcec Cecho. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373823 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-16 13:26:12 +00:00
Tsz-wo Sze
3cab01ba6e HADOOP-8700. Use enum to define the checksum constants in DataChecksum. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373683 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-16 01:32:10 +00:00
Alejandro Abdelnur
8fcad7e8e9 MAPREDUCE-4511. Add IFile readahead (ahmed via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373669 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 23:08:40 +00:00
Thomas Graves
dfad16fed2 MAPREDUCE-4053. Counters group names deprecation is wrong, iterating over group names deprecated names don't show up (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372636 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-13 21:48:33 +00:00
Alejandro Abdelnur
4bca22005f HADOOP-8681. add support for HTTPS to the web UIs. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371525 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-09 22:52:56 +00:00
Arun Murthy
e1fdf62123 YARN-1. Promote YARN to be a sub-project of Apache Hadoop. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370666 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-08 05:22:27 +00:00
Alejandro Abdelnur
117145f1ad MAPREDUCE-4498. Remove hsqldb jar from Hadoop runtime classpath. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1369906 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-06 18:14:47 +00:00
Jonathan Turner Eagles
72789336b2 MAPREDUCE-4503. Should throw InvalidJobConfException if duplicates found in cacheArchives or cacheFiles (Robert Evans via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1369197 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-03 20:29:29 +00:00
Siddharth Seth
3eb0cb27e8 MAPREDUCE-3289. Make use of fadvise in the NM's shuffle handler. (Contributed by Todd Lipcon and Siddharth Seth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1368718 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-02 21:55:37 +00:00
Thomas Graves
28320c5220 MAPREDUCE-4504. SortValidator writes to wrong directory (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1368294 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-01 22:14:13 +00:00
Jonathan Turner Eagles
5938c86352 MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name) (Robert Evans via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367789 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 21:26:08 +00:00
Thomas Graves
9d42fb2e8e MAPREDUCE-4457. mr job invalid transition TA_TOO_MANY_FETCH_FAILURE at FAILED (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367771 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 20:52:41 +00:00
Thomas Graves
735b50e8bd MAPREDUCE-4493. Distibuted Cache Compatability Issues (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367713 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 19:20:03 +00:00
Robert Joseph Evans
a37272e0e3 MAPREDUCE-4496. AM logs link is missing user name (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367601 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 15:08:47 +00:00
Thomas Graves
e9bf6c56c5 MAPREDUCE-4456. LocalDistributedCacheManager can get an ArrayIndexOutOfBounds when creating symlinks (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367581 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 14:46:57 +00:00
Thomas Graves
a3e695ab2c MAPREDUCE-4375. Show Configuration Tracability in MR UI (bobby via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367539 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 13:39:41 +00:00
Thomas Graves
d87b545165 MAPREDUCE-4423. Potential infinite fetching of map output (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366258 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-27 01:48:09 +00:00
Alejandro Abdelnur
9d16c9354b MAPREDUCE-4417. add support for encrypted shuffle (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365979 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-26 13:23:05 +00:00