big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,690 Commits 2 Branches 2 Tags 264 MiB
d1963ad4bd
Commit Graph

402 Commits

Author SHA1 Message Date
Jason Darrell Lowe
0939c49368 MAPREDUCE-5317. Stale files left behind for failed jobs. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506154 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-23 16:52:24 +00:00
Arun Murthy
ac914f79bc YARN-918. Remove ApplicationAttemptId from RegisterApplicationMasterRequestProto. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504735 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-19 00:57:40 +00:00
Arun Murthy
32bc200d54 YARN-701. Use application tokens irrespective of secure or non-secure mode. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504604 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-18 19:03:48 +00:00
Bikas Saha
bf876f70fb MAPREDUCE-5398. MR changes for YARN-513 (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503934 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 22:51:12 +00:00
Jason Darrell Lowe
44ea6e3ad5 Back out revision 1503499 for MAPREDUCE-5317. Stale files left behind for failed jobs 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503744 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 15:05:16 +00:00
Jason Darrell Lowe
8a037edbfb MAPREDUCE-5317. Stale files left behind for failed jobs. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503499 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-15 22:24:37 +00:00
Jason Darrell Lowe
cf4d0cf189 MAPREDUCE-5358. MRAppMaster throws invalid transitions for JobImpl. Contributed by Devaraj K 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499425 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-03 14:41:18 +00:00
Vinod Kumar Vavilapalli
1a06175440 YARN-851. Share NMTokens using NMTokenCache (api-based) between AMRMClient and NMClient instead of memory based approach which is used currently. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495247 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-21 00:08:13 +00:00
Siddharth Seth
84862b54aa MAPREDUCE-5334. Fix failing unit tests - TestContainerLauncher, TestContainerLauncherImpl. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494820 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-20 00:35:51 +00:00
Jonathan Turner Eagles
1ab4659b6b MAPREDUCE-5084. fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and org.apache.hadoop.mapreduce.v2.hs.webapp (Aleksey Gorshkov via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494676 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-19 16:22:12 +00:00
Vinod Kumar Vavilapalli
243bcd367f YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 23:19:49 +00:00
Vinod Kumar Vavilapalli
f4d80e91ae YARN-841. Move Auxiliary service to yarn-api, annotate and document it. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494031 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 06:20:37 +00:00
Arun Murthy
acc0d3eb52 YARN-840. Moved ProtoUtils to yarn.api.records.pb.impl. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494027 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 05:47:54 +00:00
Siddharth Seth
2b14656ab5 YARN-805. Fix javadoc and annotations on classes in the yarn-api package. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493992 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 03:10:11 +00:00
Vinod Kumar Vavilapalli
5d1b453b85 YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 23:13:17 +00:00
Vinod Kumar Vavilapalli
7d0c43d7cf YARN-822. Renamed ApplicationToken to be AMRMToken, and similarly the corresponding TokenSelector and SecretManager. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493889 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 19:11:29 +00:00
Vinod Kumar Vavilapalli
cd30058193 YARN-837. Moved yarn.ClusterInfo into MapReduce project as it doesn't belong to YARN. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493676 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 08:13:48 +00:00
Vinod Kumar Vavilapalli
4662b8b443 YARN-833. Moved Graph and VisualizeStateMachine into yarn.state package. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493654 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 06:51:51 +00:00
Vinod Kumar Vavilapalli
b9efe6bd4a HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common for usage across all Hadoop projects. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493650 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 06:39:33 +00:00
Vinod Kumar Vavilapalli
d4a811edb2 YARN-826. Moved Clock and SystemClock into yarn.util package. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493639 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 04:49:13 +00:00
Arun Murthy
77e134d13f YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 02:27:18 +00:00
Alejandro Abdelnur
789c03438c MAPREDUCE-5171. Expose blacklisted nodes from the MR AM REST API. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493600 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 22:15:37 +00:00
Arun Murthy
98d97d316c YARN-787. Removed minimum resource from RegisterApplicationMasterResponse. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493509 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 13:59:41 +00:00
Vinod Kumar Vavilapalli
c2d0cf1b20 YARN-821. Renamed setFinishApplicationStatus to setFinalApplicationStatus in FinishApplicationMasterRequest for consistency. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493315 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-15 04:57:04 +00:00
Alejandro Abdelnur
079ed1c9e1 MAPREDUCE-5310. MRAM should not normalize allocation request capabilities. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493207 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 19:06:50 +00:00
Vinod Kumar Vavilapalli
fdc9412a81 YARN-806. Moved ContainerExitStatus from yarn.api to yarn.api.records. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493138 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 16:10:37 +00:00
Vinod Kumar Vavilapalli
c0cd68d8fb YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 23:25:14 +00:00
Vinod Kumar Vavilapalli
b64572b06b MAPREDUCE-5199. Removing ApplicationTokens file as it is no longer needed. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492848 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 20:20:33 +00:00
Vinod Kumar Vavilapalli
0928502029 YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran. 
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 15:54:38 +00:00
Bikas Saha
9fcfbf5f51 YARN-759. Create Command enum in AllocateResponse (bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490470 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-06 23:33:48 +00:00
Bikas Saha
2051fd5ee2 YARN-750. Allow for black-listing resources in YARN API and Impl in CS (acmurthy via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490392 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-06 18:46:59 +00:00
Vinod Kumar Vavilapalli
978012b9b6 YARN-755. Renamed AllocateResponse.reboot to AllocateResponse.resync. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489295 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 05:53:47 +00:00
Vinod Kumar Vavilapalli
a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth. 
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:05:50 +00:00
Jason Darrell Lowe
78daceaea4 MAPREDUCE-5283. Over 10 different tests have near identical implementations of AppContext. Committed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489271 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 01:50:15 +00:00
Vinod Kumar Vavilapalli
d33534c4fb YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He. 
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 00:34:18 +00:00
Siddharth Seth
a2205a3b72 YARN-720 and MAPREDUCE-5291. container-log4j.properties should not refer to mapreduce properties. Update MRApp to use YARN properties for log setup. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488829 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 02:35:34 +00:00
Arun Murthy
f27e484636 YARN-749. Rename ResourceRequest.(get,set)HostName to ResourceRequest.(get,set)ResourceName. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488806 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-02 22:53:58 +00:00
Vinod Kumar Vavilapalli
a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He. 
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 21:43:17 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Siddharth Seth
f3f7a11336 MAPREDUCE-5286. Change MapReduce to use ContainerTokenIdentifier instead of the entire Container in the startContainer call - YARN-684. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488087 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 04:17:33 +00:00
Vinod Kumar Vavilapalli
131bfc91a6 MAPREDUCE-5261. Reverting the patch as it is no longer needed. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488032 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 21:53:34 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Siddharth Seth
b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli
643155cbee MAPREDUCE-5270. Migrated MR app from using BuilderUtil factory methods to individual record factory methods. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1486271 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-25 01:46:14 +00:00
Vinod Kumar Vavilapalli
259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 03:22:54 +00:00
Vinod Kumar Vavilapalli
4a4e42d919 MAPREDUCE-5261. Fix issues in TestRMContainerAllocator after YARN-617. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485079 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 06:24:17 +00:00
Vinod Kumar Vavilapalli
9cf436aef5 MAPREDUCE-5257. Fix issues in TestContainerLauncherImpl after YARN-617. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484349 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-19 21:15:11 +00:00
Jason Darrell Lowe
ca2265b581 MAPREDUCE-4927. Historyserver 500 error due to NPE when accessing specific counters page for failed job. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483974 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 20:19:48 +00:00
Vinod Kumar Vavilapalli
ca80246731 YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 06:36:33 +00:00
Vinod Kumar Vavilapalli
4d8e350750 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 23:43:53 +00:00
Siddharth Seth
1a119f87b4 YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 04:11:55 +00:00
Vinod Kumar Vavilapalli
14d856be57 MAPREDUCE-5240. Fix a bug in MRAppMaster because of which OutputCommitter could not access credentials set by the user. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482175 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 02:48:53 +00:00
Vinod Kumar Vavilapalli
7359dc32d3 MAPREDUCE-4067. Changed MRClientProtocol api to throw IOException only (Xuan Gong via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481695 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-13 03:34:25 +00:00
Vinod Kumar Vavilapalli
2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Vinod Kumar Vavilapalli
9c4f86879c YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli
6e37a7e5e1 MAPREDUCE-5205. Fixed MR App to load tokens correctly. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479016 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-04 00:27:47 +00:00
Vinod Kumar Vavilapalli
71be8ad325 MAPREDUCE-5145. Changed default max-attempts to be more than one for MR jobs inline with YARN. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477889 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-01 00:42:32 +00:00
Vinod Kumar Vavilapalli
fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Siddharth Seth
e96077aaf4 MAPREDUCE-5181. RMCommunicator should not use AMToken from the env. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471816 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 02:50:16 +00:00
Vinod Kumar Vavilapalli
27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00
Thomas White
009af54d51 MAPREDUCE-5146. application classloader may be used too early to load classes. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470694 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-22 20:07:06 +00:00
Arun Murthy
794f9bb3e4 MAPREDUCE-5066. Added a timeout for the job.end.notification.url. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470216 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 19:18:24 +00:00
Thomas Graves
febc52be37 MAPREDUCE-5147. Maven build should create hadoop-mapreduce-client-app-VERSION.jar directly (Robert Parker via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470035 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 20:50:48 +00:00
Vinod Kumar Vavilapalli
8e1c2823fc YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong. 
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 01:33:13 +00:00
Vinod Kumar Vavilapalli
cb78a65a15 MAPREDUCE-5152. Make MR App to simply pass through the container from RM instead of extracting and populating information itself to start any container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469544 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 20:13:40 +00:00
Vinod Kumar Vavilapalli
0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:54:09 +00:00
Vinod Kumar Vavilapalli
e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong. 
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:28:51 +00:00
Jason Darrell Lowe
8e86ffd91e MAPREDUCE-5136. TestJobImpl->testJobNoTasks fails with IBM JAVA. Contributed by Amir Sanjar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467061 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:12:59 +00:00
Jason Darrell Lowe
d04e65b797 MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466948 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 16:27:24 +00:00
Siddharth Seth
6a1c41111e MAPREDUCE-5079. Changes job recovery to restore state directly from job history, instaed of simulating state machine events. Contributed by Jason Lowe and Robert Parker. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466767 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 04:52:38 +00:00
Arun Murthy
7f13207ed1 MAPREDUCE-5129. Allow tags to JobHistory for deeper analytics. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465188 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-06 05:30:56 +00:00
Jason Darrell Lowe
fc75d3f3dc MAPREDUCE-5086. MR app master deletes staging dir when sent a reboot command from the RM. Contributed by Jian He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464255 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 01:56:49 +00:00
Bikas Saha
520033b1cd YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462271 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-28 19:44:28 +00:00
Robert Joseph Evans
a0a844222d MAPREDUCE-4875. coverage fixing for org.apache.hadoop.mapred (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461146 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 14:16:13 +00:00
Vinod Kumar Vavilapalli
46315a2d91 MAPREDUCE-5062. Fix MR AM to read max-retries from the RM. Contributed by *Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460923 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 22:33:43 +00:00
Siddharth Seth
cd2c5fab8b MAPREDUCE-3872. Fix an event handling races in ContainerLauncherImpl. (Contributed by Robert Kanter) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459547 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 21:55:31 +00:00
Siddharth Seth
6610212dd3 MAPREDUCE-5078. TestMRAppMaster fails on Windows due to mismatched path separators. (Contributed by Chris Nauroth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459544 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 21:44:13 +00:00
Hitesh Shah
1bd345d6e3 YARN-396. Rationalize AllocateResponse in RM Scheduler API. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459040 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-20 20:44:35 +00:00
Robert Joseph Evans
f7df0cb7df MAPREDUCE-4972. Coverage fixing for org.apache.hadoop.mapreduce.jobhistory (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458906 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-20 16:05:33 +00:00
Robert Joseph Evans
7d7553c4eb MAPREDUCE-5042. Reducer unable to fetch for a map task that was recovered (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457119 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-15 21:09:25 +00:00
Robert Joseph Evans
1b6f0582fb YARN-237. Refreshing the RM page forgets how many rows I had in my Datatables (jian he via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456536 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-14 16:56:10 +00:00
Thomas Graves
7122cb5bf4 MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1454156 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-07 23:02:28 +00:00
Suresh Srinivas
638801cce1 HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 19:15:18 +00:00
Robert Joseph Evans
ec13f1eb3a MAPREDUCE-5043. Fetch failure processing can cause AM event queue to backup and eventually OOM (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452372 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-04 16:32:18 +00:00
Jonathan Turner Eagles
38c4e17ec1 MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451826 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-02 03:30:58 +00:00
Siddharth Seth
979fb054f8 MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450956 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-27 21:03:19 +00:00
Thomas White
0b73dde6ce MAPREDUCE-4951. Container preemption interpreted as task failure. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448615 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-21 11:56:08 +00:00
Jonathan Turner Eagles
c8f35bc3d2 MAPREDUCE-5009. Killing the Task Attempt slated for commit does not clear the value from the Task commitAttempt member (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1447965 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-19 23:35:56 +00:00
Siddharth Seth
43204f0c83 MAPREDUCE-5000. Fixes getCounters when speculating by fixing the selection of the best attempt for a task. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445871 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 19:20:31 +00:00
Jason Darrell Lowe
390d1fbef8 MAPREDUCE-4992. AM hangs in RecoveryService when recovering tasks with speculative attempts. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445456 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 02:11:36 +00:00
Jason Darrell Lowe
c2d6407070 MAPREDUCE-4989. JSONify DataTables input data for Attempts page. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445448 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 01:15:48 +00:00
Siddharth Seth
35832053bf MAPREDUCE-4671. AM does not tell the RM about container requests which are no longer needed. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443323 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-07 07:07:11 +00:00
Hitesh Shah
b1baf11fb2 YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441668 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-02 00:43:54 +00:00
Vinod Kumar Vavilapalli
28c308d5e8 MAPREDUCE-4893. Fixed MR ApplicationMaster to do optimal assignment of containers to get maximum locality. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440749 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-31 00:28:10 +00:00
Siddharth Seth
c163dc2fce MAPREDUCE-4838. Add additional fields like Locality, Avataar to the JobHistory logs. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439714 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-29 00:21:58 +00:00
Siddharth Seth
74ffc7a74d MAPREDUCE-4946. Fix a performance problem for large jobs by reducing the number of map completion event type conversions. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437103 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 19:07:08 +00:00
Siddharth Seth
6a2f2551fd YARN-135. Client tokens should be per app-attempt, and should be unregistered on App-finish. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433570 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 18:33:38 +00:00
Jason Darrell Lowe
dd6aa0cf12 MAPREDUCE-4936. JobImpl uber checks for cpu are wrong. Contributed by Arun C Murthy 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433445 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 14:45:08 +00:00
Thomas Graves
e7aeb9d129 MAPREDUCE-4934. Maven RAT plugin is not checking all source files (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432932 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-14 14:35:45 +00:00
Jason Darrell Lowe
b16dfc125d MAPREDUCE-4848. TaskAttemptContext cast error during AM recovery. Contributed by Jerry Chen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431131 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 22:56:09 +00:00