big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,836 Commits 2 Branches 2 Tags 264 MiB
552e081a07
Commit Graph

395 Commits

Author SHA1 Message Date
Arun Murthy
f27e484636 YARN-749. Rename ResourceRequest.(get,set)HostName to ResourceRequest.(get,set)ResourceName. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488806 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-02 22:53:58 +00:00
Vinod Kumar Vavilapalli
a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He. 
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 21:43:17 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Alejandro Abdelnur
781e82ca9a YARN-392. Make it possible to specify hard locality constraints in resource requests. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488326 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 17:26:18 +00:00
Siddharth Seth
2692675fc3 YARN-684. ContainerManager.startContainer should use ContainerTokenIdentifier instead of the entire Container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488085 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 04:13:16 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Siddharth Seth
b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli
29902cd53c YARN-638. Modified ResourceManager to restore RMDelegationTokens after restarting. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487720 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:14:26 +00:00
Vinod Kumar Vavilapalli
5420f287cc YARN-578. Fixed NM to use SecureIOUtils for reading and aggregating logs. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487672 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 23:14:59 +00:00
Jason Darrell Lowe
31c96829c7 YARN-512. Log aggregation root directory check is more expensive than it needs to be. Contributed by Maysam Yabandeh 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487498 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 14:25:04 +00:00
Vinod Kumar Vavilapalli
259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 03:22:54 +00:00
Vinod Kumar Vavilapalli
61885df2fa YARN-711. Copied BuilderUtil methods in individual API records as BuilderUtils is going to be dismantled. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485888 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 22:47:33 +00:00
Vinod Kumar Vavilapalli
43876770d9 YARN-563. Add the concept of an application-type for each application. Contributed by Mayank Bansal. 
MAPREDUCE-5246. Specify application-type at the time of job submission after YARN-563. Contributed by Mayank Bansal.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485790 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 18:02:04 +00:00
Vinod Kumar Vavilapalli
1fac1ac98c YARN-708. Moved RecordFactory classes to hadoop-yarn-api, and put some miscellaneous fixes to the interfaces. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485478 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 22:51:04 +00:00
Christopher Douglas
f9ab715880 YARN-568. Followup; remove an unused field added to Allocation 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484375 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-20 05:46:43 +00:00
Vinod Kumar Vavilapalli
ca80246731 YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 06:36:33 +00:00
Robert Joseph Evans
3641f938f6 YARN-690. RM exits on token cancel/renew problems (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483578 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-16 21:41:49 +00:00
Vinod Kumar Vavilapalli
065747efab YARN-628. Fix the way YarnRemoteException is being unrolled to extract out the underlying exception. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483207 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-16 06:58:34 +00:00
Siddharth Seth
1a119f87b4 YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 04:11:55 +00:00
Vinod Kumar Vavilapalli
68148989bf YARN-590. Added an optional mesage to be returned by ResourceMaanger when RM asks an RM to shutdown/resync etc so that NMs can log this message locally for better debuggability. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481234 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 23:37:44 +00:00
Vinod Kumar Vavilapalli
a35c7fd80b YARN-663. Changed ResourceTracker API and LocalizationProtocol API to throw YarnRemoteException and IOException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481215 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:58:59 +00:00
Vinod Kumar Vavilapalli
2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Alejandro Abdelnur
f19465f72f YARN-598. Add virtual cores to queue metrics. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480816 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 22:23:50 +00:00
Alejandro Abdelnur
51ccb87031 YARN-655. Fair scheduler metrics should subtract allocated memory from available memory. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480809 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 22:15:58 +00:00
Alejandro Abdelnur
e0562e3d07 YARN-637. FS: maxAssign is not honored. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480802 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 21:52:03 +00:00
Alejandro Abdelnur
c7bfd17ad1 YARN-507. Add interface visibility and stability annotations to FS interfaces/classes. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480799 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 21:47:21 +00:00
Christopher Douglas
505fe26539 YARN-568. Add support for work preserving preemption to the FairScheduler. 
Contributed by Carlo Curino and Sandy Ryza


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480778 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 21:21:42 +00:00
Vinod Kumar Vavilapalli
9c4f86879c YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli
0727ecaf50 YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli
74231f0276 YARN-582. Changed ResourceManager to recover Application token and client tokens for app attempt so that RM can be restarted while preserving current applications. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480168 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 06:20:22 +00:00
Vinod Kumar Vavilapalli
9eb53b9c84 YARN-632. Changed ContainerManager api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479740 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 00:42:48 +00:00
Vinod Kumar Vavilapalli
92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli
c0fa9d2231 YARN-645. Moved RMDelegationTokenSecretManager from yarn-server-common to yarn-server-resourcemanager where it really belongs. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479022 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-04 01:12:17 +00:00
Vinod Kumar Vavilapalli
fc275fed66 YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 22:44:12 +00:00
Vinod Kumar Vavilapalli
46b2898a1a YARN-618. Modified RM_INVALID_IDENTIFIER to be -1 instead of zero. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478230 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-01 23:23:20 +00:00
Vinod Kumar Vavilapalli
6de09af244 YARN-599. Refactoring submitApplication in ClientRMService and RMAppManager to separate out various validation checks depending on whether they rely on RM configuration or not. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477478 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-30 05:44:06 +00:00
Suresh Srinivas
27d4592771 YARN-506. Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477408 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 23:09:28 +00:00
Vinod Kumar Vavilapalli
d9ce42479e YARN-576. Modified ResourceManager to reject NodeManagers that don't satisy minimum resource requirements. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476824 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 20:03:37 +00:00
Suresh Srinivas
fe3853484e HADOOP-9290. Some tests cannot load native library on windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476586 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:24:02 +00:00
Vinod Kumar Vavilapalli
95fa3e82e4 YARN-363. Add webapps/proxy directory without which YARN proxy-server fails when started in stand-alone mode. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476330 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 18:24:30 +00:00
Vinod Kumar Vavilapalli
f0ac5a849c YARN-591. Moved RM recovery related records out of public API as they do not belong there. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476324 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 18:02:45 +00:00
Vinod Kumar Vavilapalli
635f96e74e YARN-562. Missed files from previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476038 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 04:42:34 +00:00
Vinod Kumar Vavilapalli
fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Thomas White
d10428cab2 YARN-289. Fair scheduler allows reservations that won't fit on node. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475681 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 09:15:51 +00:00
Thomas White
a43c344eee YARN-595. Refactor fair scheduler to use common Resources. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475670 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 08:33:43 +00:00
Vinod Kumar Vavilapalli
448df18646 YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah. 
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 05:42:22 +00:00
Siddharth Seth
be256a1819 YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471814 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 02:46:47 +00:00
Vinod Kumar Vavilapalli
a2f97fd9cb YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471608 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 19:42:47 +00:00
Vinod Kumar Vavilapalli
cd975d4b64 YARN-581. Added a test to verify that app delegation tokens are restored after RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471187 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 22:49:44 +00:00
Vinod Kumar Vavilapalli
27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00
Vinod Kumar Vavilapalli
ffd7dbb34a YARN-583. Moved application level local resources to be localized under the filecache sub-directory under application directory. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470812 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 04:37:45 +00:00
Bikas Saha
e24a9b83f1 YARN-594. Update test and add comments in YARN-534 (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470243 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 22:04:08 +00:00
Vinod Kumar Vavilapalli
a91067fc5e YARN-542. Changed the default global AM max-attempts value to be not one. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470094 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 00:35:29 +00:00
Vinod Kumar Vavilapalli
c570309b07 YARN-547. Fixed race conditions in public and private resource localization which used to cause duplicate downloads. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470076 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 22:35:43 +00:00
Bikas Saha
edcfd4527c YARN-593. container launch on Windows does not correctly populate classpath with new process's environment variables and localized resources (Chris Nauroth via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469998 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 19:29:22 +00:00
Vinod Kumar Vavilapalli
44bf8525a5 YARN-493. Fixed some shell related flaws in YARN on Windows. Contributed by Chris Nauroth. 
HADOOP-9486. Promoted Windows and Shell related utils from YARN to Hadoop Common. Contributed by Chris Nauroth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469667 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 02:14:58 +00:00
Vinod Kumar Vavilapalli
8e1c2823fc YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong. 
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 01:33:13 +00:00
Alejandro Abdelnur
df55edd5de YARN-482. FS: Extend SchedulingMode to intermediate queues. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469506 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 18:11:21 +00:00
Vinod Kumar Vavilapalli
d219c904d3 YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469492 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 17:49:22 +00:00
Bikas Saha
ef9f251679 YARN-514.Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469059 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 20:19:43 +00:00
Vinod Kumar Vavilapalli
0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:54:09 +00:00
Arun Murthy
c2592021f3 YARN-412. Fixed FifoScheduler to check hostname of a NodeManager rather than its host:port during scheduling which caused incorrect locality for containers. Contributed by Roger Hoover. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467244 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-12 12:12:08 +00:00
Vinod Kumar Vavilapalli
e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong. 
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:28:51 +00:00
Vinod Kumar Vavilapalli
4234bc87b3 YARN-539. Addressed memory leak of LocalResource objects NM when a resource localization fails. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466756 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 02:08:11 +00:00
Vinod Kumar Vavilapalli
2e3b56f6e9 YARN-495. Changed NM reboot behaviour to be a simple resync - kill all containers and re-register with RM. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466752 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 02:00:47 +00:00
Vinod Kumar Vavilapalli
add00d6d74 YARN-487. Modify path manipulation in LocalDirsHandlerService to let TestDiskFailures pass on Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466746 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 01:00:38 +00:00
Vinod Kumar Vavilapalli
7d00d3d20f YARN-534. Change RM restart recovery to also account for AM max-attempts configuration after the restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466208 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 20:16:19 +00:00
Vinod Kumar Vavilapalli
3a54a5653b YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466196 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 19:56:10 +00:00
Vinod Kumar Vavilapalli
b96d18bd23 YARN-99. Modify private distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465853 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 01:33:38 +00:00
Bikas Saha
16fedf5473 YARN-479. NM retry behavior for connection to RM should be similar for lost heartbeats (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465731 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-08 19:17:16 +00:00
Bikas Saha
bc6777dd5b YARN-193. Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465067 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 17:45:44 +00:00
Jason Darrell Lowe
18757c1267 YARN-525. make CS node-locality-delay refreshable. Contributed by Thomas Graves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465009 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 15:30:24 +00:00
Vinod Kumar Vavilapalli
0836d68abf YARN-532. Change RMAdmin and Localization client protocol PB implementations to implement closeable so that they can be stopped when needed via RPC.stopProxy(). Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464788 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 23:30:15 +00:00
Vinod Kumar Vavilapalli
1fd462b118 YARN-536. Removed the unused objects ContainerStatus and ContainerStatus from Container which also don't belong to the container. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464271 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 03:41:18 +00:00
Eli Collins
2d02b7741d Revert YARN-516 per HADOOP-9357. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464181 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 19:42:12 +00:00
Vinod Kumar Vavilapalli
3e9200ddde YARN-101. Fix NodeManager heartbeat processing to not lose track of completed containers in case of dropped heartbeats. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464105 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 16:57:07 +00:00
Vinod Kumar Vavilapalli
e67e3ff05d YARN-467. Modify public distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463823 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 05:00:28 +00:00
Bikas Saha
d40b154d27 YARN-382. SchedulerUtils improve way normalizeRequest sets the resource capabilities (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463653 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 18:07:07 +00:00
Vinod Kumar Vavilapalli
bcce9354bc YARN-447. Move ApplicationComparator in CapacityScheduler to use comparator in ApplicationId. Contributed by Nemon Lou. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463405 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 06:59:13 +00:00
Vinod Kumar Vavilapalli
17be263e9b YARN-516. Fix failure in TestContainerLocalizer caused by HADOOP-9357. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463362 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 02:16:35 +00:00
Vinod Kumar Vavilapalli
7ca9fe73a0 YARN-309. Changed NodeManager to obtain heart-beat interval from the ResourceManager. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463346 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 01:04:07 +00:00
Jason Darrell Lowe
2fd56e33b4 YARN-515. Node Manager not getting the master key. Contributed by Robert Joseph Evans 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462632 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-29 20:20:19 +00:00
Thomas Graves
01aabf7363 YARN-460. CS user left in list of active users for the queue even when application finished (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462486 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-29 14:36:53 +00:00
Bikas Saha
520033b1cd YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462271 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-28 19:44:28 +00:00
Alejandro Abdelnur
967b62f5d2 YARN-24. Nodemanager fails to start if log aggregation enabled and namenode unavailable. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461891 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-27 23:14:05 +00:00
Vinod Kumar Vavilapalli
d0bbff6c32 YARN-209. Fix CapacityScheduler to trigger application-activation when the cluster capacity changes. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461773 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-27 18:38:28 +00:00
Thomas White
414458ff9d YARN-496. Fair scheduler configs are refreshed inconsistently in reinitialize. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461614 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-27 14:42:41 +00:00
Vinod Kumar Vavilapalli
d60c2fa17f YARN-474. Fix CapacityScheduler to trigger application-activation when am-resource-percent configuration is refreshed. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461402 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-27 03:44:34 +00:00
Siddharth Seth
66e90b205a YARN-440. Flatten RegisterNodeManagerResponse. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461256 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 18:20:36 +00:00
Alejandro Abdelnur
e74d1f0435 YARN-469. Make scheduling mode in FS pluggable. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460961 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 03:25:26 +00:00
Vinod Kumar Vavilapalli
0757364014 YARN-378. Fix RM to make the AM max attempts/retries to be configurable per application by clients. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460895 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 21:11:33 +00:00
Siddharth Seth
18e08a8f51 YARN-439. Flatten NodeHeartbeatResponse. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460811 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 18:28:50 +00:00
Siddharth Seth
6a482a88b8 YARN-71. Fix the NodeManager to clean up local-dirs on restart. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460808 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 18:23:59 +00:00
Hitesh Shah
b15b44e722 YARN-470. Support a way to disable resource monitoring on the NodeManager. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460001 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-22 21:28:31 +00:00
Hitesh Shah
0cda8267d5 YARN-491. TestContainerLogsPage fails on Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459526 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 21:04:59 +00:00
Hitesh Shah
92fb7cdcde YARN-488. TestContainerManagerSecurity fails on Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459514 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 20:38:43 +00:00
Hitesh Shah
1bd345d6e3 YARN-396. Rationalize AllocateResponse in RM Scheduler API. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459040 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-20 20:44:35 +00:00
Jason Darrell Lowe
67699c2d18 YARN-200. yarn log does not output all needed information, and is in a binary format. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458466 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-19 19:29:15 +00:00
Hitesh Shah
2ba66a54f5 YARN-196. Nodemanager should be more robust in handling connection failure to ResourceManager when a cluster is started. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457038 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-15 17:58:58 +00:00
Robert Joseph Evans
9ec88e8e29 YARN-468. coverage fix for org.apache.hadoop.yarn.server.webproxy.amfilter (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456458 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-14 14:52:33 +00:00
Vinod Kumar Vavilapalli
44eb4c31b8 YARN-198. Added a link to RM pages from the NodeManager web app. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1455800 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-13 04:24:35 +00:00
Thomas Graves
5f2c518c95 YARN-443. allow OS scheduling priority of NM to be different than the containers it launches (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1454411 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-08 14:46:26 +00:00
Hitesh Shah
2e4382b7c3 YARN-429. capacity-scheduler config missing from yarn-test artifact. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453501 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 19:26:57 +00:00
Suresh Srinivas
638801cce1 HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 19:15:18 +00:00
Jonathan Turner Eagles
21066255da YARN-227. Application expiration difficult to debug for end-users (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453080 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-05 23:12:17 +00:00
Jason Darrell Lowe
5889f54ad1 YARN-345. Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452548 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-04 21:47:39 +00:00
Siddharth Seth
83d8065867 YARN-376. Fixes a bug which would prevent the NM knowing about completed containers and applications. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451473 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-01 05:59:54 +00:00
Alejandro Abdelnur
5d6eca08bd MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does not exist. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451452 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-01 01:28:18 +00:00
Kihwal Lee
27c8a87a54 YARN-269. Resource Manager not logging the health_check_script result when taking it out. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451341 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-28 21:02:06 +00:00
Robert Joseph Evans
0057600a82 YARN-426. Failure to download a public resource prevents further downloads (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450807 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-27 15:30:10 +00:00
Siddharth Seth
7ea82ca558 YARN-365. Change NM heartbeat handling to not generate a scheduler event on each heartbeat. (Contributed by Xuan Gong) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450007 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-26 03:32:10 +00:00
Jonathan Turner Eagles
320f7bdcb8 YARN-236. RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448406 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-20 20:57:22 +00:00
Thomas Graves
efaaf58605 YARN-400. RM can return null application resource usage report leading to NPE in client (Jason Lowe via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448241 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-20 15:35:06 +00:00
Alejandro Abdelnur
6bcbf492a2 HADOOP-9117. replace protoc ant plugin exec with a maven plugin. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445956 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 22:13:44 +00:00
Siddharth Seth
7bc6040ebb YARN-391. Formatting fixes for LCEResourceHandler classes. Contributed by Steve Loughran) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445882 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 19:40:36 +00:00
Thomas Graves
666667464a YARN-249. Capacity Scheduler web page should show list of active users per queue like it used to (in 1.x) (Ravi Prakash via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445673 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 15:40:01 +00:00
Tsz-wo Sze
a63d50f79a YARN-377. Use the new StringUtils methods added by HADOOP-9252 and fix TestContainersMonitor. Contributed by Chris Nauroth 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443796 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-07 23:53:49 +00:00
Siddharth Seth
ab16a37572 YARN-355. Fixes a bug where RM app submission could jam under load. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443131 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 19:03:52 +00:00
Daryn Sharp
d272918426 YARN-357. App submission should not be synchronized (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443016 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 15:26:30 +00:00
Arun Murthy
38bb4ad738 YARN-370. Fix SchedulerUtils to correctly round up the resource for containers. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442840 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 04:24:56 +00:00
Siddharth Seth
5d679c4f43 YARN-360. Allow apps to concurrently register tokens for renewal. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442441 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-05 00:26:16 +00:00
Hitesh Shah
b1baf11fb2 YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441668 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-02 00:43:54 +00:00
Thomas Graves
6384684299 YARN-343. Capacity Scheduler maximum-capacity value -1 is invalid (Xuan Gong via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440457 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-30 15:15:23 +00:00
Jason Darrell Lowe
2e9e7601e1 YARN-354. WebAppProxyServer exits immediately after startup. Contributed by Liang Xie 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437775 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-23 22:05:55 +00:00
Thomas White
345bcee066 YARN-319. Submitting a job to a fair scheduler queue for which the user does not have permission causes the client to wait forever. Contributed by shenhong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437336 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-23 10:53:05 +00:00
Hitesh Shah
29ddce96a2 YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437245 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-23 00:57:37 +00:00
Alejandro Abdelnur
f85b583731 YARN-302. Fair scheduler assignmultiple should default to false. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434996 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 00:48:22 +00:00
Todd Lipcon
e322e0a05f HADOOP-9215. when using cmake-2.6, libhadoop.so doesn't get created (only libhadoop.so.1.0.0). Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434530 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-17 01:37:41 +00:00
Siddharth Seth
6a2f2551fd YARN-135. Client tokens should be per app-attempt, and should be unregistered on App-finish. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433570 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 18:33:38 +00:00
Thomas White
72e631098d YARN-336. Fair scheduler FIFO scheduling within a queue only allows 1 app at a time. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433526 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 17:09:48 +00:00
Thomas White
8447557333 YARN-335. Fair scheduler doesn't check whether rack needs containers before assigning to node. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433484 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 16:12:33 +00:00
Hitesh Shah
d7f9f9d177 YARN-330. Fix flakey test: TestNodeManagerShutdown#testKillContainersOnShutdown. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433138 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-14 21:27:39 +00:00
Thomas Graves
ce9b0a05b0 YARN-334. Maven RAT plugin is not checking all source files (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432931 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-14 14:34:51 +00:00
Thomas Graves
106e2e27ff YARN-325. RM CapacityScheduler can deadlock when getQueueInfo() is called and a container is completing (Arun C Murthy via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431070 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 21:00:47 +00:00
Siddharth Seth
ba5b19fb5d YARN-320. RM should always be able to renew its own tokens. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431020 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 19:33:06 +00:00
Arun Murthy
4539263971 YARN-2. Enhanced CapacityScheduler to account for CPU alongwith memory for multi-dimensional resource scheduling. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430682 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 05:08:23 +00:00
Vinod Kumar Vavilapalli
9385dd50c7 YARN-253. Fixed container-launch to not fail when there are no local resources to localize. Contributed by Tom White. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430269 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-08 13:20:07 +00:00
Vinod Kumar Vavilapalli
2cd41855d5 YARN-170. Change NodeManager stop to be reentrant. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429796 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-07 13:43:52 +00:00
Vinod Kumar Vavilapalli
917ff03b2c YARN-315. Using the common security token protobuf definition from hadoop common. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429748 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-07 11:08:21 +00:00
Robert Joseph Evans
64e4fb983e MAPREDUCE-4819. AM can rerun job after reporting final job status to the client (bobby and Bikas Saha via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429114 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:35:56 +00:00
Thomas Graves
dd8fc7e499 YARN-50. Implement renewal / cancellation of Delegation Tokens(Siddharth Seth via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429085 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:15:32 +00:00
Thomas White
8cd026d365 YARN-301. Fair scheduler throws ConcurrentModificationException when iterating over app's priorities. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428400 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 15:26:55 +00:00
Thomas White
0768f96a83 YARN-300. After YARN-271, fair scheduler can infinite loop and not schedule any application. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428387 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 15:07:22 +00:00
Thomas White
19a291a0d6 YARN-288. Fair scheduler queue doesn't accept any jobs when ACLs are configured. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428362 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 14:32:01 +00:00
Thomas White
ba90c9c867 YARN-192. Node update causes NPE in the fair scheduler. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428314 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 12:29:13 +00:00
Jason Darrell Lowe
16cf201042 YARN-293. Node Manager leaks LocalizerRunner object for every Container. Contributed by Robert Joseph Evans 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428095 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-02 22:41:03 +00:00
Robert Joseph Evans
4b9f0443cb YARN-225. Proxy Link in RM UI thows NPE in Secure mode (Devaraj K via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1426515 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-28 14:21:07 +00:00
Vinod Kumar Vavilapalli
5a408bdeac YARN-285. Added a temporary plugin interface for RM to be able to redirect to JobHistory server for apps that it no longer tracks. Contributed by Derek Dagit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425210 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-22 00:26:29 +00:00
Thomas Graves
a806a2aac6 YARN-280. RM does not reject app submission with invalid tokens (Daryn Sharp via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425079 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 18:49:47 +00:00
Alejandro Abdelnur
a276f68b92 YARN-283. Fair scheduler fails to get queue info without root prefix. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424998 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 16:20:12 +00:00
Alejandro Abdelnur
5014a4b133 YARN-282. Fair scheduler web UI double counts Apps Submitted. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424995 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 16:12:20 +00:00
Alejandro Abdelnur
f121d1e333 YARN-278. Fair scheduler maxRunningApps config causes no apps to make progress. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424989 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 16:07:11 +00:00
Alejandro Abdelnur
39ec437268 YARN-272. Fair scheduler log messages try to print objects without overridden toString methods. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424984 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 16:01:48 +00:00
Thomas White
ee007d3f38 YARN-271. Fair scheduler hits IllegalStateException trying to reserve different apps on same node. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424945 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 15:10:24 +00:00
Luke Lu
1a49c85438 YARN-223. Update process tree instead of getting new process trees. (Radim Kolar via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424244 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-20 00:20:53 +00:00
Arun Murthy
6cd0736cc5 YARN-230. RM Restart phase 1 - includes support for saving/restarting all applications on an RM bounce. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1423758 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-19 04:21:18 +00:00
Alejandro Abdelnur
595d52cf37 YARN-254. Update fair scheduler web UI for hierarchical queues. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1423742 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-19 03:04:39 +00:00
Alejandro Abdelnur
5032a694ed YARN-3. Add support for CPU isolation/monitoring of containers. (adferguson via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1423706 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-18 22:58:32 +00:00
Alejandro Abdelnur
be8ef2ec33 YARN-264. y.s.rm.DelegationTokenRenewer attempts to renew token even after removing an app. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1421575 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-13 22:16:29 +00:00
Thomas White
618fa6cd59 YARN-267. Fix fair scheduler web UI. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420704 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-12 14:35:01 +00:00
Jason Darrell Lowe
b72b36009a YARN-266. RM and JHS Web UIs are blank because AppsBlock is not escaping string properly. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420232 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-11 15:45:10 +00:00
Jason Darrell Lowe
2d1237cbb0 YARN-258. RM web page UI shows Invalid Date for start and finish times. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1417948 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-06 15:39:30 +00:00
Thomas White
235749a8ab YARN-72. NM should handle cleaning up containers when it shuts down. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1416484 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-03 12:12:11 +00:00
Jason Darrell Lowe
1573c1159a YARN-251. Proxy URI generation fails for blank tracking URIs. Contributed by Tom White 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1415742 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-30 17:07:40 +00:00
Thomas White
ae6f1123f5 YARN-187. Add hierarchical queues to the fair scheduler. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1415592 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-30 12:03:25 +00:00
Arun Murthy
1943fdbec6 YARN-229. Remove old unused RM recovery code. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1415029 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-29 01:56:33 +00:00
Thomas White
dd0ba9c0cc MAPREDUCE-4778. Fair scheduler event log is only written if directory exists on HDFS. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414729 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-28 14:40:40 +00:00
Thomas White
fcbd55d092 YARN-222. Fair scheduler should create queue for each user by default. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414655 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-28 12:53:29 +00:00
Thomas White
d9050e1208 YARN-224. Fair scheduler logs too many nodeUpdate INFO messages. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414647 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-28 12:27:30 +00:00
Robert Joseph Evans
f0a6d06485 YARN-219. NM should aggregate logs when application finishes. (bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411289 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-19 16:41:11 +00:00
Alejandro Abdelnur
576f96f0ac YARN-184. Remove unnecessary locking in fair scheduler, and address findbugs excludes. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410826 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-18 05:00:56 +00:00
Robert Joseph Evans
a23e8b0929 YARN-151. Browser thinks RM main page JS is taking too long (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410564 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-16 20:56:08 +00:00
Robert Joseph Evans
ab5039d4f1 YARN-214. RMContainerImpl does not handle event EXPIRE at state RUNNING (jeagles via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410522 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-16 18:26:19 +00:00
Robert Joseph Evans
1dedca8c9a YARN-188. Coverage fixing for CapacityScheduler (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409827 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-15 15:15:51 +00:00
Jason Darrell Lowe
397c261433 YARN-216. Remove jquery theming support. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409313 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-14 19:22:11 +00:00
Jason Darrell Lowe
6db6e00649 YARN-212. NM state machine ignores an APPLICATION_CONTAINER_FINISHED event when it shouldn't. Contributed by Nathan Roberts 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408812 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-13 15:59:13 +00:00
Jonathan Turner Eagles
9e3af0ccd2 YARN-206. TestApplicationCleanup.testContainerCleanup occasionally fails. (jlowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407606 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 19:52:28 +00:00
Thomas White
933a6d2c1e YARN-183. Clean up fair scheduler code. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407433 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 12:38:10 +00:00
Robert Joseph Evans
8a1f333707 YARN-186. Coverage fixing LinuxContainerExecutor (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407171 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-08 17:22:52 +00:00
Thomas White
5a324b18ed YARN-169. Update log4j.appender.EventCounter to use org.apache.hadoop.log.metrics.EventCounter. Contributed by Anthony Rojas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1406999 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-08 10:33:29 +00:00
Arun Murthy
fb5b96dfc3 YARN-201. Fix CapacityScheduler to be less conservative for starved off-switch requests. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1406834 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-07 21:56:18 +00:00
Vinod Kumar Vavilapalli
a124297cf0 YARN-189. Fixed a deadlock between RM's ApplicationMasterService and the dispatcher. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404431 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-01 00:32:37 +00:00
Robert Joseph Evans
ab8fa651f7 YARN-166. capacity scheduler doesn't allow capacity < 1.0 (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404247 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-31 16:56:23 +00:00
Robert Joseph Evans
9f55dd3a54 YARN-159. RM web ui applications page should be sorted to display last app first (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404228 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-31 16:22:34 +00:00
Robert Joseph Evans
c15c192ad0 YARN-165. RM should point tracking URL to RM web page for app when AM fails (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404211 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-31 15:47:11 +00:00
Thomas White
b54e794fb0 YARN-145. Add a Web UI to the fair share scheduler. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1402657 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-26 20:55:57 +00:00
Robert Joseph Evans
90ba993bc7 YARN-180. Capacity scheduler - containers that get reserved create container token to early (acmurthy and bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401703 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-24 14:16:49 +00:00
Thomas Graves
cc523683cf YARN-177. CapacityScheduler - adding a queue while the RM is running has wacky results (acmurthy vai tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401668 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-24 13:21:09 +00:00
Vinod Kumar Vavilapalli
ca4bc1a417 YARN-181. Fixed eclipse settings broken by capacity-scheduler.xml move via YARN-140. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401504 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-23 22:48:49 +00:00
Siddharth Seth
88d326f0a4 YARN-179. Fix some unit test failures. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401481 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-23 21:06:36 +00:00
Vinod Kumar Vavilapalli
4da8e2613e YARN-174. Modify NodeManager to pass the user's configuration even when rebooting. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401086 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-22 21:25:12 +00:00
Alejandro Abdelnur
d0a2d55b1a YARN-140. Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler. (ahmed via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400335 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 23:16:07 +00:00
Siddharth Seth
f79ae91414 YARN-136. Make ClientToAMTokenSecretManager part of RMContext (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400278 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 20:37:30 +00:00
Robert Joseph Evans
ae15ad6202 YARN-163. Retrieving container log via NM webapp can hang with multibyte characters in log (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400189 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 17:34:15 +00:00
Alejandro Abdelnur
a4f30ff53c reverting commit 1399898 (HADOOP-8887) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399972 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 05:38:26 +00:00
Alejandro Abdelnur
53c598ab11 HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399898 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-18 22:40:41 +00:00
Vinod Kumar Vavilapalli
6cfa710bad YARN-32. Fix TestApplicationTokens to not depend on test order and thus pass on JDK7. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399795 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-18 19:12:13 +00:00
Vinod Kumar Vavilapalli
ab727f4966 YARN-43. Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399066 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-17 01:24:50 +00:00
Thomas White
3d76be910d YARN-146. Add unit tests for computing fair share in the fair scheduler. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396972 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-11 10:07:01 +00:00
Siddharth Seth
0e2d93557e YARN-150. Fixes AppRejectedTransition does not unregister a rejected app-attempt from the ApplicationMasterService (Contributed by Bikas Saha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396429 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-10 00:08:25 +00:00