big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,432 Commits 2 Branches 2 Tags 264 MiB
3ee5949912
Commit Graph

216 Commits

Author SHA1 Message Date
Alejandro Abdelnur
c1b635ed48 YARN-326. Add multi-resource scheduling to the fair scheduler. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489070 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 17:33:55 +00:00
Arun Murthy
f27e484636 YARN-749. Rename ResourceRequest.(get,set)HostName to ResourceRequest.(get,set)ResourceName. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488806 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-02 22:53:58 +00:00
Vinod Kumar Vavilapalli
a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He. 
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 21:43:17 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Alejandro Abdelnur
781e82ca9a YARN-392. Make it possible to specify hard locality constraints in resource requests. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488326 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 17:26:18 +00:00
Siddharth Seth
2692675fc3 YARN-684. ContainerManager.startContainer should use ContainerTokenIdentifier instead of the entire Container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488085 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 04:13:16 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Siddharth Seth
b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli
29902cd53c YARN-638. Modified ResourceManager to restore RMDelegationTokens after restarting. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487720 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:14:26 +00:00
Vinod Kumar Vavilapalli
259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 03:22:54 +00:00
Vinod Kumar Vavilapalli
61885df2fa YARN-711. Copied BuilderUtil methods in individual API records as BuilderUtils is going to be dismantled. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485888 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 22:47:33 +00:00
Vinod Kumar Vavilapalli
43876770d9 YARN-563. Add the concept of an application-type for each application. Contributed by Mayank Bansal. 
MAPREDUCE-5246. Specify application-type at the time of job submission after YARN-563. Contributed by Mayank Bansal.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485790 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 18:02:04 +00:00
Christopher Douglas
f9ab715880 YARN-568. Followup; remove an unused field added to Allocation 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484375 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-20 05:46:43 +00:00
Vinod Kumar Vavilapalli
ca80246731 YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 06:36:33 +00:00
Robert Joseph Evans
3641f938f6 YARN-690. RM exits on token cancel/renew problems (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483578 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-16 21:41:49 +00:00
Vinod Kumar Vavilapalli
065747efab YARN-628. Fix the way YarnRemoteException is being unrolled to extract out the underlying exception. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483207 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-16 06:58:34 +00:00
Siddharth Seth
1a119f87b4 YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 04:11:55 +00:00
Vinod Kumar Vavilapalli
68148989bf YARN-590. Added an optional mesage to be returned by ResourceMaanger when RM asks an RM to shutdown/resync etc so that NMs can log this message locally for better debuggability. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481234 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 23:37:44 +00:00
Vinod Kumar Vavilapalli
a35c7fd80b YARN-663. Changed ResourceTracker API and LocalizationProtocol API to throw YarnRemoteException and IOException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481215 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:58:59 +00:00
Vinod Kumar Vavilapalli
2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Alejandro Abdelnur
f19465f72f YARN-598. Add virtual cores to queue metrics. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480816 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 22:23:50 +00:00
Alejandro Abdelnur
51ccb87031 YARN-655. Fair scheduler metrics should subtract allocated memory from available memory. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480809 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 22:15:58 +00:00
Alejandro Abdelnur
e0562e3d07 YARN-637. FS: maxAssign is not honored. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480802 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 21:52:03 +00:00
Alejandro Abdelnur
c7bfd17ad1 YARN-507. Add interface visibility and stability annotations to FS interfaces/classes. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480799 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 21:47:21 +00:00
Christopher Douglas
505fe26539 YARN-568. Add support for work preserving preemption to the FairScheduler. 
Contributed by Carlo Curino and Sandy Ryza


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480778 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 21:21:42 +00:00
Vinod Kumar Vavilapalli
9c4f86879c YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli
0727ecaf50 YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli
74231f0276 YARN-582. Changed ResourceManager to recover Application token and client tokens for app attempt so that RM can be restarted while preserving current applications. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480168 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 06:20:22 +00:00
Vinod Kumar Vavilapalli
92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli
c0fa9d2231 YARN-645. Moved RMDelegationTokenSecretManager from yarn-server-common to yarn-server-resourcemanager where it really belongs. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479022 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-04 01:12:17 +00:00
Vinod Kumar Vavilapalli
6de09af244 YARN-599. Refactoring submitApplication in ClientRMService and RMAppManager to separate out various validation checks depending on whether they rely on RM configuration or not. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477478 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-30 05:44:06 +00:00
Vinod Kumar Vavilapalli
d9ce42479e YARN-576. Modified ResourceManager to reject NodeManagers that don't satisy minimum resource requirements. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476824 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 20:03:37 +00:00
Vinod Kumar Vavilapalli
f0ac5a849c YARN-591. Moved RM recovery related records out of public API as they do not belong there. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476324 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 18:02:45 +00:00
Vinod Kumar Vavilapalli
635f96e74e YARN-562. Missed files from previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476038 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 04:42:34 +00:00
Vinod Kumar Vavilapalli
fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Thomas White
d10428cab2 YARN-289. Fair scheduler allows reservations that won't fit on node. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475681 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 09:15:51 +00:00
Thomas White
a43c344eee YARN-595. Refactor fair scheduler to use common Resources. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475670 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 08:33:43 +00:00
Vinod Kumar Vavilapalli
448df18646 YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah. 
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 05:42:22 +00:00
Siddharth Seth
be256a1819 YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471814 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 02:46:47 +00:00
Vinod Kumar Vavilapalli
a2f97fd9cb YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471608 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 19:42:47 +00:00
Vinod Kumar Vavilapalli
cd975d4b64 YARN-581. Added a test to verify that app delegation tokens are restored after RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471187 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 22:49:44 +00:00
Vinod Kumar Vavilapalli
27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00
Bikas Saha
e24a9b83f1 YARN-594. Update test and add comments in YARN-534 (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470243 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 22:04:08 +00:00
Vinod Kumar Vavilapalli
a91067fc5e YARN-542. Changed the default global AM max-attempts value to be not one. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470094 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 00:35:29 +00:00
Alejandro Abdelnur
df55edd5de YARN-482. FS: Extend SchedulingMode to intermediate queues. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469506 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 18:11:21 +00:00
Vinod Kumar Vavilapalli
d219c904d3 YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469492 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 17:49:22 +00:00
Bikas Saha
ef9f251679 YARN-514.Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469059 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 20:19:43 +00:00
Vinod Kumar Vavilapalli
0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:54:09 +00:00
Arun Murthy
c2592021f3 YARN-412. Fixed FifoScheduler to check hostname of a NodeManager rather than its host:port during scheduling which caused incorrect locality for containers. Contributed by Roger Hoover. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467244 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-12 12:12:08 +00:00
Vinod Kumar Vavilapalli
e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong. 
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:28:51 +00:00
Vinod Kumar Vavilapalli
2e3b56f6e9 YARN-495. Changed NM reboot behaviour to be a simple resync - kill all containers and re-register with RM. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466752 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 02:00:47 +00:00
Vinod Kumar Vavilapalli
7d00d3d20f YARN-534. Change RM restart recovery to also account for AM max-attempts configuration after the restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466208 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 20:16:19 +00:00
Bikas Saha
bc6777dd5b YARN-193. Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465067 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 17:45:44 +00:00
Jason Darrell Lowe
18757c1267 YARN-525. make CS node-locality-delay refreshable. Contributed by Thomas Graves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465009 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 15:30:24 +00:00
Vinod Kumar Vavilapalli
1fd462b118 YARN-536. Removed the unused objects ContainerStatus and ContainerStatus from Container which also don't belong to the container. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464271 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 03:41:18 +00:00
Bikas Saha
d40b154d27 YARN-382. SchedulerUtils improve way normalizeRequest sets the resource capabilities (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463653 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 18:07:07 +00:00
Vinod Kumar Vavilapalli
bcce9354bc YARN-447. Move ApplicationComparator in CapacityScheduler to use comparator in ApplicationId. Contributed by Nemon Lou. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463405 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 06:59:13 +00:00
Vinod Kumar Vavilapalli
7ca9fe73a0 YARN-309. Changed NodeManager to obtain heart-beat interval from the ResourceManager. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463346 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 01:04:07 +00:00
Thomas Graves
01aabf7363 YARN-460. CS user left in list of active users for the queue even when application finished (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462486 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-29 14:36:53 +00:00
Bikas Saha
520033b1cd YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462271 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-28 19:44:28 +00:00
Vinod Kumar Vavilapalli
d0bbff6c32 YARN-209. Fix CapacityScheduler to trigger application-activation when the cluster capacity changes. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461773 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-27 18:38:28 +00:00
Thomas White
414458ff9d YARN-496. Fair scheduler configs are refreshed inconsistently in reinitialize. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461614 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-27 14:42:41 +00:00
Vinod Kumar Vavilapalli
d60c2fa17f YARN-474. Fix CapacityScheduler to trigger application-activation when am-resource-percent configuration is refreshed. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461402 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-27 03:44:34 +00:00
Siddharth Seth
66e90b205a YARN-440. Flatten RegisterNodeManagerResponse. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461256 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 18:20:36 +00:00
Alejandro Abdelnur
e74d1f0435 YARN-469. Make scheduling mode in FS pluggable. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460961 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 03:25:26 +00:00
Vinod Kumar Vavilapalli
0757364014 YARN-378. Fix RM to make the AM max attempts/retries to be configurable per application by clients. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460895 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 21:11:33 +00:00
Siddharth Seth
18e08a8f51 YARN-439. Flatten NodeHeartbeatResponse. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460811 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 18:28:50 +00:00
Hitesh Shah
1bd345d6e3 YARN-396. Rationalize AllocateResponse in RM Scheduler API. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459040 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-20 20:44:35 +00:00
Jonathan Turner Eagles
21066255da YARN-227. Application expiration difficult to debug for end-users (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453080 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-05 23:12:17 +00:00
Siddharth Seth
83d8065867 YARN-376. Fixes a bug which would prevent the NM knowing about completed containers and applications. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451473 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-01 05:59:54 +00:00
Alejandro Abdelnur
5d6eca08bd MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does not exist. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451452 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-01 01:28:18 +00:00
Kihwal Lee
27c8a87a54 YARN-269. Resource Manager not logging the health_check_script result when taking it out. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451341 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-28 21:02:06 +00:00
Siddharth Seth
7ea82ca558 YARN-365. Change NM heartbeat handling to not generate a scheduler event on each heartbeat. (Contributed by Xuan Gong) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450007 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-26 03:32:10 +00:00
Thomas Graves
efaaf58605 YARN-400. RM can return null application resource usage report leading to NPE in client (Jason Lowe via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448241 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-20 15:35:06 +00:00
Thomas Graves
666667464a YARN-249. Capacity Scheduler web page should show list of active users per queue like it used to (in 1.x) (Ravi Prakash via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445673 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 15:40:01 +00:00
Siddharth Seth
ab16a37572 YARN-355. Fixes a bug where RM app submission could jam under load. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443131 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 19:03:52 +00:00
Daryn Sharp
d272918426 YARN-357. App submission should not be synchronized (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443016 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 15:26:30 +00:00
Arun Murthy
38bb4ad738 YARN-370. Fix SchedulerUtils to correctly round up the resource for containers. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442840 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 04:24:56 +00:00
Siddharth Seth
5d679c4f43 YARN-360. Allow apps to concurrently register tokens for renewal. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442441 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-05 00:26:16 +00:00
Hitesh Shah
b1baf11fb2 YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441668 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-02 00:43:54 +00:00
Thomas Graves
6384684299 YARN-343. Capacity Scheduler maximum-capacity value -1 is invalid (Xuan Gong via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440457 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-30 15:15:23 +00:00
Thomas White
345bcee066 YARN-319. Submitting a job to a fair scheduler queue for which the user does not have permission causes the client to wait forever. Contributed by shenhong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437336 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-23 10:53:05 +00:00
Hitesh Shah
29ddce96a2 YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437245 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-23 00:57:37 +00:00
Alejandro Abdelnur
f85b583731 YARN-302. Fair scheduler assignmultiple should default to false. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434996 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 00:48:22 +00:00
Siddharth Seth
6a2f2551fd YARN-135. Client tokens should be per app-attempt, and should be unregistered on App-finish. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433570 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 18:33:38 +00:00
Thomas White
72e631098d YARN-336. Fair scheduler FIFO scheduling within a queue only allows 1 app at a time. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433526 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 17:09:48 +00:00
Thomas White
8447557333 YARN-335. Fair scheduler doesn't check whether rack needs containers before assigning to node. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433484 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-15 16:12:33 +00:00
Thomas Graves
ce9b0a05b0 YARN-334. Maven RAT plugin is not checking all source files (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432931 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-14 14:34:51 +00:00
Thomas Graves
106e2e27ff YARN-325. RM CapacityScheduler can deadlock when getQueueInfo() is called and a container is completing (Arun C Murthy via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431070 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 21:00:47 +00:00
Siddharth Seth
ba5b19fb5d YARN-320. RM should always be able to renew its own tokens. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431020 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 19:33:06 +00:00
Arun Murthy
4539263971 YARN-2. Enhanced CapacityScheduler to account for CPU alongwith memory for multi-dimensional resource scheduling. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430682 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 05:08:23 +00:00
Thomas Graves
dd8fc7e499 YARN-50. Implement renewal / cancellation of Delegation Tokens(Siddharth Seth via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429085 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:15:32 +00:00
Thomas White
8cd026d365 YARN-301. Fair scheduler throws ConcurrentModificationException when iterating over app's priorities. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428400 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 15:26:55 +00:00
Thomas White
0768f96a83 YARN-300. After YARN-271, fair scheduler can infinite loop and not schedule any application. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428387 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 15:07:22 +00:00
Thomas White
19a291a0d6 YARN-288. Fair scheduler queue doesn't accept any jobs when ACLs are configured. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428362 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 14:32:01 +00:00
Thomas White
ba90c9c867 YARN-192. Node update causes NPE in the fair scheduler. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428314 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-03 12:29:13 +00:00
Thomas Graves
a806a2aac6 YARN-280. RM does not reject app submission with invalid tokens (Daryn Sharp via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425079 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 18:49:47 +00:00
Alejandro Abdelnur
a276f68b92 YARN-283. Fair scheduler fails to get queue info without root prefix. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424998 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 16:20:12 +00:00
Alejandro Abdelnur
5014a4b133 YARN-282. Fair scheduler web UI double counts Apps Submitted. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424995 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 16:12:20 +00:00
Alejandro Abdelnur
f121d1e333 YARN-278. Fair scheduler maxRunningApps config causes no apps to make progress. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424989 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-21 16:07:11 +00:00