big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,269 Commits 2 Branches 2 Tags 264 MiB
8314674947
Commit Graph

463 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
1fa6ab249b YARN-1459. Changed ResourceManager to depend its service initialization on the configuration-provider mechanism during startup too. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566791 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 22:50:15 +00:00
Vinod Kumar Vavilapalli
8b2336fcef YARN-1660. Simplified the RM HA configuration to accept and be able to simply depend just on 
configuration properties of the form yarn.resourcemanager.hostname.RMID and use the default
ports for all service addresses. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 03:18:39 +00:00
Vinod Kumar Vavilapalli
30294a2196 YARN-1665. Simplify the configuration of RM HA by having better default values. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565517 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:39:32 +00:00
Zhijie Shen
943b2190d7 YARN-1689. Made RMAppAttempt get killed when RMApp is at ACCEPTED. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565497 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 00:18:46 +00:00
Sanford Ryza
e6eccf20de YARN-1499. Fair Scheduler changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564856 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 18:09:07 +00:00
Zhijie Shen
ebe0c17a95 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564633 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:55:00 +00:00
Vinod Kumar Vavilapalli
b812af964d YARN-1669. Modified RM HA handling of protocol level service-ACLS to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564549 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-04 22:46:52 +00:00
Vinod Kumar Vavilapalli
24fa232707 YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564100 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 22:10:56 +00:00
Vinod Kumar Vavilapalli
eff1e809f2 YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564043 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:16:28 +00:00
Vinod Kumar Vavilapalli
3e7d56678c YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564032 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:53:46 +00:00
Sanford Ryza
cb5e0787a6 YARN-1498 addendum to fix findbugs warning 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564018 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:30:55 +00:00
Vinod Kumar Vavilapalli
ca72e11158 YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564002 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:07:55 +00:00
Sanford Ryza
1079c37128 YARN-1504. RM changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563371 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 04:03:36 +00:00
Jonathan Turner Eagles
5d4bd6a15a YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563157 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 16:00:39 +00:00
Sanford Ryza
981679e6d7 YARN-1498. Common scheduler changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563021 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 00:14:47 +00:00
Sanford Ryza
1cac66ce68 YARN-1617. Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563004 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 22:40:22 +00:00
Karthik Kambatla
f1cef4cd29 YARN-1618. Fix invalid RMApp transition from NEW to FINAL_SAVING (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562529 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 18:02:40 +00:00
Sanford Ryza
7f2b01a742 YARN-1629. IndexOutOfBoundsException in MaxRunningAppsEnforcer (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561996 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 08:34:55 +00:00
Vinod Kumar Vavilapalli
cbee889711 YARN-321. Merging YARN-321 branch to trunk. 
svn merge ../branches/YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561452 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 04:51:10 +00:00
Karthik Kambatla
482993ac53 YARN-1573. ZK store should use a private password for root-node-acls. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560594 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 06:22:05 +00:00
Sanford Ryza
163348701c YARN-1623. Include queue name in RegisterApplicationMasterResponse (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560545 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:25:03 +00:00
Sanford Ryza
8ceddeb0c3 YARN-1607. TestRM relies on the scheduler assigning multiple containers in a single node update (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560533 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:08:04 +00:00
Sanford Ryza
1a7e158844 YARN-1624. QueuePlacementPolicy format is not easily readable via a JAXB parser (Aditya Acharya via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560470 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 18:51:23 +00:00
Karthik Kambatla
e0ce554b86 YARN-1616. RMFatalEventDispatcher should log the cause of the event (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559902 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-21 02:01:51 +00:00
Sanford Ryza
67cbde3008 YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559050 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 08:56:22 +00:00
Sanford Ryza
7274b5ff93 YARN-1351. Invalid string format in Fair Scheduler log warn message (Konstantin Weitz via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558597 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 23:09:32 +00:00
Junping Du
4d413350d7 YARN-1603. Remove two *.orig files which were unexpectedly committed. (Zhijie Shen via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558340 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 11:05:15 +00:00
Karthik Kambatla
d13c2eeca6 YARN-1598. HA-related rmadmin commands don't work on a secure cluster (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558251 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 00:57:28 +00:00
Sanford Ryza
f4fd050dee YARN-1567. In Fair Scheduler, allow empty queues to change between leaf and parent on allocation file reload (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558228 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-14 22:33:56 +00:00
Vinod Kumar Vavilapalli
7c8b654ba5 YARN-888. Cleaned up POM files so that non-leaf modules don't include any dependencies and thus compact the dependency list for leaf modules. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557801 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-13 18:34:02 +00:00
Vinod Kumar Vavilapalli
25bc68d15e YARN-1041. Added the ApplicationMasterProtocol API for applications to use the ability in ResourceManager to optionally not kill containers when the ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557318 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 07:07:17 +00:00
Jian He
f677175f35 YARN-1166. Fixed app-specific and attempt-specific QueueMetrics to be triggered by accordingly app event and attempt event. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557296 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 01:15:49 +00:00
Karthik Kambatla
6608b75829 YARN-1574. RMDispatcher should be reset on transition to standby. (Xuan Gong via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557248 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 20:15:00 +00:00
Karthik Kambatla
a0ecb98a77 YARN-1033. Expose RM active/standby state to Web UI and REST API (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557181 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 16:35:52 +00:00
Vinod Kumar Vavilapalli
1393581bce YARN-1490. Introduced the ability to make ResourceManager optionally not kill all containers when an ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557143 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 15:04:39 +00:00
Karthik Kambatla
49ec397340 YARN-1568. Rename clusterid to clusterId in ActiveRMInfoProto (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556435 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-08 03:30:49 +00:00
Vinod Kumar Vavilapalli
4931545f76 YARN-1482. Modified WebApplicationProxy to make it work across ResourceManager fail-over. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556380 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 22:16:30 +00:00
Vinod Kumar Vavilapalli
c3cc855d27 YARN-1029. Added embedded leader election in the ResourceManager. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556103 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 01:56:11 +00:00
Vinod Kumar Vavilapalli
a45017a5f1 YARN-1493. Changed ResourceManager and Scheduler interfacing to recognize app-attempts separately from apps. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554896 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-02 20:19:45 +00:00
Vinod Kumar Vavilapalli
bb2e2fee60 YARN-1549. Fixed a bug in ResourceManager's ApplicationMasterService that was causing unamanged AMs to not finish correctly. Contributed by haosdent. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554886 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-02 19:54:07 +00:00
Vinod Kumar Vavilapalli
50480f892a YARN-1522. Fixed a race condition in the test TestApplicationCleanup that was causing it to randomly fail. Contributed by Liyin Liang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554328 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-30 22:21:55 +00:00
Karthik Kambatla
e7120079bd YARN-1481. Reverting addendum patch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553994 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-29 05:43:31 +00:00
Vinod Kumar Vavilapalli
8f0bf54d34 YARN-1541. Changed ResourceManager to invalidate ApplicationMaster host/port information once an AM crashes. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553772 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-28 01:09:07 +00:00
Karthik Kambatla
defeef6fe4 YARN-1481. Addendum patch to fix synchronization in AdminService 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553738 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-27 21:10:14 +00:00
Karthik Kambatla
1c9ac9adeb YARN-1523. Use StandbyException instead of RMNotYetReadyException (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553616 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-27 03:28:23 +00:00
Karthik Kambatla
d841a0f474 YARN-1172. Convert SecretManagers in RM to services (Tsuyoshi OZAWA via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553431 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-26 00:21:56 +00:00
Vinod Kumar Vavilapalli
93907baa0b YARN-1307. Redesign znode structure for Zookeeper based RM state-store for better organization and scalability. Contributed by Tsuyoshi OZAWA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552209 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-19 02:33:05 +00:00
Vinod Kumar Vavilapalli
b774d7b3de YARN-1446. Changed client API to retry killing application till RM acknowledges so as to account for RM crashes/failover. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551444 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 02:16:20 +00:00
Karthik Kambatla
a6754bbb81 YARN-1451. TestResourceManager relies on the scheduler assigning multiple containers in a single node update. (Sandy Ryza via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551440 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 01:47:23 +00:00
Vinod Kumar Vavilapalli
49ad07af97 YARN-312. Introduced ResourceManagerAdministrationProtocol changes to support changing resources on node. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551403 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 23:36:16 +00:00
Vinod Kumar Vavilapalli
1182ca04d4 YARN-1485. Modified RM HA configuration validation to also ensure that service-address configuration are configured for every RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550854 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 23:54:44 +00:00
Jian He
28db28d491 YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550613 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 01:44:16 +00:00
Jian He
791e43c53b Reverting YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550594 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 23:45:01 +00:00
Jian He
38c32ed239 YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550579 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 23:07:17 +00:00
Vinod Kumar Vavilapalli
8a79be36bd YARN-1325. Modified RM HA configuration validation to also ensure that multiple RMs are configured. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550524 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 20:50:11 +00:00
Vinod Kumar Vavilapalli
4ff1a8203b YARN-408. Change CapacityScheduler to not disable delay-scheduling by default. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550245 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 20:14:30 +00:00
Karthik Kambatla
94e2e78ab7 YARN-1481. Move internal services logic from AdminService to ResourceManager. (vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550167 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 15:13:46 +00:00
Vinod Kumar Vavilapalli
48fb53bc49 YARN-1405. Fixed ResourceManager to not hang when init/start fails with an exception w.r.t state-store. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548992 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 04:30:32 +00:00
Vinod Kumar Vavilapalli
305ae48136 YARN-1378. Implemented a cleaner of old finished applications from the RM state-store. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548990 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 04:17:51 +00:00
Sanford Ryza
e1d3670f4c YARN-807. When querying apps by queue, iterating over all apps is inefficient and limiting (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548983 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 03:09:34 +00:00
Sanford Ryza
eccd7b2093 YARN-546. Allow disabling the Fair Scheduler event log. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548360 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-06 01:32:27 +00:00
Sanford Ryza
6d5f8ebed6 YARN-1403. Separate out configuration loading from QueueManager in the Fair Scheduler (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548006 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 03:26:11 +00:00
Vinod Kumar Vavilapalli
7fe953ed90 YARN-1454. Fixed test failure issue with TestRMRestart. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547651 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-04 00:22:12 +00:00
Vinod Kumar Vavilapalli
9c95015bb4 YARN-895. Changed RM state-store to not crash immediately if RM restarts while the state-store is down. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547538 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-03 19:28:02 +00:00
Vinod Kumar Vavilapalli
c58ae266e9 YARN-1318. Promoted AdminService to an Always-On service and merged it into RMHAProtocolService. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547212 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 22:18:44 +00:00
Sanford Ryza
61b6ed73f8 YARN-1241: Include missing files 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546625 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-29 19:08:48 +00:00
Sanford Ryza
7545d8bf99 YARN-1241. In Fair Scheduler, maxRunningApps does not work for non-leaf queues. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546623 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-29 19:02:41 +00:00
Vinod Kumar Vavilapalli
6369c8d819 YARN-1239. Modified ResourceManager state-store implementations to start storing version numbers. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546229 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 23:22:33 +00:00
Vinod Kumar Vavilapalli
6c243fceac YARN-1416. Fixed a few invalid transitions in RMApp, RMAppAttempt and in some tests. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545448 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-25 23:10:34 +00:00
Sanford Ryza
38f94dc16d YARN-1423. Support queue placement by secondary group in the Fair Scheduler (Ted Malaska via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545157 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-25 06:04:26 +00:00
Bikas Saha
8313697752 YARN-1425. TestRMRestart fails because MockRM.waitForState(AttemptId) uses current attempt instead of the attempt passed as argument (Omkar Vinit Joshi via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543952 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 21:16:39 +00:00
Bikas Saha
8caae1d5a6 YARN-744. Race condition in ApplicationMasterService.allocate .. It might process same allocate request twice resulting in additional containers getting allocated. (Omkar Vinit Joshi via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543707 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 05:06:15 +00:00
Sanford Ryza
2f6b393543 YARN-1407. RM Web UI and REST APIs should uniformly use YarnApplicationState (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543675 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 00:47:18 +00:00
Sanford Ryza
c78c0e277f YARN-786: Addendum so that RMAppAttemptImpl#getApplicationResourceUsageReport won't return null 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543597 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 22:22:00 +00:00
Sanford Ryza
9ff520b7d6 YARN-584. In scheduler web UIs, queues unexpand on refresh. (Harshit Daga via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543350 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 09:00:20 +00:00
Vinod Kumar Vavilapalli
512475e56f YARN-674. Fixed ResourceManager to renew DelegationTokens on submission asynchronously to work around potential slowness in state-store. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543312 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 05:20:58 +00:00
Vinod Kumar Vavilapalli
cfa783141f YARN-1210. Changed RM to start new app-attempts on RM restart only after ensuring that previous AM exited or after expiry time. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543310 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 05:17:20 +00:00
Vinod Kumar Vavilapalli
4341562622 YARN-709. Added tests to verify validity of delegation tokens and logging of appsummary after RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543269 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 00:50:34 +00:00
Jason Darrell Lowe
e77aec001f YARN-1419. TestFifoScheduler.testAppAttemptMetrics fails intermittently under jdk7. Contributed by Jonathan Eagles 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543117 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-18 19:24:05 +00:00
Bikas Saha
f7efa0b86e YARN-1411. HA config shouldn't affect NodeManager RPC addresses (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542367 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-15 19:02:30 +00:00
Sanford Ryza
6ac4ac05f7 YARN-1392: Add new files 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542106 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 22:13:07 +00:00
Sanford Ryza
3858b9018e YARN-1392. Allow sophisticated app-to-queue placement policies in the Fair Scheduler (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542105 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 22:12:13 +00:00
Bikas Saha
797159bbd4 YARN-1222. Make improvements in ZKRMStateStore for fencing (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541995 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 17:53:25 +00:00
Sanford Ryza
72c6d6255a YARN-1387. RMWebServices should use ClientRMService for filtering applications (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540851 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-11 21:49:43 +00:00
Vinod Kumar Vavilapalli
e28015ed1b YARN-1121. Changed ResourceManager's state-store to drain all events on shut-down. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540232 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-09 00:55:21 +00:00
Luke Lu
6990355e57 YARN-311. RM/scheduler support for dynamic resource configuration. (Junping Du via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539134 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 21:23:53 +00:00
Vinod Kumar Vavilapalli
4550eac099 YARN-1374. Changed ResourceManager to start the preemption policy monitors as active services. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539089 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 18:25:30 +00:00
Sanford Ryza
d12010e658 YARN-1388. Fair Scheduler page always displays blank fair share (Liyin Liang via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538855 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 02:14:23 +00:00
Sanford Ryza
da317f2ea5 YARN-786. Expose application resource usage in RM REST API (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537809 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-01 02:54:28 +00:00
Sanford Ryza
30007fd686 YARN-1290. Let continuous scheduling achieve more balanced task assignment (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537731 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-31 22:04:31 +00:00
Vinod Kumar Vavilapalli
9f4d4e27fb YARN-891. Modified ResourceManager state-store to remember completed applications so that clients can get information about them post RM-restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537560 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-31 17:25:06 +00:00
Alejandro Abdelnur
990dcae717 YARN-1343. NodeManagers additions/restarts are not reported as node updates in AllocateResponse responses to AMs. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537368 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-31 02:54:39 +00:00
Bikas Saha
03510d00f4 YARN-1068. Add admin support for HA operations (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536888 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 21:03:54 +00:00
Sanford Ryza
b4a5476d95 YARN-1333: Add missing file SchedulerAppUtils 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535900 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-26 00:31:47 +00:00
Sanford Ryza
025f171947 YARN-1333. Support blacklisting in the Fair Scheduler (Tsuyoshi Ozawa via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535899 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-26 00:27:40 +00:00
Sanford Ryza
6772d07fdc YARN-1335. Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535582 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-24 23:15:25 +00:00
Bikas Saha
116b459d22 YARN-1305. RMHAProtocolService#serviceInit should handle HAUtil's IllegalArgumentException (Tsuyoshi Ozawa via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534884 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 01:10:17 +00:00
Sanford Ryza
3baff29b8c YARN-1330. Fair Scheduler: defaultQueueSchedulingPolicy does not take effect (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534861 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 23:53:32 +00:00
Sanford Ryza
755def8fe9 YARN-1315. TestQueueACLs should also test FairScheduler (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534508 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 06:22:46 +00:00
Sanford Ryza
dc523bd182 YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534315 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 18:45:38 +00:00