big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,873 Commits 2 Branches 2 Tags 264 MiB
990cffdcfa
Commit Graph

325 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
990cffdcfa YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568501 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:02 +00:00
Vinod Kumar Vavilapalli
d0a5e43de7 YARN-1417. Modified RM to generate container-tokens not at creation time, but at allocation time so as to prevent RM 
from shelling out containers with expired tokens. Contributed by Omkar Vinit Joshi and Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568060 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 22:02:11 +00:00
Vinod Kumar Vavilapalli
2b916db60b YARN-1676. Modified RM HA handling of user-to-group mappings to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568041 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 21:23:33 +00:00
Jian He
dab635980d YARN-1345. Remove FINAL_SAVING state from YarnApplicationAttemptState. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567820 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 00:59:19 +00:00
Karthik Kambatla
41ec3cce76 YARN-1641. ZK store should attempt a write periodically to ensure it is still Active. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567628 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 14:09:13 +00:00
Vinod Kumar Vavilapalli
1fa6ab249b YARN-1459. Changed ResourceManager to depend its service initialization on the configuration-provider mechanism during startup too. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566791 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 22:50:15 +00:00
Vinod Kumar Vavilapalli
8b2336fcef YARN-1660. Simplified the RM HA configuration to accept and be able to simply depend just on 
configuration properties of the form yarn.resourcemanager.hostname.RMID and use the default
ports for all service addresses. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 03:18:39 +00:00
Vinod Kumar Vavilapalli
30294a2196 YARN-1665. Simplify the configuration of RM HA by having better default values. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565517 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:39:32 +00:00
Zhijie Shen
943b2190d7 YARN-1689. Made RMAppAttempt get killed when RMApp is at ACCEPTED. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565497 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 00:18:46 +00:00
Sanford Ryza
e6eccf20de YARN-1499. Fair Scheduler changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564856 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 18:09:07 +00:00
Zhijie Shen
ebe0c17a95 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564633 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:55:00 +00:00
Vinod Kumar Vavilapalli
b812af964d YARN-1669. Modified RM HA handling of protocol level service-ACLS to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564549 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-04 22:46:52 +00:00
Vinod Kumar Vavilapalli
24fa232707 YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564100 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 22:10:56 +00:00
Vinod Kumar Vavilapalli
eff1e809f2 YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564043 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:16:28 +00:00
Vinod Kumar Vavilapalli
3e7d56678c YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564032 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:53:46 +00:00
Vinod Kumar Vavilapalli
ca72e11158 YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564002 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:07:55 +00:00
Sanford Ryza
1079c37128 YARN-1504. RM changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563371 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 04:03:36 +00:00
Jonathan Turner Eagles
5d4bd6a15a YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563157 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 16:00:39 +00:00
Sanford Ryza
981679e6d7 YARN-1498. Common scheduler changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563021 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 00:14:47 +00:00
Karthik Kambatla
f1cef4cd29 YARN-1618. Fix invalid RMApp transition from NEW to FINAL_SAVING (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562529 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 18:02:40 +00:00
Sanford Ryza
7f2b01a742 YARN-1629. IndexOutOfBoundsException in MaxRunningAppsEnforcer (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561996 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 08:34:55 +00:00
Vinod Kumar Vavilapalli
cbee889711 YARN-321. Merging YARN-321 branch to trunk. 
svn merge ../branches/YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561452 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 04:51:10 +00:00
Sanford Ryza
163348701c YARN-1623. Include queue name in RegisterApplicationMasterResponse (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560545 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:25:03 +00:00
Sanford Ryza
8ceddeb0c3 YARN-1607. TestRM relies on the scheduler assigning multiple containers in a single node update (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560533 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:08:04 +00:00
Sanford Ryza
1a7e158844 YARN-1624. QueuePlacementPolicy format is not easily readable via a JAXB parser (Aditya Acharya via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560470 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 18:51:23 +00:00
Junping Du
4d413350d7 YARN-1603. Remove two *.orig files which were unexpectedly committed. (Zhijie Shen via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558340 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 11:05:15 +00:00
Sanford Ryza
f4fd050dee YARN-1567. In Fair Scheduler, allow empty queues to change between leaf and parent on allocation file reload (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558228 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-14 22:33:56 +00:00
Vinod Kumar Vavilapalli
25bc68d15e YARN-1041. Added the ApplicationMasterProtocol API for applications to use the ability in ResourceManager to optionally not kill containers when the ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557318 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 07:07:17 +00:00
Jian He
f677175f35 YARN-1166. Fixed app-specific and attempt-specific QueueMetrics to be triggered by accordingly app event and attempt event. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557296 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 01:15:49 +00:00
Karthik Kambatla
6608b75829 YARN-1574. RMDispatcher should be reset on transition to standby. (Xuan Gong via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557248 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 20:15:00 +00:00
Karthik Kambatla
a0ecb98a77 YARN-1033. Expose RM active/standby state to Web UI and REST API (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557181 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 16:35:52 +00:00
Vinod Kumar Vavilapalli
1393581bce YARN-1490. Introduced the ability to make ResourceManager optionally not kill all containers when an ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557143 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 15:04:39 +00:00
Vinod Kumar Vavilapalli
c3cc855d27 YARN-1029. Added embedded leader election in the ResourceManager. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556103 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 01:56:11 +00:00
Vinod Kumar Vavilapalli
a45017a5f1 YARN-1493. Changed ResourceManager and Scheduler interfacing to recognize app-attempts separately from apps. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554896 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-02 20:19:45 +00:00
Vinod Kumar Vavilapalli
50480f892a YARN-1522. Fixed a race condition in the test TestApplicationCleanup that was causing it to randomly fail. Contributed by Liyin Liang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554328 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-30 22:21:55 +00:00
Vinod Kumar Vavilapalli
8f0bf54d34 YARN-1541. Changed ResourceManager to invalidate ApplicationMaster host/port information once an AM crashes. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553772 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-28 01:09:07 +00:00
Karthik Kambatla
d841a0f474 YARN-1172. Convert SecretManagers in RM to services (Tsuyoshi OZAWA via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553431 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-26 00:21:56 +00:00
Vinod Kumar Vavilapalli
93907baa0b YARN-1307. Redesign znode structure for Zookeeper based RM state-store for better organization and scalability. Contributed by Tsuyoshi OZAWA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552209 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-19 02:33:05 +00:00
Vinod Kumar Vavilapalli
b774d7b3de YARN-1446. Changed client API to retry killing application till RM acknowledges so as to account for RM crashes/failover. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551444 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 02:16:20 +00:00
Karthik Kambatla
a6754bbb81 YARN-1451. TestResourceManager relies on the scheduler assigning multiple containers in a single node update. (Sandy Ryza via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551440 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 01:47:23 +00:00
Vinod Kumar Vavilapalli
1182ca04d4 YARN-1485. Modified RM HA configuration validation to also ensure that service-address configuration are configured for every RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550854 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 23:54:44 +00:00
Jian He
28db28d491 YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550613 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 01:44:16 +00:00
Jian He
791e43c53b Reverting YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550594 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 23:45:01 +00:00
Jian He
38c32ed239 YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550579 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 23:07:17 +00:00
Vinod Kumar Vavilapalli
8a79be36bd YARN-1325. Modified RM HA configuration validation to also ensure that multiple RMs are configured. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550524 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 20:50:11 +00:00
Vinod Kumar Vavilapalli
4ff1a8203b YARN-408. Change CapacityScheduler to not disable delay-scheduling by default. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550245 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 20:14:30 +00:00
Vinod Kumar Vavilapalli
48fb53bc49 YARN-1405. Fixed ResourceManager to not hang when init/start fails with an exception w.r.t state-store. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548992 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 04:30:32 +00:00
Vinod Kumar Vavilapalli
305ae48136 YARN-1378. Implemented a cleaner of old finished applications from the RM state-store. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548990 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 04:17:51 +00:00
Sanford Ryza
e1d3670f4c YARN-807. When querying apps by queue, iterating over all apps is inefficient and limiting (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548983 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 03:09:34 +00:00
Sanford Ryza
eccd7b2093 YARN-546. Allow disabling the Fair Scheduler event log. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548360 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-06 01:32:27 +00:00