big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,943 Commits 2 Branches 2 Tags 264 MiB
5cb489f9d3
Commit Graph

789 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
8aab8533a1 YARN-1800. Fixed NodeManager to gracefully handle RejectedExecutionException in the public-localizer thread-pool. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576545 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 23:33:56 +00:00
Karthik Kambatla
477ed62b3f YARN-1821. NPE on registerNodeManager if the request has containers for UnmanagedAMs (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576525 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 22:18:54 +00:00
Vinod Kumar Vavilapalli
0edda25373 YARN-1764. Modified YarnClient to correctly handle failover of ResourceManager after the submitApplication call goes through. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576160 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 00:52:03 +00:00
Vinod Kumar Vavilapalli
e4190b2a97 YARN-1788. Fixed a bug in ResourceManager to set the apps-completed and apps-killed metrics correctly for killed applications. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576072 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 20:36:34 +00:00
Jian He
7da07461ff YARN-1793. Fixed ClientRMService#forceKillApplication not killing unmanaged application. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576023 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 17:58:55 +00:00
Sanford Ryza
e2530de4ea YARN-1790. Fair Scheduler UI not showing apps table (bc Wong via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575487 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 05:12:45 +00:00
Vinod Kumar Vavilapalli
8497b870af YARN-1410. Added tests to validate that clients can fail-over to a new RM 
after getting an application-ID but before submission and can still submit
to the newly active RM with no issues. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575478 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 04:43:29 +00:00
Vinod Kumar Vavilapalli
0b1304d098 YARN-1781. Modified NodeManagers to allow admins to specify max disk utilization for local disks so as to be able to offline full disks. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575463 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 00:52:06 +00:00
Vinod Kumar Vavilapalli
1c4047b0e4 YARN-1783. Fixed a bug in NodeManager's status-updater that was losing completed container statuses when NodeManager is forced to resync by the ResourceManager. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575437 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 22:36:47 +00:00
Karthik Kambatla
b06cc16f7d YARN-1774. FS: Submitting to non-leaf queue throws NPE. (Anubhav Dhoot and Karthik Kambatla via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575415 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 21:34:19 +00:00
Karthik Kambatla
9b15c5b11a YARN-1525. Web UI should redirect to active RM when HA is enabled. (Cindy Li via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575166 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 04:34:16 +00:00
Vinod Kumar Vavilapalli
f4eb6ddec9 YARN-1780. Improved logging in the Timeline client and server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575141 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 03:52:53 +00:00
Jian He
8cd23c5b48 YARN-1752. Fixed ApplicationMasterService to reject unregister request if AM did not register before. Contributed by Rohith Sharma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574623 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 19:04:58 +00:00
Karthik Kambatla
2e7c512270 YARN-1785. FairScheduler treats app lookup failures as ERRORs. (bc Wong via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574604 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 18:22:25 +00:00
Vinod Kumar Vavilapalli
82e8c8bd71 YARN-1766. Fixed a bug in ResourceManager to use configuration loaded from the configuration-provider when booting up. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574252 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 23:35:20 +00:00
Vinod Kumar Vavilapalli
88245b6a41 YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574190 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 20:39:06 +00:00
Vinod Kumar Vavilapalli
40464fba22 YARN-1730. Implemented simple write-locking in the LevelDB based timeline-store. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574145 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 17:32:41 +00:00
Zhijie Shen
33714d9ad6 YARN-1729. Made TimelineWebServices deserialize the string primary- and secondary-filters param into the JSON-compatible object. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573825 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 00:20:43 +00:00
Vinod Kumar Vavilapalli
5bf449e402 YARN-1748. Excluded core-site.xml from hadoop-yarn-server-tests package's jar and thus avoid breaking downstream tests. Contributed by Sravya Tirukkovalur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573795 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 23:09:08 +00:00
Vinod Kumar Vavilapalli
c26bdddcc3 YARN-1675. Added the previously missed new file. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573736 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 20:39:01 +00:00
Vinod Kumar Vavilapalli
a60434368c YARN-1758. Fixed ResourceManager to not mandate the presence of site specific configuration files and thus fix failures in downstream tests. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573695 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 19:41:52 +00:00
Karthik Kambatla
8caeb0af9d YARN-1760. Addendum patch. TestRMAdminService assumes Capacity Scheduler. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573134 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-01 08:46:29 +00:00
Karthik Kambatla
1ba26e3e68 YARN-1528. Allow setting auth for ZK connections. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573014 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 17:34:08 +00:00
Vinod Kumar Vavilapalli
94b29b3348 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572711 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 19:23:55 +00:00
Zhijie Shen
faddccc216 YARN-1301. Added the INFO level log of the non-empty blacklist additions and removals inside ApplicationMasterService. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572400 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 02:07:16 +00:00
Vinod Kumar Vavilapalli
0fbc7fe816 YARN-1588. Enhanced RM and the scheduling protocol to also send NMTokens of transferred containers from previous app-attempts to new AMs after YARN-1490. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572230 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:20:07 +00:00
Junping Du
c7142e7761 YARN-1561. Fix a generic type warning in FairScheduler. (Chen He via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571924 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 05:14:51 +00:00
Karthik Kambatla
5a42e1b7c3 YARN-1760. TestRMAdminService assumes CapacityScheduler. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571777 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 18:06:45 +00:00
Vinod Kumar Vavilapalli
ad70f26b1f YARN-1734. Fixed ResourceManager to update the configurations when it transits from standby to active mode so as to assimilate any changes that happened while it was in standby mode. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571539 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 02:07:19 +00:00
Vinod Kumar Vavilapalli
d07f855892 YARN-1686. Fixed NodeManager to properly handle any errors during re-registration after a RESYNC and thus avoid hanging. Contributed by Rohith Sharma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571474 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-24 22:41:24 +00:00
Sanford Ryza
1ab2f5a916 YARN-1678. Fair scheduler gabs incessantly about reservations (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571468 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-24 22:35:27 +00:00
Vinod Kumar Vavilapalli
e167e585e9 YARN-1749. Updated application-history related configs to reflect the latest reality and to be consistently named. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570948 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 02:04:58 +00:00
Vinod Kumar Vavilapalli
84425fb435 YARN-1687. Renamed user-facing records for the timeline-service to be simply named after 'timeline' instead of 'apptimeline'. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570922 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-22 20:55:06 +00:00
Vinod Kumar Vavilapalli
e06226126c YARN-1732. Changed types of related-entities and primary-filters in the timeline-service to be sets instead of maps. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570914 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-22 20:46:54 +00:00
Karthik Kambatla
87dce7b2d2 YARN-1470. Add audience annotations to MiniYARNCluster. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570708 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 21:20:59 +00:00
Zhijie Shen
da20095284 YARN-1071. Enabled ResourceManager to recover cluster metrics numDecommissionedNMs after restarting. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570469 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 06:46:24 +00:00
Vinod Kumar Vavilapalli
772ead791c YARN-1398. Fixed a deadlock in ResourceManager between users requesting queue-acls and completing containers. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570415 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 02:20:20 +00:00
Karthik Kambatla
12a8aa1510 YARN-1736. FS: AppSchedulable.assignContainer's priority argument is redundant. (Naren Koneru via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570336 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 20:09:54 +00:00
Vinod Kumar Vavilapalli
5fd5c9900c YARN-713. Fixed ResourceManager to not crash while building tokens when DNS issues happen transmittently. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569979 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 23:39:13 +00:00
Sanford Ryza
0e3e91a06d YARN-1718. Fix a couple isTerminals in Fair Scheduler queue placement rules (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569928 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 21:36:51 +00:00
Vinod Kumar Vavilapalli
9da9f7d4d8 YARN-1666. Modified RM HA handling of include/exclude node-lists to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569856 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:52:57 +00:00
Jonathan Turner Eagles
996acc834e YARN-1479. Invalid NaN values in Hadoop REST API JSON response (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569853 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:49:21 +00:00
Jian He
3a7e7b3671 YARN-1428. Fixed RM to write the final state of RMApp/RMAppAttempt to the application history store in the transition to the final state. (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569585 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 01:19:07 +00:00
Vinod Kumar Vavilapalli
6b03174aae YARN-1590. Fixed ResourceManager, web-app proxy and MR JobHistoryServer to expand _HOST properly in their kerberos principles. Contributed by Mohammad Kamrul Islam. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569537 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 21:50:37 +00:00
Sanford Ryza
bbbe808a51 YARN-1724. Race condition in Fair Scheduler when continuous scheduling is turned on (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569447 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 18:00:56 +00:00
Sanford Ryza
e60f5b6c40 YARN-1721. When moving app between queues in Fair Scheduler, grab lock on FSSchedulerApp (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569443 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 17:51:40 +00:00
Vinod Kumar Vavilapalli
990cffdcfa YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568501 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:02 +00:00
Vinod Kumar Vavilapalli
d0a5e43de7 YARN-1417. Modified RM to generate container-tokens not at creation time, but at allocation time so as to prevent RM 
from shelling out containers with expired tokens. Contributed by Omkar Vinit Joshi and Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568060 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 22:02:11 +00:00
Vinod Kumar Vavilapalli
2b916db60b YARN-1676. Modified RM HA handling of user-to-group mappings to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568041 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 21:23:33 +00:00
Jian He
dab635980d YARN-1345. Remove FINAL_SAVING state from YarnApplicationAttemptState. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567820 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 00:59:19 +00:00
Zhijie Shen
0185a57847 YARN-1578. Fixed reading incomplete application attempt and container data in FileSystemApplicationHistoryStore. Contributed by Shinichi Yamashita. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567816 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 00:08:47 +00:00
Sanford Ryza
bc939f7ae6 YARN-1692. ConcurrentModificationException in fair scheduler AppSchedulable (Sangjin Lee via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567788 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 23:04:42 +00:00
Karthik Kambatla
41ec3cce76 YARN-1641. ZK store should attempt a write periodically to ensure it is still Active. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567628 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 14:09:13 +00:00
Zhijie Shen
1b5254a4de YARN-1719. Fixed the root path related Jersey warnings produced in ATSWebServices. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567498 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 02:13:23 +00:00
Sanford Ryza
9024ad4aa0 YARN-1697. NodeManager reports negative running containers (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567356 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-11 20:14:30 +00:00
Vinod Kumar Vavilapalli
7fce641c49 YARN-1698. Fixed default TimelineStore in code to match what is documented in yarn-default.xml. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566937 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-11 01:05:16 +00:00
Vinod Kumar Vavilapalli
1fa6ab249b YARN-1459. Changed ResourceManager to depend its service initialization on the configuration-provider mechanism during startup too. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566791 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 22:50:15 +00:00
Zhijie Shen
23b2e43f5d YARN-1635. Implemented a Leveldb based ApplicationTimelineStore. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565868 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 02:15:46 +00:00
Karthik Kambatla
d57c6e0fe7 YARN-1672. YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds (Naren Koneru via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565866 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 01:55:33 +00:00
Vinod Kumar Vavilapalli
8b2336fcef YARN-1660. Simplified the RM HA configuration to accept and be able to simply depend just on 
configuration properties of the form yarn.resourcemanager.hostname.RMID and use the default
ports for all service addresses. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 03:18:39 +00:00
Vinod Kumar Vavilapalli
30294a2196 YARN-1665. Simplify the configuration of RM HA by having better default values. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565517 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:39:32 +00:00
Zhijie Shen
943b2190d7 YARN-1689. Made RMAppAttempt get killed when RMApp is at ACCEPTED. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565497 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 00:18:46 +00:00
Zhijie Shen
24775c10ef YARN-1628. Fixed the test failure in TestContainerManagerSecurity. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565094 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 07:21:42 +00:00
Sanford Ryza
e6eccf20de YARN-1499. Fair Scheduler changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564856 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 18:09:07 +00:00
Vinod Kumar Vavilapalli
484faadffe YARN-1636. Augmented Application-history server's web-services to also expose new APIs for retrieving and storing timeline information. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564829 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 17:02:26 +00:00
Zhijie Shen
ebe0c17a95 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564633 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:55:00 +00:00
Vinod Kumar Vavilapalli
96578f0e01 YARN-1634. Added a testable in-memory implementation of ApplicationTimelineStore. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564583 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 00:32:02 +00:00
Vinod Kumar Vavilapalli
b812af964d YARN-1669. Modified RM HA handling of protocol level service-ACLS to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564549 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-04 22:46:52 +00:00
Vinod Kumar Vavilapalli
24fa232707 YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564100 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 22:10:56 +00:00
Vinod Kumar Vavilapalli
eff1e809f2 YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564043 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:16:28 +00:00
Vinod Kumar Vavilapalli
eac066b13f YARN-1659. Defined the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564037 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:07:01 +00:00
Vinod Kumar Vavilapalli
3e7d56678c YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564032 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:53:46 +00:00
Sanford Ryza
cb5e0787a6 YARN-1498 addendum to fix findbugs warning 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564018 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:30:55 +00:00
Vinod Kumar Vavilapalli
ca72e11158 YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564002 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:07:55 +00:00
Sanford Ryza
1079c37128 YARN-1504. RM changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563371 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 04:03:36 +00:00
Jonathan Turner Eagles
5d4bd6a15a YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563157 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 16:00:39 +00:00
Sanford Ryza
981679e6d7 YARN-1498. Common scheduler changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563021 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 00:14:47 +00:00
Sanford Ryza
1cac66ce68 YARN-1617. Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563004 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 22:40:22 +00:00
Karthik Kambatla
f1cef4cd29 YARN-1618. Fix invalid RMApp transition from NEW to FINAL_SAVING (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562529 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 18:02:40 +00:00
Sanford Ryza
7f2b01a742 YARN-1629. IndexOutOfBoundsException in MaxRunningAppsEnforcer (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561996 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 08:34:55 +00:00
Suresh Srinivas
33a47743a5 HADOOP-10255. Rename HttpServer to HttpServer2 to retain older HttpServer in branch-2 for compatibility. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561959 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 07:32:52 +00:00
Vinod Kumar Vavilapalli
cbee889711 YARN-321. Merging YARN-321 branch to trunk. 
svn merge ../branches/YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561452 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 04:51:10 +00:00
Jason Darrell Lowe
3497e76e19 YARN-1575. Public localizer crashes with "Localized unkown resource". Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561110 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-24 18:54:48 +00:00
Karthik Kambatla
482993ac53 YARN-1573. ZK store should use a private password for root-node-acls. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560594 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 06:22:05 +00:00
Sanford Ryza
163348701c YARN-1623. Include queue name in RegisterApplicationMasterResponse (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560545 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:25:03 +00:00
Sanford Ryza
8ceddeb0c3 YARN-1607. TestRM relies on the scheduler assigning multiple containers in a single node update (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560533 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:08:04 +00:00
Sanford Ryza
1a7e158844 YARN-1624. QueuePlacementPolicy format is not easily readable via a JAXB parser (Aditya Acharya via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560470 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 18:51:23 +00:00
Karthik Kambatla
e0ce554b86 YARN-1616. RMFatalEventDispatcher should log the cause of the event (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559902 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-21 02:01:51 +00:00
Sanford Ryza
67cbde3008 YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559050 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 08:56:22 +00:00
Jason Darrell Lowe
a6ea460a91 MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j (syslog). Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558948 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 22:56:09 +00:00
Karthik Kambatla
752441434c YARN-1608. LinuxContainerExecutor has a few DEBUG messages at INFO level (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558874 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 18:42:25 +00:00
Sanford Ryza
7274b5ff93 YARN-1351. Invalid string format in Fair Scheduler log warn message (Konstantin Weitz via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558597 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 23:09:32 +00:00
Junping Du
4d413350d7 YARN-1603. Remove two *.orig files which were unexpectedly committed. (Zhijie Shen via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558340 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 11:05:15 +00:00
Karthik Kambatla
d13c2eeca6 YARN-1598. HA-related rmadmin commands don't work on a secure cluster (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558251 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 00:57:28 +00:00
Sanford Ryza
f4fd050dee YARN-1567. In Fair Scheduler, allow empty queues to change between leaf and parent on allocation file reload (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558228 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-14 22:33:56 +00:00
Vinod Kumar Vavilapalli
7c8b654ba5 YARN-888. Cleaned up POM files so that non-leaf modules don't include any dependencies and thus compact the dependency list for leaf modules. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557801 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-13 18:34:02 +00:00
Vinod Kumar Vavilapalli
25bc68d15e YARN-1041. Added the ApplicationMasterProtocol API for applications to use the ability in ResourceManager to optionally not kill containers when the ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557318 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 07:07:17 +00:00
Jian He
f677175f35 YARN-1166. Fixed app-specific and attempt-specific QueueMetrics to be triggered by accordingly app event and attempt event. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557296 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 01:15:49 +00:00
Karthik Kambatla
6608b75829 YARN-1574. RMDispatcher should be reset on transition to standby. (Xuan Gong via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557248 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 20:15:00 +00:00
Karthik Kambatla
a0ecb98a77 YARN-1033. Expose RM active/standby state to Web UI and REST API (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557181 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 16:35:52 +00:00
Vinod Kumar Vavilapalli
1393581bce YARN-1490. Introduced the ability to make ResourceManager optionally not kill all containers when an ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557143 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 15:04:39 +00:00
Karthik Kambatla
49ec397340 YARN-1568. Rename clusterid to clusterId in ActiveRMInfoProto (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556435 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-08 03:30:49 +00:00
Vinod Kumar Vavilapalli
4931545f76 YARN-1482. Modified WebApplicationProxy to make it work across ResourceManager fail-over. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556380 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 22:16:30 +00:00
Jian He
332c2704d4 YARN-1293. Fixed TestContainerLaunch#testInvalidEnvSyntaxDiagnostics failure caused by non-English system locale. Contributed by Tsuyoshi OZAWA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556318 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 19:00:21 +00:00
Jason Darrell Lowe
785c12056c YARN-1409. NonAggregatingLogHandler can throw RejectedExecutionException. Contributed by Tsuyoshi OZAWA 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556282 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 17:11:09 +00:00
Vinod Kumar Vavilapalli
c3cc855d27 YARN-1029. Added embedded leader election in the ResourceManager. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556103 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 01:56:11 +00:00
Karthik Kambatla
2cddd21db9 YARN-1559. Race between ServerRMProxy and ClientRMProxy setting RMProxy#INSTANCE. (kasha and vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555970 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 18:40:15 +00:00
Vinod Kumar Vavilapalli
a45017a5f1 YARN-1493. Changed ResourceManager and Scheduler interfacing to recognize app-attempts separately from apps. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554896 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-02 20:19:45 +00:00
Vinod Kumar Vavilapalli
bb2e2fee60 YARN-1549. Fixed a bug in ResourceManager's ApplicationMasterService that was causing unamanged AMs to not finish correctly. Contributed by haosdent. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554886 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-02 19:54:07 +00:00
Vinod Kumar Vavilapalli
50480f892a YARN-1522. Fixed a race condition in the test TestApplicationCleanup that was causing it to randomly fail. Contributed by Liyin Liang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554328 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-30 22:21:55 +00:00
Karthik Kambatla
e7120079bd YARN-1481. Reverting addendum patch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553994 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-29 05:43:31 +00:00
Karthik Kambatla
1152e23ed0 YARN-1463. Tests should avoid starting http-server where possible or creates spnego keytab/principals (vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553790 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-28 06:55:30 +00:00
Vinod Kumar Vavilapalli
8f0bf54d34 YARN-1541. Changed ResourceManager to invalidate ApplicationMaster host/port information once an AM crashes. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553772 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-28 01:09:07 +00:00
Karthik Kambatla
defeef6fe4 YARN-1481. Addendum patch to fix synchronization in AdminService 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553738 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-27 21:10:14 +00:00
Karthik Kambatla
1c9ac9adeb YARN-1523. Use StandbyException instead of RMNotYetReadyException (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553616 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-27 03:28:23 +00:00
Karthik Kambatla
d841a0f474 YARN-1172. Convert SecretManagers in RM to services (Tsuyoshi OZAWA via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553431 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-26 00:21:56 +00:00
Vinod Kumar Vavilapalli
93907baa0b YARN-1307. Redesign znode structure for Zookeeper based RM state-store for better organization and scalability. Contributed by Tsuyoshi OZAWA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552209 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-19 02:33:05 +00:00
Vinod Kumar Vavilapalli
ca125153b3 YARN-1028. Added FailoverProxyProvider capability to ResourceManager to help with RM failover. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551739 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 22:32:38 +00:00
Vinod Kumar Vavilapalli
b774d7b3de YARN-1446. Changed client API to retry killing application till RM acknowledges so as to account for RM crashes/failover. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551444 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 02:16:20 +00:00
Karthik Kambatla
a6754bbb81 YARN-1451. TestResourceManager relies on the scheduler assigning multiple containers in a single node update. (Sandy Ryza via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551440 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 01:47:23 +00:00
Vinod Kumar Vavilapalli
49ad07af97 YARN-312. Introduced ResourceManagerAdministrationProtocol changes to support changing resources on node. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551403 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 23:36:16 +00:00
Vinod Kumar Vavilapalli
5a1b33507b YARN-1145. Fixed a potential file-handle leak in the web interface for displaying aggregated logs. Contributed by Rohith Sharma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551326 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 19:27:48 +00:00
Vinod Kumar Vavilapalli
d38fb71d00 YARN-1505. Fixed Webapplication proxy server to not hardcode its bind address. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551314 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 19:09:44 +00:00
Vinod Kumar Vavilapalli
1182ca04d4 YARN-1485. Modified RM HA configuration validation to also ensure that service-address configuration are configured for every RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550854 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 23:54:44 +00:00
Jian He
28db28d491 YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550613 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 01:44:16 +00:00
Jian He
791e43c53b Reverting YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550594 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 23:45:01 +00:00
Jian He
38c32ed239 YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550579 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 23:07:17 +00:00
Vinod Kumar Vavilapalli
8a79be36bd YARN-1325. Modified RM HA configuration validation to also ensure that multiple RMs are configured. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550524 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 20:50:11 +00:00
Vinod Kumar Vavilapalli
4ff1a8203b YARN-408. Change CapacityScheduler to not disable delay-scheduling by default. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550245 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 20:14:30 +00:00
Karthik Kambatla
94e2e78ab7 YARN-1481. Move internal services logic from AdminService to ResourceManager. (vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550167 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 15:13:46 +00:00
Vinod Kumar Vavilapalli
48fb53bc49 YARN-1405. Fixed ResourceManager to not hang when init/start fails with an exception w.r.t state-store. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548992 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 04:30:32 +00:00
Vinod Kumar Vavilapalli
305ae48136 YARN-1378. Implemented a cleaner of old finished applications from the RM state-store. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548990 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 04:17:51 +00:00
Sanford Ryza
e1d3670f4c YARN-807. When querying apps by queue, iterating over all apps is inefficient and limiting (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548983 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 03:09:34 +00:00
Sanford Ryza
eccd7b2093 YARN-546. Allow disabling the Fair Scheduler event log. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548360 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-06 01:32:27 +00:00
Arun Murthy
ef1f556cef YARN-1438. Ensure container diagnostics includes exception from container launch. Contributed by Steve Loughran. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548336 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-06 00:25:02 +00:00
Karthik Kambatla
368d9769f4 YARN-1181. Augment MiniYARNCluster to support HA mode (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548330 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 23:48:37 +00:00
Sanford Ryza
6d5f8ebed6 YARN-1403. Separate out configuration loading from QueueManager in the Fair Scheduler (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548006 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 03:26:11 +00:00
Vinod Kumar Vavilapalli
7fe953ed90 YARN-1454. Fixed test failure issue with TestRMRestart. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547651 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-04 00:22:12 +00:00
Vinod Kumar Vavilapalli
9c95015bb4 YARN-895. Changed RM state-store to not crash immediately if RM restarts while the state-store is down. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547538 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-03 19:28:02 +00:00
Vinod Kumar Vavilapalli
c58ae266e9 YARN-1318. Promoted AdminService to an Always-On service and merged it into RMHAProtocolService. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547212 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 22:18:44 +00:00
Sanford Ryza
61b6ed73f8 YARN-1241: Include missing files 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546625 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-29 19:08:48 +00:00
Sanford Ryza
7545d8bf99 YARN-1241. In Fair Scheduler, maxRunningApps does not work for non-leaf queues. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546623 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-29 19:02:41 +00:00
Vinod Kumar Vavilapalli
6369c8d819 YARN-1239. Modified ResourceManager state-store implementations to start storing version numbers. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546229 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 23:22:33 +00:00
Jing Zhao
2214871d91 HDFS-5545. Allow specifying endpoints for listeners in HttpServer. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546151 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 18:20:14 +00:00
Vinod Kumar Vavilapalli
6c243fceac YARN-1416. Fixed a few invalid transitions in RMApp, RMAppAttempt and in some tests. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545448 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-25 23:10:34 +00:00
Sanford Ryza
38f94dc16d YARN-1423. Support queue placement by secondary group in the Fair Scheduler (Ted Malaska via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545157 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-25 06:04:26 +00:00
Bikas Saha
d1fe9e4142 YARN-1053. Diagnostic message from ContainerExitEvent is ignored in ContainerImpl (Omkar Vinit Joshi via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543973 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 22:25:59 +00:00
Bikas Saha
8313697752 YARN-1425. TestRMRestart fails because MockRM.waitForState(AttemptId) uses current attempt instead of the attempt passed as argument (Omkar Vinit Joshi via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543952 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 21:16:39 +00:00
Bikas Saha
8caae1d5a6 YARN-744. Race condition in ApplicationMasterService.allocate .. It might process same allocate request twice resulting in additional containers getting allocated. (Omkar Vinit Joshi via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543707 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 05:06:15 +00:00
Sanford Ryza
2f6b393543 YARN-1407. RM Web UI and REST APIs should uniformly use YarnApplicationState (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543675 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 00:47:18 +00:00
Sanford Ryza
c78c0e277f YARN-786: Addendum so that RMAppAttemptImpl#getApplicationResourceUsageReport won't return null 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543597 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 22:22:00 +00:00
Sanford Ryza
9ff520b7d6 YARN-584. In scheduler web UIs, queues unexpand on refresh. (Harshit Daga via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543350 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 09:00:20 +00:00
Vinod Kumar Vavilapalli
512475e56f YARN-674. Fixed ResourceManager to renew DelegationTokens on submission asynchronously to work around potential slowness in state-store. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543312 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 05:20:58 +00:00
Vinod Kumar Vavilapalli
cfa783141f YARN-1210. Changed RM to start new app-attempts on RM restart only after ensuring that previous AM exited or after expiry time. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543310 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 05:17:20 +00:00
Vinod Kumar Vavilapalli
4341562622 YARN-709. Added tests to verify validity of delegation tokens and logging of appsummary after RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543269 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 00:50:34 +00:00
Jason Darrell Lowe
e77aec001f YARN-1419. TestFifoScheduler.testAppAttemptMetrics fails intermittently under jdk7. Contributed by Jonathan Eagles 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543117 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-18 19:24:05 +00:00
Bikas Saha
f7efa0b86e YARN-1411. HA config shouldn't affect NodeManager RPC addresses (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542367 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-15 19:02:30 +00:00
Sanford Ryza
6ac4ac05f7 YARN-1392: Add new files 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542106 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 22:13:07 +00:00
Sanford Ryza
3858b9018e YARN-1392. Allow sophisticated app-to-queue placement policies in the Fair Scheduler (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542105 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 22:12:13 +00:00
Sanford Ryza
1a76ccbbc4 YARN-1401. With zero sleep-delay-before-sigkill.ms, no signal is ever sent (Gera Shegalov via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542038 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 19:52:14 +00:00
Bikas Saha
797159bbd4 YARN-1222. Make improvements in ZKRMStateStore for fencing (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541995 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 17:53:25 +00:00
Jonathan Turner Eagles
656e859f5c YARN-1386. NodeManager mistakenly loses resources and relocalizes them (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541375 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 03:17:23 +00:00
Sanford Ryza
72c6d6255a YARN-1387. RMWebServices should use ClientRMService for filtering applications (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540851 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-11 21:49:43 +00:00
Vinod Kumar Vavilapalli
e28015ed1b YARN-1121. Changed ResourceManager's state-store to drain all events on shut-down. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540232 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-09 00:55:21 +00:00
Luke Lu
6990355e57 YARN-311. RM/scheduler support for dynamic resource configuration. (Junping Du via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539134 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 21:23:53 +00:00
Vinod Kumar Vavilapalli
4550eac099 YARN-1374. Changed ResourceManager to start the preemption policy monitors as active services. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539089 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 18:25:30 +00:00
Sanford Ryza
d12010e658 YARN-1388. Fair Scheduler page always displays blank fair share (Liyin Liang via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538855 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 02:14:23 +00:00
Sanford Ryza
da317f2ea5 YARN-786. Expose application resource usage in RM REST API (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537809 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-01 02:54:28 +00:00
Sanford Ryza
30007fd686 YARN-1290. Let continuous scheduling achieve more balanced task assignment (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537731 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-31 22:04:31 +00:00
Vinod Kumar Vavilapalli
9f4d4e27fb YARN-891. Modified ResourceManager state-store to remember completed applications so that clients can get information about them post RM-restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537560 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-31 17:25:06 +00:00
Alejandro Abdelnur
990dcae717 YARN-1343. NodeManagers additions/restarts are not reported as node updates in AllocateResponse responses to AMs. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537368 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-31 02:54:39 +00:00
Chris Nauroth
218ef59d89 YARN-1357. TestContainerLaunch.testContainerEnvVariables fails on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537293 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-30 20:49:12 +00:00
Bikas Saha
03510d00f4 YARN-1068. Add admin support for HA operations (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536888 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 21:03:54 +00:00
Sanford Ryza
defa7af4d3 YARN-1109. Demote NodeManager "Sending out status for container" logs to debug (haosdent via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536573 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 00:54:12 +00:00
Sanford Ryza
b4a5476d95 YARN-1333: Add missing file SchedulerAppUtils 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535900 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-26 00:31:47 +00:00
Sanford Ryza
025f171947 YARN-1333. Support blacklisting in the Fair Scheduler (Tsuyoshi Ozawa via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535899 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-26 00:27:40 +00:00
Sanford Ryza
6772d07fdc YARN-1335. Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535582 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-24 23:15:25 +00:00
Bikas Saha
116b459d22 YARN-1305. RMHAProtocolService#serviceInit should handle HAUtil's IllegalArgumentException (Tsuyoshi Ozawa via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534884 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 01:10:17 +00:00
Sanford Ryza
3baff29b8c YARN-1330. Fair Scheduler: defaultQueueSchedulingPolicy does not take effect (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534861 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 23:53:32 +00:00
Jonathan Turner Eagles
66aa806305 YARN-1183. MiniYARNCluster shutdown takes several minutes intermittently (Andrey Klochkov via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534800 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 21:24:46 +00:00
Sanford Ryza
755def8fe9 YARN-1315. TestQueueACLs should also test FairScheduler (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534508 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 06:22:46 +00:00
Sanford Ryza
dc523bd182 YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534315 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 18:45:38 +00:00
Sanford Ryza
881d1d5c7e Reverting "YARN-1258: Move to 2.2.1 in CHANGES.txt" because it contained unintended changes 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534308 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 18:37:32 +00:00
Sanford Ryza
65bda6d168 YARN-1258: Move to 2.2.1 in CHANGES.txt 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534307 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 18:31:57 +00:00
Vinod Kumar Vavilapalli
7f97fd1319 YARN-1185. Fixed FileSystemRMStateStore to not leave partial files that prevent subsequent ResourceManager recovery. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533803 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-19 18:14:14 +00:00
Sanford Ryza
6a55ed333a YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text file busy errors. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532532 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-15 21:14:47 +00:00
Sanford Ryza
989db9ee55 YARN-1182. MiniYARNCluster creates and inits the RM/NM only on start() (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532109 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-14 21:42:31 +00:00
Sanford Ryza
cd7e7c3cdd YARN-1259. In Fair Scheduler web UI, queue num pending and num active apps switched. (Robert Kanter via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532094 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-14 21:06:01 +00:00
Sanford Ryza
2caa899882 YARN-305. Fair scheduler logs too many "Node offered to app" messages. (Lohit Vijayarenu via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531817 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-14 08:51:34 +00:00
Sanford Ryza
5bed5bb829 YARN-1044. used/min/max resources do not display info in the scheduler page (Sangjin Lee via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531447 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-11 22:13:57 +00:00
Sanford Ryza
f218527fff YARN-1265. Fair Scheduler chokes on unhealthy node reconnect (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531146 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-10 22:49:56 +00:00
Devarajulu K
22b332ff46 YARN-879. Fixed tests w.r.t o.a.h.y.server.resourcemanager.Application. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530902 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-10 09:47:11 +00:00
Vinod Kumar Vavilapalli
9b9ddf29e2 YARN-1283. Fixed RM to give a fully-qualified proxy URL for an application so that clients don't need to do scheme-mangling. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530819 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 22:49:06 +00:00
Daryn Sharp
b858b812b7 HADOOP-9470. eliminate duplicate FQN tests in different Hadoop modules (Ivan A. Veselovsky via daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530667 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 15:09:37 +00:00
Sanford Ryza
643b1a4019 YARN-1258. Allow configuring the Fair Scheduler root queue (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530542 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 09:19:14 +00:00
Sanford Ryza
5a87d89366 Add missing file TestCgroupsLCEResourcesHandler for YARN-1284. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530493 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 05:07:41 +00:00
Sanford Ryza
726c3538a7 YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed containers. (Alejandro Abdelnur via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530492 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 05:05:17 +00:00
Jason Darrell Lowe
3de419a478 YARN-465. fix coverage org.apache.hadoop.yarn.server.webproxy. Contributed by Aleksey Gorshkov and Andrey Klochkov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530095 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-07 22:21:38 +00:00
Vinod Kumar Vavilapalli
21181b6553 YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related 
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529662 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 18:43:36 +00:00
Vinod Kumar Vavilapalli
44f7ee7192 YARN-1278. Fixed NodeManager to not delete local resources for apps on resync command from RM - a bug caused by YARN-1149. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529657 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 18:32:16 +00:00