big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,875 Commits 2 Branches 2 Tags 264 MiB
d16470025a
Commit Graph

546 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
bad021534c YARN-1281. Fixed TestZKRMStateStoreZKClientConnections to not fail intermittently due to ZK-client timeouts. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588369 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 20:57:15 +00:00
Junping Du
bd43d2481e YARN-1947. TestRMDelegationTokens#testRMDTMasterKeyStateOnRollingMasterKey is failing intermittently. (Jian He via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588365 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 20:27:37 +00:00
Jian He
38d5ca2e47 YARN-1934. Fixed a potential NPE in ZKRMStateStore caused by handling Disconnected event from ZK. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587776 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-16 02:02:18 +00:00
Zhijie Shen
44b6261bfa YARN-1892. Improved some logs in the scheduler. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587717 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-15 20:37:44 +00:00
Vinod Kumar Vavilapalli
eb7b33c298 YARN-1928. Fixed a race condition in TestAMRMRPCNodeUpdates which caused it to fail occassionally. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587114 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-13 22:40:16 +00:00
Vinod Kumar Vavilapalli
c6b70f4760 YARN-1933. Fixed test issues with TestAMRestart and TestNodeHealthService. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587104 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-13 21:51:38 +00:00
Sanford Ryza
9274626a11 YARN-1923. Make Fair Scheduler resource ratio calculations terminate faster (Anubhav Dhoot via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586796 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 23:24:40 +00:00
Zhijie Shen
dc84800184 YARN-1924. Made ZKRMStateStore updateApplication(Attempt)StateInternal work when Application(Attempt) state hasn't been stored before. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586547 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 03:36:36 +00:00
Vinod Kumar Vavilapalli
791f2f6f1b YARN-1910. Fixed a race condition in TestAMRMTokens that causes the test to fail more often on Windows. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586192 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 01:36:50 +00:00
Kihwal Lee
c3f6d71726 YARN-1907. TestRMApplicationHistoryWriter#testRMWritingMassiveHistory intermittently fails. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585992 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 15:33:25 +00:00
Karthik Kambatla
1e1200d594 YARN-1784. TestContainerAllocation assumes CapacityScheduler. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585894 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 05:49:24 +00:00
Arun Murthy
05da90ee2f YARN-1898. Addendum patch to ensure /jmx and /metrics are re-directed to Active RM. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584954 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-05 02:51:08 +00:00
Jian He
15ac36a216 YARN-1837. Fixed TestMoveApplication#testMoveRejectedByScheduler failure. Contributed by Hong Zhiguo 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584862 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-04 18:42:35 +00:00
Zhijie Shen
b0180afc9c YARN-1898. Made Standby RM links conf, stacks, logLevel, metrics, jmx, logs and static not be redirected to Active RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583833 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-02 00:01:50 +00:00
Sanford Ryza
7bd62b8da0 YARN-1889. In Fair Scheduler, avoid creating objects on each call to AppSchedulable comparator (Hong Zhiguo via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583491 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-31 23:40:45 +00:00
Jonathan Turner Eagles
6d2e1fde63 YARN-1883. TestRMAdminService fails due to inconsistent entries in UserGroups (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582862 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 19:30:08 +00:00
Tsz-wo Sze
3f7aa79e58 HDFS-6168. Remove a deprecated constructor and the deprecated methods reportChecksumFailure, getDelegationToken(Text), renewDelegationToken and cancelDelegationToken from DistributedFileSystem. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582856 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 18:29:31 +00:00
Jian He
71c2b159ab YARN-1521. Mark Idempotent/AtMostOnce annotations to the APIs in ApplicationClientProtcol, ResourceManagerAdministrationProtocol and ResourceTrackerProtocol so that they work in HA scenario. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581678 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 03:43:41 +00:00
Vinod Kumar Vavilapalli
8a9ae9e3ec YARN-1867. Fixed a bug in ResourceManager that was causing invalid ACL checks in the web-services after fail-over. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581662 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 03:03:17 +00:00
Vinod Kumar Vavilapalli
d14eff7d38 YARN-1866. Fixed an issue with renewal of RM-delegation tokens on restart or fail-over. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581448 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 18:21:55 +00:00
Jian He
0febaa1b3d YARN-1852. Fixed RMAppAttempt to not resend AttemptFailed/AttemptKilled events to already recovered Failed/Killed RMApps. Contributed by Rohith Sharmaks 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580997 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 19:04:57 +00:00
Zhijie Shen
ef9634bf79 YARN-1577. Made UnmanagedAMLauncher do launchAM after the attempt reaches the LAUNCHED state. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580164 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 04:35:46 +00:00
Jian He
ce56616037 YARN-1776. Fixed DelegationToken renewal to survive RM failover. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580154 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 04:12:57 +00:00
Karthik Kambatla
cbb3914c89 YARN-1536. Cleanup: Get rid of ResourceManager#get*SecretManager() methods and use the RMContext methods instead. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580128 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 02:35:52 +00:00
Vinod Kumar Vavilapalli
dc594101cc YARN-1854. Fixed test failure in TestRMHA#testStartAndTransitions. Contributed by Rohith Sharma KS. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580097 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:29:43 +00:00
Jian He
f67218809c YARN-1849. Fixed NPE in ResourceTrackerService#registerNodeManager for UAM. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580077 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 22:53:49 +00:00
Vinod Kumar Vavilapalli
4224e61340 YARN-1811. Fixed AMFilters in YARN to correctly accept requests from either web-app proxy or the RMs when HA is enabled. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579877 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 04:20:46 +00:00
Vinod Kumar Vavilapalli
764af69aa0 YARN-1640. Fixed manual failover of ResourceManagers to work correctly in secure clusters. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579510 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-20 02:40:21 +00:00
Karthik Kambatla
bfafc1d0ce YARN-1705. Reset cluster-metrics on transition to standby. (Rohith via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579014 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-18 19:00:46 +00:00
Karthik Kambatla
e52c1535b8 YARN-1846. TestRM#testNMTokenSentForNormalContainer assumes CapacityScheduler. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578763 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-18 06:56:15 +00:00
Vinod Kumar Vavilapalli
57cdf8626a YARN-1512. Enhanced CapacityScheduler to be able to decouple scheduling from node-heartbeats. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578722 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-18 02:53:20 +00:00
Vinod Kumar Vavilapalli
92317e3459 YARN-1839. Fixed handling of NMTokens in ResourceManager such that containers launched by AMs running on the same machine as the AM are correctly propagated. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578631 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 22:22:15 +00:00
Vinod Kumar Vavilapalli
81a456e638 YARN-1591. Fixed AsyncDispatcher to handle interrupts on shutdown in a sane manner and thus fix failure of TestResourceTrackerService. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578628 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 22:17:43 +00:00
Vinod Kumar Vavilapalli
8314674947 YARN-1685. Fixed few bugs related to handling of containers' log-URLs on ResourceManager and history-service. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578602 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 21:36:21 +00:00
Jonathan Turner Eagles
7944eab64a YARN-1136. Replace junit.framework.Assert with org.junit.Assert (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578539 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 20:03:35 +00:00
Jian He
bc00bc966b YARN-1830. Fixed TestRMRestart#testQueueMetricsOnRMRestart failure due to race condition when app is submitted. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578486 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 18:03:06 +00:00
Jonathan Turner Eagles
95b7f50813 YARN-1833. TestRMAdminService Fails in trunk and branch-2 (Mit Desais via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577737 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 21:58:42 +00:00
Vinod Kumar Vavilapalli
d9cdcb9474 YARN-1658. Modified web-app framework to let standby RMs redirect web-service calls to the active RM. Contributed by Cindy Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577408 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 02:39:59 +00:00
Zhijie Shen
6af044bb75 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577052 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 07:42:29 +00:00
Jason Darrell Lowe
135700a611 YARN-1789. ApplicationSummary does not escape newlines in the app name. Contributed by Tsuyoshi OZAWA 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576960 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 22:25:01 +00:00
Vinod Kumar Vavilapalli
197217c95d YARN-1816. Fixed ResourceManager to get RMApp correctly handle ATTEMPT_FINISHED event at ACCEPTED state that can happen after RM restarts. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576911 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 21:18:55 +00:00
Vinod Kumar Vavilapalli
4de17c6052 YARN-1812. Fixed ResourceManager to synchrously renew tokens after recovery and thus recover app itself synchronously and avoid races with resyncing NodeManagers. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576843 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 18:25:33 +00:00
Arun Murthy
4ce0e4bf2e YARN-1444. Fix CapacityScheduler to deal with cases where applications specify host/rack requests without off-switch request. Contributed by Wangda Tan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576751 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 14:36:27 +00:00
Karthik Kambatla
477ed62b3f YARN-1821. NPE on registerNodeManager if the request has containers for UnmanagedAMs (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576525 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 22:18:54 +00:00
Vinod Kumar Vavilapalli
0edda25373 YARN-1764. Modified YarnClient to correctly handle failover of ResourceManager after the submitApplication call goes through. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576160 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 00:52:03 +00:00
Vinod Kumar Vavilapalli
e4190b2a97 YARN-1788. Fixed a bug in ResourceManager to set the apps-completed and apps-killed metrics correctly for killed applications. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576072 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 20:36:34 +00:00
Jian He
7da07461ff YARN-1793. Fixed ClientRMService#forceKillApplication not killing unmanaged application. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576023 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 17:58:55 +00:00
Sanford Ryza
e2530de4ea YARN-1790. Fair Scheduler UI not showing apps table (bc Wong via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575487 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 05:12:45 +00:00
Vinod Kumar Vavilapalli
8497b870af YARN-1410. Added tests to validate that clients can fail-over to a new RM 
after getting an application-ID but before submission and can still submit
to the newly active RM with no issues. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575478 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 04:43:29 +00:00
Karthik Kambatla
b06cc16f7d YARN-1774. FS: Submitting to non-leaf queue throws NPE. (Anubhav Dhoot and Karthik Kambatla via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575415 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 21:34:19 +00:00
Karthik Kambatla
9b15c5b11a YARN-1525. Web UI should redirect to active RM when HA is enabled. (Cindy Li via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575166 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 04:34:16 +00:00
Jian He
8cd23c5b48 YARN-1752. Fixed ApplicationMasterService to reject unregister request if AM did not register before. Contributed by Rohith Sharma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574623 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 19:04:58 +00:00
Karthik Kambatla
2e7c512270 YARN-1785. FairScheduler treats app lookup failures as ERRORs. (bc Wong via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574604 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 18:22:25 +00:00
Vinod Kumar Vavilapalli
82e8c8bd71 YARN-1766. Fixed a bug in ResourceManager to use configuration loaded from the configuration-provider when booting up. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574252 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 23:35:20 +00:00
Vinod Kumar Vavilapalli
88245b6a41 YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574190 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 20:39:06 +00:00
Vinod Kumar Vavilapalli
c26bdddcc3 YARN-1675. Added the previously missed new file. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573736 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 20:39:01 +00:00
Vinod Kumar Vavilapalli
a60434368c YARN-1758. Fixed ResourceManager to not mandate the presence of site specific configuration files and thus fix failures in downstream tests. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573695 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 19:41:52 +00:00
Karthik Kambatla
8caeb0af9d YARN-1760. Addendum patch. TestRMAdminService assumes Capacity Scheduler. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573134 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-01 08:46:29 +00:00
Karthik Kambatla
1ba26e3e68 YARN-1528. Allow setting auth for ZK connections. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573014 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 17:34:08 +00:00
Zhijie Shen
faddccc216 YARN-1301. Added the INFO level log of the non-empty blacklist additions and removals inside ApplicationMasterService. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572400 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 02:07:16 +00:00
Vinod Kumar Vavilapalli
0fbc7fe816 YARN-1588. Enhanced RM and the scheduling protocol to also send NMTokens of transferred containers from previous app-attempts to new AMs after YARN-1490. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572230 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:20:07 +00:00
Junping Du
c7142e7761 YARN-1561. Fix a generic type warning in FairScheduler. (Chen He via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571924 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 05:14:51 +00:00
Karthik Kambatla
5a42e1b7c3 YARN-1760. TestRMAdminService assumes CapacityScheduler. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571777 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 18:06:45 +00:00
Vinod Kumar Vavilapalli
ad70f26b1f YARN-1734. Fixed ResourceManager to update the configurations when it transits from standby to active mode so as to assimilate any changes that happened while it was in standby mode. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571539 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 02:07:19 +00:00
Sanford Ryza
1ab2f5a916 YARN-1678. Fair scheduler gabs incessantly about reservations (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571468 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-24 22:35:27 +00:00
Vinod Kumar Vavilapalli
e167e585e9 YARN-1749. Updated application-history related configs to reflect the latest reality and to be consistently named. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570948 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 02:04:58 +00:00
Zhijie Shen
da20095284 YARN-1071. Enabled ResourceManager to recover cluster metrics numDecommissionedNMs after restarting. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570469 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 06:46:24 +00:00
Vinod Kumar Vavilapalli
772ead791c YARN-1398. Fixed a deadlock in ResourceManager between users requesting queue-acls and completing containers. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570415 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 02:20:20 +00:00
Karthik Kambatla
12a8aa1510 YARN-1736. FS: AppSchedulable.assignContainer's priority argument is redundant. (Naren Koneru via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570336 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 20:09:54 +00:00
Vinod Kumar Vavilapalli
5fd5c9900c YARN-713. Fixed ResourceManager to not crash while building tokens when DNS issues happen transmittently. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569979 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 23:39:13 +00:00
Sanford Ryza
0e3e91a06d YARN-1718. Fix a couple isTerminals in Fair Scheduler queue placement rules (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569928 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 21:36:51 +00:00
Vinod Kumar Vavilapalli
9da9f7d4d8 YARN-1666. Modified RM HA handling of include/exclude node-lists to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569856 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:52:57 +00:00
Jonathan Turner Eagles
996acc834e YARN-1479. Invalid NaN values in Hadoop REST API JSON response (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569853 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:49:21 +00:00
Jian He
3a7e7b3671 YARN-1428. Fixed RM to write the final state of RMApp/RMAppAttempt to the application history store in the transition to the final state. (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569585 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 01:19:07 +00:00
Vinod Kumar Vavilapalli
6b03174aae YARN-1590. Fixed ResourceManager, web-app proxy and MR JobHistoryServer to expand _HOST properly in their kerberos principles. Contributed by Mohammad Kamrul Islam. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569537 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 21:50:37 +00:00
Sanford Ryza
bbbe808a51 YARN-1724. Race condition in Fair Scheduler when continuous scheduling is turned on (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569447 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 18:00:56 +00:00
Sanford Ryza
e60f5b6c40 YARN-1721. When moving app between queues in Fair Scheduler, grab lock on FSSchedulerApp (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569443 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 17:51:40 +00:00
Vinod Kumar Vavilapalli
990cffdcfa YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568501 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:02 +00:00
Vinod Kumar Vavilapalli
d0a5e43de7 YARN-1417. Modified RM to generate container-tokens not at creation time, but at allocation time so as to prevent RM 
from shelling out containers with expired tokens. Contributed by Omkar Vinit Joshi and Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568060 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 22:02:11 +00:00
Vinod Kumar Vavilapalli
2b916db60b YARN-1676. Modified RM HA handling of user-to-group mappings to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568041 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 21:23:33 +00:00
Jian He
dab635980d YARN-1345. Remove FINAL_SAVING state from YarnApplicationAttemptState. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567820 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 00:59:19 +00:00
Sanford Ryza
bc939f7ae6 YARN-1692. ConcurrentModificationException in fair scheduler AppSchedulable (Sangjin Lee via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567788 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 23:04:42 +00:00
Karthik Kambatla
41ec3cce76 YARN-1641. ZK store should attempt a write periodically to ensure it is still Active. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567628 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 14:09:13 +00:00
Vinod Kumar Vavilapalli
1fa6ab249b YARN-1459. Changed ResourceManager to depend its service initialization on the configuration-provider mechanism during startup too. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566791 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 22:50:15 +00:00
Vinod Kumar Vavilapalli
8b2336fcef YARN-1660. Simplified the RM HA configuration to accept and be able to simply depend just on 
configuration properties of the form yarn.resourcemanager.hostname.RMID and use the default
ports for all service addresses. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 03:18:39 +00:00
Vinod Kumar Vavilapalli
30294a2196 YARN-1665. Simplify the configuration of RM HA by having better default values. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565517 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:39:32 +00:00
Zhijie Shen
943b2190d7 YARN-1689. Made RMAppAttempt get killed when RMApp is at ACCEPTED. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565497 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 00:18:46 +00:00
Sanford Ryza
e6eccf20de YARN-1499. Fair Scheduler changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564856 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 18:09:07 +00:00
Zhijie Shen
ebe0c17a95 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564633 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:55:00 +00:00
Vinod Kumar Vavilapalli
b812af964d YARN-1669. Modified RM HA handling of protocol level service-ACLS to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564549 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-04 22:46:52 +00:00
Vinod Kumar Vavilapalli
24fa232707 YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564100 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 22:10:56 +00:00
Vinod Kumar Vavilapalli
eff1e809f2 YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564043 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:16:28 +00:00
Vinod Kumar Vavilapalli
3e7d56678c YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564032 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:53:46 +00:00
Sanford Ryza
cb5e0787a6 YARN-1498 addendum to fix findbugs warning 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564018 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:30:55 +00:00
Vinod Kumar Vavilapalli
ca72e11158 YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564002 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:07:55 +00:00
Sanford Ryza
1079c37128 YARN-1504. RM changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563371 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 04:03:36 +00:00
Jonathan Turner Eagles
5d4bd6a15a YARN-1632. TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563157 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 16:00:39 +00:00
Sanford Ryza
981679e6d7 YARN-1498. Common scheduler changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563021 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 00:14:47 +00:00
Sanford Ryza
1cac66ce68 YARN-1617. Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563004 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 22:40:22 +00:00
Karthik Kambatla
f1cef4cd29 YARN-1618. Fix invalid RMApp transition from NEW to FINAL_SAVING (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562529 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 18:02:40 +00:00