big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,888 Commits 2 Branches 2 Tags 264 MiB
e285b98f0f
Commit Graph

551 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
e285b98f0f YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:43:22 +00:00
Thomas Graves
1f9a0fd927 YARN-2072. RM/NM UIs and webservices are missing vcore information. (Nathan Roberts via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605162 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 19:34:34 +00:00
Karthik Kambatla
c0991d11eb YARN-2109. Fix TestRM to work with both schedulers. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605142 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 17:30:53 +00:00
Karthik Kambatla
db4d277117 YARN-2192. TestRMHA fails when run with a mix of Schedulers. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605138 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 17:05:35 +00:00
Sanford Ryza
29c102cad0 YARN-2111. In FairScheduler.attemptScheduling, we don't count containers as assigned if they have 0 memory but non-zero cores (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605113 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 15:40:39 +00:00
Vinod Kumar Vavilapalli
d16470025a YARN-2074. Changed ResourceManager to not count AM preemptions towards app failures. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605106 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 15:15:12 +00:00
Jian He
59b5e9fa15 YARN-2191. Added a new test to ensure NM will clean up completed applications in the case of RM restart. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604949 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-23 22:52:38 +00:00
Karthik Kambatla
6fcbf9b848 YARN-2187. FairScheduler: Disable max-AM-share check by default. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604321 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-21 07:30:07 +00:00
Haohui Mai
22cbcd16cb HADOOP-10717. Revert r1603571. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603620 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 20:37:25 +00:00
Steve Loughran
bae5837d3b HADOOP-10717. Missing JSP support in Jetty 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603534 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 17:08:45 +00:00
Jian He
95897ca14b YARN-1885. Fixed a bug that RM may not send application-clean-up signal to NMs where the completed applications previously ran in case of RM restart. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603028 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 23:56:12 +00:00
Karthik Kambatla
082c2ab8b8 YARN-2159. Better logging in SchedulerNode#allocateContainer. (Ray Chiang via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603003 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 20:45:38 +00:00
Vinod Kumar Vavilapalli
dc7dd1fa19 YARN-1702. Added kill app functionality to RM web services. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602298 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 21:31:52 +00:00
Karthik Kambatla
4bc91b44c9 YARN-2155. FairScheduler: Incorrect threshold check for preemption. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602295 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 21:23:32 +00:00
Jian He
c6e29a9f06 YARN-2125. Changed ProportionalCapacityPreemptionPolicy to log CSV in debug level. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601980 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-11 18:31:28 +00:00
Jian He
710a8693e5 YARN-2124. Fixed NPE in ProportionalCapacityPreemptionPolicy. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601964 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-11 17:30:18 +00:00
Karthik Kambatla
5de6f72054 YARN-1424. RMAppAttemptImpl should return the DummyApplicationResourceUsageReport for all invalid accesses. (Ray Chiang via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601742 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-10 19:03:06 +00:00
Jian He
c94f2cec3a Augmented RMStateStore with state machine. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601491 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-09 19:44:31 +00:00
Vinod Kumar Vavilapalli
424fd9494f YARN-1368. Added core functionality of recovering container state into schedulers after ResourceManager Restart so as to preserve running work in the cluster. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601303 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-09 03:09:21 +00:00
Karthik Kambatla
9958072d53 YARN-2132. ZKRMStateStore.ZKAction#runWithRetries doesn't log the exception it encounters. (Vamsee Yarlagadda via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601065 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-07 06:12:59 +00:00
Karthik Kambatla
85d4c787e0 YARN-2128. FairScheduler: Incorrect calculation of amResource usage. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601050 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-07 01:21:33 +00:00
Karthik Kambatla
969bef3c77 YARN-2122. In AllocationFileLoaderService, the reloadThread should be created in init() and started in start(). (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601045 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-07 00:49:20 +00:00
Junping Du
0ceb742549 YARN-1977. Add tests on getApplicationRequest with filtering start time range. (Contributed by Junping Du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600644 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-05 13:15:44 +00:00
Karthik Kambatla
d1d6bb70f4 YARN-2061. Revisit logging levels in ZKRMStateStore. (Ray Chiang via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600498 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-04 21:11:09 +00:00
Sanford Ryza
16caa3fd18 YARN-1913. With Fair Scheduler, cluster can logjam when all resources are consumed by AMs (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599400 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-03 00:56:48 +00:00
Karthik Kambatla
0aad2d56df YARN-1550. NPE in FairSchedulerAppsBlock#render. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599345 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-02 20:22:52 +00:00
Karthik Kambatla
a4ba451802 YARN-1474. Make schedulers services. (Tsuyoshi Ozawa via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598908 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 19:33:09 +00:00
Vinod Kumar Vavilapalli
23c325ad47 YARN-2115. Replaced RegisterNodeManagerRequest's ContainerStatus with a new NMContainerStatus which has more information that is needed for work-preserving RM-restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598790 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 00:20:50 +00:00
Karthik Kambatla
49a3a0cd0c YARN-2054. Better defaults for YARN ZK configs for retries and retry-inteval when HA is enabled. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598630 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:24:49 +00:00
Arpit Agarwal
4a4868e523 HADOOP-10448. Support pluggable mechanism to specify proxy user settings (Contributed by Benoy Antony) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598396 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 20:52:01 +00:00
Sanford Ryza
342da5b4d3 YARN-596. Use scheduling policies throughout the queue hierarchy to decide which containers to preempt (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598197 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 04:01:24 +00:00
Sanford Ryza
edfbc8ad4a YARN-2105. Fix TestFairScheduler after YARN-2012. (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597902 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-27 23:46:22 +00:00
Karthik Kambatla
7dd378c274 YARN-2096. Race in TestRMRestart#testQueueMetricsOnRMRestart. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597223 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 23:51:00 +00:00
Sanford Ryza
a00b2d4f37 YARN-2073. Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597209 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 22:52:46 +00:00
Sanford Ryza
6c56612af5 YARN-2012. Fair Scheduler: allow default queue placement rule to take an arbitrary queue (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597204 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 22:38:52 +00:00
Karthik Kambatla
619ec833fc YARN-2089. FairScheduler: QueuePlacementPolicy and QueuePlacementRule are missing audience annotations. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596765 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 06:13:24 +00:00
Vinod Kumar Vavilapalli
82f3454f5a YARN-2017. Merged some of the common scheduler code. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596753 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 05:32:26 +00:00
Jian He
0f9147c857 YARN-2053. Fixed a bug in AMS to not add null NMToken into NMTokens list from previous attempts for work-preserving AM restart. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595116 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-16 06:22:22 +00:00
Sanford Ryza
561c014e77 YARN-2042. String shouldn't be compared using == in QueuePlacementRule#NestedUserQueue#getQueueForApp (Chen He via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594482 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-14 06:55:04 +00:00
Sanford Ryza
84dfae2f8a YARN-1986. In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE (Hong Zhiguo via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594476 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-14 06:41:20 +00:00
Christopher Douglas
45b42676f9 YARN-1957. Consider the max capacity of the queue when computing the ideal 
capacity for preemption. Contributed by Carlo Curino


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594414 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 23:15:27 +00:00
Jonathan Turner Eagles
1c48142807 YARN-1981. Nodemanager version is not updated when a node reconnects (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594358 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 20:03:58 +00:00
Vinod Kumar Vavilapalli
2f87e77cb7 YARN-1861. Fixed a bug in RM to reset leader-election on fencing that was causing both RMs to be stuck in standby mode when automatic failover is enabled. Contributed by Karthik Kambatla and Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594356 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 20:00:44 +00:00
Jian He
41344a4a69 YARN-1975. Fix yarn application CLI to print the scheme of the tracking url of failed/killed applications. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593874 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-12 00:43:35 +00:00
Junping Du
ca95af7d23 YARN-2011. Fix typo and warning in TestLeafQueue (Contributed by Chen He) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593804 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 15:13:29 +00:00
Sanford Ryza
1c5902fba2 YARN-1864. Add missing file FSQueueType.java 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593191 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-08 07:22:42 +00:00
Sanford Ryza
cfc97a4e88 YARN-1864. Fair Scheduler Dynamic Hierarchical User Queues (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593190 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-08 07:21:11 +00:00
Arpit Agarwal
f4b687b873 YARN-2018. TestClientRMService.testTokenRenewalWrongUser fails after HADOOP-10562. (Contributed by Ming Ma) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592783 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-06 15:45:49 +00:00
Junping Du
2ad1cee5da YARN-1201. TestAMAuthorization fails with local hostname cannot be resolved. (Wangda Tan via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592197 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-03 13:03:27 +00:00
Haohui Mai
d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-02 22:26:02 +00:00