big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,742 Commits 2 Branches 2 Tags 264 MiB
16caa3fd18
Commit Graph

527 Commits

Author SHA1 Message Date
Sanford Ryza
16caa3fd18 YARN-1913. With Fair Scheduler, cluster can logjam when all resources are consumed by AMs (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599400 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-03 00:56:48 +00:00
Karthik Kambatla
0aad2d56df YARN-1550. NPE in FairSchedulerAppsBlock#render. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599345 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-02 20:22:52 +00:00
Karthik Kambatla
a4ba451802 YARN-1474. Make schedulers services. (Tsuyoshi Ozawa via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598908 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 19:33:09 +00:00
Vinod Kumar Vavilapalli
23c325ad47 YARN-2115. Replaced RegisterNodeManagerRequest's ContainerStatus with a new NMContainerStatus which has more information that is needed for work-preserving RM-restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598790 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 00:20:50 +00:00
Karthik Kambatla
49a3a0cd0c YARN-2054. Better defaults for YARN ZK configs for retries and retry-inteval when HA is enabled. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598630 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:24:49 +00:00
Arpit Agarwal
4a4868e523 HADOOP-10448. Support pluggable mechanism to specify proxy user settings (Contributed by Benoy Antony) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598396 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 20:52:01 +00:00
Sanford Ryza
342da5b4d3 YARN-596. Use scheduling policies throughout the queue hierarchy to decide which containers to preempt (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598197 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 04:01:24 +00:00
Sanford Ryza
edfbc8ad4a YARN-2105. Fix TestFairScheduler after YARN-2012. (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597902 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-27 23:46:22 +00:00
Karthik Kambatla
7dd378c274 YARN-2096. Race in TestRMRestart#testQueueMetricsOnRMRestart. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597223 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 23:51:00 +00:00
Sanford Ryza
a00b2d4f37 YARN-2073. Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597209 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 22:52:46 +00:00
Sanford Ryza
6c56612af5 YARN-2012. Fair Scheduler: allow default queue placement rule to take an arbitrary queue (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597204 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 22:38:52 +00:00
Karthik Kambatla
619ec833fc YARN-2089. FairScheduler: QueuePlacementPolicy and QueuePlacementRule are missing audience annotations. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596765 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 06:13:24 +00:00
Vinod Kumar Vavilapalli
82f3454f5a YARN-2017. Merged some of the common scheduler code. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596753 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 05:32:26 +00:00
Jian He
0f9147c857 YARN-2053. Fixed a bug in AMS to not add null NMToken into NMTokens list from previous attempts for work-preserving AM restart. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595116 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-16 06:22:22 +00:00
Sanford Ryza
561c014e77 YARN-2042. String shouldn't be compared using == in QueuePlacementRule#NestedUserQueue#getQueueForApp (Chen He via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594482 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-14 06:55:04 +00:00
Sanford Ryza
84dfae2f8a YARN-1986. In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE (Hong Zhiguo via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594476 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-14 06:41:20 +00:00
Christopher Douglas
45b42676f9 YARN-1957. Consider the max capacity of the queue when computing the ideal 
capacity for preemption. Contributed by Carlo Curino


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594414 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 23:15:27 +00:00
Jonathan Turner Eagles
1c48142807 YARN-1981. Nodemanager version is not updated when a node reconnects (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594358 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 20:03:58 +00:00
Vinod Kumar Vavilapalli
2f87e77cb7 YARN-1861. Fixed a bug in RM to reset leader-election on fencing that was causing both RMs to be stuck in standby mode when automatic failover is enabled. Contributed by Karthik Kambatla and Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594356 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 20:00:44 +00:00
Jian He
41344a4a69 YARN-1975. Fix yarn application CLI to print the scheme of the tracking url of failed/killed applications. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593874 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-12 00:43:35 +00:00
Junping Du
ca95af7d23 YARN-2011. Fix typo and warning in TestLeafQueue (Contributed by Chen He) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593804 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 15:13:29 +00:00
Sanford Ryza
1c5902fba2 YARN-1864. Add missing file FSQueueType.java 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593191 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-08 07:22:42 +00:00
Sanford Ryza
cfc97a4e88 YARN-1864. Fair Scheduler Dynamic Hierarchical User Queues (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593190 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-08 07:21:11 +00:00
Arpit Agarwal
f4b687b873 YARN-2018. TestClientRMService.testTokenRenewalWrongUser fails after HADOOP-10562. (Contributed by Ming Ma) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592783 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-06 15:45:49 +00:00
Junping Du
2ad1cee5da YARN-1201. TestAMAuthorization fails with local hostname cannot be resolved. (Wangda Tan via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592197 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-03 13:03:27 +00:00
Haohui Mai
d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-02 22:26:02 +00:00
Vinod Kumar Vavilapalli
7a241aee90 YARN-1929. Fixed a deadlock in ResourceManager that occurs when failover happens right at the time of shutdown. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1591071 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-29 19:49:44 +00:00
Jason Darrell Lowe
a9775b4e49 YARN-738. TestClientRMTokens is failing irregularly while running all yarn tests. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1591030 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-29 17:47:11 +00:00
Jing Zhao
6d4c7df434 HADOOP-10535. Make the retry numbers in ActiveStandbyElector configurable. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589905 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-24 23:03:30 +00:00
Jason Darrell Lowe
c02ea74551 YARN-1975. Used resources shows escaped html in CapacityScheduler and FairScheduler page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589859 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-24 20:02:44 +00:00
Chris Nauroth
84388525a3 YARN-1970. Prepare YARN codebase for JUnit 4.11. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589001 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-21 23:31:18 +00:00
Vinod Kumar Vavilapalli
bad021534c YARN-1281. Fixed TestZKRMStateStoreZKClientConnections to not fail intermittently due to ZK-client timeouts. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588369 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 20:57:15 +00:00
Junping Du
bd43d2481e YARN-1947. TestRMDelegationTokens#testRMDTMasterKeyStateOnRollingMasterKey is failing intermittently. (Jian He via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588365 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 20:27:37 +00:00
Jian He
38d5ca2e47 YARN-1934. Fixed a potential NPE in ZKRMStateStore caused by handling Disconnected event from ZK. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587776 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-16 02:02:18 +00:00
Zhijie Shen
44b6261bfa YARN-1892. Improved some logs in the scheduler. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587717 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-15 20:37:44 +00:00
Vinod Kumar Vavilapalli
eb7b33c298 YARN-1928. Fixed a race condition in TestAMRMRPCNodeUpdates which caused it to fail occassionally. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587114 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-13 22:40:16 +00:00
Vinod Kumar Vavilapalli
c6b70f4760 YARN-1933. Fixed test issues with TestAMRestart and TestNodeHealthService. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587104 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-13 21:51:38 +00:00
Sanford Ryza
9274626a11 YARN-1923. Make Fair Scheduler resource ratio calculations terminate faster (Anubhav Dhoot via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586796 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 23:24:40 +00:00
Zhijie Shen
dc84800184 YARN-1924. Made ZKRMStateStore updateApplication(Attempt)StateInternal work when Application(Attempt) state hasn't been stored before. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586547 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 03:36:36 +00:00
Vinod Kumar Vavilapalli
791f2f6f1b YARN-1910. Fixed a race condition in TestAMRMTokens that causes the test to fail more often on Windows. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586192 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 01:36:50 +00:00
Kihwal Lee
c3f6d71726 YARN-1907. TestRMApplicationHistoryWriter#testRMWritingMassiveHistory intermittently fails. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585992 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 15:33:25 +00:00
Karthik Kambatla
1e1200d594 YARN-1784. TestContainerAllocation assumes CapacityScheduler. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585894 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 05:49:24 +00:00
Arun Murthy
05da90ee2f YARN-1898. Addendum patch to ensure /jmx and /metrics are re-directed to Active RM. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584954 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-05 02:51:08 +00:00
Jian He
15ac36a216 YARN-1837. Fixed TestMoveApplication#testMoveRejectedByScheduler failure. Contributed by Hong Zhiguo 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584862 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-04 18:42:35 +00:00
Zhijie Shen
b0180afc9c YARN-1898. Made Standby RM links conf, stacks, logLevel, metrics, jmx, logs and static not be redirected to Active RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583833 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-02 00:01:50 +00:00
Sanford Ryza
7bd62b8da0 YARN-1889. In Fair Scheduler, avoid creating objects on each call to AppSchedulable comparator (Hong Zhiguo via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583491 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-31 23:40:45 +00:00
Jonathan Turner Eagles
6d2e1fde63 YARN-1883. TestRMAdminService fails due to inconsistent entries in UserGroups (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582862 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 19:30:08 +00:00
Tsz-wo Sze
3f7aa79e58 HDFS-6168. Remove a deprecated constructor and the deprecated methods reportChecksumFailure, getDelegationToken(Text), renewDelegationToken and cancelDelegationToken from DistributedFileSystem. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582856 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 18:29:31 +00:00
Jian He
71c2b159ab YARN-1521. Mark Idempotent/AtMostOnce annotations to the APIs in ApplicationClientProtcol, ResourceManagerAdministrationProtocol and ResourceTrackerProtocol so that they work in HA scenario. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581678 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 03:43:41 +00:00
Vinod Kumar Vavilapalli
8a9ae9e3ec YARN-1867. Fixed a bug in ResourceManager that was causing invalid ACL checks in the web-services after fail-over. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581662 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 03:03:17 +00:00