big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8,276 Commits 2 Branches 2 Tags 264 MiB
86840834ed
Commit Graph

1129 Commits

Author SHA1 Message Date
Jian He
c3084d6c16 YARN-2389. Added functionality for schedulers to kill all applications in a queue. Contributed by Subramaniam Venkatraman Krishnan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618294 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 23:53:57 +00:00
Jian He
7360cec692 YARN-2378. Added support for moving applications across queues in CapacityScheduler. Contributed by Subramaniam Venkatraman Krishnan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618106 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 06:00:31 +00:00
Allen Wittenauer
0350ea3c72 YARN-1918. Typo in description and error message for yarn.resourcemanager.cluster-id (Anandha L Ranganathan via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618070 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 21:53:12 +00:00
Zhijie Shen
a9023c2736 YARN-2397. Avoided loading two authentication filters for RM and TS web interfaces. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618054 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 21:17:20 +00:00
Zhijie Shen
1a535f8dcd YARN-2070. Made DistributedShell publish the short user name to the timeline server. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617837 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 21:07:18 +00:00
Zhijie Shen
41aa4badf8 YARN-2277. Added cross-origin support for the timeline server web services. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617832 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 20:29:23 +00:00
Karthik Kambatla
5197f8c3c5 YARN-1370. Fair scheduler to re-populate container allocation state. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617645 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 01:38:59 +00:00
Karthik Kambatla
4239695588 YARN-2399. Delete old versions of files. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617619 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 22:51:57 +00:00
Karthik Kambatla
486e718fc1 YARN-2399. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617600 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 21:43:27 +00:00
Zhijie Shen
7ad7ab1723 YARN-2317. Updated the document about how to write YARN applications. Contributed by Li Lu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617594 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 21:37:34 +00:00
Jian He
0c9b8f2d7f YARN-2373. Changed WebAppUtils to use Configuration#getPassword for accessing SSL passwords. Contributed by Larry McCay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617555 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 18:29:42 +00:00
Junping Du
c2febdcbaa YARN-1337. Recover containers upon nodemanager restart. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617448 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 10:56:13 +00:00
Jian He
c4dc685343 YARN-2138. Cleaned up notifyDone* APIs in RMStateStore. Contributed by Varun Saxena 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617341 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 18:24:24 +00:00
Xuan Gong
946be75704 YARN-2400: Addendum fix for TestAMRestart failure. Contributed by Jian He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617333 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 17:42:53 +00:00
Karthik Kambatla
da7b508ffc YARN-2361. RMAppAttempt state machine entries for KILLED state has duplicate event entries. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617190 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 01:42:26 +00:00
Karthik Kambatla
bdd3e2ce49 YARN-2337. ResourceManager sets ClientRMService in RMContext multiple times. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617183 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 00:13:27 +00:00
Junping Du
e91d099c4a YARN-2302. Refactor TimelineWebServices. (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617055 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-10 07:21:15 +00:00
Xuan Gong
743f7f30da YARN-2400. Fixed TestAMRestart fails intermittently. Contributed by Jian He: 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617028 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-09 23:31:11 +00:00
Zhijie Shen
ee3825e278 YARN-1954. Added waitFor to AMRMClient(Async). Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617002 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-09 18:44:51 +00:00
Karthik Kambatla
a7643f4de7 YARN-2026. Fair scheduler: Consider only active queues for computing fairshare. (Ashwin Shankar via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616915 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-09 02:10:00 +00:00
Xuan Gong
eeb4acd955 YARN-2212: ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616892 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:38:24 +00:00
Karthik Kambatla
6d39525f6b YARN-2352. Add missing file. FairScheduler: Collect metrics on duration of critical methods that affect performance. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616784 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 14:38:18 +00:00
Karthik Kambatla
14864e9c7c YARN-2352. FairScheduler: Collect metrics on duration of critical methods that affect performance. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616769 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 14:17:54 +00:00
Jian He
8437df8ba9 YARN-2008. Fixed CapacityScheduler to calculate headroom based on max available capacity instead of configured max capacity. Contributed by Craig Welch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616580 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 20:00:04 +00:00
Zhijie Shen
3bfdb0091d YARN-2288. Made persisted data in LevelDB timeline store be versioned. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616540 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 17:02:26 +00:00
Zhijie Shen
be6360593b YARN-2388. Fixed TestTimelineWebServices failure due to HADOOP-10791. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616405 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 05:36:50 +00:00
Karthik Kambatla
8feddc4c84 YARN-2359. Application hangs when it fails to launch AM container. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616375 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 00:06:17 +00:00
Jian He
9ebeac16eb YARN-2374. Fixed TestDistributedShell#testDSShell failure due to hostname dismatch. Contributed by Varun Vasudev 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616302 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 18:58:50 +00:00
Junping Du
b47f65214c YARN-2298. Move TimelineClient to yarn-common project (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616100 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 05:48:32 +00:00
Junping Du
b8f151231b YARN-1354. Recover applications upon nodemanager restart. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615550 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-04 13:25:37 +00:00
Junping Du
04ae560370 YARN-2370. Fix comment in o.a.h.y.server.resourcemanager.schedulerAppSchedulingInfo (Contributed by Wenwu Peng) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615469 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-04 01:15:46 +00:00
Jian He
a41c314373 YARN-2343. Improve NMToken expire exception message. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615270 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 23:44:48 +00:00
Junping Du
1141edc424 YARN-2051. Fix bug in PBimpls and add more unit tests with reflection. (Contributed by Binglin Chang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615025 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 04:16:08 +00:00
Xuan Gong
e52f67e389 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 20:06:02 +00:00
Zhijie Shen
1d6e178144 YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614838 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 09:27:43 +00:00
Jian He
7e54b1c6d9 YARN-2354. DistributedShell may allocate more containers than client specified after AM restarts. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614538 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 03:58:59 +00:00
Karthik Kambatla
c0b49ff107 YARN-2328. FairScheduler: Verify update and continuous scheduling threads are stopped when the scheduler is stopped. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614432 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 17:41:52 +00:00
Zhijie Shen
d6532d3a77 YARN-2247. Made RM web services authenticate users via kerberos and delegation token. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613821 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 17:55:06 +00:00
Aaron Myers
5d4677b57b YARN-1796. container-executor shouldn't require o-r permissions. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613548 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-26 01:51:35 +00:00
Jian He
d4fec34933 YARN-2211. Persist AMRMToken master key in RMStateStore for RM recovery. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613515 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 20:42:37 +00:00
Karthik Kambatla
1e553858f9 YARN-2214. FairScheduler: preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness. (Ashwin Shankar via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613459 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 16:13:07 +00:00
Devarajulu K
2050e0dad6 YARN-1342. Recover container tokens upon nodemanager restart. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612995 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 05:02:00 +00:00
Zhijie Shen
06e5c5cb2d YARN-2300. Improved the documentation of the sample requests for RM REST API - submitting an app. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 03:12:45 +00:00
Jason Darrell Lowe
28fca92521 YARN-2147. client lacks delegation token exception details when application submit fails. Contributed by Chen He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612950 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 21:40:57 +00:00
Sanford Ryza
c88402f36d YARN-2313. Livelock can occur in FairScheduler when there are lots of running apps (Tsuyoshi Ozawa via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612769 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 05:00:52 +00:00
Karthik Kambatla
ff77582991 YARN-2273. NPE in ContinuousScheduling thread when we lose a node. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612720 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 22:44:38 +00:00
Karthik Kambatla
df2abcb372 YARN-2131. Addendum2: Document -format-state-store. Add a way to format the RMStateStore. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612634 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 18:11:11 +00:00
Jian He
f6c723ff0c YARN-2295. Refactored DistributedShell to use public APIs of protocol records. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612626 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 17:49:14 +00:00
Zhijie Shen
eac0701c96 YARN-2319. Made the MiniKdc instance start/close before/after the class of TestRMWebServicesDelegationTokens. Contributed by Wenwu Peng. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612588 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 15:15:29 +00:00
Junping Du
afb9394c91 YARN-2242. Addendum patch. Improve exception information on AM launch crashes. (Contributed by Li Lu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612565 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 13:07:23 +00:00