carlo curino
|
eb3e40b833
|
YARN-1707. Introduce APIs to add/remove/resize queues in the CapacityScheduler. Contributed by Carlo Curino and Subru Krishnan
(cherry picked from commit aac47fda7fecda9fc18ade34d633eca895865a70)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
|
2014-10-03 15:41:02 -07:00 |
|
Karthik Kambatla
|
80d11eb68e
|
YARN-2635. TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS. (Wei Yan and kasha via kasha)
|
2014-10-03 11:49:49 -07:00 |
|
Jian He
|
054f285526
|
YARN-2628. Capacity scheduler with DominantResourceCalculator carries out reservation even though slots are free. Contributed by Varun Vasudev
|
2014-10-02 15:13:33 -07:00 |
|
Karthik Kambatla
|
5e0b49da9c
|
YARN-2254. TestRMWebServicesAppsModification should run against both CS and FS. (Zhihai Xu via kasha)
|
2014-10-02 10:15:04 -07:00 |
|
Zhijie Shen
|
52bbe0f11b
|
YARN-2630. Prevented previous AM container status from being acquired by the current restarted AM. Contributed by Jian He.
|
2014-10-01 15:38:11 -07:00 |
|
Jian He
|
bbff96be48
|
YARN-2602. Fixed possible NPE in ApplicationHistoryManagerOnTimelineStore. Contributed by Zhijie Shen
|
2014-09-30 16:44:17 -07:00 |
|
Karthik Kambatla
|
14d60dadc2
|
YARN-2594. Potential deadlock in RM when querying ApplicationResourceUsageReport. (Wangda Tan via kasha)
|
2014-09-30 16:12:27 -07:00 |
|
Zhijie Shen
|
9582a50176
|
YARN-2627. Added the info logs of attemptFailuresValidityInterval and number of previous failed attempts. Contributed by Xuan Gong.
|
2014-09-30 12:57:45 -07:00 |
|
Jason Lowe
|
9c22065109
|
YARN-1769. CapacityScheduler: Improve reservations. Contributed by Thomas Graves
|
2014-09-29 14:12:18 +00:00 |
|
Jian He
|
5391919b09
|
YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du.
|
2014-09-26 17:48:41 -07:00 |
|
Karthik Kambatla
|
f4357240a6
|
YARN-2608. FairScheduler: Potential deadlocks in loading alloc files and clock access. (Wei Yan via kasha)
|
2014-09-25 17:42:51 -07:00 |
|
Jason Lowe
|
8269bfa613
|
YARN-2523. ResourceManager UI showing negative value for "Decommissioned Nodes" field. Contributed by Rohith
|
2014-09-25 22:37:05 +00:00 |
|
Zhijie Shen
|
72b0881ca6
|
YARN-2546. Made REST API for application creation/submission use numeric and boolean types instead of the string of them. Contributed by Varun Vasudev.
|
2014-09-24 17:57:32 -07:00 |
|
Zhijie Shen
|
c86674a3a4
|
YARN-2581. Passed LogAggregationContext to NM via ContainerTokenIdentifier. Contributed by Xuan Gong.
|
2014-09-24 17:50:26 -07:00 |
|
Karthik Kambatla
|
39c87344e1
|
YARN-2596. TestWorkPreservingRMRestart fails with FairScheduler. (kasha)
|
2014-09-24 15:23:31 -07:00 |
|
Karthik Kambatla
|
f5578207d2
|
YARN-2252. Intermittent failure of TestFairScheduler.testContinuousScheduling. (Ratandeep Ratti and kasha via kasha)
|
2014-09-23 00:03:16 -07:00 |
|
Karthik Kambatla
|
568d3dc2bb
|
YARN-1959. Fix headroom calculation in FairScheduler. (Anubhav Dhoot via kasha)
|
2014-09-22 23:49:39 -07:00 |
|
Karthik Kambatla
|
43efdd30b5
|
YARN-2539. FairScheduler: Set the default value for maxAMShare to 0.5. (Wei Yan via kasha)
|
2014-09-22 16:09:52 -07:00 |
|
Karthik Kambatla
|
23e17ce0ff
|
YARN-2540. FairScheduler: Queue filters not working on scheduler page in RM UI. (Ashwin Shankar via kasha)
|
2014-09-22 15:40:47 -07:00 |
|
Jian He
|
0a641496c7
|
YARN-1372. Ensure all completed containers are reported to the AMs across RM restart. Contributed by Anubhav Dhoot
|
2014-09-22 10:30:53 -07:00 |
|
Karthik Kambatla
|
9721e2c1fe
|
YARN-2453. TestProportionalCapacityPreemptionPolicy fails with FairScheduler. (Zhihai Xu via kasha)
|
2014-09-21 23:13:45 -07:00 |
|
Karthik Kambatla
|
c50fc92502
|
YARN-2452. TestRMApplicationHistoryWriter fails with FairScheduler. (Zhihai Xu via kasha)
|
2014-09-21 13:15:04 -07:00 |
|
Jian He
|
444acf8ea7
|
YARN-2565. Fixed RM to not use FileSystemApplicationHistoryStore unless explicitly set. Contributed by Zhijie Shen
|
2014-09-19 11:26:29 -07:00 |
|
Zhijie Shen
|
6fe5c6b746
|
YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He.
|
2014-09-18 21:56:56 -07:00 |
|
Jason Lowe
|
a337f0e354
|
YARN-2561. MR job client cannot reconnect to AM after NM restart. Contributed by Junping Du
|
2014-09-18 21:34:40 +00:00 |
|
Jason Lowe
|
9ea7b6c063
|
YARN-2363. Submitted applications occasionally lack a tracking URL. Contributed by Jason Lowe
|
2014-09-18 20:13:16 +00:00 |
|
Vinod Kumar Vavilapalli
|
485c96e3cb
|
YARN-2001. Added a time threshold for RM to wait before starting container allocations after restart/failover. Contributed by Jian He.
|
2014-09-18 11:03:12 -07:00 |
|
Jian He
|
ee21b13cbd
|
YARN-2559. Fixed NPE in SystemMetricsPublisher when retrieving FinalApplicationStatus. Contributed by Zhijie Shen
|
2014-09-17 21:44:15 -07:00 |
|
junping_du
|
90a0c03f0a
|
YARN-1250. Generic history service should support application-acls. (Contributed by Zhijie Shen)
|
2014-09-16 18:20:49 -07:00 |
|
Vinod Kumar Vavilapalli
|
14e2639fd0
|
YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong.
|
2014-09-13 18:04:05 -07:00 |
|
XuanGong
|
e65ae575a0
|
YARN-2456. Possible livelock in CapacityScheduler when RM is recovering
apps. Contributed by Jian He
|
2014-09-12 15:21:46 -07:00 |
|
Jian He
|
3122daa802
|
YARN-2229. Changed the integer field of ContainerId to be long type. Contributed by Tsuyoshi OZAWA
|
2014-09-12 10:33:33 -07:00 |
|
Jason Lowe
|
78b048393a
|
YARN-2484. FileSystemRMStateStore#readFile/writeFile should close FSData(In|Out)putStream in final block. Contributed by Tsuyoshi OZAWA
|
2014-09-12 14:09:14 +00:00 |
|
Zhijie Shen
|
469ea3dcef
|
YARN-2538. Added logs when RM sends roll-overed AMRMToken to AM. Contributed by Xuan Gong.
|
2014-09-11 22:19:00 -07:00 |
|
junping_du
|
6b8b1608e6
|
YARN-2033. Merging generic-history into the Timeline Store (Contributed by Zhijie Shen)
|
2014-09-12 10:04:51 +08:00 |
|
Karthik Kambatla
|
c11ada5ea6
|
YARN-2534. FairScheduler: Potential integer overflow calculating totalMaxShare. (Zhihai Xu via kasha)
|
2014-09-11 12:06:06 -07:00 |
|
Jian He
|
83be3ad444
|
YARN-415. Capture aggregate memory allocation at the app-level for chargeback. Contributed by Eric Payne & Andrey Klochkov
|
2014-09-10 18:20:54 -07:00 |
|
Jian He
|
cbfe26370b
|
YARN-2158. Fixed TestRMWebServicesAppsModification#testSingleAppKill test failure. Contributed by Varun Vasudev
|
2014-09-10 12:47:34 -07:00 |
|
XUAN
|
47bdfa044a
|
YARN-2459. RM crashes if App gets rejected for any reason and HA is enabled. Contributed by Jian He
|
2014-09-10 11:44:41 -07:00 |
|
Vinod Kumar Vavilapalli
|
b67d5ba784
|
YARN-2448. Changed ApplicationMasterProtocol to expose RM-recognized resource types to the AMs. Contributed by Varun Vasudev.
|
2014-09-10 10:15:47 -07:00 |
|
Karthik Kambatla
|
3072c83b38
|
YARN-1458. FairScheduler: Zero weight can lead to livelock. (Zhihai Xu via kasha)
|
2014-09-10 08:26:14 -07:00 |
|
Karthik Kambatla
|
1dcaba9a7a
|
YARN-2394. FairScheduler: Configure fairSharePreemptionThreshold per queue. (Wei Yan via kasha)
|
2014-09-03 10:27:36 -07:00 |
|
Karthik Kambatla
|
0f34e6f387
|
YARN-2395. FairScheduler: Preemption timeout should be configurable per queue. (Wei Yan via kasha)
|
2014-08-30 01:17:13 -07:00 |
|
Jian He
|
5c14bc426b
|
YARN-1506. Changed RMNode/SchedulerNode to update resource with event notification. Contributed by Junping Du
|
2014-08-29 23:05:51 -07:00 |
|
Karthik Kambatla
|
270a271f53
|
YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the scheduler page. (Ashwin Shankar and Wei Yan via kasha)
|
2014-08-29 17:15:38 -07:00 |
|
Jian He
|
c686aa3533
|
YARN-2447. RM web service app submission doesn't pass secrets correctly. Contributed by Varun Vasudev
|
2014-08-29 11:40:47 -07:00 |
|
Hitesh Shah
|
3de66011c2
|
YARN-2450. Fix typos in log messages. Contributed by Ray Chiang.
|
2014-08-29 11:16:36 -07:00 |
|
Karthik Kambatla
|
fa80ca49bd
|
YARN-2405. NPE in FairSchedulerAppsBlock. (Tsuyoshi Ozawa via kasha)
|
2014-08-28 23:21:37 -07:00 |
|
Jian
|
7b3e27ab73
|
YARN-2406. Move RM recovery related proto to yarn_server_resourcemanager_recovery.proto. Contributed by Tsuyoshi OZAWA
|
2014-08-28 21:47:26 -07:00 |
|
Karthik Kambatla
|
d16bfd1d0f
|
YARN-1326. RM should log using RMStore at startup time. (Tsuyoshi Ozawa via kasha)
|
2014-08-27 01:43:58 -07:00 |
|
Karthik Kambatla
|
0097b15e21
|
YARN-2393. FairScheduler: Add the notion of steady fair share. (Wei Yan via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619845 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-22 15:44:47 +00:00 |
|
Jason Darrell Lowe
|
4236c6600e
|
YARN-2434. RM should not recover containers from previously failed attempt when AM restart is not enabled. Contributed by Jian He
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619614 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-21 22:41:34 +00:00 |
|
Karthik Kambatla
|
e1dd210fa5
|
YARN-1919. Potential NPE in EmbeddedElectorService#stop. (Tsuyoshi Ozawa via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619251 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-20 22:10:52 +00:00 |
|
Zhijie Shen
|
f6a778c372
|
YARN-2249. Avoided AM release requests being lost on work preserving RM restart. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618972 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-19 20:33:49 +00:00 |
|
Jian He
|
375c221960
|
YARN-2409. RM ActiveToStandBy transition missing stoping previous rmDispatcher. Contributed by Rohith
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618915 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-19 17:49:39 +00:00 |
|
Jian He
|
519c4be95a
|
YARN-2411. Support simple user and group mappings to queues. Contributed by Ram Venkatesh
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618542 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-18 06:08:45 +00:00 |
|
Jian He
|
c3084d6c16
|
YARN-2389. Added functionality for schedulers to kill all applications in a queue. Contributed by Subramaniam Venkatraman Krishnan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618294 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-15 23:53:57 +00:00 |
|
Jian He
|
7360cec692
|
YARN-2378. Added support for moving applications across queues in CapacityScheduler. Contributed by Subramaniam Venkatraman Krishnan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618106 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-15 06:00:31 +00:00 |
|
Zhijie Shen
|
a9023c2736
|
YARN-2397. Avoided loading two authentication filters for RM and TS web interfaces. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618054 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-14 21:17:20 +00:00 |
|
Karthik Kambatla
|
5197f8c3c5
|
YARN-1370. Fair scheduler to re-populate container allocation state. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617645 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-13 01:38:59 +00:00 |
|
Karthik Kambatla
|
4239695588
|
YARN-2399. Delete old versions of files. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617619 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-12 22:51:57 +00:00 |
|
Karthik Kambatla
|
486e718fc1
|
YARN-2399. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617600 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-12 21:43:27 +00:00 |
|
Junping Du
|
c2febdcbaa
|
YARN-1337. Recover containers upon nodemanager restart. (Contributed by Jason Lowe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617448 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-12 10:56:13 +00:00 |
|
Jian He
|
c4dc685343
|
YARN-2138. Cleaned up notifyDone* APIs in RMStateStore. Contributed by Varun Saxena
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617341 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-11 18:24:24 +00:00 |
|
Xuan Gong
|
946be75704
|
YARN-2400: Addendum fix for TestAMRestart failure. Contributed by Jian He
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617333 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-11 17:42:53 +00:00 |
|
Karthik Kambatla
|
da7b508ffc
|
YARN-2361. RMAppAttempt state machine entries for KILLED state has duplicate event entries. (Zhihai Xu via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617190 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-11 01:42:26 +00:00 |
|
Karthik Kambatla
|
bdd3e2ce49
|
YARN-2337. ResourceManager sets ClientRMService in RMContext multiple times. (Zhihai Xu via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617183 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-11 00:13:27 +00:00 |
|
Xuan Gong
|
743f7f30da
|
YARN-2400. Fixed TestAMRestart fails intermittently. Contributed by Jian He:
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617028 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-09 23:31:11 +00:00 |
|
Karthik Kambatla
|
a7643f4de7
|
YARN-2026. Fair scheduler: Consider only active queues for computing fairshare. (Ashwin Shankar via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616915 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-09 02:10:00 +00:00 |
|
Xuan Gong
|
eeb4acd955
|
YARN-2212: ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616892 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-08 21:38:24 +00:00 |
|
Karthik Kambatla
|
6d39525f6b
|
YARN-2352. Add missing file. FairScheduler: Collect metrics on duration of critical methods that affect performance. (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616784 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-08 14:38:18 +00:00 |
|
Karthik Kambatla
|
14864e9c7c
|
YARN-2352. FairScheduler: Collect metrics on duration of critical methods that affect performance. (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616769 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-08 14:17:54 +00:00 |
|
Jian He
|
8437df8ba9
|
YARN-2008. Fixed CapacityScheduler to calculate headroom based on max available capacity instead of configured max capacity. Contributed by Craig Welch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616580 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-07 20:00:04 +00:00 |
|
Karthik Kambatla
|
8feddc4c84
|
YARN-2359. Application hangs when it fails to launch AM container. (Zhihai Xu via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616375 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-07 00:06:17 +00:00 |
|
Junping Du
|
b8f151231b
|
YARN-1354. Recover applications upon nodemanager restart. (Contributed by Jason Lowe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615550 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-04 13:25:37 +00:00 |
|
Junping Du
|
04ae560370
|
YARN-2370. Fix comment in o.a.h.y.server.resourcemanager.schedulerAppSchedulingInfo (Contributed by Wenwu Peng)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615469 13f79535-47bb-0310-9956-ffa450edef68
|
2014-08-04 01:15:46 +00:00 |
|
Xuan Gong
|
e52f67e389
|
YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-31 20:06:02 +00:00 |
|
Zhijie Shen
|
1d6e178144
|
YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614838 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-31 09:27:43 +00:00 |
|
Karthik Kambatla
|
c0b49ff107
|
YARN-2328. FairScheduler: Verify update and continuous scheduling threads are stopped when the scheduler is stopped. (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614432 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-29 17:41:52 +00:00 |
|
Zhijie Shen
|
d6532d3a77
|
YARN-2247. Made RM web services authenticate users via kerberos and delegation token. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613821 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-27 17:55:06 +00:00 |
|
Jian He
|
d4fec34933
|
YARN-2211. Persist AMRMToken master key in RMStateStore for RM recovery. Contributed by Xuan Gong
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613515 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-25 20:42:37 +00:00 |
|
Karthik Kambatla
|
1e553858f9
|
YARN-2214. FairScheduler: preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness. (Ashwin Shankar via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613459 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-25 16:13:07 +00:00 |
|
Jason Darrell Lowe
|
28fca92521
|
YARN-2147. client lacks delegation token exception details when application submit fails. Contributed by Chen He
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612950 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-23 21:40:57 +00:00 |
|
Sanford Ryza
|
c88402f36d
|
YARN-2313. Livelock can occur in FairScheduler when there are lots of running apps (Tsuyoshi Ozawa via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612769 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-23 05:00:52 +00:00 |
|
Karthik Kambatla
|
ff77582991
|
YARN-2273. NPE in ContinuousScheduling thread when we lose a node. (Wei Yan via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612720 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-22 22:44:38 +00:00 |
|
Zhijie Shen
|
eac0701c96
|
YARN-2319. Made the MiniKdc instance start/close before/after the class of TestRMWebServicesDelegationTokens. Contributed by Wenwu Peng.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612588 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-22 15:15:29 +00:00 |
|
Junping Du
|
afb9394c91
|
YARN-2242. Addendum patch. Improve exception information on AM launch crashes. (Contributed by Li Lu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612565 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-22 13:07:23 +00:00 |
|
Karthik Kambatla
|
266a4ff6a5
|
YARN-2131. Addendum. Add a way to format the RMStateStore. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612443 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-22 01:30:36 +00:00 |
|
Sanford Ryza
|
7c71a3b876
|
YARN-2323. FairShareComparator creates too many Resource objects (Hong Zhiguo via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612187 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-21 05:00:27 +00:00 |
|
Karthik Kambatla
|
8871d8ed9f
|
YARN-2244. FairScheduler missing handling of containers for unknown application attempts. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611840 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-19 00:12:05 +00:00 |
|
Xuan Gong
|
f1b831ccfb
|
YARN-2208. AMRMTokenManager need to have a way to roll over AMRMToken. Contributed by Xuan Gong
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611820 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-18 21:46:29 +00:00 |
|
Jian He
|
3c193811ca
|
YARN-2219. Addendum patch for YARN-2219
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611240 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-17 03:28:39 +00:00 |
|
Vinod Kumar Vavilapalli
|
bda23181bf
|
YARN-2219. Changed ResourceManager to avoid AMs and NMs getting exceptions after RM recovery but before scheduler learns about apps and app-attempts. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611222 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-17 00:14:56 +00:00 |
|
Vinod Kumar Vavilapalli
|
030580387a
|
YARN-2233. Implemented ResourceManager web-services to create, renew and cancel delegation tokens. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610876 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-15 23:00:17 +00:00 |
|
Mayank Bansal
|
43589a8df7
|
YARN-1408 Preemption caused Invalid State Event: ACQUIRED at KILLED and caused a task timeout for 30mins. (Sunil G via mayank)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610860 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-15 21:48:58 +00:00 |
|
Vinod Kumar Vavilapalli
|
c6cc6a6a8e
|
YARN-2260. Fixed ResourceManager's RMNode to correctly remember containers when nodes resync during work-preserving RM restart. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610557 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-14 23:32:03 +00:00 |
|
Karthik Kambatla
|
4d211d5fde
|
YARN-2274. FairScheduler: Add debug information about cluster capacity, availability and reservations. (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609942 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-12 15:10:19 +00:00 |
|
Jian He
|
c9fb040c87
|
YARN-2181. Added preemption info to logs and RM web UI. Contributed by Wangda Tan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609561 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-10 20:03:35 +00:00 |
|
Karthik Kambatla
|
8fbca62a90
|
YARN-2131. Add a way to format the RMStateStore. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609278 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-09 19:58:43 +00:00 |
|
Zhijie Shen
|
12c4197b35
|
YARN-2158. Improved assertion messages of TestRMWebServicesAppsModification. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608667 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-08 05:50:04 +00:00 |
|
Sanford Ryza
|
5644f529f3
|
YARN-2250. FairScheduler.findLowestCommonAncestorQueue returns null when queues not identical (Krisztian Horvath via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607872 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-04 15:16:43 +00:00 |
|
Junping Du
|
5cb489f9d3
|
YARN-2242. Improve exception information on AM launch crashes. (Contributed by Li Lu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607655 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-03 14:15:19 +00:00 |
|
Vinod Kumar Vavilapalli
|
45b191e38c
|
YARN-2232. Fixed ResourceManager to allow DelegationToken owners to be able to cancel their own tokens in secure mode. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607484 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-02 21:36:42 +00:00 |
|
Karthik Kambatla
|
447c1c233b
|
YARN-2241. ZKRMStateStore: On startup, show nicer messages if znodes already exist. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607473 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-02 20:33:26 +00:00 |
|
Mayank Bansal
|
03a25d2cc1
|
YARN-2022 Preempting an Application Master container can be kept as least priority when multiple applications are marked for preemption by ProportionalCapacityPreemptionPolicy (Sunil G via mayank)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607227 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-02 01:54:47 +00:00 |
|
Vinod Kumar Vavilapalli
|
075ff276ca
|
YARN-1713. Added get-new-app and submit-app functionality to RM web services. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607216 13f79535-47bb-0310-9956-ffa450edef68
|
2014-07-02 00:23:07 +00:00 |
|
Xuan Gong
|
e5ae7c55d1
|
TestRMApplicationHistoryWriter sometimes fails in trunk. Contributed by Zhijie Shen
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606835 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-30 16:51:22 +00:00 |
|
Jian He
|
b0c51504c4
|
YARN-2052. Embedded an epoch number in container id to ensure the uniqueness of container id after RM restarts. Contributed by Tsuyoshi OZAWA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606557 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-29 18:24:03 +00:00 |
|
Jian He
|
b717d44b52
|
YARN-614. Changed ResourceManager to not count disk failure, node loss and RM restart towards app failures. Contributed by Xuan Gong
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606407 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-28 23:37:46 +00:00 |
|
Zhijie Shen
|
55a0aa0bad
|
YARN-2201. Made TestRMWebServicesAppsModification be independent of the changes on yarn-default.xml. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606285 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-28 03:30:44 +00:00 |
|
Jason Darrell Lowe
|
73927ba695
|
YARN-2104. Scheduler queue filter failed to work because index of queue column changed. Contributed by Wangda Tan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606265 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-27 23:11:12 +00:00 |
|
Karthik Kambatla
|
f911f5495b
|
YARN-2204. Addendum patch. TestAMRestart#testAMRestartWithExistingContainers assumes CapacityScheduler. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606168 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-27 18:09:41 +00:00 |
|
Ravi Prakash
|
f084170a0a
|
YARN 2163. WebUI: Order of AppId in apps table should be consistent with ApplicationId.compareTo()
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605964 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-27 06:43:46 +00:00 |
|
Vinod Kumar Vavilapalli
|
9571db19eb
|
YARN-2171. Improved CapacityScheduling to not lock on nodemanager-count when AMs heartbeat in. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605616 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-25 21:56:42 +00:00 |
|
Karthik Kambatla
|
1a3a7e0c1a
|
YARN-2204. TestAMRestart#testAMRestartWithExistingContainers assumes CapacityScheduler. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605548 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-25 18:50:53 +00:00 |
|
Jian He
|
c3f1c30e65
|
YARN-1365. Changed ApplicationMasterService to allow an app to re-register after RM restart. Contributed by Anubhav Dhoot
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605263 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-25 04:42:39 +00:00 |
|
Vinod Kumar Vavilapalli
|
e285b98f0f
|
YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-24 21:43:22 +00:00 |
|
Thomas Graves
|
1f9a0fd927
|
YARN-2072. RM/NM UIs and webservices are missing vcore information. (Nathan Roberts via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605162 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-24 19:34:34 +00:00 |
|
Karthik Kambatla
|
c0991d11eb
|
YARN-2109. Fix TestRM to work with both schedulers. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605142 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-24 17:30:53 +00:00 |
|
Karthik Kambatla
|
db4d277117
|
YARN-2192. TestRMHA fails when run with a mix of Schedulers. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605138 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-24 17:05:35 +00:00 |
|
Sanford Ryza
|
29c102cad0
|
YARN-2111. In FairScheduler.attemptScheduling, we don't count containers as assigned if they have 0 memory but non-zero cores (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605113 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-24 15:40:39 +00:00 |
|
Vinod Kumar Vavilapalli
|
d16470025a
|
YARN-2074. Changed ResourceManager to not count AM preemptions towards app failures. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605106 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-24 15:15:12 +00:00 |
|
Jian He
|
59b5e9fa15
|
YARN-2191. Added a new test to ensure NM will clean up completed applications in the case of RM restart. Contributed by Wangda Tan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604949 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-23 22:52:38 +00:00 |
|
Karthik Kambatla
|
6fcbf9b848
|
YARN-2187. FairScheduler: Disable max-AM-share check by default. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604321 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-21 07:30:07 +00:00 |
|
Haohui Mai
|
22cbcd16cb
|
HADOOP-10717. Revert r1603571.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603620 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-18 20:37:25 +00:00 |
|
Steve Loughran
|
bae5837d3b
|
HADOOP-10717. Missing JSP support in Jetty
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603534 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-18 17:08:45 +00:00 |
|
Jian He
|
95897ca14b
|
YARN-1885. Fixed a bug that RM may not send application-clean-up signal to NMs where the completed applications previously ran in case of RM restart. Contributed by Wangda Tan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603028 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-16 23:56:12 +00:00 |
|
Karthik Kambatla
|
082c2ab8b8
|
YARN-2159. Better logging in SchedulerNode#allocateContainer. (Ray Chiang via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603003 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-16 20:45:38 +00:00 |
|
Vinod Kumar Vavilapalli
|
dc7dd1fa19
|
YARN-1702. Added kill app functionality to RM web services. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602298 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-12 21:31:52 +00:00 |
|
Karthik Kambatla
|
4bc91b44c9
|
YARN-2155. FairScheduler: Incorrect threshold check for preemption. (Wei Yan via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602295 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-12 21:23:32 +00:00 |
|
Jian He
|
c6e29a9f06
|
YARN-2125. Changed ProportionalCapacityPreemptionPolicy to log CSV in debug level. Contributed by Wangda Tan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601980 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-11 18:31:28 +00:00 |
|
Jian He
|
710a8693e5
|
YARN-2124. Fixed NPE in ProportionalCapacityPreemptionPolicy. Contributed by Wangda Tan
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601964 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-11 17:30:18 +00:00 |
|
Karthik Kambatla
|
5de6f72054
|
YARN-1424. RMAppAttemptImpl should return the DummyApplicationResourceUsageReport for all invalid accesses. (Ray Chiang via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601742 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-10 19:03:06 +00:00 |
|
Jian He
|
c94f2cec3a
|
Augmented RMStateStore with state machine. Contributed by Binglin Chang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601491 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-09 19:44:31 +00:00 |
|
Vinod Kumar Vavilapalli
|
424fd9494f
|
YARN-1368. Added core functionality of recovering container state into schedulers after ResourceManager Restart so as to preserve running work in the cluster. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601303 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-09 03:09:21 +00:00 |
|
Karthik Kambatla
|
9958072d53
|
YARN-2132. ZKRMStateStore.ZKAction#runWithRetries doesn't log the exception it encounters. (Vamsee Yarlagadda via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601065 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-07 06:12:59 +00:00 |
|
Karthik Kambatla
|
85d4c787e0
|
YARN-2128. FairScheduler: Incorrect calculation of amResource usage. (Wei Yan via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601050 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-07 01:21:33 +00:00 |
|
Karthik Kambatla
|
969bef3c77
|
YARN-2122. In AllocationFileLoaderService, the reloadThread should be created in init() and started in start(). (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601045 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-07 00:49:20 +00:00 |
|
Junping Du
|
0ceb742549
|
YARN-1977. Add tests on getApplicationRequest with filtering start time range. (Contributed by Junping Du)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600644 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-05 13:15:44 +00:00 |
|
Karthik Kambatla
|
d1d6bb70f4
|
YARN-2061. Revisit logging levels in ZKRMStateStore. (Ray Chiang via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600498 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-04 21:11:09 +00:00 |
|
Sanford Ryza
|
16caa3fd18
|
YARN-1913. With Fair Scheduler, cluster can logjam when all resources are consumed by AMs (Wei Yan via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599400 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-03 00:56:48 +00:00 |
|
Karthik Kambatla
|
0aad2d56df
|
YARN-1550. NPE in FairSchedulerAppsBlock#render. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599345 13f79535-47bb-0310-9956-ffa450edef68
|
2014-06-02 20:22:52 +00:00 |
|
Karthik Kambatla
|
a4ba451802
|
YARN-1474. Make schedulers services. (Tsuyoshi Ozawa via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598908 13f79535-47bb-0310-9956-ffa450edef68
|
2014-05-31 19:33:09 +00:00 |
|
Vinod Kumar Vavilapalli
|
23c325ad47
|
YARN-2115. Replaced RegisterNodeManagerRequest's ContainerStatus with a new NMContainerStatus which has more information that is needed for work-preserving RM-restart. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598790 13f79535-47bb-0310-9956-ffa450edef68
|
2014-05-31 00:20:50 +00:00 |
|
Karthik Kambatla
|
49a3a0cd0c
|
YARN-2054. Better defaults for YARN ZK configs for retries and retry-inteval when HA is enabled. (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598630 13f79535-47bb-0310-9956-ffa450edef68
|
2014-05-30 15:24:49 +00:00 |
|
Arpit Agarwal
|
4a4868e523
|
HADOOP-10448. Support pluggable mechanism to specify proxy user settings (Contributed by Benoy Antony)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598396 13f79535-47bb-0310-9956-ffa450edef68
|
2014-05-29 20:52:01 +00:00 |
|
Sanford Ryza
|
342da5b4d3
|
YARN-596. Use scheduling policies throughout the queue hierarchy to decide which containers to preempt (Wei Yan via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598197 13f79535-47bb-0310-9956-ffa450edef68
|
2014-05-29 04:01:24 +00:00 |
|
Sanford Ryza
|
edfbc8ad4a
|
YARN-2105. Fix TestFairScheduler after YARN-2012. (Ashwin Shankar via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597902 13f79535-47bb-0310-9956-ffa450edef68
|
2014-05-27 23:46:22 +00:00 |
|
Karthik Kambatla
|
7dd378c274
|
YARN-2096. Race in TestRMRestart#testQueueMetricsOnRMRestart. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597223 13f79535-47bb-0310-9956-ffa450edef68
|
2014-05-23 23:51:00 +00:00 |
|
Sanford Ryza
|
a00b2d4f37
|
YARN-2073. Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597209 13f79535-47bb-0310-9956-ffa450edef68
|
2014-05-23 22:52:46 +00:00 |
|