big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,817 Commits 2 Branches 2 Tags 264 MiB
772ea7b41b
Commit Graph

1191 Commits

Author SHA1 Message Date
Jason Lowe
91828fef6b YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne 2015-12-16 23:22:31 +00:00
Wangda Tan
9b856d9787 YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda) 2015-12-16 13:22:37 -08:00
Wangda Tan
7faa406f27 YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 2015-12-16 13:19:40 -08:00
Wangda Tan
79c41b1d83 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 2015-12-16 13:18:19 -08:00
Junping Du
50bd067e1d YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 2015-12-16 10:57:39 -08:00
Zhihai Xu
2aaed10327 YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun 2015-12-15 00:17:21 -08:00
Jian He
1cb3299b48 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:51:23 -08:00
Wangda Tan
07b0fb996a YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda) 2015-12-14 11:24:30 -08:00
Wangda Tan
6cb0af3c39 YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda) 2015-12-14 10:52:46 -08:00
Arun Suresh
7fb212e5e6 YARN-4358 addendum patch to fix javadoc error 2015-12-12 22:22:55 -08:00
rohithsharmaks
a5e2e1ecb0 YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks) 2015-12-09 11:31:51 +05:30
Wangda Tan
7e4715186d YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 2015-12-08 14:25:16 -08:00
Chris Douglas
9f50e13d5d YARN-4248. Followup patch adding asf-licence exclusions for json test files 2015-12-08 12:08:04 -08:00
=
c25a635459 YARN-4248. REST API for submit/update/delete Reservations. (curino) 2015-12-07 13:33:28 -08:00
Jonathan Eagles
4ff973f96a YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 2015-12-07 15:04:48 -06:00
Xuan
4546c7582b YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong
 2015-12-07 12:24:55 -08:00
Steve Loughran
65f395226b HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:43:35 +00:00
Arun Suresh
742632e346 YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh) 2015-12-05 21:26:16 -08:00
Jian He
755dda8dd8 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 2015-12-03 17:45:31 -08:00
Wangda Tan
a2c3bfc8c1 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 2015-12-03 14:28:32 -08:00
Jian He
9f77ccad73 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2015-12-03 12:48:50 -08:00
Jian He
6b9a5beb2b YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 2015-12-02 11:07:18 -08:00
Tsuyoshi Ozawa
28dfe721b8 YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang. 2015-11-24 19:24:01 +09:00
Karthik Kambatla
52948bb20b YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 2015-11-24 13:47:17 +05:30
Jian He
8676a118a1 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 2015-11-23 17:19:48 -08:00
Jason Lowe
d36b6e045f YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 2015-11-23 20:30:26 +00:00
Arun Suresh
da1016365a YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh) 2015-11-21 09:59:41 -08:00
Wangda Tan
2346fa3141 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 2015-11-20 15:55:50 -08:00
Jason Lowe
060cdcbe5d YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 2015-11-20 23:12:29 +00:00
Arun Suresh
6a61928fb7 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 2015-11-17 15:50:34 -08:00
Jian He
fcd7888029 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit 8fbea531d7.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:18:44 -08:00
Wangda Tan
7f55a18071 YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 2015-11-12 11:23:40 -08:00
Wangda Tan
796638d9bc YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda) 2015-11-12 11:09:37 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
6351d3fa63 YARN-4183. Reverting the patch to fix behaviour change. 
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit c293c58954.
 2015-11-11 10:40:43 -08:00
Jian He
8fbea531d7 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 2015-11-09 10:43:45 -08:00
Jian He
e5b1733e04 YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena 2015-10-29 15:42:57 -07:00
Jonathan Eagles
c293c58954 YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 2015-10-29 16:41:10 -05:00
Arun Suresh
58d1df585c YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh) 2015-10-29 13:42:09 -07:00
Rohith Sharma K S
656c8f9527 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks) 2015-10-29 12:22:44 +05:30
Wangda Tan
56e4f6237a YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda) 2015-10-26 16:44:39 -07:00
Wangda Tan
6f606214e7 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 2015-10-26 16:36:34 -07:00
Wangda Tan
3cc73773eb YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 2015-10-26 13:07:39 -07:00
Jason Lowe
33a03af3c3 YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee 2015-10-26 19:53:03 +00:00
Rohith Sharma K S
5acdde4744 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 2015-10-26 15:42:42 +05:30
Arun Suresh
ab8eb8770c YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh) 2015-10-24 22:53:10 -07:00
Akira Ajisaka
7781fe1b9e YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka) 2015-10-24 11:54:42 +09:00
Jason Lowe
d3a34a4f38 YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G 2015-10-23 20:57:01 +00:00
Ming Ma
934d96a334 YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma) 2015-10-23 08:36:33 -07:00
Jonathan Eagles
f8adeb712d YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 2015-10-23 10:34:08 -05:00
Junping Du
0fce5f9a49 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 2015-10-22 13:41:09 -07:00
Zhihai Xu
960201b79b YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong 2015-10-22 12:28:03 -07:00
Anubhav Dhoot
2798723a54 YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan. 2015-10-22 06:51:00 -07:00
Arun Suresh
506d1b1dbc YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh) 2015-10-20 16:46:14 -07:00
Arun Suresh
7e2837f830 YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh) 2015-10-19 20:00:38 -07:00
Jian He
f9da5cdb2b YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G 2015-10-16 15:26:27 -07:00
Wangda Tan
4337b263aa YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda) 2015-10-16 15:06:28 -07:00
Jian He
cf23f2c2b5 YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena 2015-10-15 17:12:46 -07:00
rohithsharmaks
d6c8bad869 YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks) 2015-10-14 16:11:34 +05:30
Jian He
9849c8b386 YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left. Contributed by Meng Ding 2015-10-12 11:51:33 -07:00
Zhihai Xu
049c6e8dc0 YARN-4201. AMBlacklist does not work for minicluster. Contributed by Jun Gong. 2015-10-12 00:14:25 -07:00
Devaraj K
db93047881 YARN-3964. Support NodeLabelsProvider at Resource Manager side. 
Contributed by Dian Fu.
 2015-10-11 11:21:29 +05:30
Wangda Tan
def374e666 YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda) 2015-10-09 16:38:59 -07:00
Karthik Kambatla
4aa9b3e75c MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 2015-10-09 07:37:39 -07:00
Jason Lowe
a0bca2b5ad YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 2015-10-09 14:17:38 +00:00
Rohith Sharma K S
8f195387a4 YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned for a user. (Anubhav Dhoot via rohithsharmaks) 2015-10-09 10:09:26 +05:30
Rohith Sharma K S
3793cbe4c3 YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks) 2015-10-07 10:12:14 +05:30
Rohith Sharma K S
9156fc60c6 YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks) 2015-10-07 09:34:59 +05:30
Wangda Tan
29a582ada0 YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request. (Naganarasimha G R via wangda) 2015-10-06 11:56:04 -07:00
Harsh J
c918f7be5e HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian. 2015-10-03 18:37:58 +05:30
Xuan
8f08532bde YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 2015-10-02 18:50:47 -07:00
Karthik Kambatla
a0b5a0a419 YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) 2015-09-29 07:55:34 -07:00
Anubhav Dhoot
9735afe967 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 2015-09-28 16:13:41 -07:00
Jason Lowe
9f53a95ff6 YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G 2015-09-28 22:55:20 +00:00
Anubhav Dhoot
fb2e525c07 YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot) 2015-09-28 09:05:45 -07:00
Rohith Sharma K S
a9aafad12b YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) 2015-09-24 12:13:22 +05:30
Jian He
b3f6b641dc YARN-4171. Fix findbugs warnings in YARN-1197 branch. Contributed by Wangda Tan 2015-09-23 13:29:38 -07:00
Jian He
89cab1ba5f YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 2015-09-23 13:29:38 -07:00
Jian He
5f5a968d65 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:37 -07:00
Jian He
83a18add10 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 2015-09-23 13:29:36 -07:00
Tsuyoshi Ozawa
dfd807afab HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 2015-09-22 12:54:29 +09:00
Rohith Sharma K S
c9cb6a5960 YARN-4167. NPE on RMActiveServices#serviceStop when store is null. (Bibin A Chundatt via rohithsharmaks) 2015-09-21 09:59:30 +05:30
Arun Suresh
94dec5a916 YARN-3920. FairScheduler container reservation on a node should be configurable to limit it to large containers (adhoot via asuresh) 2015-09-18 14:02:55 -07:00
Wangda Tan
9bc913a35c YARN-3212. RMNode State Transition Update with DECOMMISSIONING state. (Junping Du via wangda) 2015-09-18 10:04:17 -07:00
Rohith Sharma K S
723c31d45b YARN-4135. Improve the assertion message in MockRM while failing after waiting for the state.(Nijel S F via rohithsharmaks) 2015-09-18 08:44:10 +05:30
Jian He
6c6e734f0b YARN-4034. Render cluster Max Priority in scheduler metrics in RM web UI. Contributed by Rohith Sharma K S 2015-09-17 14:55:50 +08:00
Jian He
452079af8b YARN-4078. Add getPendingResourceRequestForAttempt in YarnScheduler interface. Contributed by Naganarasimha G R 2015-09-16 14:59:20 +08:00
Wangda Tan
ae5308fe1d YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda) 2015-09-15 11:40:50 -07:00
Junping Du
73e3a49eb0 YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du) 2015-09-15 07:56:47 -07:00
Jian He
5468baa80a YARN-3635. Refactored current queue mapping implementation in CapacityScheduler to use a generic PlacementManager framework. Contributed by Wangda Tan 2015-09-15 15:39:20 +08:00
Jian He
e1b1d7e4ae YARN-4126. RM should not issue delegation tokens in unsecure mode. Contributed by Bibin A Chundatt 2015-09-14 14:09:19 +08:00
Karthik Kambatla
332b520a48 YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha) 2015-09-13 18:07:43 -07:00
Karthik Kambatla
81df7b586a YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha) 2015-09-13 17:03:15 -07:00
Steve Loughran
7269906254 HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:26 +01:00
Robert Kanter
ea4bb2749f YARN-4145. Make RMHATestBase abstract so its not run when running all tests under that namespace (adhoot via rkanter) 2015-09-11 11:46:10 -07:00
Jian He
6f72f1e600 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 2015-09-08 09:35:46 +08:00
Xuan
9b78e6e33d YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when 
state-store error occurs. Contributed by Jian He
 2015-09-07 17:45:47 -07:00
Wangda Tan
bcc85e3bab YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat. (Hong Zhiguo via wangda) 2015-09-04 15:13:53 -07:00
Jason Lowe
6eaca2e363 YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li 2015-09-04 15:30:53 +00:00
Varun Vasudev
40d222e862 YARN-4103. RM WebServices missing scheme for appattempts logLinks. Contributed by Jonathan Eagles. 2015-09-04 14:31:51 +05:30
Varun Vasudev
b469ac531a YARN-3970. Add REST api support for Application Priority. Contributed by Naganarasimha G R. 2015-09-03 16:40:10 +05:30