Arun Suresh
da1016365a
YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh)
2015-11-21 09:59:41 -08:00
Wangda Tan
2346fa3141
YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda)
2015-11-20 15:55:50 -08:00
Jason Lowe
060cdcbe5d
YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li
2015-11-20 23:12:29 +00:00
Xuan
f114e728da
YARN-2859.addendum: fix the remaining issue from the previous patch
2015-11-19 10:08:38 -08:00
Arun Suresh
6a61928fb7
YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh)
2015-11-17 15:50:34 -08:00
Jian He
fcd7888029
Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan"
...
This reverts commit 8fbea531d7
.
Conflicts:
hadoop-yarn-project/CHANGES.txt
2015-11-16 20:18:44 -08:00
Junping Du
855d52927b
YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe.
2015-11-15 04:43:57 -08:00
Wangda Tan
7f55a18071
YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda)
2015-11-12 11:23:40 -08:00
Wangda Tan
796638d9bc
YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda)
2015-11-12 11:09:37 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
6351d3fa63
YARN-4183. Reverting the patch to fix behaviour change.
...
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"
This reverts commit c293c58954
.
2015-11-11 10:40:43 -08:00
Jian He
8fbea531d7
YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan
2015-11-09 10:43:45 -08:00
Tsuyoshi Ozawa
ce31b22739
YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena.
2015-10-30 18:00:20 +09:00
Jian He
e5b1733e04
YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena
2015-10-29 15:42:57 -07:00
Jonathan Eagles
c293c58954
YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)
2015-10-29 16:41:10 -05:00
Arun Suresh
58d1df585c
YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh)
2015-10-29 13:42:09 -07:00
Jason Lowe
e2267de207
YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena
2015-10-29 16:34:25 +00:00
Rohith Sharma K S
656c8f9527
YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks)
2015-10-29 12:22:44 +05:30
Xuan
27414dac66
YARN-2859. ApplicationHistoryServer binds to default port 8188 in
...
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli
2015-10-28 10:32:16 -07:00
Tsuyoshi Ozawa
96677bef00
YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula.
2015-10-27 12:35:01 +09:00
Akira Ajisaka
8a68630dd1
YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka)
2015-10-27 10:28:01 +09:00
Wangda Tan
56e4f6237a
YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda)
2015-10-26 16:44:39 -07:00
Wangda Tan
6f606214e7
YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda)
2015-10-26 16:36:34 -07:00
Wangda Tan
3cc73773eb
YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda)
2015-10-26 13:07:39 -07:00
Jason Lowe
33a03af3c3
YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee
2015-10-26 19:53:03 +00:00
Rohith Sharma K S
5acdde4744
YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks)
2015-10-26 15:42:42 +05:30
Tsuyoshi Ozawa
ce60b4fc8b
YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula.
2015-10-26 16:45:11 +09:00
Allen Wittenauer
1aa735c188
YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw)
2015-10-25 21:43:23 -07:00
Arun Suresh
ab8eb8770c
YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh)
2015-10-24 22:53:10 -07:00
Akira Ajisaka
7781fe1b9e
YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka)
2015-10-24 11:54:42 +09:00
Jason Lowe
d3a34a4f38
YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G
2015-10-23 20:57:01 +00:00
Ming Ma
934d96a334
YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma)
2015-10-23 08:36:33 -07:00
Jonathan Eagles
f8adeb712d
YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles)
2015-10-23 10:34:08 -05:00
Junping Du
0fce5f9a49
YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong.
2015-10-22 13:41:09 -07:00
Zhihai Xu
960201b79b
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong
2015-10-22 12:28:03 -07:00
Anubhav Dhoot
2798723a54
YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan.
2015-10-22 06:51:00 -07:00
Jonathan Eagles
25f8f801d1
YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles)
2015-10-21 15:38:08 -05:00
Arun Suresh
506d1b1dbc
YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh)
2015-10-20 16:46:14 -07:00
Arun Suresh
7e2837f830
YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh)
2015-10-19 20:00:38 -07:00
Varun Vasudev
151eab2a12
YARN-4267. Add additional logging to container launch implementations in container-executor. Contributed by Sidharta Seethana.
2015-10-19 20:56:01 +05:30
Varun Vasudev
e39ae0e676
YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana.
2015-10-19 20:19:22 +05:30
Steve Loughran
e2d59e2c7b
YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel)
2015-10-18 12:03:57 +01:00
Jian He
f9da5cdb2b
YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G
2015-10-16 15:26:27 -07:00
Wangda Tan
4337b263aa
YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda)
2015-10-16 15:06:28 -07:00
Jian He
cf23f2c2b5
YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena
2015-10-15 17:12:46 -07:00
Varun Vasudev
63020c54c1
YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana.
2015-10-15 13:41:20 +05:30
Steve Loughran
dfa7848463
HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel)
2015-10-14 20:25:47 +01:00
rohithsharmaks
d6c8bad869
YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks)
2015-10-14 16:11:34 +05:30
Varun Vasudev
da1ee078f9
YARN-4255. container-executor does not clean up docker operation command files. Contributed by Sidharta Seethana.
2015-10-14 15:25:21 +05:30
Varun Vasudev
fb9c1519b3
YARN-4252. Log container-executor invocation details when exit code is non-zero. Contributed by Sidharta Seethana.
2015-10-14 15:18:08 +05:30
Varun Vasudev
8d59293089
YARN-4253. Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor. Contributed by Sidharta Seethana.
2015-10-14 14:58:10 +05:30
Steve Loughran
c59af2fdf8
HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel)
2015-10-13 21:49:28 +01:00
Varun Vasudev
5b6bae0094
YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana
2015-10-13 12:48:30 +05:30
Jian He
9849c8b386
YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left. Contributed by Meng Ding
2015-10-12 11:51:33 -07:00
Zhihai Xu
049c6e8dc0
YARN-4201. AMBlacklist does not work for minicluster. Contributed by Jun Gong.
2015-10-12 00:14:25 -07:00
Devaraj K
db93047881
YARN-3964. Support NodeLabelsProvider at Resource Manager side.
...
Contributed by Dian Fu.
2015-10-11 11:21:29 +05:30
Wangda Tan
def374e666
YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda)
2015-10-09 16:38:59 -07:00
Karthik Kambatla
4aa9b3e75c
MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha)
2015-10-09 07:37:39 -07:00
Jason Lowe
a0bca2b5ad
YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S
2015-10-09 14:17:38 +00:00
Rohith Sharma K S
8f195387a4
YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned for a user. (Anubhav Dhoot via rohithsharmaks)
2015-10-09 10:09:26 +05:30
Jason Lowe
8d226225d0
YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu
2015-10-08 22:25:34 +00:00
Rohith Sharma K S
3793cbe4c3
YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks)
2015-10-07 10:12:14 +05:30
Rohith Sharma K S
9156fc60c6
YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks)
2015-10-07 09:34:59 +05:30
Wangda Tan
29a582ada0
YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request. (Naganarasimha G R via wangda)
2015-10-06 11:56:04 -07:00
Wangda Tan
30ac69c6bd
YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda)
2015-10-05 15:47:21 -07:00
Vinod Kumar Vavilapalli
f1c19b9365
HADOOP-12441. Fixed shell-kill command behaviour to work correctly on some Linux distributions after HADOOP-12317. Contributed by Wangda Tan.
2015-10-05 10:56:55 -07:00
Harsh J
c918f7be5e
HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian.
2015-10-03 18:37:58 +05:30
Xuan
8f08532bde
YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma
2015-10-02 18:50:47 -07:00
Jason Lowe
fdf02d1f26
YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu
2015-10-02 20:20:31 +00:00
Jason Lowe
854d25b0c3
YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu
2015-09-30 14:59:44 +00:00
Karthik Kambatla
a0b5a0a419
YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha)
2015-09-29 07:55:34 -07:00
Anubhav Dhoot
9735afe967
YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot)
2015-09-28 16:13:41 -07:00
Jason Lowe
9f53a95ff6
YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G
2015-09-28 22:55:20 +00:00
Anubhav Dhoot
fb2e525c07
YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot)
2015-09-28 09:05:45 -07:00
Xuan
d1b9b85244
YARN-3624. ApplicationHistoryServer should not reverse the order of the
...
filters it gets. Contributed by Mit Desai
2015-09-24 15:01:17 -07:00
Rohith Sharma K S
a9aafad12b
YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks)
2015-09-24 12:13:22 +05:30
Rohith Sharma K S
8ed0d4b744
YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for absent container. (Bibin A Chundatt via rohithsharmaks)
2015-09-24 11:24:14 +05:30
Jian He
b3f6b641dc
YARN-4171. Fix findbugs warnings in YARN-1197 branch. Contributed by Wangda Tan
2015-09-23 13:29:38 -07:00
Jian He
89cab1ba5f
YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan
2015-09-23 13:29:38 -07:00
Jian He
c57eac5dfe
YARN-3868. Recovery support for container resizing. Contributed by Meng Ding
2015-09-23 13:29:38 -07:00
Jian He
c3dc1af072
YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding
2015-09-23 13:29:37 -07:00
Jian He
c59ae4eeb1
YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan
2015-09-23 13:29:37 -07:00
Jian He
5f5a968d65
YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding
2015-09-23 13:29:37 -07:00
Jian He
ffd820c27a
YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan
2015-09-23 13:29:37 -07:00
Jian He
83a18add10
YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan)
2015-09-23 13:29:36 -07:00
Jason Lowe
692d51c09d
YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is enabled. Contributed by Mit Desai
2015-09-23 16:38:55 +00:00
Jason Lowe
c890c51a91
YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. Contributed by Zhihai Xu
2015-09-23 15:42:01 +00:00
Tsuyoshi Ozawa
dfd807afab
HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.
2015-09-22 12:54:29 +09:00
Rohith Sharma K S
c9cb6a5960
YARN-4167. NPE on RMActiveServices#serviceStop when store is null. (Bibin A Chundatt via rohithsharmaks)
2015-09-21 09:59:30 +05:30
Arun Suresh
94dec5a916
YARN-3920. FairScheduler container reservation on a node should be configurable to limit it to large containers (adhoot via asuresh)
2015-09-18 14:02:55 -07:00
Wangda Tan
9bc913a35c
YARN-3212. RMNode State Transition Update with DECOMMISSIONING state. (Junping Du via wangda)
2015-09-18 10:04:17 -07:00
Steve Loughran
a7201d635f
YARN-2597 MiniYARNCluster should propagate reason for AHS not starting
2015-09-18 09:45:35 +01:00
Rohith Sharma K S
723c31d45b
YARN-4135. Improve the assertion message in MockRM while failing after waiting for the state.(Nijel S F via rohithsharmaks)
2015-09-18 08:44:10 +05:30
Jian He
6c6e734f0b
YARN-4034. Render cluster Max Priority in scheduler metrics in RM web UI. Contributed by Rohith Sharma K S
2015-09-17 14:55:50 +08:00
Jian He
452079af8b
YARN-4078. Add getPendingResourceRequestForAttempt in YarnScheduler interface. Contributed by Naganarasimha G R
2015-09-16 14:59:20 +08:00
Xuan
34ef1a092b
YARN-4149. yarn logs -am should provide an option to fetch all the log
...
files. Contributed by Varun Vasudev
2015-09-15 14:36:30 -07:00
Jason Lowe
8c1cdb17a0
YARN-4158. Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers. Contributed by Zhihai Xu
2015-09-15 20:21:33 +00:00
Wangda Tan
ae5308fe1d
YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda)
2015-09-15 11:40:50 -07:00
Junping Du
73e3a49eb0
YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du)
2015-09-15 07:56:47 -07:00
Jian He
5468baa80a
YARN-3635. Refactored current queue mapping implementation in CapacityScheduler to use a generic PlacementManager framework. Contributed by Wangda Tan
2015-09-15 15:39:20 +08:00
Wangda Tan
e2a0270217
YARN-4151. Fix findbugs errors in hadoop-yarn-server-common module. (Meng Ding via wangda)
2015-09-14 16:00:43 -07:00
Jian He
e1b1d7e4ae
YARN-4126. RM should not issue delegation tokens in unsecure mode. Contributed by Bibin A Chundatt
2015-09-14 14:09:19 +08:00
Karthik Kambatla
332b520a48
YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha)
2015-09-13 18:07:43 -07:00
Karthik Kambatla
81df7b586a
YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha)
2015-09-13 17:03:15 -07:00
Steve Loughran
7269906254
HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel).
2015-09-13 14:25:26 +01:00
Robert Kanter
ea4bb2749f
YARN-4145. Make RMHATestBase abstract so its not run when running all tests under that namespace (adhoot via rkanter)
2015-09-11 11:46:10 -07:00
Varun Vasudev
486d5cb803
YARN-4136. LinuxContainerExecutor loses info when forwarding ResourceHandlerException. Contributed by Bibin A Chundatt.
2015-09-11 14:37:48 +05:30
Wangda Tan
77666105b4
YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda)
2015-09-10 09:30:09 -07:00
Zhihai Xu
16b9037dc1
YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe.
2015-09-08 12:29:54 -07:00
Jian He
6f72f1e600
YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil
2015-09-08 09:35:46 +08:00
Xuan
9b78e6e33d
YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when
...
state-store error occurs. Contributed by Jian He
2015-09-07 17:45:47 -07:00
Varun Vasudev
1dbd8e34a7
YARN-3591. Resource localization on a bad disk causes subsequent containers failure. Contributed by Lavkesh Lahngir.
2015-09-07 11:32:12 +05:30
Wangda Tan
bcc85e3bab
YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat. (Hong Zhiguo via wangda)
2015-09-04 15:13:53 -07:00
Jason Lowe
6eaca2e363
YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li
2015-09-04 15:30:53 +00:00
Varun Vasudev
40d222e862
YARN-4103. RM WebServices missing scheme for appattempts logLinks. Contributed by Jonathan Eagles.
2015-09-04 14:31:51 +05:30
Varun Vasudev
b469ac531a
YARN-3970. Add REST api support for Application Priority. Contributed by Naganarasimha G R.
2015-09-03 16:40:10 +05:30
Jian He
09c64ba1ba
YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong
2015-09-02 17:45:23 -07:00
Rohith Sharma K S
7d6687fe76
YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks)
2015-09-02 15:22:48 +05:30
Rohith Sharma K S
095ab9ab5f
YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor. (Naganarasimha G R via rohithsharmaks)
2015-09-02 14:13:33 +05:30
Varun Vasudev
bf669b6d9f
YARN-4082. Container shouldn't be killed when node's label updated. Contributed by Wangda Tan.
2015-09-01 14:19:11 +05:30
Jian He
a3fd2ccc86
YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong
2015-08-31 17:33:24 -07:00
Junping Du
beb65c9465
YARN-1556. NPE getting application report with a null appId. Contributed by Weiwei Yang.
2015-08-28 05:57:34 -07:00
Jian He
a9c8ea71aa
YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S
2015-08-27 13:25:53 -07:00
Jian He
57c7ae1aff
YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S
2015-08-24 20:36:44 -07:00
Rohith Sharma K S
feaf034994
YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks)
2015-08-24 11:25:07 +05:30
Xuan
b71c6006f5
YARN-221. Addendum patch to compilation issue which is caused by missing
...
AllContainerLogAggregationPolicy. Contributed by Xuan Gong
2015-08-23 16:46:30 -07:00
Xuan
37e1c3d82a
YARN-221. NM should provide a way for AM to tell it not to aggregate
...
logs. Contributed by Ming Ma
2015-08-22 16:25:24 -07:00
Rohith Sharma K S
22de7c1dca
YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface
2015-08-21 10:51:11 +05:30
Wangda Tan
fc07464d1a
YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R)
2015-08-20 11:51:03 -07:00
Xuan
22dc5fc209
YARN-4028. AppBlock page key update and diagnostics value null on
...
recovery. Contributed by Bibin A Chundatt
2015-08-18 22:53:03 -07:00
Zhihai Xu
14215c8ef8
YARN-4057. If ContainersMonitor is not enabled, only print related log info one time. Contributed by Jun Gong.
2015-08-18 11:36:02 -07:00
Zhihai Xu
3a76a010b8
YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao.
2015-08-18 10:36:40 -07:00
Karthik Kambatla
13604bd5f1
YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha)
2015-08-16 15:08:53 -07:00
Karthik Kambatla
def12933b3
YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha)
2015-08-16 06:24:16 -07:00
Jian He
0a030546e2
YARN-3987. Send AM container completed msg to NM once AM finishes. Contributed by sandflee
2015-08-13 16:22:53 -07:00
Jian He
7a445fcfab
YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe
2015-08-13 16:02:57 -07:00
Jian He
38aed1a94e
YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong
2015-08-13 14:46:08 -07:00
Robert Kanter
dc2340c60e
YARN-4031. Add JvmPauseMonitor to ApplicationHistoryServer and WebAppProxyServer (djp via rkanter)
2015-08-12 15:47:47 -07:00
Jian He
e5003be907
YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan
2015-08-12 15:07:50 -07:00
rohithsharmaks
1c12adb71f
YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks)
2015-08-12 14:45:41 +05:30
Xuan
3ae716fa69
YARN-3999. RM hangs on draing events. Contributed by Jian He
2015-08-11 18:25:11 -07:00
Jian He
fa1d84ae27
YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G
2015-08-10 20:51:54 -07:00
Wangda Tan
cf9d3c9256
YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy. (Sunil G via wangda)
2015-08-10 14:54:55 -07:00
Wangda Tan
4bc42d76e7
YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda)
2015-08-07 09:46:57 -07:00
Rohith Sharma K S
b6265d39c5
YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks)
2015-08-07 10:43:41 +05:30
Carlo Curino
8572a5a14b
YARN-3974. Refactor the reservation system test cases to use parameterized base test. (subru via curino)
2015-08-02 01:55:31 -07:00
Junping Du
cfee02b3bd
YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter.
2015-08-06 06:49:45 -07:00
Arun Suresh
154c9d2e42
YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh)
2015-08-05 23:14:14 -07:00
rohithsharmaks
df9e7280db
YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G)
2015-08-06 10:43:37 +05:30
Jian He
ba2313d614
YARN-3983. Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend container allocation logic. Contributed by Wangda Tan
2015-08-05 13:47:40 -07:00
Arun Suresh
f271d37735
YARN-3736. Add RMStateStore apis to store and load accepted reservations for failover (adhoot via asuresh)
2015-08-05 12:57:12 -07:00
Xuan
c3364ca8e7
YARN-4004. container-executor should print output of docker logs if the
...
docker container exits with non-0 exit status. Contributed by Varun
Vasudev
2015-08-03 18:10:11 -07:00
Xuan
0306d902f5
YARN-3543. ApplicationReport should be able to tell whether the
...
Application is AM managed or not. Contributed by Rohith Sharma K S
2015-08-03 15:46:00 -07:00
Jason Lowe
469cfcd695
YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo
2015-08-03 15:53:32 +00:00
Jonathan Eagles
3cd02b9522
YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles)
2015-08-03 10:38:05 -05:00
Jason Lowe
32e490b6c0
YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt
2015-07-31 17:37:24 +00:00
Zhihai Xu
ab80e27703
YARN-433. When RM is catching up with node updates then it should not expire acquired containers. Contributed by Xuan Gong
2015-07-30 21:57:11 -07:00
Wangda Tan
91b42e7d6e
YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda)
2015-07-30 10:00:31 -07:00
Karthik Kambatla
5205a330b3
YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand. (Hong Zhiguo via kasha)
2015-07-29 09:42:32 -07:00
Xuan
f170934215
YARN-3982. container-executor parsing of container-executor.cfg broken
...
in trunk and branch-2. Contributed by Varun Vasudev
2015-07-27 23:45:58 -07:00
Jian He
3572ebd738
YARN-3846. RM Web UI queue filter is not working for sub queue. Contributed by Mohammad Shahid Khan
2015-07-27 17:12:05 -07:00
Varun Vasudev
3e6fce91a4
YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana.
2015-07-27 11:57:40 -07:00
Varun Vasudev
f36835ff9b
YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab.
2015-07-27 10:14:51 -07:00
ccurino
156f24ead0
YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations. (Jonathan Yaniv and Ishai Menache via curino)
2015-07-25 07:39:47 -07:00
Wangda Tan
a3bd7b4a59
YARN-3973. Recent changes to application priority management break reservation system from YARN-1051 (Carlo Curino via wangda)
2015-07-24 16:44:18 -07:00
Jason Lowe
ff9c13e0a7
YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu
2015-07-24 22:14:39 +00:00
Jian He
83fe34ac08
YARN-3026. Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp. Contributed by Wangda Tan
2015-07-24 14:00:25 -07:00
Karthik Kambatla
d19d187753
YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500. (Anubhav Dhoot via kasha)
2015-07-24 11:44:37 -07:00
Xuan
fbd6063269
YARN-3967. Fetch the application report from the AHS if the RM does not
...
know about it. Contributed by Mit Desai
2015-07-24 10:15:54 -07:00
carlo curino
0fcb4a8cf2
YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino)
2015-07-23 19:33:59 -07:00
Rohith Sharma K S
e202efaf93
YARN-3845. Scheduler page does not render RGBA color combinations in IE11. (Contributed by Mohammad Shahid Khan)
2015-07-24 12:43:06 +05:30
Robert Kanter
1d3026e7b3
YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos that include it (adhoot via rkanter)
2015-07-23 14:46:54 -07:00
Wangda Tan
3bba180051
YARN-3941. Proportional Preemption policy should try to avoid sending duplicate PREEMPT_CONTAINER event to scheduler. (Sunil G via wangda)
2015-07-23 10:07:57 -07:00
Junping Du
ee98d6354b
YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He.
2015-07-22 17:52:35 -07:00
Wangda Tan
06e5dd2c84
YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda)
2015-07-22 12:01:41 -07:00
Wangda Tan
76ec26de80
YARN-3932. SchedulerApplicationAttempt#getResourceUsageReport and UserInfo should based on total-used-resources. (Bibin A Chundatt via wangda)
2015-07-22 11:54:02 -07:00
Wangda Tan
c39ca541f4
YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda)
2015-07-21 09:57:23 -07:00
Colin Patrick Mccabe
419c51d233
YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe)
2015-07-17 11:38:59 -07:00
Jonathan Eagles
7faae0e6fe
YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles)
2015-07-17 11:02:11 -05:00
Arun Suresh
9b272ccae7
YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh)
2015-07-17 04:31:34 -07:00
Wangda Tan
3540d5fe4b
YARN-3885. ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level. (Ajith S via wangda)
2015-07-16 16:13:32 -07:00
Arun Suresh
ac94ba3e18
YARN-3453. Ensure preemption logic in FairScheduler uses DominantResourceCalculator in DRF queues to prevent unnecessary thrashing. (asuresh)
2015-07-14 00:23:55 -07:00
Akira Ajisaka
19295b36d9
YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula.
2015-07-13 17:52:13 +09:00
Wangda Tan
5ed1fead6b
YARN-3894. RM startup should fail for wrong CS xml NodeLabel capacity configuration. (Bibin A Chundatt via wangda)
2015-07-12 21:52:11 -07:00
Wangda Tan
1df39c1efc
YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda)
2015-07-11 10:26:46 -07:00
Zhijie Shen
1ea36299a4
YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola.
2015-07-10 18:58:10 -07:00
Ming Ma
08244264c0
YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma)
2015-07-10 08:30:10 -07:00
Xuan
5214876792
YARN-3888. ApplicationMaster link is broken in RM WebUI when appstate is
...
NEW. Contributed by Bibin A Chundatt
2015-07-09 21:37:33 -07:00
carlo curino
0e602fa3a1
YARN-3800. Reduce storage footprint for ReservationAllocation. Contributed by Anubhav Dhoot.
2015-07-09 16:51:59 -07:00
Karthik Kambatla
527c40e4d6
YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha)
2015-07-09 09:35:14 -07:00
Varun Vasudev
63d0365088
YARN-2194. Addendum patch to fix failing unit test in TestPrivilegedOperationExecutor. Contributed by Sidharta Seethana.
2015-07-09 11:51:59 +05:30
Jian He
c9dd2cada0
YARN-3892. Fixed NPE on RMStateStore#serviceStop when CapacityScheduler#serviceInit fails. Contributed by Bibin A Chundatt
2015-07-07 14:16:21 -07:00
Varun Vasudev
c40bdb56a7
YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan.
2015-07-07 16:59:29 +05:30
Devaraj K
af63427c6d
YARN-3837. javadocs of TimelineAuthenticationFilterInitializer give wrong
...
prefix for auth options. Contributed by Bibin A Chundatt.
2015-07-07 12:06:30 +05:30
Devaraj K
37d7395773
YARN-3875. FSSchedulerNode#reserveResource() doesn't print Application Id
...
properly in log. Contributed by Bibin A Chundatt.
2015-07-02 10:20:31 +05:30
Wangda Tan
0e4b06690f
YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda)
2015-07-01 17:32:22 -07:00
Jason Lowe
b5cdf78e8e
YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena
2015-07-01 21:13:32 +00:00
Devaraj K
80a68d6056
YARN-3830. AbstractYarnScheduler.createReleaseCache may try to clean a
...
null attempt. Contributed by nijel.
2015-07-01 19:03:44 +05:30
Colin Patrick Mccabe
d0cc0380b5
YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison via Colin P. McCabe)
2015-06-30 16:24:19 -07:00
Jian He
62e583c7dc
YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti
2015-06-29 13:37:32 -07:00
Devaraj K
b543d1a390
YARN-3859. LeafQueue doesn't print user properly for application add.
...
Contributed by Varun Saxena.
2015-06-28 10:04:50 +05:30
Xuan
fe6c1bd73a
YARN-2871. TestRMRestart#testRMRestartGetApplicationList sometime fails
...
in trunk. Contributed by zhihai xu
2015-06-26 19:43:59 -07:00
Jason Lowe
40b256949a
YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena
2015-06-26 15:47:07 +00:00
Devaraj K
57f1a01eda
YARN-3826. Race condition in ResourceTrackerService leads to wrong
...
diagnostics messages. Contributed by Chengbing Liu.
2015-06-25 16:13:59 +05:30
Jonathan Eagles
4c659ddbf7
YARN-3360. Add JMX metrics to TimelineDataManager (Jason Lowe via jeagles)
2015-06-24 16:16:52 -05:00
rohithsharmaks
dd4b387d96
YARN-3790. usedResource from rootQueue metrics may get stale data for FS scheduler after recovering the container (Zhihai Xu via rohithsharmaks)
2015-06-24 23:00:14 +05:30
Jason Lowe
8d58512d6e
YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula
2015-06-24 16:37:39 +00:00
Jason Lowe
2a20dd9b61
YARN-3809. Failed to launch new attempts because ApplicationMasterLauncher's threads all hang. Contributed by Jun Gong
2015-06-24 16:23:48 +00:00
Robert Kanter
99271b7621
YARN-3835. hadoop-yarn-server-resourcemanager test package bundles core-site.xml, yarn-site.xml (vamsee via rkanter)
2015-06-22 18:02:27 -07:00
Karthik Kambatla
5ebf2817e5
YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha)
2015-06-22 17:47:47 -07:00
Xuan
6c7a9d502a
YARN-3834. Scrub debug logging of tokens during resource localization. Contributed by Chris Nauroth
2015-06-21 17:13:44 -07:00
Xuan
5b5bb8dcdc
YARN-3802. Two RMNodes for the same NodeId are used in RM sometimes
...
after NM is reconnected. Contributed by zhihai xu
2015-06-18 14:37:49 -07:00
Xuan
a826d432f9
YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl. Contributed by Varun Saxena
2015-06-17 16:23:27 -07:00
Devaraj K
ebb9a82519
YARN-3148. Allow CORS related headers to passthrough in
...
WebAppProxyServlet. Contributed by Varun Saxena.
2015-06-17 15:56:18 +05:30
Xuan
e27d5a13b0
YARN-3714. AM proxy filter can not get RM webapp address from
...
yarn.resourcemanager.hostname.rm-id. Contributed by Masatake Iwasaki
2015-06-16 14:06:16 -07:00
Devaraj K
b039e69bb0
YARN-3789. Improve logs for LeafQueue#activateApplications(). Contributed
...
by Bibin A Chundatt.
2015-06-16 14:03:22 +05:30
Devaraj K
d8dcfa98e3
YARN-3794. TestRMEmbeddedElector fails because of ambiguous LOG reference.
...
Contributed by Chengbing Liu.
2015-06-12 13:42:49 +05:30
Xuan
5583f88bf7
YARN-3785. Support for Resource as an argument during submitApp call in
...
MockRM test class. Contributed by Sunil G
2015-06-10 21:40:48 -07:00
Zhijie Shen
8d0ef31632
YARN-3787. Allowed generic history service to load a number of applications whose started time is within the given range. Contributed by Xuan Gong.
2015-06-08 23:14:58 -07:00
Xuan
2b2465dfac
YARN-3778. Fix Yarn resourcemanger CLI usage. Contributed by Brahma Reddy Battula
2015-06-08 15:43:03 -07:00
Jian He
960b8f19ca
YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla
2015-06-08 14:50:58 -07:00
Devaraj K
126321eded
YARN-3747. TestLocalDirsHandlerService should delete the created test
...
directory logDir2. Contributed by David Moore.
2015-06-08 15:32:13 +05:30
Devaraj K
c7ee6c151c
YARN-3780. Should use equals when compare Resource in
...
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.
2015-06-08 11:54:55 +05:30
Karthik Kambatla
bd69ea408f
YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation and container reservation. (Zhihai Xu via kasha)
2015-06-07 11:37:52 -07:00
Xuan
3e000a919f
YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong
2015-06-05 12:48:52 -07:00
Karthik Kambatla
75885852cc
YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha)
2015-06-05 09:39:41 -07:00
Zhijie Shen
18dd01d6bf
YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong.
2015-06-04 14:47:08 -07:00
Jian He
1970ca7cbc
YARN-2392. Add more diags about app retry limits on AM failures. Contributed by Steve Loughran
2015-06-04 11:14:09 -07:00
Jian He
6ad4e59cfc
YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan
2015-06-04 10:52:59 -07:00
Wangda Tan
ebd797c48f
YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda)
2015-06-04 10:22:57 -07:00
Junping Du
d7e7f6aa03
YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K.
2015-06-04 04:59:27 -07:00
Xuan
5766a04428
YARN-3749. We should make a copy of configuration when init
...
MiniYARNCluster with multiple RMs. Contributed by Chun Chen
2015-06-03 17:20:15 -07:00
Zhijie Shen
bc85959edd
Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong."
...
This reverts commit 4a9ec1a824
.
2015-06-03 14:15:56 -07:00
Karthik Kambatla
edb9cd0f7a
YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha)
2015-06-03 13:47:24 -07:00
Jason Lowe
e13b671aa5
YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks
2015-06-03 19:44:07 +00:00
Zhijie Shen
dbc4f64937
YARN-3751. Fixed AppInfo to check if used resources are null. Contributed by Sunil G.
2015-06-03 11:51:41 -07:00
Wangda Tan
5cc3fced95
YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda)
2015-05-31 16:31:09 -07:00
Xuan
9686261ecb
YARN-2900. Application (Attempt and Container) Not Found in AHS results
...
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai
2015-05-31 15:37:05 -07:00
Xuan
da05361c3b
Revert "YARN-2900. Application (Attempt and Container) Not Found in AHS results"
...
This reverts commit 06f8e9caba
.
2015-05-31 00:26:04 -07:00
Xuan
06f8e9caba
YARN-2900. Application (Attempt and Container) Not Found in AHS results
...
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai
2015-05-30 21:56:19 -07:00
Zhijie Shen
4a9ec1a824
YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong.
2015-05-30 21:05:36 -07:00
Karthik Kambatla
a8acdd65b3
YARN-3467. Expose allocatedMB, allocatedVCores, and runningContainers metrics on running Applications in RM Web UI. (Anubhav Dhoot via kasha)
2015-05-30 13:36:24 -07:00
Zhijie Shen
eb6bf91eea
YARN-3740. Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS. Contributed by Xuan Gong.
2015-05-29 18:17:48 -07:00
Robert Kanter
6aec13cb33
YARN-3713. Remove duplicate function call storeContainerDiagnostics in ContainerDiagnosticsUpdateTransition (zxu via rkanter)
2015-05-29 15:34:37 -07:00
Karthik Kambatla
3ae2a62501
YARN-3547. FairScheduler: Apps that have no resource demand should not participate scheduling. (Xianyin Xin via kasha)
2015-05-29 15:17:22 -07:00
Andrew Wang
7673d4f205
HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang)
2015-05-29 14:54:18 -07:00
Vinod Kumar Vavilapalli
9acd24fec4
Fixed more FilesSystemRMStateStore issues. Contributed by Vinod Kumar Vavilapalli.
2015-05-28 15:25:56 -07:00
Zhijie Shen
54504133f4
YARN-3700. Made generic history service load a number of latest applications according to the parameter or the configuration. Contributed by Xuan Gong.
2015-05-27 16:52:22 -07:00
Allen Wittenauer
d6e3164d4a
YARN-2355. MAX_APP_ATTEMPTS_ENV may no longer be a useful env var for a container (Darrell Taylor via aw)
2015-05-27 16:40:56 -07:00
cnauroth
4102e5882e
YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch.
2015-05-27 14:31:49 -07:00
Wangda Tan
ec0a852a37
YARN-3647. RMWebServices api's should use updated api from CommonNodeLabelsManager to get NodeLabel object. (Sunil G via wangda)
2015-05-27 12:51:20 -07:00
Wangda Tan
cdbd66be11
YARN-3686. CapacityScheduler should trim default_node_label_expression. (Sunil G via wangda)
2015-05-26 15:58:47 -07:00
Jian He
10732d515f
YARN-3632. Ordering policy should be allowed to reorder an application when demand changes. Contributed by Craig Welch
2015-05-26 12:00:51 -07:00
Vinod Kumar Vavilapalli
500a1d9c76
YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev.
2015-05-26 11:38:35 -07:00
Tsuyoshi Ozawa
9a3d617b63
YARN-2336. Fair scheduler's REST API returns a missing '[' bracket JSON for deep queue tree. Contributed by Kenji Kikushima and Akira Ajisaka.
2015-05-26 19:07:40 +09:00
Jian He
446d51591e
YARN-3707. RM Web UI queue filter doesn't work. Contributed by Wangda Tan
2015-05-22 15:58:22 -07:00
Xuan
455b3acf0e
YARN-3701. Isolating the error of generating a single app report when
...
getting all apps from generic history service. Contributed by Zhijie
Shen
2015-05-22 14:34:39 -07:00
Vinayakumar B
55ed6558ba
HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin)
2015-05-22 16:45:19 +05:30
Junping Du
132d909d4a
YARN-3594. WintuilsProcessStubExecutor.startStreamReader leaks streams. Contributed by Lars Francke.
2015-05-22 04:23:25 -07:00
Vinod Kumar Vavilapalli
53fafcf061
YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more extensible mechanism of context objects. Contributed by Sidharta Seethana.
2015-05-21 15:50:23 -07:00
Karthik Kambatla
4513761869
YARN-3675. FairScheduler: RM quits when node removal races with continuous-scheduling on the same node. (Anubhav Dhoot via kasha)
2015-05-21 13:44:42 -07:00
Jian He
6329bd00fa
YARN-3654. ContainerLogsPage web UI should not have meta-refresh. Contributed by Xuan Gong
2015-05-20 17:20:21 -07:00
Jian He
8966d42179
YARN-3609. Load node labels from storage inside RM serviceStart. Contributed by Wangda Tan
2015-05-20 16:30:07 -07:00
Tsuyoshi Ozawa
7401e5b5e8
YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager. Contributed by Vinod Kumar Vavilapalli.
2015-05-20 08:59:26 +09:00
Wangda Tan
563eb1ad2a
YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda)
2015-05-19 16:54:38 -07:00
Wangda Tan
b37da52a1c
YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda)
2015-05-19 16:34:17 -07:00
Colin Patrick Mccabe
470c87dbc6
HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe)
2015-05-19 10:50:15 -07:00
Ravi Prakash
c97f32e7b9
YARN-3302. TestDockerContainerExecutor should run automatically if it can detect docker in the usual place (Ravindra Kumar Naik via raviprak)
2015-05-19 10:28:11 -07:00
Xuan
76afd28862
YARN-3541. Add version info on timeline service / generic history web UI and REST API. Contributed by Zhijie Shen
2015-05-18 13:17:16 -07:00
Xuan
b0ad644083
YARN-3526. ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster. Contributed by Weiwei Yang
2015-05-15 22:39:49 -07:00
Jason Lowe
f7e051c431
YARN-2421. RM still allocates containers to an app in the FINISHING state. Contributed by Chang Li
2015-05-15 22:09:30 +00:00
Junping Du
03a293aed6
YARN-3505 addendum: fix an issue in previous patch.
2015-05-15 06:39:39 -07:00
Vinod Kumar Vavilapalli
9a2a9553ee
Fixing HDFS state-store. Contributed by Arun Suresh.
2015-05-14 16:13:51 -07:00
Ravi Prakash
53fe4eff09
YARN-1519. Check in container-executor if sysconf is implemented before using it (Radim Kolar and Eric Payne via raviprak)
2015-05-14 15:55:37 -07:00
Junping Du
15ccd967ee
YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong.
2015-05-14 10:58:12 -07:00
Wangda Tan
0e85044e26
YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda)
2015-05-13 17:00:36 -07:00
Jason Lowe
711d77cc54
YARN-3641. NodeManager: stopRecoveryStore() shouldn't be skipped when exceptions happen in stopping NM's sub-services. Contributed by Junping Du
2015-05-13 21:06:47 +00:00
Wangda Tan
7f19e7a254
YARN-3521. Support return structured NodeLabel objects in REST API (Sunil G via wangda)
2015-05-13 13:43:17 -07:00
Xuan
0f95921447
YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch
2015-05-13 13:10:53 -07:00
Wangda Tan
341a476812
YARN-2921. Fix MockRM/MockAM#waitForState sleep too long. (Tsuyoshi Ozawa via wangda)
2015-05-13 13:06:07 -07:00
Zhijie Shen
fcd0702c10
YARN-3539. Updated timeline server documentation and marked REST APIs evolving. Contributed by Steve Loughran.
2015-05-12 21:13:22 -07:00
Karthik Kambatla
fe0df59627
YARN-3613. TestContainerManagerSecurity should init and start Yarn cluster in setup instead of individual methods. (nijel via kasha)
2015-05-12 10:45:33 -07:00
Devaraj K
5c2f05cd9b
YARN-3629. NodeID is always printed as "null" in node manager
...
initialization log. Contributed by nijel.
2015-05-12 22:20:25 +05:30
Devaraj K
8badd82ce2
YARN-3513. Remove unused variables in ContainersMonitorImpl and add debug
...
log for overall resource usage by all containers. Contributed by
Naganarasimha G R.
2015-05-12 16:54:38 +05:30
Wangda Tan
d6f6741296
YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda)
2015-05-11 17:31:15 -07:00
Junping Du
7e543c27fa
YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak.
2015-05-11 05:27:07 -07:00
Karthik Kambatla
a60f78e98e
YARN-3395. FairScheduler: Trim whitespaces when using username for queuename. (Zhihai Xu via kasha)
2015-05-09 15:41:20 -07:00
Karthik Kambatla
70fb37cd79
YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha)
2015-05-09 14:34:54 -07:00
Xuan
6471d18bc7
YARN-1912. ResourceLocalizer started without any jvm memory control.
...
Contributed by Masatake Iwasaki
2015-05-08 20:01:21 -07:00
Xuan
333f9a896d
YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup. Contributed by zhihai xu
2015-05-08 17:52:54 -07:00
Karthik Kambatla
2fb44c8aaf
YARN-3271. FairScheduler: Move tests related to max-runnable-apps from TestFairScheduler to TestAppRunnability. (nijel via kasha)
2015-05-08 16:39:10 -07:00
Robert Kanter
5658998845
YARN-3473. Fix RM Web UI configuration for some properties (rchiang via rkanter)
2015-05-08 16:23:10 -07:00
Jason Lowe
25e2b02122
YARN-3476. Nodemanager can fail to delete local logs if log aggregation fails. Contributed by Rohith
2015-05-08 22:45:52 +00:00
Jian He
d0b73287c2
YARN-3604. Fixed ZKRMStateStore#removeApplication to also disable watch. Contributed Zhihai Xu
2015-05-08 15:12:33 -07:00
Xuan
088156de43
YARN-2331. Distinguish shutdown during supervision vs. shutdown for
...
rolling upgrade. Contributed by Jason Lowe
2015-05-08 15:10:43 -07:00
Vinod Kumar Vavilapalli
f7a74d2491
YARN-3018. Unified the default value for the configuration property yarn.scheduler.capacity.node-locality-delay in code and default xml file. Contributed by Nijel SF.
2015-05-08 14:45:06 -07:00
Wangda Tan
f55d21d894
YARN-3593. Add label-type and Improve "DEFAULT_PARTITION" in Node Labels Page. (Naganarasimha G R via wangda)
2015-05-08 12:04:50 -07:00
Thomas Graves
5d708a4725
YARN-3600. AM container link is broken (Naganarasimha G R via tgraves
2015-05-08 16:35:40 +00:00
Robert (Bobby) Evans
bcf2890502
YARN-644: Basic null check is not performed on passed in arguments before using them in ContainerManagerImpl.startContainer
2015-05-08 11:11:01 -05:00
Devaraj K
241a72af0d
YARN-2784. Make POM project names consistent. Contributed by Rohith.
2015-05-08 19:23:53 +05:30
Tsuyoshi Ozawa
d18f10ad1b
MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi.
2015-05-08 15:59:46 +09:00
Devaraj K
ef3d66d462
YARN-3358. Audit log not present while refreshing Service ACLs.
...
Contributed by Varun Saxena.
2015-05-08 12:14:10 +05:30
Devaraj K
a28cd02ba0
YARN-3592. Fix typos in RMNodeLabelsManager. Contributed by Sunil G.
2015-05-08 10:54:36 +05:30
Akira Ajisaka
b167fe7605
YARN-1832. Fix wrong MockLocalizerStatus#equals implementation. Contributed by Hong Zhiguo.
2015-05-08 11:14:45 +09:00
Jian He
f489a4ec96
YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan
2015-05-07 17:35:41 -07:00
Jian He
b88700dcd0
YARN-3584. Fixed attempt diagnostics format shown on the UI. Contributed by nijel
2015-05-07 13:54:07 -07:00
Zhijie Shen
daf3e4ef8b
YARN-3448. Added a rolling time-to-live LevelDB timeline store implementation. Contributed by Jonathan Eagles.
2015-05-07 10:01:51 -07:00
Akira Ajisaka
918af8efff
YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula.
2015-05-07 13:33:03 +09:00
Vinod Kumar Vavilapalli
4c7b9b6abe
YARN-3385. Fixed a race-condition in ResourceManager's ZooKeeper based state-store to avoid crashing on duplicate deletes. Contributed by Zhihai Xu.
2015-05-06 17:51:17 -07:00
Junping Du
31b627b2a8
YARN-3580. [JDK8] TestClientRMService.testGetLabelsToNodes fails. Contributed by Robert Kanter.
2015-05-06 16:51:05 -07:00
Robert Kanter
b72507810a
YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter)
2015-05-06 14:19:06 -07:00
Jian He
0d3188fd25
YARN-3301. Fixed the format issue of the new RM attempt web page. Contributed by Xuan Gong
2015-05-06 13:45:00 -07:00
Xuan
a583a40693
YARN-3582. NPE in WebAppProxyServlet. Contributed by Jian He
2015-05-05 21:42:19 -07:00
Jian He
e4c3b52c89
YARN-3343. Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate timeout. Contributed by Rohith Sharmaks
2015-05-05 11:33:47 -07:00
Xuan
b7dd3a4f04
YARN-2123. Progress bars in Web UI always at 100% due to non-US locale.
...
Contributed by Akira AJISAKA
2015-05-05 10:40:16 -07:00
Junping Du
3810242062
YARN-3396. Handle URISyntaxException in ResourceLocalizationService. (Contributed by Brahma Reddy Battula)
2015-05-05 10:18:23 -07:00
Jason Lowe
9356cf8676
YARN-3552. RM Web UI shows -1 running containers for completed apps. Contributed by Rohith
2015-05-05 13:42:32 +00:00
Jian He
d701acc9c6
YARN-2725. Added test cases of retrying creating znode in ZKRMStateStore. Contributed by Tsuyoshi Ozawa
2015-05-04 16:13:29 -07:00
Wangda Tan
71f4de220c
YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner (Devaraj K via wangda)
2015-05-04 15:49:19 -07:00
Jason Lowe
8f65c793f2
YARN-3097. Logging of resource recovery on NM restart has redundancies. Contributed by Eric Payne
2015-05-04 15:31:15 +00:00
Gera Shegalov
f8204e241d
YARN-2893. AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream. (Zhihai Xu via gera)
2015-05-01 18:18:55 -07:00
Robert Kanter
ac7d152901
YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter)
2015-05-01 16:39:21 -07:00
Vinod Kumar Vavilapalli
1b3b9e5c31
YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan.
2015-04-30 21:41:07 -07:00
Zhijie Shen
7e8639fda4
YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong.
2015-04-30 13:05:55 -07:00
Wangda Tan
e2e8f77118
YARN-3564. Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable fails randomly. (Jian He via wangda)
2015-04-30 11:03:19 -07:00
Jian He
4c1af156ae
YARN-3533. Test: Fix launchAM in MockRM to wait for attempt to be scheduled. Contributed by Anubhav Dhoot
2015-04-29 14:50:01 -07:00
tgraves
2e215484bd
YARN-3517. RM web ui for dumping scheduler logs should be for admins only (Varun Vasudev via tgraves)
2015-04-29 21:25:42 +00:00
Karthik Kambatla
8f82970e0c
YARN-3485. FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies. (kasha)
2015-04-28 21:00:35 -07:00
Wangda Tan
db1b674b50
YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda)
2015-04-27 16:24:38 -07:00
Xuan
7f07c4d810
YARN-3530. ATS throws exception on trying to filter results without
...
otherinfo. Contributed by zhijie shen
2015-04-27 10:36:42 -07:00
Karthik Kambatla
47279c3228
YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha)
2015-04-26 09:13:46 -07:00
Jian He
d497f6ea2b
YARN-2498. Respect labels in preemption policy of capacity scheduler for inter-queue preemption. Contributed by Wangda Tan
2015-04-24 17:03:13 -07:00
Tsuyoshi Ozawa
4a3dabd94f
YARN-3406. Display count of running containers in the RM's Web UI. Contributed by Ryu Kobayashi.
2015-04-25 07:17:11 +09:00
Jason Lowe
5e093f0d40
YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula
2015-04-24 22:02:53 +00:00
Jian He
d03dcb9635
YARN-3387. Previous AM's container completed status couldn't pass to current AM if AM and RM restarted during the same time. Contributed by Sandflee
2015-04-24 12:13:29 -07:00
Xuan
eee9facbba
YARN-3511. Add errors and warnings page to ATS. Contributed by Varun Vasudev
2015-04-24 09:41:59 -07:00
Xuan
0b3f8957a8
YARN-3516. killing ContainerLocalizer action doesn't take effect when
...
private localizer receives FETCH_FAILURE status. Contributed by zhihai
xu
2015-04-23 16:40:40 -07:00
Vinod Kumar Vavilapalli
f5fe35e297
YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv)
2015-04-23 11:19:55 -07:00
Wangda Tan
395205444e
YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda)
2015-04-23 10:47:15 -07:00
tgraves
189a63a719
YARN-3434. Interaction between reservations and userlimit can result in significant ULF violation
2015-04-23 14:39:25 +00:00
Steve Loughran
d9bcf99aef
YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel)
2015-04-23 09:22:48 +01:00
Vinod Kumar Vavilapalli
a100be685c
YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana.
2015-04-22 17:26:13 -07:00
Junping Du
fad9d7e85b
New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K
2015-04-22 10:07:20 -07:00
Jian He
674c7ef649
YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev
2015-04-21 20:57:02 -07:00
Jian He
bdd90110e6
YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. Contributed by Rohith Sharmaks
2015-04-21 20:06:20 -07:00
Wangda Tan
e71d0d87d9
YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda)
2015-04-21 17:51:22 -07:00
Tsuyoshi Ozawa
105afd5477
YARN-3495. Confusing log generated by FairScheduler. Contributed by Brahma Reddy Battula.
2015-04-22 05:47:59 +09:00
Wangda Tan
44872b76fc
YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler. (Craig Welch via wangda)
2015-04-20 17:12:32 -07:00
Jian He
497c86b485
YARN-3136. Fixed a synchronization problem of AbstractYarnScheduler#getTransferredContainers. Contributed by Sunil G
2015-04-18 12:46:37 -07:00
Wangda Tan
f47a5763ac
YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda)
2015-04-17 17:22:27 -07:00
Wangda Tan
f65eeb412d
YARN-3493. RM fails to come up with error "Failed to load/recover state" when mem settings are changed. (Jian He via wangda)
2015-04-17 17:11:22 -07:00