big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
19,062 Commits 2 Branches 2 Tags 264 MiB
2d6195c0af
Commit Graph

4720 Commits

Author SHA1 Message Date
Weiwei Yang
2c95eb8111 YARN-8225. YARN precommit build failing in TestPlacementConstraintTransformations. Contributed by Shane Kumpf. 2018-04-28 17:37:37 +08:00
Wangda Tan
92c5331423 YARN-8005. Add unit tests for queue priority with dominant resource calculator. (Zian Chen via wangda) 
Change-Id: I17a645f20869a1e5d86fa7a325c93fec908b91dc
 2018-04-27 13:08:43 -07:00
Wangda Tan
b1833d9ba2 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
 2018-04-27 13:07:38 -07:00
Rohith Sharma K S
ef3ecc308d YARN-8221. RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 2018-04-27 22:58:10 +05:30
Eric Yang
24a5ccbf4b YARN-7781. Update YARN service documentation. 
Contributed by Gour Saha
 2018-04-27 12:38:30 -04:00
Eric Yang
14b47990af YARN-8204. Added a flag to disable YARN service upgrade. 
Contributed by Chandni Singh
 2018-04-27 12:24:43 -04:00
Billie Rinaldi
9ab3f97089 YARN-8211. Yarn registry dns log finds BufferUnderflowException on port ping. Contributed by Eric Yang 2018-04-27 07:58:43 -07:00
Billie Rinaldi
914d6b5cb1 YARN-8208. Add log statement for Docker client configuration file at INFO level. Contributed by Yesha Vora 2018-04-27 06:10:32 -07:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Sunil G
71220d218d YARN-8004. Add unit tests for inter queue preemption for dominant resource calculator. Contributed by Zian Chen. 2018-04-27 10:42:30 +05:30
Sunil G
7fc09c4f72 YARN-8215. ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM. Contributed by Rohith Sharma K S. 2018-04-27 10:33:42 +05:30
Sunil G
1634de0fc1 YARN-8205. Application State is not updated to ATS if AM launching is delayed. Contributed by Rohith Sharma K S. 2018-04-27 10:26:57 +05:30
Eric Yang
93979db840 YARN-8122. Added component health monitoring for YARN service. 
Contributed by Gour Saha
 2018-04-26 19:17:32 -04:00
Billie Rinaldi
598483e496 YARN-8140. Improve log message when launch cmd is ran for stopped yarn service. Contributed by Eric Yang 2018-04-26 13:54:40 -07:00
Eric Yang
4a7369b095 YARN-7939. Added support to upgrade a component instance. 
Contributed by Chandni Singh
 2018-04-26 13:54:40 -07:00
Wangda Tan
2a0fa50f9d YARN-8193. YARN RM hangs abruptly (stops allocating resources) when running successive applications. (Zian Chen via wangda) 
Change-Id: Ia83dd2499ee9000b9e09ae5a932f21a13c0ddee6
 2018-04-26 13:54:40 -07:00
Sunil G
7ac0abdc13 YARN-8188. RM Nodes UI data table index for sorting column need to be corrected post Application tags display. Contributed by Weiwei Yang. 2018-04-26 13:54:40 -07:00
Wangda Tan
ebedd69e25 YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda) 
Change-Id: I347871d672001653a3afe2e99adefd74e0d798cd
 2018-04-26 13:54:40 -07:00
Eric Yang
3b1dad9123 YARN-8196. Updated documentation for enabling YARN Service REST API. 
Contributed by Billie Rinaldi
 2018-04-26 13:54:40 -07:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Junping Du
56788d759f YARN-7598. Document how to use classpath isolation for aux-services in YARN. Contributed by Xuan Gong. 2018-04-24 18:29:14 +08:00
Robert Kanter
7ab08a9c37 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 2018-04-23 15:44:15 -07:00
Sunil G
83e5f25d5e YARN-8177. Documentation changes for auto creation of Leaf Queues with node label. Contributed by Suma Shivaprasad. 2018-04-23 13:36:27 +05:30
Sunil G
b6d94fad08 YARN-7956. [UI2] Avoid duplicating Components link under Services/<ServiceName>/Components. Contributed by Yesha Vora. 2018-04-23 13:32:09 +05:30
Konstantinos Karanasos
28e2244390 YARN-8111. Simplify PlacementConstraints API by removing allocationTagToIntraApp. Contributed by Weiwei Yang. 2018-04-20 12:24:48 -07:00
Jason Lowe
766544c0b0 YARN-7786. NullPointerException while launching ApplicationMaster. Contributed by lujie 2018-04-20 13:22:20 -05:00
Eric Yang
85585f9eef YARN-8064. Bugfix for clean up container-executor cmd helper file. 
Contributed by Eric Badger
 2018-04-20 13:49:33 -04:00
Sunil G
7fd878813b Revert "YARN-7956. [UI2] Avoid duplicating Components link under Services/<ServiceName>/Components. Contributed by Yesha Vora." 
This reverts commit a11ce19f47.
 2018-04-20 22:33:54 +05:30
Sunil G
a11ce19f47 YARN-7956. [UI2] Avoid duplicating Components link under Services/<ServiceName>/Components. Contributed by Yesha Vora. 2018-04-20 22:28:36 +05:30
Haibo Chen
860cc28a5a YARN-8185. Improve log in DirectoryCollection constructor. (Yufei Gu via Haibo Chen) 2018-04-20 07:41:24 -07:00
Sunil G
55e4252fc5 YARN-7830. [UI2] Post selecting grid view in Attempt page, attempt info page should also be opened with grid view. Contributed by Gergely Novák. 2018-04-20 20:05:05 +05:30
Rohith Sharma K S
b0753cc2c0 YARN-8189. [UI2] Nodes page column headers are half truncated. Contributed by Sunil G. 2018-04-20 19:36:26 +05:30
Rohith Sharma K S
87e09859f1 YARN-8182. [UI2] Proxy- Clicking on nodes under Nodes HeatMap gives 401 error. Contributed by Sunil G. 2018-04-20 19:35:17 +05:30
Inigo Goiri
da5bcf5f7d YARN-8186. [Router] Federation: routing getAppState REST invocations transparently to multiple RMs. Contributed by Giovanni Matteo Fumarola. 2018-04-19 16:49:29 -07:00
Sunil G
7d06806dfd YARN-6827. [ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart. Contributed by Rohith Sharma K S. 2018-04-20 00:05:53 +05:30
Sunil G
c6d7d3eb05 YARN-5888. [UI2] Improve unit tests for new YARN UI. Contributed by Akhil PB. 2018-04-19 21:37:35 +05:30
BELUGA BEHR
351e509a47
YARN-8169. Cleanup RackResolver.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-19 16:07:14 +09:00
Shashikant Banerjee
e4c39f3247
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-19 08:35:38 +09:00
Rohith Sharma K S
034da8fc97 YARN-8171. [UI2] AM Node link from attempt page should not redirect to new tab. Contributed by Sunil G. 2018-04-18 13:39:41 +05:30
Rohith Sharma K S
241d985467 YARN-8145. yarn rmadmin -getGroups doesn't return updated groups for user. Contributed by Sunil G. 2018-04-18 13:39:41 +05:30
Yufei
d426b980ac YARN-7966. Remove method AllocationConfiguration#getQueueAcl and related unit tests. Contributed by Sen Zhao. 2018-04-17 22:53:32 -07:00
Inigo Goiri
db1bba857a YARN-8096. Wrong condition in AmIpFilter#getProxyAddresses() to update the proxy IP list. Contributed by Oleksandr Shevchenko. 2018-04-17 13:08:01 -07:00
Haibo Chen
bb92bfb4ef YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 11:15:20 -07:00
Jason Lowe
1d6e43da51 YARN-7996. Allow user supplied Docker client configurations with YARN native services. Contributed by Shane Kumpf 2018-04-17 08:57:31 -05:00
Haibo Chen
86dfb70875 Revert "YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)" 
This reverts commit 0c14b6f0d8.
 2018-04-17 06:04:38 -07:00
Haibo Chen
0c14b6f0d8 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 05:59:03 -07:00
Sunil G
427ad7ecc4 YARN-8126. Support auto-spawning of admin configured services during bootstrap of RM. Contributed by Rohith Sharma K S. 2018-04-17 15:08:31 +05:30
Inigo Goiri
f1461b2487 YARN-8164. Fix a potential NPE in AbstractSchedulerPlanFollower. Contributed by lujie. 2018-04-16 17:32:19 -07:00
Inigo Goiri
dd5e18c4ae YARN-8165. Incorrect queue name logging in AbstractContainerAllocator. Contributed by Weiwei Yan. 2018-04-16 16:52:17 -07:00
Inigo Goiri
60e5c1b516 YARN-7773. YARN Federation used Mysql as state store throw exception, Unknown column 'homeSubCluster' in 'field list'. Contributed by Yiran Wu. 2018-04-16 16:26:54 -07:00
Haibo Chen
49f9acae40 YARN-8162. Remove Method DirectoryCollection#verifyDirUsingMkdir. (Yufei Gu via Haibo Chen) 2018-04-16 15:05:04 -07:00
Eric Yang
2d0662cfd5 YARN-8060. Added default readiness check for service components. 
Contributed by Billie Rinaldi
 2018-04-16 14:24:26 -04:00
Wangda Tan
896b473f1b YARN-8138. Add unit test to validate queue priority preemption works under node partition. (Zian Chen via wangda) 
Change-Id: Ibebfab98a714c12c2dc643b6d7b9754a7f813632
(cherry picked from commit 6ee62e6b1c9b4bc3447ce870446068e626b1a492)
 2018-04-14 11:08:36 -07:00
Rohith Sharma K S
669eb7bdea YARN-8156. Increase the default value of yarn.timeline-service.app-collector.linger-period.ms. Contributed by Charan Hebri. 2018-04-14 10:31:28 +05:30
Eric Yang
9031a76d44 YARN-8142. Improve SIGTERM handling for YARN Service Application Master. 
Contributed by Billie Rinaldi
 2018-04-13 15:34:33 -04:00
Weiwei Yang
226bedc023 YARN-8153. Guaranteed containers always stay in SCHEDULED on NM after restart. Contributed by Yang Wang. 2018-04-13 13:17:37 +08:00
Weiwei Yang
375654c36a YARN-8154. Fix missing titles in PlacementConstraints document. Contributed by Weiwei Yang. 2018-04-13 13:06:47 +08:00
Billie Rinaldi
53b3e59473 YARN-7936. Add default service AM Xmx. Contributed by Jian He 2018-04-12 12:38:00 -07:00
Eric E Payne
18844599ae YARN-8147. TestClientRMService#testGetApplications sporadically fails. Contributed by Jason Lowe 2018-04-12 17:53:57 +00:00
Jason Lowe
4571351ccc MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 2018-04-12 11:12:46 -05:00
Eric E Payne
6bb128dfb8 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 2018-04-12 16:04:23 +00:00
Haibo Chen
b8597858b1 YARN-7931. [atsv2 read acls] Include domain table creation as part of schema creator. (Vrushali C via Haibo Chen) 2018-04-12 06:38:30 -07:00
Billie Rinaldi
933477e9e0 YARN-7221. Add security check for privileged docker container. Contributed by Eric Yang 2018-04-11 11:24:23 -07:00
Weiwei Yang
7eb783e263 YARN-8127. Resource leak when async scheduling is enabled. Contributed by Tao Yang. 2018-04-11 17:15:25 +08:00
Rohith Sharma K S
c048711099 YARN-7941. Transitive dependencies for component are not resolved. Contributed by Billie Rinaldi. 2018-04-11 09:18:50 +05:30
Wangda Tan
d919eb6efa YARN-8133. Doc link broken for yarn-service from overview page. (Rohith Sharma K S via wangda) 
Change-Id: Iacf9a004585dd59e1c0cd8f8c618a38f047cc0fe
 2018-04-10 17:33:15 -07:00
Wangda Tan
2bf9cc2c73 YARN-8116. Nodemanager fails with NumberFormatException: For input string: . (Chandni Singh via wangda) 
Change-Id: Idd30cfca59982d3fc6e47aa1b88f844a78fae94d
 2018-04-10 17:32:38 -07:00
Eric Yang
c467f311d0 YARN-7973. Added ContainerRelaunch feature for Docker containers. 
Contributed by Shane Kumpf
 2018-04-10 19:25:00 -04:00
Eric Yang
d553799030 YARN-7984. Improved YARN service stop/destroy and clean up. 
Contributed by Billie Rinaldi
 2018-04-10 17:40:49 -04:00
Sunil G
7c1e77dda4 YARN-7804. [UI2] Refresh action on Grid view page should not be redirected to graph view. Contributed by Gergely Novák. 2018-04-10 16:09:09 +05:30
Jason Lowe
907919d28c YARN-7667. Docker Stop grace period should be configurable. Contributed by Eric Badger 2018-04-09 17:19:21 -05:00
Sunil G
821b0de4c5 YARN-7574. Add support for Node Labels on Auto Created Leaf Queue Template. Contributed by Suma Shivaprasad. 2018-04-09 21:17:22 +05:30
bibinchundatt
eb47c3de74 YARN-7905. Parent directory permission incorrect during public localization. Contributed by Bilwa S T. 2018-04-07 12:26:29 +05:30
Wangda Tan
00ebec89f1 YARN-1151. Ability to configure auxiliary services from HDFS-based JAR files. (Xuan Gong via wangda) 
Change-Id: Ied37ff11e507fc86847753ba79486652c8fadfe9
 2018-04-06 21:25:57 -07:00
Wangda Tan
d4e63ccca0 YARN-8048. Support auto-spawning of admin configured services during bootstrap of RM (Rohith Sharma K S via wangda) 
Change-Id: I2d8d61ccad55e1118009294d7e17822df3cd0fd5
 2018-04-06 21:24:58 -07:00
Subru Krishnan
00905efab2 YARN-8110. AMRMProxy recover should catch for all throwable to avoid premature exit. (Botong Huang via Subru). 2018-04-06 16:31:16 -07:00
Haibo Chen
024d7c0870 YARN-8107. Give an informative message when incorrect format is used in ATSv2 filter attributes. (Rohith Sharma K S via Haibo Chen) 2018-04-06 09:39:01 -07:00
Sunil G
b17dc9f5f5 YARN-8083. [UI2] All YARN related configurations are paged together in conf page. Contributed by Gergely Novák. 2018-04-06 21:53:14 +05:30
Wangda Tan
6cf023f9b7 Added CHANGES/RELEASES/Jdiff for 3.1.0 release 
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
 2018-04-05 15:52:39 -07:00
Rohith Sharma K S
f32d6275ba YARN-8119. [UI2] Timeline Server address' url scheme should be removed while accessing via KNOX. Contributed by Sunil G. 2018-04-05 23:32:35 +05:30
Haibo Chen
f8b8bd53c4 YARN-6936. [Atsv2] Retrospect storing entities into sub application table from client perspective. (Rohith Sharma K S via Haibo Chen) 2018-04-05 10:23:42 -07:00
Vrushali C
345e7624d5 YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 2018-04-04 15:08:03 -07:00
Haibo Chen
3087e89135 YARN-7946. Update TimelineServerV2 doc as per YARN-7919. (Haibo Chen) 2018-04-04 11:59:31 -07:00
Konstantinos Karanasos
7853ec8d2f YARN-8013. Support application tags when defining application namespaces for placement constraints. Contributed by Weiwei Yang. 2018-04-04 10:51:58 -07:00
Sunil G
42cd367c93 YARN-8115. [UI2] URL data like nodeHTTPAddress must be encoded in UI before using to access NM. Contributed by Sreenath Somarajapuram. 2018-04-04 22:13:14 +05:30
Wangda Tan
b779f4f0f6 YARN-8106. Update LogAggregationIndexedFileController to use readFull instead read to avoid IOException while loading log meta. (Prabhu Joseph via wangda) 
Change-Id: I63a65f73f8d1636e2c99ed9c8c2bbd05efcff80f
 2018-04-03 21:06:24 -07:00
Sunil G
f7a17b029d YARN-7764. Findbugs warning: Resource#getResources may expose internal representation. Contributed by Weiwei Yang. 2018-04-04 09:22:35 +05:30
Miklos Szegedi
2d06d885c8 YARN-8035. Uncaught exception in ContainersMonitorImpl during relaunch due to the process ID changing. Contributed by Shane Kumpf. 2018-04-03 10:01:00 -07:00
Haibo Chen
93d47a0ed5 YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 2018-04-03 07:59:20 -07:00
Wangda Tan
c78cb18c61 YARN-8091. Revisit checkUserAccessToQueue RM REST API. (wangda) 
Change-Id: I5fab3fe229c34e967487b7327c7b3c8ddf7cb795
 2018-04-02 15:22:05 -07:00
Jason Lowe
54a81211ea YARN-8082. Include LocalizedResource size information in the NM download log for localization. Contributed by Kuhu Shukla 2018-04-02 10:38:13 -05:00
Wangda Tan
a0bde7d525 YARN-7142. Support placement policy in yarn native services. (Gour Saha via wangda) 
Change-Id: I166c67a7a34430627c17365f60bac75b6da1b434
 2018-04-02 07:26:01 -07:00
Weiwei Yang
1606dad133 YARN-7497. Add file system based scheduler configuration store. Contributed by Jiandan Yang. 2018-03-30 21:41:33 +08:00
Wangda Tan
e7e20190df YARN-8086. ManagedParentQueue with no leaf queues cause JS error in new UI. (Suma Shivaprasad via wangda) 
Change-Id: I6d82af91adec02698afddde6883b1fe1924e6041
 2018-03-29 15:03:31 -07:00
Eric Yang
8fb00c3fce YARN-6257. Fix CapacityScheduler REST API JSON output. 
Contributed By Tao Yang
 2018-03-29 17:36:34 -04:00
Sunil G
7a59d60e0c YARN-8085. ResourceProfilesManager should be set in RMActiveServiceContext. Contributed by Tao Yang. 2018-03-29 21:41:16 +05:30
Sunil G
431076f637 YARN-8076. Support to specify application tags in distributed shell. Contributed by Weiwei Yang. 2018-03-29 15:55:39 +05:30
Eric Yang
3d185d62fc YARN-8069. Clean up example hostnames for RegistryDNS. 
Contributed by Billie Rinaldi
 2018-03-28 20:17:37 -04:00
Zhe Zhang
0b1c2b5fe1 YARN-7623. Fix the CapacityScheduler Queue configuration documentation. Contributed by Jonathan Hung. 2018-03-28 11:41:20 -07:00