big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,671 Commits 2 Branches 2 Tags 264 MiB
1943db5571
Commit Graph

5425 Commits

Author SHA1 Message Date
Eric Yang
a33ef4fd31 YARN-8867. Added resource localization status to YARN service status call. 
Contributed by Chandni Singh
 2019-01-24 18:43:21 -05:00
Weiwei Yang
a4bd64e724 YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu. 2019-01-24 17:53:43 +08:00
Sunil G
c726445990 YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 2019-01-24 15:01:43 +05:30
Weiwei Yang
721d5c2a5f YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph. 2019-01-23 17:59:36 +08:00
Weiwei Yang
e3e076dc78 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 2019-01-23 14:14:49 +08:00
Eric Yang
2fa9389c2e YARN-9146. Added REST API to configure auxiliary service. 
Contributed by Billie Rinaldi
 2019-01-22 18:24:43 -05:00
Weiwei Yang
d43df31751 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 2019-01-22 10:23:32 +08:00
Weiwei Yang
abde1e1f58 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang. 2019-01-21 16:54:14 +08:00
Takanobu Asanuma
27aa6e8899 HADOOP-16046. [JDK 11] Correct the compiler exclusion of org/apache/hadoop/yarn/webapp/hamlet/** classes for >= Java 9. Contributed by Devaraj K. 2019-01-21 10:54:58 +09:00
Eric Yang
824dfa3b09 YARN-8489. Support "dominant" component concept in YARN service. 
Contributed by Zac Zhou
 2019-01-18 20:23:04 -05:00
Eric Yang
8c7f6b2d4d YARN-9197. Add safe guard against NPE for component instance failure. 
Contributed by kyungwan nam
 2019-01-18 13:49:56 -05:00
Wangda Tan
6d7eedfd28 YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda) 
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
 2019-01-17 14:20:10 -08:00
rahul3
0a46baecd3
YARN-9203. Fix typos in yarn-default.xml. 
This closes #437

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-01-17 16:18:33 +09:00
Akira Ajisaka
104ef5df36
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 2019-01-16 14:36:36 +09:00
Vrushali C
713ded6b15 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks 2019-01-15 21:28:10 -08:00
Vrushali C
6be39230a6 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks 2019-01-15 21:25:37 -08:00
Akira Ajisaka
614af50625
YARN-9179. Fix NPE in AbstractYarnScheduler#updateNewContainerInfo. 2019-01-15 12:59:59 +09:00
Jason Lowe
6a923464af YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R 2019-01-08 16:54:05 -06:00
Weiwei Yang
0921b706f7 YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan. 2019-01-08 14:30:53 +08:00
Giovanni Matteo Fumarola
489411579c YARN-9169. Add metrics for queued opportunistic and guaranteed containers. Contributed by Abhishek Modi. 2019-01-07 15:16:55 -08:00
Wangda Tan
0a01d49917 YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: Id8af27134d3286a7a10d85eda9be25df9689d0e7
 2019-01-07 12:07:26 -08:00
Akira Ajisaka
5db7c49062
YARN-9162. Fix TestRMAdminCLI#testHelp. Contributed by Ayush Saxena. 2019-01-07 17:32:28 +09:00
Weiwei Yang
944cf87223 YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg. 2019-01-07 15:57:31 +08:00
Weiwei Yang
d3321fbed0 Revert "YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi." 
This reverts commit 650b9cbe16.
 2019-01-07 08:58:09 +08:00
Wangda Tan
2c02aa6ec2 YARN-9160. [Submarine] Document 'PYTHONPATH' environment variable setting when using -localization options. (Zhankun Tang via wangda) 
Change-Id: I115b6710eb11501e21ab3e9e7cd1a87f322e5cf8
 2019-01-06 10:25:35 -08:00
Wangda Tan
8f004fe4e0 YARN-9141. [Submarine] JobStatus outputs with system UTC clock, not local clock. (Zac Zhou via wangda) 
Change-Id: Ib2b17cf7860ddc9188f06498086de3326bc09519
 2019-01-06 10:24:29 -08:00
Weiwei Yang
650b9cbe16 YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi. 2019-01-06 11:29:59 +08:00
Giovanni Matteo Fumarola
6e35f7130f YARN-9166. Fix logging for preemption of Opportunistic containers for Guaranteed containers. Contributed by Abhishek Modi. 2019-01-04 11:16:41 -08:00
Rohith Sharma K S
8c6978c3ba YARN-6149. Allow port range to be specified while starting NM Timeline collector manager. Contributed by Abhishek Modi. 2019-01-04 14:07:02 +05:30
Rohith Sharma K S
573b158791 YARN-8567. Fetching yarn logs fails for long running application if it is not present in timeline store. Contributed by Tarun Parimi. 2019-01-04 14:01:35 +05:30
Sunil G
f4906ac019 YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang. 2019-01-04 12:10:00 +05:30
Eric Yang
dfceffa70d YARN-9147. Rmove auxiliary services when manifest file is removed. 
Contributed by Billie Rinaldi
 2019-01-03 12:57:21 -05:00
Weiwei Yang
cfe89e6f96 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 2019-01-03 23:56:28 +08:00
Akira Ajisaka
128f340f4f
YARN-8894. Improve InMemoryPlan#toString. Contributed by BELUGA BEHR. 2018-12-28 17:43:47 +09:00
Akira Ajisaka
15fe1b34dd
YARN-8878. Remove StringBuffer from ManagedParentQueue.java. Contributed by BELUGA BEHR. 2018-12-28 17:29:22 +09:00
Sunil G
742fd978b5 YARN-9090. [Submarine] Update submarine installation script document. Contributed by Xun Liu. 2018-12-26 08:26:48 +05:30
Erik Krogen
091ad974cd HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Botong Huang
657aa433e2 YARN 9108. Fix FederationIntercepter merge home and secondary allocate response typo. Contributed by Abhishek Modi. 2018-12-22 12:41:49 -08:00
Akira Ajisaka
e98e963e51
YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena. 2018-12-22 13:25:39 +09:00
Eric Yang
f82922dcfa YARN-5168. Added exposed port information for Docker container. 
Contributed by Xun Liu
 2018-12-21 19:44:07 -05:00
Eric Yang
ea724181d6 YARN-9132. Added file permission check for auxiliary services manifest file. 
Contributed by Billie Rinaldi
 2018-12-21 14:56:39 -05:00
Weiwei Yang
f659485ee8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 10:56:42 +08:00
Eric Yang
a80d321074 YARN-9152. Improved AuxServices REST API output. 
Contributed by Billie Rinaldi
 2018-12-20 19:21:55 -05:00
Eric Yang
7affa3053c YARN-9131. Updated document usage for dynamic auxiliary service. 
Contributed by Billie Rinaldi
 2018-12-20 18:45:49 -05:00
Inigo Goiri
5df9fb16b9 YARN-9130. Add Bind_HOST configuration for Yarn Web Proxy. Contributed by Rong Tang. 2018-12-19 18:58:00 -08:00
Billie Rinaldi
2e544dc921 YARN-9129. Ensure flush after printing to log plus additional cleanup. Contributed by Eric Yang 2018-12-19 16:45:23 -08:00
Eric Yang
e815fd9c49 YARN-9126. Fix container clean up for reinitialization. 
Contributed by Chandni Singh
 2018-12-19 14:55:56 -05:00
Eric Yang
ccdd982e51 YARN-9084. Reset container state and defer readiness check for upgrade. 
Contributed by Chandni Singh
 2018-12-18 18:02:03 -05:00
Eric Yang
c7a5a4435e YARN-9075. Add ability to register/remove auxiliary service without restart node manager. 
Contributed by Billie Rinaldi
 2018-12-18 17:05:51 -05:00
Billie Rinaldi
a6e9d27c29 YARN-9117. Add a check for insecure setup for container terminal. Contributed by Eric Yang 2018-12-18 12:49:10 -08:00
Billie Rinaldi
c5c73182e5 YARN-9072. Send exit command to terminate docker exec on connection close. Contributed by Eric Yang 2018-12-18 10:06:33 -08:00
Eric Yang
71e0b0d800 YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. 
Contributed by Tarun Parimi
 2018-12-17 12:04:25 -05:00
Vidura Mudalige
04c03476c9
YARN-9095. Removed Unused field from Resource: NUM_MANDATORY_RESOURCES 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-12-15 16:07:27 +09:00
Billie Rinaldi
4f622337ee YARN-9091. Improve terminal message when connection is refused. Contributed by Eric Yang 2018-12-14 19:55:50 -08:00
Billie Rinaldi
42297f7d99 YARN-8963. Add flag to disable interactive shell. Contributed by Eric Yang 2018-12-14 18:44:36 -08:00
Billie Rinaldi
aab310978f YARN-9089. Add Terminal Link to Service component instance page for UI2. Contributed by Eric Yang 2018-12-14 18:38:03 -08:00
Akira Ajisaka
26e55d4ceb
YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang. 2018-12-15 10:29:47 +09:00
Eric Yang
b2d7204ed0 YARN-9125. Fixed Carriage Return detection in Docker container launch command. 
Contributed by Billie Rinaldi
 2018-12-14 17:52:26 -05:00
Billie Rinaldi
ca90b2480e YARN-9032. Support sh shell for interactive container shell at command line. Contributed by Eric Yang 2018-12-14 14:08:22 -08:00
Billie Rinaldi
72e7c6a489 YARN-8962. Add ability to use interactive shell with normal yarn container. Contributed by Eric Yang 2018-12-12 18:18:57 -08:00
Wangda Tan
37eb919c59 YARN-8885. [DevicePlugin] Support NM APIs to query device resource allocation. (Zhankun Tang via wangda) 
Change-Id: I2a9870709b512af1ac6c09c9701d0b3c0791ff32
 2018-12-12 11:45:47 -08:00
Wangda Tan
9fba6cc247 YARN-9112. [Submarine] Support polling applicationId when it's not ready in cluster. (Zhankun Tang via wangda) 
Change-Id: I73d73f3d631b28fb9866faa56571839b13824a97
 2018-12-12 11:44:58 -08:00
Wangda Tan
61bdcb7b2b YARN-9015. [DevicePlugin] Add an interface for device plugin to provide customized scheduler. (Zhankun Tang via wangda) 
Change-Id: Ib2e4ae47a6f29bb3082c1f8520cf5a52ca720979
 2018-12-12 11:44:22 -08:00
Wangda Tan
c771fe6e10 YARN-8714. [Submarine] Support files/tarballs to be localized for a training job. (Zhankun Tang via wangda) 
Change-Id: I845131273e52a9d81dbc813ea6d4af06b205e334
 2018-12-12 11:43:23 -08:00
Wangda Tan
8a5361116e YARN-9078. [Submarine] Clean up the code of CliUtils#parseResourcesString. (Zhankun Tang via wangda) 
Change-Id: I21ac5757b6115d55ec3157ba25db1b65bc85a37b
 2018-12-12 11:42:44 -08:00
Sunil G
5a03c8b9ba YARN-9114. [UI2] log service redirect url need to support user name. Contributed by Akhil PB 2018-12-12 14:19:49 +05:30
Haibo Chen
fb55e5201e YARN-9008. Extend YARN distributed shell with file localization feature. (Contributed by Peter Bacsko) 2018-12-11 14:13:34 -08:00
Haibo Chen
881230da21 YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth) 2018-12-11 11:41:32 -08:00
Haibo Chen
ac578c0e82 YARN-9087. Improve logging for initialization of Resource plugins. (Contributed by Szilard Nemeth) 2018-12-10 11:12:54 -08:00
Haibo Chen
64411a6ff7 YARN-8738. FairScheduler should not parse negative maxResources or minResources values as positive. (Contributed by Szilard Nemeth) 2018-12-10 11:06:50 -08:00
Weiwei Yang
1c09a10e96 YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters. 2018-12-10 11:55:24 +08:00
Billie Rinaldi
154449fbd8 YARN-8914. Add xtermjs to YARN UI2. Contributed by Eric Yang and Akhil PB 2018-12-07 10:56:17 -08:00
Jonathan Hung
cc51607ccd YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 2018-12-07 10:45:37 -08:00
Haibo Chen
5d4a432660 YARN-9025. TestFairScheduler#testChildMaxResources is flaky. (Contributed by Szilard Nemeth) 2018-12-05 16:36:13 -08:00
Haibo Chen
912b1f9d64 YARN-9019. Ratio calculation of ResourceCalculator implementations could return NaN. (Contributed by Szilard Nemeth) 2018-12-05 15:16:41 -08:00
Haibo Chen
b3c75c1f1d YARN-8985. Improve debug log in FSParentQueue when assigning container. (Contributed by Wilfred Spiegelenburg) 2018-12-05 15:02:57 -08:00
Haibo Chen
2bba329c14 YARN-8994. Fix race condition between move app and queue cleanup in Fair Scheduler. (Contributed by Wilfred Spiegelenburg) 2018-12-05 14:38:15 -08:00
Eric Yang
1b790f4dd1 YARN-9071. Improved status update for reinitialized containers. 
Contributed by Chandni Singh
 2018-12-05 17:00:56 -05:00
Eric Yang
1dabb31cdf YARN-9057. Removed third party class bundle from CSI jar file. 
Contributed by Weiwei Yang
 2018-12-05 15:56:10 -05:00
Wangda Tan
228156cfd1 Revert "YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)" 
This reverts commit 46d6e00166.
 2018-12-04 14:13:06 -08:00
Yufei Gu
e89941fdbb YARN-9041. Performance Optimization of method FSPreemptionThread#identifyContainersToPreempt. Contributed by Wanqiang Ji. 2018-12-04 10:08:45 -08:00
Jonathan Hung
6d7b44c489 YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 2018-11-30 10:13:52 -08:00
Wangda Tan
bad12031f6 YARN-9010. Fix the incorrect trailing slash deletion in constructor method of CGroupsHandlerImpl. (Zhankun Tang via wangda) 
Change-Id: Iaecc66d57781cc10f19ead4647e47fc9556676da
 2018-11-29 14:56:07 -08:00
bibinchundatt
07142f54a8 YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 2018-11-29 22:02:59 +05:30
bibinchundatt
a68d766e87 YARN-8948. PlacementRule interface should be for all YarnSchedulers. Contributed by Bibin A Chundatt. 2018-11-29 21:43:34 +05:30
Weiwei Yang
efc4d91cbe YARN-9067. YARN Resource Manager is running OOM because of leak of Configuration Object. Contributed by Eric Yang. 2018-11-29 09:34:14 +08:00
Wangda Tan
fe7dab8ef5 YARN-8989. [YARN-8851] Move DockerCommandPlugin volume related APIs' invocation from DockerLinuxContainerRuntime#prepareContainer to #launchContainer. (Zhankun Tang via wangda) 
Change-Id: Ia6d532c687168448416dfdf46f0ac34bff20e6ca
 2018-11-28 15:03:06 -08:00
Wangda Tan
897643928c YARN-8975. [Submarine] Use predefined Charset object StandardCharsets.UTF_8 instead of String UTF-8. (Zhankun Tang via wangda) 
Change-Id: If6c7904aa17895e543cfca245264249eb7328bdc
 2018-11-28 14:39:06 -08:00
Wangda Tan
8ebeda98a9 YARN-8974. Improve the assertion message in TestGPUResourceHandler. (Zhankun Tang via wangda) 
Change-Id: I4eb58e9d251d5f54e7feffc4fbb813b4f5ae4b1b
 2018-11-28 14:36:30 -08:00
Wangda Tan
9ed87567ad YARN-9061. Improve the GPU/FPGA module log message of container-executor. (Zhankun Tang via wangda) 
Change-Id: Iece9b47438357077a53984a820d4d6423f480518
 2018-11-28 14:31:31 -08:00
Wangda Tan
579ef4be06 YARN-8882. [YARN-8851] Add a shared device mapping manager (scheduler) for device plugins. (Zhankun Tang via wangda) 
Change-Id: I9435136642c3d556971a357bf687f69df90bb45e
 2018-11-28 14:09:52 -08:00
Wangda Tan
9de8e8d049 YARN-9030. Log aggregation changes to handle filesystems which do not support setting permissions. (Suma Shivaprasad via wangda) 
Change-Id: I80f1e8196b8624e24d74494719fdedfd7061dced
 2018-11-28 13:36:21 -08:00
Jason Lowe
3ce99e32f7 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 2018-11-28 08:46:11 -06:00
Suma Shivaprasad
b3a052d199 YARN-9044. LogsCLI should contact ATSv2 for -am option. Contributed by Rohith Sharma K S 2018-11-28 00:46:53 -08:00
Suma Shivaprasad
7dc272199f YARN-9034. ApplicationCLI should have option to take clusterId. Contributed by Rohith Sharma K S. 2018-11-28 00:42:14 -08:00
Eric Yang
f657a2a661 YARN-9056. Improved YARN service upgrade state logic for readiness check. 
Contributed by Chandni Singh
 2018-11-27 18:36:59 -05:00
Eric Yang
33e0df4b35 YARN-8986. Added port publish for Docker container running with bridge network. 
Contributed by Charo Zhang
 2018-11-26 19:45:05 -05:00
Giovanni Matteo Fumarola
0d8406135f YARN-9054. Fix FederationStateStoreFacade#buildGetSubClustersCacheRequest. Contributed by Bibin A Chundatt. 2018-11-26 10:57:38 -08:00
Rohith Sharma K S
e223a790a7 YARN-9042. Fix javadoc error in deviceplugin package. Contributed by Zhankun Tang. 2018-11-22 20:04:30 +05:30
Weiwei Yang
176bb3f812 YARN-8984. AMRMClient#OutstandingSchedRequests leaks when AllocationTags is null or empty. Contributed by Yang Wang. 2018-11-22 16:52:29 +08:00
Sunil G
5fe1dbf195 YARN-8964. [UI2] YARN ui2 should use clusters/{cluster name} for all ATSv2 REST APIs. Contributed by Akhil PB. 2018-11-22 10:25:22 +05:30
Haibo Chen
a41b648e98 YARN-8992. Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue. (Contributed by Wilfred Spiegelenburg) 2018-11-20 14:12:37 -08:00
Billie Rinaldi
49824ed260 YARN-8838. Check that container user is same as websocket user for interactive shell. Contributed by Eric Yang 2018-11-20 11:12:24 -08:00
Sunil G
5fb14e0635 YARN-8953. [CSI] CSI driver adaptor module support in NodeManager. Contributed by Weiwei Yang. 2018-11-20 08:02:11 +05:30
Billie Rinaldi
27ffec7ba7 YARN-8778. Add command line interface to invoke interactive docker shell. Contributed by Eric Yang 2018-11-19 17:59:12 -08:00
Wangda Tan
8b23814415 YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda) 
Change-Id: I21b1addc9c32817650ea744a8f2e6b5602f2f4d4
 2018-11-19 08:55:30 -08:00
Wangda Tan
6357803645 YARN-8881. [YARN-8851] Add basic pluggable device plugin framework. (Zhankun Tang via wangda) 
Change-Id: If9a2f68cd4713b4ec932cdeda68106f17437c3d3
 2018-11-19 08:54:31 -08:00
Weiwei Yang
d027a24f03 YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 2018-11-18 23:18:26 +08:00
Giovanni Matteo Fumarola
abd6d48c46 YARN-8860. Federation client intercepter class contains unwanted character. Contributed by Abhishek Modi. 2018-11-16 10:23:06 -08:00
Rohith Sharma K S
ee3355be3c YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report. 2018-11-16 17:58:19 +05:30
Inigo Goiri
eeaf8edaa7 YARN-8856. TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError. Contributed by Sushil Ks. 2018-11-15 09:25:25 -08:00
Eric Yang
21ec4bdaef YARN-8672. Improve token filename management for localization. 
Contributed by Chandni Singh
 2018-11-14 15:22:01 -05:00
Wangda Tan
fcbd205cc3 YARN-9001. [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs. (Zac Zhou via wangda) 
Change-Id: Ic3d6c1e439df9cdf74448b345b925343224efe51
 2018-11-13 13:13:27 -08:00
Wangda Tan
076b795b2e YARN-8918. [Submarine] Correct method usage of str.subString in CliUtils. (Zhankun Tang via wangda) 
Change-Id: Id1f11dbab3aa838dee3a0ec8b8fd5dc32f5dd946
 2018-11-13 11:24:15 -08:00
Eric Yang
4c465f5535 YARN-8761. Service AM support for decommissioning component instances. 
Contributed by Billie Rinaldi
 2018-11-12 19:53:10 -05:00
Giovanni Matteo Fumarola
e269c3fb5a YARN-8997. [Submarine] Small refactors of modifier, condition check and redundant local variables. Contributed by Zhankun Tang. 2018-11-12 15:06:43 -08:00
Billie Rinaldi
1f9c4f32e8 YARN-8776. Implement Container Exec feature in LinuxContainerExecutor. Contributed by Eric Yang 2018-11-12 10:42:30 -08:00
Sunil G
42f3a7082a YARN-8877. [CSI] Extend service spec to allow setting resource attributes. Contributed by Weiwei Yang. 2018-11-12 21:09:30 +05:30
Weiwei Yang
c741109522 YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt. 2018-11-12 18:18:23 +08:00
Sunil G
4e7284443e YARN-8902. [CSI] Add volume manager that manages CSI volume lifecycle. Contributed by Weiwei Yang. 2018-11-12 11:57:02 +05:30
Botong Huang
b5ec85d966 YARN-8933. [AMRMProxy] Fix potential empty fields in allocation response, move SubClusterTimeout to FederationInterceptor. Contributed by Botong Huang. 2018-11-11 11:12:53 -08:00
Eric Yang
2664248797 YARN-9002. Improve keytab loading for YARN Service. 
Contributed by Gour Saha
 2018-11-10 01:52:19 -05:00
Haibo Chen
524a7523c4 YARN-8990. Fix fair scheduler race condition in app submit and queue cleanup. (Contributed by Wilfred Spiegelenburg) 2018-11-08 16:02:48 -08:00
Weiwei Yang
e1bbf7dcdf YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang. 2018-11-08 17:47:18 +08:00
Weiwei Yang
f8c72d7b3a YARN-8880. Add configurations for pluggable plugin framework. Contributed by Zhankun Tang. 2018-11-08 12:23:00 +08:00
Weiwei Yang
c96cbe8659 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 2018-11-07 22:42:22 +08:00
Takanobu Asanuma
8dc1f6dbf7 YARN-8866. Fix a parsing error for crossdomain.xml. 2018-11-07 18:26:07 +09:00
Weiwei Yang
482716e5a4 YARN-8976. Remove redundant modifiers in interface ApplicationConstants. Contributed by Zhankun Tang. 2018-11-07 10:48:07 +08:00
Akira Ajisaka
951c98f890
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 2018-11-07 11:17:35 +09:00
Eric Yang
08d69d91f2 YARN-8957. Add Serializable interface to ComponentContainers. 
Contributed by Zhankun Tang
 2018-11-06 14:46:05 -05:00
Jason Lowe
ab6aa4c726 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 2018-11-06 08:40:59 -06:00
Weiwei Yang
5d6554c722 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 2018-11-06 14:50:09 +08:00
Weiwei Yang
c7fcca0d7e YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 2018-11-06 13:14:57 +08:00
bibinchundatt
f84a278baa YARN-8905. [Router] Add JvmMetricsInfo and pause monitor. Contributed by Bilwa S T. 2018-11-03 20:35:31 +05:30
Giovanni Matteo Fumarola
989715ec50 YARN-8893. [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM client. Contributed by Botong Huang. 2018-11-02 15:30:08 -07:00
Giovanni Matteo Fumarola
aed836efbf YARN-8897. LoadBasedRouterPolicy throws NPE in case of sub cluster unavailability. Contributed by Bilwa S T. 2018-11-02 11:27:11 -07:00
Giovanni Matteo Fumarola
babc946d40 YARN-8954. Reservations list field in ReservationListInfo is not accessible. Contributed by Oleksandr Shevchenko. 2018-11-02 11:10:08 -07:00
Jonathan Hung
2ab611d48b YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 2018-11-01 14:22:00 -07:00
Steve Loughran
d174b91635 HADOOP-15895. [JDK9+] Add missing javax.annotation-api dependency to hadoop-yarn-csi. 
Contributed by Takanobu Asanuma.
 2018-11-01 12:56:20 +00:00
Akira Ajisaka
f747f5b06c
HADOOP-15886. Fix findbugs warnings in RegistryDNS.java. 2018-10-31 10:01:31 +09:00
Shane Kumpf
277a3d8d9f YARN-6729. Clarify documentation on how to enable cgroup support. Contributed by Zhankun Tang 2018-10-30 11:36:55 -06:00
Sunil G
d36012b69f YARN-8854. Upgrade jquery datatable version references to v1.10.19. Contributed by Akhil PB. 2018-10-30 22:56:46 +05:30
Rohith Sharma K S
486b9a4a75 YARN-7754. [Atsv2] Update document for running v1 and v2 TS. Contributed by Suma Shivaprasad. 2018-10-30 11:35:01 +05:30
Rohith Sharma K S
a283da2167 YARN-8871. Document ATSv2 integrated LogWebService. Contributed by Suma Shivaprasad. 2018-10-30 11:34:20 +05:30
Rohith Sharma K S
4ec4ec6971 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0. 2018-10-30 11:30:08 +05:30
Weiwei Yang
78ea897b67 YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang. 2018-10-29 12:04:32 +08:00
Weiwei Yang
1d90a0dd23 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 2018-10-29 11:53:10 +08:00
Billie Rinaldi
d07e873b7d YARN-8569. Create an interface to provide cluster information to application. Contributed by Eric Yang 2018-10-26 17:57:05 -07:00
Robert Kanter
f76e3c3db7 YARN-8930. CGroup-based strict container memory enforcement does not work with CGroupElasticMemoryController (haibochen via rkanter) 2018-10-25 11:09:47 -07:00
Weiwei Yang
ddc1e0b727 YARN-8941. Fix findbugs warnings in hadoop-yarn-csi. Contributed by Weiwei Yang. 2018-10-25 07:55:51 +08:00
Eric Yang
c16c49b8c3 YARN-8587. Added retries for fetching docker exit code. 
Contributed by Charo Zhang
 2018-10-24 17:29:46 -04:00
Robert Kanter
69b328943e YARN-8929. DefaultOOMHandler should only pick running containers to kill upon oom events (haibochen via rkanter) 2018-10-24 13:15:50 -07:00
Eric Yang
bbc6dcd3d0 YARN-8938. Updated YARN service upgrade document. 
Contributed by Chandni Singh
 2018-10-24 11:51:44 -04:00
Haibo Chen
766b78ee07 YARN-8911. ContainerScheduler incorrectly uses percentage number as the cpu resource utlization. 2018-10-24 07:58:26 -07:00
Weiwei Yang
553044818d YARN-8939. Javadoc build fails in hadoop-yarn-csi. Contributed by Weiwei Yang. 2018-10-24 18:02:45 +08:00
Eric Yang
a0c0b793c3 YARN-8814. Yarn Service Upgrade: Update the swagger definition. 
Contributed by Chandni Singh
 2018-10-23 19:04:58 -04:00
Haibo Chen
823bb5dda8 MAPREDUCE-4669. MRAM web UI does not work with HTTPS. (Contributed by Robert Kanter) 2018-10-23 15:28:13 -07:00
Jason Lowe
93fb3b4b9c YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 2018-10-23 12:49:15 -05:00
Rohith Sharma K S
0b62983c5a YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 2018-10-23 12:58:41 +05:30
Sunil G
0efddd85f0 YARN-8873. [YARN-8811] Add CSI java-based client library. Contributed by Weiwei Yang. 2018-10-23 11:46:06 +05:30
Eric Yang
e98a506227 YARN-8923. Cleanup references to ENV file type in YARN service code. 
Contributed by Suma Shivaprasad
 2018-10-22 18:39:57 -04:00
Eric Yang
292c9e017f YARN-8542. Added YARN service REST API to list containers. 
Contributed by Chandni Singh
 2018-10-22 15:59:52 -04:00
Eric Yang
66e7a2c787 YARN-8922. Fixed test-container-executor test setup and clean up. 
Contributed by Robert Kanter
 2018-10-22 15:28:58 -04:00
Sunil G
e50334513c YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 2018-10-22 14:15:51 +05:30
Eric Yang
e2a9fa8448 HADOOP-15821. Move YARN Registry to Hadoop Registry. 
Contributed by Íñigo Goiri
 2018-10-19 19:46:48 -04:00
Eric Yang
47ad98b2e1 YARN-8910. Fixed misleading log statement when container max retries is infinite. 
Contributed by Chandni Singh
 2018-10-19 13:49:04 -04:00
Wangda Tan
5e02b4915b YARN-8916. Define a constant docker string in ContainerRuntimeConstants.java for better maintainability. (Zhankun Tang via wangda) 
Change-Id: I1349e740037f81afdbe30edbe741f20e88fd0a90
 2018-10-19 09:49:26 -07:00
Wangda Tan
071b9e19aa YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda) 
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
 2018-10-19 09:49:26 -07:00
Haibo Chen
285d2c0753 YARN-8449. RM HA for AM web server HTTPS Support. (Contributed by Robert Kanter) 2018-10-18 21:24:36 -07:00
Weiwei Yang
13cc0f50ea YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 2018-10-19 09:55:59 +08:00
Eric Yang
66f059ed1d YARN-8618. Added detection for non-upgradable service. 
Contributed by Chandni Singh
 2018-10-18 19:59:11 -04:00
Wangda Tan
780be14f07 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
 2018-10-18 10:58:21 -07:00
Wangda Tan
a457a8951a YARN-8456. Fix a configuration handling bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable. (Zhankun Tang via wangda) 
Change-Id: Iff150ea98ba0c60d448474fd940eb121afce6965
 2018-10-18 10:57:11 -07:00
Wangda Tan
bfb88b10f4 YARN-6098. Add documentation for Delete Queue. (Suma Shivaprasad via wangda) 
Change-Id: I23bf7ca0acbe1168a7f805ceccdd62fe41bfeb35
 2018-10-18 10:55:39 -07:00
Eric Yang
ba7e81667c YARN-8687. Update YARN service file type in documentation. 
Contributed by Suma Shivaprasad
 2018-10-18 12:02:10 -04:00
Haibo Chen
32fe351bb6 YARN-8864. NM incorrectly logs container user as the user who sent a start/stop container request in its audit log. (Contributed by Wilfred Spiegelenburg) 2018-10-18 08:28:07 -07:00
Sunil G
2202e00ba8 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 2018-10-18 15:23:20 +05:30
Sunil G
5085e5fa9e YARN-8759. Copy of resource-types.xml is not deleted if test fails, causes other test failures. Contributed by Antal Bálint Steinbach. 2018-10-17 16:05:08 +05:30
Haibo Chen
22f85f2927 YARN-8582. Document YARN support for HTTPS in AM Web server. (Contributed by Robert Kanter) 2018-10-16 17:23:33 -07:00
Eric Yang
3bfd214a59 YARN-8810. Fixed a YARN service bug in comparing ConfigFile object. 
Contributed by Chandni Singh
 2018-10-16 18:54:40 -04:00
Haibo Chen
84e22a6af4 YARN-8842. Expose metrics for custom resource types in QueueMetrics. (Contributed by Szilard Nemeth) 2018-10-16 14:14:30 -07:00
Wangda Tan
538250db26 YARN-8892. YARN UI2 doc changes to update security status (verified under security environment). (Sunil G via wangda) 
Change-Id: I8bc8622936861b8d6de3e42a0b75af86ad8a3961
 2018-10-16 13:51:02 -07:00
Wangda Tan
143d74775b YARN-8798. [Submarine] Job should not be submitted if --input_path option is missing. (Zhankun Tang via wangda) 
Change-Id: I7ae0e44eb5179b04a6ac861ec1c65f3b18c38f0f
 2018-10-16 13:51:02 -07:00
Wangda Tan
46d6e00166 YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda) 
Change-Id: I46e8d9fd32c7745c313030da62da41486a77b3ea
 2018-10-16 13:51:02 -07:00
Wangda Tan
ed08dd3b0c YARN-8875. [Submarine] Add documentation for submarine installation script details. (Xun Liu via wangda) 
Change-Id: I1c8d39c394e5a30f967ea514919835b951f2c124
 2018-10-16 13:51:01 -07:00
Haibo Chen
c2288ac45b YARN-8448. AM HTTPS Support for AM communication with RMWeb proxy. (Contributed by Robert Kanter) 2018-10-16 13:36:26 -07:00
Sunil G
753f149fd3 YARN-8879. Kerberos principal is needed when submitting a submarine job. Contributed by Zac Zhou. 2018-10-16 22:17:51 +05:30
Haibo Chen
f880ff418c YARN-8775. TestDiskFailures.testLocalDirsFailures sometimes can fail on concurrent File modifications. (Contributed by Antal Bálint Steinbach) 2018-10-15 09:37:20 -07:00
Sunil G
fa94d370b6 YARN-8869. YARN Service Client might not work correctly with RM REST API for Kerberos authentication. Contributed by Eric Yang. 2018-10-15 21:21:57 +05:30
Sunil G
8e5365e277 YARN-8836. Add tags and attributes in resource definition. Contributed by Weiwei Yang. 2018-10-15 15:38:42 +05:30
Vrushali C
bca928d3c7 YARN-3879 [Storage implementation] Create HDFS backing storage implementation for ATS reads. Contributed by Abhishek Modi. 2018-10-11 21:14:06 -07:00
Vrushali C
a3edfddcf7 YARN-8834 Provide Java client for fetching Yarn specific entities from TimelineReader. Contributed by Abhishek Modi 2018-10-11 20:21:00 -07:00
Vrushali C
8d1981806f YARN-5742 Serve aggregated logs of historical apps from timeline service. Contributed by Rohith Sharma KS 2018-10-11 16:26:07 -07:00
Billie Rinaldi
96d28b4750 YARN-8777. Container Executor C binary change to execute interactive docker command. Contributed by Eric Yang 2018-10-11 09:25:21 -07:00
Billie Rinaldi
aeeb0389a5 YARN-8710. Service AM should set a finite limit on NM container max retries. Contributed by Suma Shivaprasad 2018-10-11 08:59:17 -07:00
Jason Lowe
e787d65a08 YARN-8861. executorLock is misleading in ContainerLaunch. Contributed by Chandni Singh 2018-10-11 10:54:57 -05:00
Sunil G
8a37983990 YARN-8753. [UI2] Lost nodes representation missing from Nodemanagers Chart. Contributed by Yesha Vora. 2018-10-11 16:15:25 +05:30
Sunil G
d3da741fa2 YARN-8666. [UI2] Remove application tab from YARN Queue Page. Contributed by Yesha Vora. 2018-10-11 16:05:53 +05:30
Vrushali C
045069efec YARN-6989 Ensure timeline service v2 codebase gets UGI from HttpServletRequest in a consistent way. Contributed by Abhishek Modi 2018-10-10 15:17:05 -07:00
Jason Lowe
5ce70e1211 YARN-7644. NM gets backed up deleting docker containers. Contributed by Chandni Singh 2018-10-10 09:52:19 -05:00