big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,348 Commits 2 Branches 2 Tags 264 MiB
15098df744
Commit Graph

5355 Commits

Author SHA1 Message Date
Weiwei Yang
2b7f828d46 YARN-9252. Allocation Tag Namespace support in Distributed Shell. Contributed by Prabhu Joseph. 2019-02-11 14:42:21 +08:00
Masatake Iwasaki
e0ab1bdece YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 2019-02-09 00:26:50 +09:00
Eric E Payne
d1ca9432dd YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 2019-02-07 16:38:11 +00:00
Vinayakumar B
3c96a03dea YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena. 2019-02-07 13:02:35 +05:30
Rohith Sharma K S
7fa62e150c YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi. 2019-02-07 09:58:53 +05:30
Rohith Sharma K S
1e5e08d83b YARN-7627. [ATSv2] When passing a non-number as metricslimit, the error message is wrong. Contributed by Charan Hebri. 2019-02-07 09:58:53 +05:30
Weiwei Yang
711d22f166 YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph. 2019-02-06 22:36:51 +08:00
Wangda Tan
308f3168fa Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
 2019-02-05 14:03:18 -08:00
Suma Shivaprasad
2044967e75 YARN-9246 NPE when executing a command yarn node -status or -states without additional arguments. Contributed by Masahiro Tanaka 2019-02-05 09:39:38 -08:00
Rohith Sharma K S
e3ec18b0c4 YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi. 2019-02-05 13:48:04 +05:30
Sunil G
ba38db4f5b YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri. 2019-02-05 13:41:44 +05:30
Weiwei Yang
aa7ce50e55 YARN-9275. Add link to NodeAttributes doc in PlacementConstraints document. Contributed by Masatake Iwasaki. 2019-02-05 16:07:41 +08:00
Rohith Sharma K S
571838920d YARN-9149. yarn container -status misses logUrl when integrated with ATSv2. Contributed by Abhishek Modi. 2019-02-05 12:39:10 +05:30
Sunil G
604b2489a9 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 2019-02-04 12:48:28 +05:30
Weiwei Yang
69bcff380f YARN-9231. TestDistributedShell fix timeout. Contributed by Prabhu Joseph. 2019-02-02 23:27:31 +08:00
Weiwei Yang
f20b043a02 YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang. 2019-02-02 23:04:34 +08:00
Weiwei Yang
28ad20a711 YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie. 2019-02-02 23:02:45 +08:00
Vrushali C
2c13513040 YARN-8549 Adding a NoOp timeline writer and reader plugin classes for ATSv2. Contributed by Prabha Manepalli. 2019-02-01 14:26:50 -08:00
Giovanni Matteo Fumarola
f738b397ae YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi. 2019-01-31 11:24:15 -08:00
Sunil G
71c49fa60f YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth. 2019-01-31 09:25:29 +05:30
Akira Ajisaka
a3a9ae3cea
YARN-9251. Build failure for -Dhbase.profile=2.0. Contributed by Rohith Sharma K S. 2019-01-30 05:17:04 -08:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Eric E Payne
04105bbfdb YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 2019-01-29 17:22:24 +00:00
Weiwei Yang
085f0e8ae7 YARN-9086. [CSI] Run csi-driver-adaptor as aux service. Contributed by Weiwei Yang. 2019-01-29 14:53:08 +08:00
Weiwei Yang
4f63ffe444 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 2019-01-29 10:34:26 +08:00
Eric Yang
f5a95f7998 YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency. 
Contributed by Suma Shivaprasad
 2019-01-28 18:10:33 -05:00
Eric Yang
2e636dd3c4 YARN-9074. Consolidate docker removal logic in ContainerCleanup. 
Contributed by Zhaohui Xin
 2019-01-28 18:05:53 -05:00
Eric Yang
1ab69a9543 YARN-9221. Added flag to disable dynamic auxiliary service feature. 
Contributed by Billie Rinaldi
 2019-01-25 19:05:36 -05:00
Jonathan Hung
6cace58e21 YARN-9222. Print launchTime in ApplicationSummary 2019-01-25 13:17:00 -08:00
Rohith Sharma K S
84bb9808a2 YARN-7761. [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link. Contributed by Akhil PB. 2019-01-25 14:00:35 +05:30
Eric Yang
a33ef4fd31 YARN-8867. Added resource localization status to YARN service status call. 
Contributed by Chandni Singh
 2019-01-24 18:43:21 -05:00
Weiwei Yang
a4bd64e724 YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu. 2019-01-24 17:53:43 +08:00
Sunil G
c726445990 YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 2019-01-24 15:01:43 +05:30
Weiwei Yang
721d5c2a5f YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph. 2019-01-23 17:59:36 +08:00
Weiwei Yang
e3e076dc78 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 2019-01-23 14:14:49 +08:00
Eric Yang
2fa9389c2e YARN-9146. Added REST API to configure auxiliary service. 
Contributed by Billie Rinaldi
 2019-01-22 18:24:43 -05:00
Weiwei Yang
d43df31751 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 2019-01-22 10:23:32 +08:00
Weiwei Yang
abde1e1f58 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang. 2019-01-21 16:54:14 +08:00
Takanobu Asanuma
27aa6e8899 HADOOP-16046. [JDK 11] Correct the compiler exclusion of org/apache/hadoop/yarn/webapp/hamlet/** classes for >= Java 9. Contributed by Devaraj K. 2019-01-21 10:54:58 +09:00
Eric Yang
824dfa3b09 YARN-8489. Support "dominant" component concept in YARN service. 
Contributed by Zac Zhou
 2019-01-18 20:23:04 -05:00
Eric Yang
8c7f6b2d4d YARN-9197. Add safe guard against NPE for component instance failure. 
Contributed by kyungwan nam
 2019-01-18 13:49:56 -05:00
Wangda Tan
6d7eedfd28 YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda) 
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
 2019-01-17 14:20:10 -08:00
rahul3
0a46baecd3
YARN-9203. Fix typos in yarn-default.xml. 
This closes #437

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-01-17 16:18:33 +09:00
Akira Ajisaka
104ef5df36
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 2019-01-16 14:36:36 +09:00
Vrushali C
713ded6b15 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks 2019-01-15 21:28:10 -08:00
Vrushali C
6be39230a6 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks 2019-01-15 21:25:37 -08:00
Akira Ajisaka
614af50625
YARN-9179. Fix NPE in AbstractYarnScheduler#updateNewContainerInfo. 2019-01-15 12:59:59 +09:00
Jason Lowe
6a923464af YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R 2019-01-08 16:54:05 -06:00
Weiwei Yang
0921b706f7 YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan. 2019-01-08 14:30:53 +08:00
Giovanni Matteo Fumarola
489411579c YARN-9169. Add metrics for queued opportunistic and guaranteed containers. Contributed by Abhishek Modi. 2019-01-07 15:16:55 -08:00
Wangda Tan
0a01d49917 YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: Id8af27134d3286a7a10d85eda9be25df9689d0e7
 2019-01-07 12:07:26 -08:00
Akira Ajisaka
5db7c49062
YARN-9162. Fix TestRMAdminCLI#testHelp. Contributed by Ayush Saxena. 2019-01-07 17:32:28 +09:00
Weiwei Yang
944cf87223 YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg. 2019-01-07 15:57:31 +08:00
Weiwei Yang
d3321fbed0 Revert "YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi." 
This reverts commit 650b9cbe16.
 2019-01-07 08:58:09 +08:00
Wangda Tan
2c02aa6ec2 YARN-9160. [Submarine] Document 'PYTHONPATH' environment variable setting when using -localization options. (Zhankun Tang via wangda) 
Change-Id: I115b6710eb11501e21ab3e9e7cd1a87f322e5cf8
 2019-01-06 10:25:35 -08:00
Wangda Tan
8f004fe4e0 YARN-9141. [Submarine] JobStatus outputs with system UTC clock, not local clock. (Zac Zhou via wangda) 
Change-Id: Ib2b17cf7860ddc9188f06498086de3326bc09519
 2019-01-06 10:24:29 -08:00
Weiwei Yang
650b9cbe16 YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi. 2019-01-06 11:29:59 +08:00
Giovanni Matteo Fumarola
6e35f7130f YARN-9166. Fix logging for preemption of Opportunistic containers for Guaranteed containers. Contributed by Abhishek Modi. 2019-01-04 11:16:41 -08:00
Rohith Sharma K S
8c6978c3ba YARN-6149. Allow port range to be specified while starting NM Timeline collector manager. Contributed by Abhishek Modi. 2019-01-04 14:07:02 +05:30
Rohith Sharma K S
573b158791 YARN-8567. Fetching yarn logs fails for long running application if it is not present in timeline store. Contributed by Tarun Parimi. 2019-01-04 14:01:35 +05:30
Sunil G
f4906ac019 YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang. 2019-01-04 12:10:00 +05:30
Eric Yang
dfceffa70d YARN-9147. Rmove auxiliary services when manifest file is removed. 
Contributed by Billie Rinaldi
 2019-01-03 12:57:21 -05:00
Weiwei Yang
cfe89e6f96 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 2019-01-03 23:56:28 +08:00
Akira Ajisaka
128f340f4f
YARN-8894. Improve InMemoryPlan#toString. Contributed by BELUGA BEHR. 2018-12-28 17:43:47 +09:00
Akira Ajisaka
15fe1b34dd
YARN-8878. Remove StringBuffer from ManagedParentQueue.java. Contributed by BELUGA BEHR. 2018-12-28 17:29:22 +09:00
Sunil G
742fd978b5 YARN-9090. [Submarine] Update submarine installation script document. Contributed by Xun Liu. 2018-12-26 08:26:48 +05:30
Erik Krogen
091ad974cd HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Botong Huang
657aa433e2 YARN 9108. Fix FederationIntercepter merge home and secondary allocate response typo. Contributed by Abhishek Modi. 2018-12-22 12:41:49 -08:00
Akira Ajisaka
e98e963e51
YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena. 2018-12-22 13:25:39 +09:00
Eric Yang
f82922dcfa YARN-5168. Added exposed port information for Docker container. 
Contributed by Xun Liu
 2018-12-21 19:44:07 -05:00
Eric Yang
ea724181d6 YARN-9132. Added file permission check for auxiliary services manifest file. 
Contributed by Billie Rinaldi
 2018-12-21 14:56:39 -05:00
Weiwei Yang
f659485ee8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 10:56:42 +08:00
Eric Yang
a80d321074 YARN-9152. Improved AuxServices REST API output. 
Contributed by Billie Rinaldi
 2018-12-20 19:21:55 -05:00
Eric Yang
7affa3053c YARN-9131. Updated document usage for dynamic auxiliary service. 
Contributed by Billie Rinaldi
 2018-12-20 18:45:49 -05:00
Inigo Goiri
5df9fb16b9 YARN-9130. Add Bind_HOST configuration for Yarn Web Proxy. Contributed by Rong Tang. 2018-12-19 18:58:00 -08:00
Billie Rinaldi
2e544dc921 YARN-9129. Ensure flush after printing to log plus additional cleanup. Contributed by Eric Yang 2018-12-19 16:45:23 -08:00
Eric Yang
e815fd9c49 YARN-9126. Fix container clean up for reinitialization. 
Contributed by Chandni Singh
 2018-12-19 14:55:56 -05:00
Eric Yang
ccdd982e51 YARN-9084. Reset container state and defer readiness check for upgrade. 
Contributed by Chandni Singh
 2018-12-18 18:02:03 -05:00
Eric Yang
c7a5a4435e YARN-9075. Add ability to register/remove auxiliary service without restart node manager. 
Contributed by Billie Rinaldi
 2018-12-18 17:05:51 -05:00
Billie Rinaldi
a6e9d27c29 YARN-9117. Add a check for insecure setup for container terminal. Contributed by Eric Yang 2018-12-18 12:49:10 -08:00
Billie Rinaldi
c5c73182e5 YARN-9072. Send exit command to terminate docker exec on connection close. Contributed by Eric Yang 2018-12-18 10:06:33 -08:00
Eric Yang
71e0b0d800 YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. 
Contributed by Tarun Parimi
 2018-12-17 12:04:25 -05:00
Vidura Mudalige
04c03476c9
YARN-9095. Removed Unused field from Resource: NUM_MANDATORY_RESOURCES 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-12-15 16:07:27 +09:00
Billie Rinaldi
4f622337ee YARN-9091. Improve terminal message when connection is refused. Contributed by Eric Yang 2018-12-14 19:55:50 -08:00
Billie Rinaldi
42297f7d99 YARN-8963. Add flag to disable interactive shell. Contributed by Eric Yang 2018-12-14 18:44:36 -08:00
Billie Rinaldi
aab310978f YARN-9089. Add Terminal Link to Service component instance page for UI2. Contributed by Eric Yang 2018-12-14 18:38:03 -08:00
Akira Ajisaka
26e55d4ceb
YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang. 2018-12-15 10:29:47 +09:00
Eric Yang
b2d7204ed0 YARN-9125. Fixed Carriage Return detection in Docker container launch command. 
Contributed by Billie Rinaldi
 2018-12-14 17:52:26 -05:00
Billie Rinaldi
ca90b2480e YARN-9032. Support sh shell for interactive container shell at command line. Contributed by Eric Yang 2018-12-14 14:08:22 -08:00
Billie Rinaldi
72e7c6a489 YARN-8962. Add ability to use interactive shell with normal yarn container. Contributed by Eric Yang 2018-12-12 18:18:57 -08:00
Wangda Tan
37eb919c59 YARN-8885. [DevicePlugin] Support NM APIs to query device resource allocation. (Zhankun Tang via wangda) 
Change-Id: I2a9870709b512af1ac6c09c9701d0b3c0791ff32
 2018-12-12 11:45:47 -08:00
Wangda Tan
9fba6cc247 YARN-9112. [Submarine] Support polling applicationId when it's not ready in cluster. (Zhankun Tang via wangda) 
Change-Id: I73d73f3d631b28fb9866faa56571839b13824a97
 2018-12-12 11:44:58 -08:00
Wangda Tan
61bdcb7b2b YARN-9015. [DevicePlugin] Add an interface for device plugin to provide customized scheduler. (Zhankun Tang via wangda) 
Change-Id: Ib2e4ae47a6f29bb3082c1f8520cf5a52ca720979
 2018-12-12 11:44:22 -08:00
Wangda Tan
c771fe6e10 YARN-8714. [Submarine] Support files/tarballs to be localized for a training job. (Zhankun Tang via wangda) 
Change-Id: I845131273e52a9d81dbc813ea6d4af06b205e334
 2018-12-12 11:43:23 -08:00
Wangda Tan
8a5361116e YARN-9078. [Submarine] Clean up the code of CliUtils#parseResourcesString. (Zhankun Tang via wangda) 
Change-Id: I21ac5757b6115d55ec3157ba25db1b65bc85a37b
 2018-12-12 11:42:44 -08:00
Sunil G
5a03c8b9ba YARN-9114. [UI2] log service redirect url need to support user name. Contributed by Akhil PB 2018-12-12 14:19:49 +05:30
Haibo Chen
fb55e5201e YARN-9008. Extend YARN distributed shell with file localization feature. (Contributed by Peter Bacsko) 2018-12-11 14:13:34 -08:00
Haibo Chen
881230da21 YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth) 2018-12-11 11:41:32 -08:00
Haibo Chen
ac578c0e82 YARN-9087. Improve logging for initialization of Resource plugins. (Contributed by Szilard Nemeth) 2018-12-10 11:12:54 -08:00
Haibo Chen
64411a6ff7 YARN-8738. FairScheduler should not parse negative maxResources or minResources values as positive. (Contributed by Szilard Nemeth) 2018-12-10 11:06:50 -08:00