big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,653 Commits 2 Branches 2 Tags 264 MiB
4024afec28
Commit Graph

915 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
fa4915fdbb
Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
zhuqi-lucas
7d2eeaecc8 YARN-10701. The yarn.resource-types should support multi types without trimmed. Contributed by Qi Zhu. 2021-05-19 21:24:26 +08:00
Eric Badger
da2ebfa8a9 YARN-10707. Support custom resources in ResourceUtilization, and update Node GPU Utilization to use. Contributed by Qi Zhu 
(cherry picked from commit 803ac4b1a0)
 2021-04-29 17:52:03 +00:00
Jim Brennan
3b1cde2f42 YARN-10743. Add a policy for not aggregating for containers which are killed because exceeding container log size limit. Contributed by Qi Zhu. 
(cherry picked from commit 1cbe35946b)
 2021-04-23 21:38:02 +00:00
Eric Badger
1658a5140a YARN-10503. Support queue capacity in terms of absolute resources with custom 
resourceType. Contributed by Qi Zhu.
 2021-04-09 17:51:01 +00:00
Eric Badger
fb5809984e YARN-10702. Add cluster metric for amount of CPU used by RM Event Processor. 
Contributed by Jim Brennan.
 2021-04-06 23:34:35 +00:00
Brahma Reddy Battula
616a41ee32 YARN-10441. Add support for hadoop.http.rmwebapp.scheduler.page.class. Contributed by D M Murali Krishna Reddy 
(cherry picked from commit b61f52ec56)
 2021-03-30 09:52:28 +05:30
Jim Brennan
78bddd0d9f YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T. 
(cherry picked from commit 174f3a96b1)
 2021-03-23 18:23:50 +00:00
Jim Brennan
2204bafbfb [YARN-10687] Add option to disable/enable free disk space checking and percentage checking for full and not-full disks. Contributed by Qi Zhu. 
(cherry picked from commit 5f067cf0f3)
 2021-03-12 17:20:37 +00:00
Prabhu Joseph
72904c014d YARN-10361. Make custom DAO classes configurable into RMWebApp#JAXBContextResolver. 
Contributed by Bilwa ST.

(cherry picked from commit c7e71a6c0b)
 2021-02-18 14:25:16 +05:30
Prabhu Joseph
0c46ab51b5 YARN-8047. RMWebApp make external class pluggable. 
Contributed by Bilwa S T.

(cherry picked from commit 3a4d05b850)
 2021-02-18 13:59:50 +05:30
Jim Brennan
d3c7cb7c38 [YARN-10607] User environment is unable to prepend PATH when mapreduce.admin.user.env also sets PATH. Contributed by Eric Badger. 
(cherry picked from commit c22c77af43)
 2021-02-05 17:37:04 +00:00
Ayush Saxena
8378ab9f92 HADOOP-17288. Use shaded guava from thirdparty. Contributed by Ayush Saxena. #2505 2020-12-10 05:50:55 +05:30
Eric E Payne
2473e8b711 YARN-10475: Scale RM-NM heartbeat interval based on node utilization. Contributed by Jim Brennan (Jim_Brennan). 2020-11-02 17:16:28 +00:00
Jim Brennan
cdaef111d5 YARN-10471. Prevent logs for any container from becoming larger than a configurable size. Contributed by Eric Payne 
(cherry picked from commit 8ee6bc2518)
 2020-10-29 19:52:43 +00:00
Eric Yang
9176e8fe5d YARN-9809. Added node manager health status to resource manager registration call. 
Contributed by Eric Badger via eyang

(cherry picked from commit e8dc862d38)
 2020-09-28 16:41:53 +00:00
Eric E Payne
0cb1b178ec YARN-1529: Add Localization overhead metrics to NM. Contributed by Jim_Brennan. 
(cherry picked from commit e0c9653166)
 2020-07-30 16:39:56 +00:00
Eric Badger
41bcef9486 YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by 
Jim Brennan

(cherry picked from commit 48f90115b5)
 2020-07-13 23:12:18 +00:00
Akira Ajisaka
dfa7f160a5
Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
Jonathan Hung
49ae9b2137 YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 2020-04-17 10:34:59 -07:00
Jonathan Hung
e1dd78143b YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
 2020-04-09 11:06:49 -07:00
RogPodge
2d294bd575
HADOOP-16938. Make non-HA proxy providers pluggable 2020-03-25 08:06:58 -07:00
Eric Badger
2649f8b327 YARN-10173. Make pid file generation timeout configurable in case of reacquired 
container. Contributed by Adam Antal.
 2020-03-04 23:31:57 +00:00
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Inigo Goiri
5abd0148eb YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R. 2020-01-28 10:06:37 -08:00
Inigo Goiri
b4870bce3a Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R." 
This reverts commit 0696828a09.
 2020-01-21 17:45:17 -08:00
Inigo Goiri
0696828a09 YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R. 2020-01-21 13:41:01 -08:00
Eric E Payne
93233a7d6e YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 2020-01-09 17:18:44 +00:00
Szilard Nemeth
768ee22e9e YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal 2020-01-06 16:26:33 +01:00
Jonathan Hung
fddc3d55c3 YARN-10039. Allow disabling app submission from REST endpoints 2019-12-18 10:48:05 -08:00
Szilard Nemeth
631dbbc6f2 YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal 2019-12-15 17:28:04 +01:00
prabhujoseph
aa7ab2719f YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth. 2019-11-27 01:59:12 +05:30
Szilard Nemeth
52e9ee39a1 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2019-11-26 16:42:59 +01:00
Szilard Nemeth
dfdc6d6dd9 YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton 2019-11-19 21:14:15 +01:00
Eric Badger
0e22e9ab83 YARN-9562. Add Java changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-11-18 21:56:31 +00:00
Eric Badger
eef34f2d87 YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan 2019-10-25 17:13:04 +00:00
Jonathan Hung
5462d2176f YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:20:13 -07:00
bibinchundatt
425a6c89ae YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung. 2019-10-01 16:02:26 +05:30
Jonathan Hung
c2731d4b63 YARN-9730. Support forcing configured partitions to be exclusive based on app node label 2019-09-24 13:51:32 -07:00
Duo Zhang
bfe1dac29d
HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes (#1496) 
Signed-off-by:  Vinayakumar B <vinayakumarb@apache.org>
 2019-09-24 08:50:36 +08:00
Vinayakumar B
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432) 
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
 2019-09-20 16:08:30 +05:30
Sunil G
01d7924473 YARN-9814. JobHistoryServer can't delete aggregated files, if remote app root directory is created by NodeManager. Contributed by Adam Antal. 2019-09-18 09:27:41 +05:30
Eric Yang
10144a580e YARN-9728. Bugfix for escaping illegal xml characters for Resource Manager REST API. 
Contributed by Prabhu Joseph
 2019-09-10 17:04:39 -04:00
Jonathan Hung
e7d44e48f7 YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:17:07 -07:00
Tao Yang
172bcd8e01 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. Contributed by zhuqi. 2019-09-05 16:20:05 +08:00
Szilard Nemeth
e8fa192f07 YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko 2019-08-21 16:44:22 +02:00
Haibo Chen
f51702d539 YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 2019-08-06 13:52:30 -07:00
Szilard Nemeth
54ac80176e Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph 2019-08-02 13:38:06 +02:00
Adam Antal
cbcada804d YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently (#1121) 2019-08-02 13:04:34 +02:00
Szilard Nemeth
a2a8be18cb YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 2019-07-03 13:45:00 +02:00