Chris Nauroth
33293d4ba4
YARN-11360: Add number of decommissioning/shutdown nodes to YARN cluster metrics. ( #5060 )
...
(cherry picked from commit bfb84cd7f6
)
2022-10-28 18:13:57 +00:00
Steve Loughran
19f8e4f34d
YARN-11330. use secure XML parsers ( #4981 )
...
Move construction of XML parsers in YARN
modules to using the locked-down parser factory
of HADOOP-18469.
One exception: GpuDeviceInformationParser still supports DTD resolution;
all other features are disabled.
Contributed by P J Fanning
2022-10-21 14:16:22 +01:00
PJ Fanning
ea851c5e4a
HADOOP-15983. Use jersey-json that is built to use jackson2 (( #3988 )
...
Moves from com.sun.jersey 1.19 to the artifact
com.github.pjfanning:jersey-json:1.20
This allows jackson 1 to be removed from the classpath.
Contains
* HADOOP-16908. Prune Jackson 1 from the codebase and restrict
its usage for future
* HADOOP-18219. Fix shaded client test failure
These are needed for the HADOOP-15983 changes to build.
Contributed by PJ Fanning.
2022-10-20 17:37:56 +01:00
Ashutosh Gupta
3b3bd89084
YARN-11248. Add unit test for FINISHED_CONTAINERS_PULLED_BY_AM event on DECOMMISSIONING ( #4721 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f02ff1afe2
)
2022-08-16 19:07:42 +09:00
PJ Fanning
6733ba56b8
HADOOP-18332. Remove rs-api dependency by downgrading jackson to 2.12.7. ( #4552 )
...
This downgrades jackson from the version switched to in
HADOOP-18033 (2.13.0), to Jackson 2.12.7.
This removes the dependency on javax.ws.rs-api,
so avoiding runtime problems with applications using
jersey-core v1 and/or jsr311-api.
The 2.12.7 release still contains the fix for CVE-2020-36518.
Contributed by PJ Fanning
2022-07-16 18:18:52 +01:00
Szilard Nemeth
ebe3ed5cd3
YARN-10997. Revisit allocation and reservation logging. Contributed by Andras Gyori
...
(cherry picked from commit 7cb887e6c2
)
2022-07-07 21:04:15 +00:00
Benjamin Teke
838b63d836
YARN-10974. Queue filter in CS UI v1 does not work as expected. Contributed by Chengbing Liu.
2022-06-22 18:20:09 +02:00
Steve Loughran
9ca4ac0af0
HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 ( #4482 )
...
Updating the hadoop version of branch-3.3 to 3.3.9-SNAPSHOT
pending agreement on what number its future release should take.
Using 3.3.9-SNAPSHOT puts space in for other incremental releases,
while avoiding creating JIRA release ordering and autocompletion
confusion the way adding a 3.3.10 or higher version would do.
Contributed by Steve Loughran
2022-06-22 13:09:50 +01:00
zhengchenyu
d7de378b22
YARN-11172. Fix TestClientRMTokens#testDelegationToken introduced by HDFS-16563. ( #4408 )
...
Regression caused by HDFS-16563; the hdfs exception text was changed, but because it was
a YARN test doing the check, Yetus didn't notice.
Contributed by zhengchenyu
2022-06-17 19:51:56 +01:00
Ashutosh Gupta
e0732baeb8
YARN-11128. Fix comments in TestProportionalCapacityPreemptionPolicy* ( #4271 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e3e9369c1d
)
2022-05-26 15:58:47 +09:00
Owen O'Malley
1f111d6a41
YARN-11162. Set the zk acl for nodes created by ZKConfigurationStore. ( #4350 )
...
(cherry picked from commit f390edaec4
)
2022-05-24 05:17:34 +00:00
Szilard Nemeth
90ec4418c7
YARN-11141. Capacity Scheduler does not support ambiguous queue names when moving application across queues. Contributed by Andras Gyori
2022-05-18 14:34:08 +02:00
Szilard Nemeth
4f112e3138
YARN-11126. ZKConfigurationStore Java deserialisation vulnerability. Contributed by Tamas Domok
2022-05-18 14:25:35 +02:00
Viraj Jasani
bb13e228bc
HADOOP-17956. Replace all default Charset usage with UTF-8 ( #3529 )
...
Change-Id: I0094a84619ce19acf340d8dd1040cfe9bd88184e
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-04-27 10:30:07 +01:00
Steve Loughran
44e662272f
HADOOP-18198. Preparing for 3.3.4 development
...
Change-Id: I2bf19beb541739af22fced38c2545f09c4e1bd53
2022-04-12 14:09:08 +01:00
Masatake Iwasaki
160b6d106d
HADOOP-18088. Replace log4j 1.x with reload4j. ( #4052 )
...
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-04-07 08:33:13 +09:00
Fei Hui
5a38ed2f22
HADOOP-17276. Extend CallerContext to make it include many items ( #2327 )
...
Cherry-picked from d0d10f7e
by Owen O'Malley
2022-03-14 10:28:38 -07:00
Szilard Nemeth
192f53283b
YARN-10894. Follow up YARN-10237: fix the new test case in TestRMWebServicesCapacitySched. Contributed by Tamas Domok
2022-03-02 16:04:38 +01:00
Szilard Nemeth
3ef3c5a05b
YARN-11033. isAbsoluteResource is not correct for dynamically created queues. Contributed by Tamas Domok
2022-03-02 14:45:31 +01:00
Szilard Nemeth
f06f44b1c2
YARN-11014. YARN incorrectly validates maximum capacity resources on the validation API. Contributed by Benjamin Teke
2022-03-02 14:23:00 +01:00
Szilard Nemeth
935619a28c
YARN-11075. Explicitly declare serialVersionUID in LogMutation class. Contributed by Benjamin Teke
2022-03-01 18:05:04 +01:00
Ashutosh Gupta
1c99810b89
YARN-8234. Improve RM system metrics publisher's performance by pushing events to timeline server in batch ( #3793 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 00e2405fbd
)
2021-12-23 17:16:09 +09:00
Eric Payne
ccaba2561a
YARN-10178: Global Scheduler async thread crash caused by 'Comparison method violates its general contract. Contributed by Andras Gyori (gandras) and Qi Zhu (zhuqi).
...
(cherry picked from commit e2d6fd075d
)
2021-12-21 19:20:21 +00:00
Viraj Jasani
b0c1158829
HADOOP-18033. Upgrade fasterxml Jackson to 2.13.0 ( #3764 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-13 13:52:44 +09:00
Shubham Gupta
c44f109860
YARN-10438. Handle null containerId in ClientRMService#getContainerReport() ( #2313 )
...
Co-authored-by: Shubham Gupta <gshubham@microsoft.com>
(cherry picked from commit e3cd627069
)
2021-11-19 00:23:48 +09:00
Chao Sun
e079fa6577
Preparing for 3.3.3 development
2021-11-16 16:02:34 -08:00
Ahmed Hussein
742d88b1c6
YARN-1115: Provide optional means for a scheduler to check real user ACLs. Contributed by Eric Payne (epayne)
2021-10-21 17:04:29 +00:00
Benjamin Teke
700045896c
YARN-10869. CS considers only the default maximum-allocation-mb/vcore property as a maximum when it creates dynamic queues ( #3504 )
...
Co-authored-by: Benjamin Teke <bteke@cloudera.com>
2021-10-12 18:05:50 +02:00
Neil
88deac0479
YARN-10970. Standby RM should expose prom endpoint ( #3480 )
...
Reviewed-by: Adam Antal <adamantal@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4bd0c36189
)
2021-09-29 15:48:02 +09:00
Eric Badger
52ba50fd3c
YARN-10935. AM Total Queue Limit goes below per-user AM Limit if parent is full. Contributed by Eric Payne.
...
(cherry picked from commit 43f0a34dd4
)
2021-09-16 16:46:44 +00:00
Szilard Nemeth
6c68211062
YARN-10870. Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page. Contributed by Gergely Pollak
2021-09-14 18:08:34 +02:00
Szilard Nemeth
0a726250ea
YARN-10428. Zombie applications in the YARN queue using FAIR + sizebasedweight. Contributed by Guang Yang, Andras Gyori
...
(cherry picked from commit 79a46599f7
)
2021-09-01 10:44:15 +09:00
Szilard Nemeth
a272adc5fa
YARN-10789. RM HA startup can fail due to race conditions in ZKConfigurationStore. Contributed by Tarun Parimi
2021-07-29 19:21:58 +02:00
Szilard Nemeth
72801be13a
YARN-10813. Set default capacity of root for node labels. Contributed by Andras Gyori
2021-07-28 14:55:19 +02:00
zhuqi-lucas
c31618e6b9
YARN-10860. Make max container per heartbeat configs refreshable. Contributed by Eric Badger.
2021-07-22 10:12:32 +08:00
Jim Brennan
b3481062e0
YARN-10456. RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry. Contributed by Eric Payne.
...
(cherry picked from commit 632f64cadb
)
2021-07-15 14:26:03 +00:00
Jim Brennan
47b3939009
YARN-10834. Intra-queue preemption: apps that don't use defined custom resource won't be preempted. Contributed by Eric Payne.
...
(cherry picked from commit dc6f456e95
)
2021-06-28 14:55:26 +00:00
Wei-Chiu Chuang
86c28f0639
Revert "HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 ( #2959 )"
...
This reverts commit 4ffe5eb1dd
.
2021-05-24 17:37:18 +08:00
Jim Brennan
53a1c7653f
YARN-10337. Amendment to fix import as in HADOOP-17100
2021-05-19 22:00:55 +00:00
Prabhu Joseph
1b3e4cf9ce
YARN-10337. Fix failing testcase TestRMHATimelineCollectors.
...
Contributed by Bilwa S T.
(cherry picked from commit 2bbd00dff4
)
2021-05-19 21:19:05 +00:00
Wei-Chiu Chuang
fa4915fdbb
Preparing for 3.3.2 development
2021-05-19 21:52:37 +08:00
zhuqi-lucas
7d2eeaecc8
YARN-10701. The yarn.resource-types should support multi types without trimmed. Contributed by Qi Zhu.
2021-05-19 21:24:26 +08:00
Wei-Chiu Chuang
4ffe5eb1dd
HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 ( #2959 )
...
* HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
Change-Id: I0f31409923ece24a82dfba4c4610d8a38c52d9fb
* HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661 ). Contributed by Vinayakumar B.
(cherry picked from commit c4c0683dff
)
Co-authored-by: Xiaoyu Yao <xyao@apache.org>
Co-authored-by: Vinayakumar B <vinayakumarb@apache.org>
2021-05-17 18:57:46 -07:00
lujiefsi
137e20cc9b
YARN-10555. Missing access check before getAppAttempts ( #2608 )
...
Co-authored-by: lujie <lujie@foxmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d92a25b790
)
2021-05-17 13:53:27 +09:00
Peter Bacsko
051a5068dd
YARN-9615. Add dispatcher metrics to RM. Contributed by Jonathan Hung and Qi Zhu.
2021-05-11 19:23:45 +02:00
Szilard Nemeth
3d715c2e4c
YARN-9333. TestFairSchedulerPreemption.testRelaxLocalityPreemptionWithNoLessAMInRemainingNodes fails intermittently. Contributed by Peter Bacsko
...
(cherry picked from commit eacbe07b56
)
2021-05-07 14:32:17 +09:00
Szilard Nemeth
3303aa5947
YARN-10515. Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags. Contributed by Peter Bacsko
...
(cherry picked from commit 8620984b8d
)
2021-05-07 14:26:58 +09:00
Wei-Chiu Chuang
670205c541
HADOOP-17653. Do not use guava's Files.createTempDir(). ( #2945 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f1e1809029
)
2021-05-02 11:12:37 +09:00
Eric Badger
003deeeecf
YARN-10479. Can't remove all node labels after add node label without
...
nodemanager port, broken by YARN-10647. Contributed by D M Murali Krishna Reddy
(cherry picked from commit 6857a05d6a
)
2021-04-23 22:14:57 +00:00
Eric Badger
1960924d07
YARN-10723. Change CS nodes page in UI to support custom resource. Contributed by Qi Zhu
...
(cherry picked from commit 6cb90005a7
)
2021-04-20 17:46:05 +00:00