Commit Graph

27030 Commits

Author SHA1 Message Date
Susheel Gupta
271b4b25cd
Revert "YARN-11535: Jackson-dataformat-yaml should be upgraded to 2.15.2 as it may cause transitive dependency issue with 2.12.7" (#5969)
This reverts commit 35af8b9d02.
2023-08-21 17:43:08 +02:00
zhangshuyan
60867de422
HDFS-17151. EC: Fix wrong metadata in BlockInfoStriped after recovery. (#5938). Contributed by Shuyan Zhang.
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-08-21 10:05:34 +08:00
WangYuanben
7368226332
YARN-11553. Change the time unit of scCleanerIntervalMs in Router. (#5965) 2023-08-20 10:12:21 +08:00
slfan1989
ff1570acfa
YARN-10218. [GPG] Support HTTPS in GPG. (#5945) Contributed by Shilun Fan.
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-08-19 10:32:06 +08:00
zhengchenyu
136111314d
YARN-11154. Make router support proxy server. (#5946) 2023-08-19 10:29:26 +08:00
Benjamin Teke
ebc32fbcaf
YARN-11551. RM format-conf-store should delete all the content of ZKConfigStore (#5958) 2023-08-18 12:01:56 +02:00
Chunyi Yang
42b4525f75
HDFS-17156. Client may receive old state ID which will lead to inconsistent reads. (#5951)
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2023-08-18 01:56:34 +09:00
huhaiyang
65e4a66e25
HDFS-17087. Add Throttler for datanode reading block (#5845)
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-08-17 09:33:50 +08:00
Viraj Jasani
911e9e0c01
HADOOP-18832. Upgrade aws-java-sdk to 1.12.499 (#5908)
Contributed by Viraj Jasani
2023-08-16 14:34:36 +01:00
zhangshuyan
10b1d7340b
HDFS-17154. EC: Fix bug in updateBlockForPipeline after failover. (#5941). Contributed by Shuyan Zhang.
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-08-16 20:29:19 +08:00
slfan1989
ea87aa2f5b
YARN-11037. Add configurable logic to split resource request to the least loaded SC. (#5515) 2023-08-15 13:28:29 -07:00
zhangshuyan
655c3df050
HDFS-17150. EC: Fix the bug of failed lease recovery. (#5937). Contributed by Shuyan Zhang.
Reviewed-by: hfutatzhanghb <1036798979@qq.com>
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-08-15 19:54:15 +08:00
slfan1989
8d95c588d2
YARN-8934. [BackPort][GPG] Add JvmMetricsInfo and pause monitor. (#5929) Contributed by Bilwa S T. 2023-08-12 09:15:34 +08:00
hchaverri
ad2f45c64f
HDFS-17148. RBF: SQLDelegationTokenSecretManager must cleanup expired tokens in SQL (#5936) 2023-08-11 13:04:32 -07:00
zhengchenyu
59f3a16819
YARN-11153. Make proxy server support YARN federation. (#4314) 2023-08-11 13:02:58 -07:00
hfutatzhanghb
58314cbbf6
HDFS-17145. Fix description of property dfs.namenode.file.close.num-committed-allowed. (#5933)
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-08-11 13:35:41 +08:00
Benjamin Teke
85b3ea6f50
MAPREDUCE-7449: Add add-opens flag to container launch commands on JDK17 nodes (#5935) 2023-08-10 22:47:17 +02:00
Liangjun He
b6edcb9a84
HADOOP-18840. Add enQueue time to RpcMetrics (#5926). Contributed by Liangjun He.
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-08-10 10:38:48 +08:00
slfan1989
df0381f236
YARN-11536. [Federation] Router CLI Supports Batch Save the SubClusterPolicyConfiguration Of Queues. (#5862) Contributed by Shilun Fan.
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-08-10 10:27:36 +08:00
Anuj Modi
ba32ea70fd
HADOOP-18826. [ABFS] Fix for GetFileStatus("/") failure. (#5909)
Contributed by Anmol Asrani
2023-08-08 19:00:02 +01:00
rohit-kb
b1ed23654c
HADOOP-18837. Upgrade okio to 3.4.0 due to CVE-2023-3635. (#5914)
Contributed by Rohit Kumar
2023-08-08 13:37:20 +01:00
huhaiyang
5b81caf0cf
HDFS-17137. Standby/Observer NameNode skip to handle redundant replica block logic when set decrease replication. (#5913). Contributed by Haiyang Hu.
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-08-08 15:42:23 +08:00
hfutatzhanghb
b5b9a81538
HDFS-17144. Remove incorrect comment in method storeAllocatedBlock. (#5932)
Signed-off-by: Tao Li <tomscut@apache.org>
2023-08-08 09:26:51 +08:00
hchaverri
bc48e5cbe8
HDFS-17128. Updating SQLDelegationTokenSecretManager to use LoadingCache so tokens are updated frequently. (#5897) Contributed by Hector Sandoval Chaverri.
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-08-08 07:45:14 +08:00
WangYuanben
1e3e246934
HADOOP-18810. Document missing a lot of properties in core-default.xml. (#5912) Contributed by WangYuanben.
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-08-08 07:37:26 +08:00
WangYuanben
440698eb07
HADOOP-18836. Some properties are missing from hadoop-policy.xml (#5922) 2023-08-07 20:03:23 +08:00
Xing Lin
55e8301470
HDFS-17118 Fixed a couple checkstyle warnings in TestObserverReadProxyProvider (#5880) 2023-08-05 12:27:52 -07:00
slfan1989
001d353cc5
YARN-7708. BackPort [GPG] Load based policy generator. (#5902) Contributed by Young Chen. 2023-08-05 16:33:35 +08:00
slfan1989
6d3bcaa674
YARN-7402. BackPort [GPG] Fix potential connection leak in GPGUtils. (#5901) 2023-08-05 16:29:21 +08:00
susheel-gupta
8eb58630ec
YARN-11416. FS2CS should use CapacitySchedulerConfiguration in FSQueueConverterBuilder (#5320)
Co-authored-by: Susheel Gupta <38013283+susheelgupta7@users.noreply.github.com>
2023-08-04 10:32:55 +02:00
zhtttylz
600dd45e47
HDFS-17122. Rectify the table length discrepancy in the DataNode UI (#5915) 2023-08-04 13:07:19 +08:00
Susheel Gupta
35af8b9d02
YARN-11535: Jackson-dataformat-yaml should be upgraded to 2.15.2 as it may cause transitive dependency issue with 2.12.7 (#5884) 2023-08-03 16:35:27 +02:00
Peter Szucs
6e6349c793
YARN-11545. Fixed FS2CS ACL conversion when all users are allowed. (#5910) 2023-08-03 16:29:41 +02:00
WangYuanben
04c34a2171
HDFS-17113. Reconfig transfer and write bandwidth for datanode. (#5869) 2023-08-03 22:22:06 +08:00
slfan1989
81b1562c32
YARN-3660. [Addendum] Fix GPG Pom.xml Typo, (#5903) 2023-08-02 11:10:24 -07:00
zhangshuyan
5aee0e0c0a
HDFS-17134. RBF: Fix duplicate results of getListing through Router. (#5900). Contributed by Shuyan Zhang.
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-08-01 17:52:54 +08:00
Benjamin Teke
030811bf85
YARN-11543: Fix checkstyle issues after YARN-11520. (#5896)
Co-authored-by: Benjamin Teke <bteke@cloudera.com>
2023-07-31 15:44:18 +02:00
zhiubok
52d66c2c11
HDFS-17136. Fix annotation description and typo in BlockPlacementPolicyDefault Class (#5907)
Co-authored-by: huangzhaobo <huangzhaobo99@126.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-07-31 09:34:49 +08:00
huhaiyang
6cc56d40f5
HDFS-17135. Update fsck -blockId to display excess state info of blocks (#5904)
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-07-31 09:21:22 +08:00
Yang Jiandan
688f76a822
YARN-11538. CS UI: queue filter do not work as expected when submitti… (#5890) 2023-07-31 07:39:42 +08:00
slfan1989
5056c267e6
YARN-11508. [Minor] Improve UnmanagedAMPoolManager/UnmanagedApplicationManager Code (#5726) 2023-07-28 20:20:23 +08:00
Peter Szucs
aae5527c9a
MAPREDUCE-7446. Fix NegativeArraySizeException in IFile's readRawValue method (#5895) 2023-07-28 14:16:03 +02:00
huhaiyang
87c036efb3
HDFS-17116. RBF: Update invoke millisecond time as monotonicNow() in RouterSafemodeService (#5876). Contributed by Haiyang Hu.
Reviewed-by: hfutatzhanghb <1036798979@qq.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-07-28 15:15:00 +08:00
slfan1989
6d32a06125
YARN-7707. BackPort [GPG] Policy generator framework. (#5810) 2023-07-28 13:41:27 +08:00
Benjamin Teke
6dafd53626
YARN-11520. Support capacity vector for AQCv2 dynamic templates. (#5871) 2023-07-27 17:33:19 +02:00
Tamas Domok
d1c52bffbd
YARN-11521. Parameterize tests runs with legacy/non-legacy queue-mode. (#5883) 2023-07-27 15:16:34 +02:00
Sadanand Shenoy
b971222372
HDFS-17120. Support snapshot diff based copylisting for flat paths. (#5885) 2023-07-27 00:53:57 -07:00
Benjamin Teke
b1fc00d4b2
YARN-11539. Fix leaf-templates in Flexible AQC. (#5868) 2023-07-27 09:24:35 +02:00
huhaiyang
1d09dcc614
HDFS-17117. Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync. (#5877). Contributed by Haiyang Hu.
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-07-27 11:28:30 +08:00
ahmarsuhail
24f5f708df
HADOOP-18778. Fixes failing tests when CSE is enabled. (#5763)
Contributed By: Ahmar Suhail <ahmarsu@amazon.co.uk>
2023-07-26 21:56:49 +05:30