hfutatzhanghb
562c42c86a
HDFS-17242. Make congestion backoff time configurable. ( #6227 )
...
Reviewed-by: Xing Lin <xinglin@linkedin.com>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-12-13 20:56:53 +08:00
yijut2
81de229cf6
HDFS-17278. Fix order dependent flakiness in TestViewfsWithNfs3.java under hadoop-hdfs-nfs module ( #6329 ). Contributed by Ruby.
...
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-12-12 12:42:26 +05:30
caozhiqiang
37d6cada14
HDFS-17272. NNThroughputBenchmark should support specifying the base directory for multi-client test ( #6319 ). Contributed by caozhiqiang.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-12-10 13:43:04 +05:30
huhaiyang
f96361fd54
HDFS-17279. RBF: Fix link to Fedbalance document ( #6333 ). Contributed by Haiyang Hu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-12-09 02:32:04 +05:30
Jian Zhang
29b0a6c28d
HDFS-17265. RBF: Throwing an exception prevents the permit from being released when using FairnessPolicyController ( #6298 )
2023-12-08 10:18:22 -08:00
Xing Lin
607c981042
HDFS-17262. Fixed the verbose log.warn in DFSUtil.addTransferRateMetric(). ( #6290 ). Contributed by Xing Lin.
...
Reviewed-by: Ravindra Dingankar <rdingankar@linkedin.com>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-12-07 10:16:23 +08:00
Thinker313
6a22bead54
HDFS-17270. RBF: Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper client to get token in some case ( #6315 ). Contributed by lei w.
...
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-12-06 21:34:46 +08:00
huhaiyang
1fb80ef787
HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted ( #6266 ). Contributed by Haiyang Hu.
...
Reviewed-by: ZanderXu <zanderxu@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-12-04 21:19:33 +08:00
huhaiyang
9a6d00aba4
HDFS-17218. NameNode should process time out excess redundancy blocks ( #6176 ). Contributed by Haiyang Hu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-12-04 21:16:38 +08:00
huangzhaobo
2323ad24a2
HDFS-17260. Fix the logic for reconfigure slow peer enable for Namenode. ( #6279 ). Contributed by huangzhaobo99.
...
Reviewed-by: huhaiyang <huhaiyang926@126.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-12-03 15:09:55 +05:30
Palakur Eshwitha Sai
d0b460f270
HDFS-17233. The conf dfs.datanode.lifeline.interval.seconds is not considering time unit seconds ( #6286 ). Contributed by Palakur Eshwitha Sai.
...
Reviewed-by: huhaiyang <huhaiyang926@126.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-12-02 13:40:43 +05:30
Felix Nguyen
e76477e823
HDFS-17271. Fix dead DN sorting in web UI ( #6318 ). Contributed by Felix N.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-12-02 13:37:22 +05:30
LiuGuH
071f850841
HDFS-17261. RBF: Fix getFileInfo return wrong path when get mountTable path which is multi-level ( #6288 ). Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-12-01 17:05:58 +05:30
huangzhaobo
89e78a76a0
HDFS-17259. Fix typo in TestFsDatasetImpl Class. ( #6278 ). Contributed by huangzhaobo99.
2023-12-01 10:46:31 +05:30
LiuGuH
0417c1c633
HDFS-17263. RBF: Fix client ls trash path cannot get except default nameservices trash path ( #6291 ) Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-11-30 22:46:50 +08:00
PJ Fanning
f609460bda
HADOOP-18957. Use StandardCharsets.UTF_8 ( #6231 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-11-20 23:44:48 +05:30
Istvan Fajth
7a55442297
HADOOP-18956. Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and ZKSignerSecretProvider ( #6263 )
2023-11-17 01:51:43 -08:00
QI Jiale
28068aa320
HDFS-17063. Support to configure different capacity reserved for each disk of DataNode. ( #5793 ). Contributed by QI Jiale.
...
Reviewed-by : Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-11-17 11:59:04 +08:00
huhaiyang
469f20a52b
HDFS-17172. Support FSNamesystemLock Parameters reconfigurable. ( #6002 ). Contributed by Haiyang Hu.
...
Reviewed-by: ZanderXu <zanderxu@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-11-17 11:50:08 +08:00
Jian Zhang
513e6dcf14
HDFS-17232. RBF: Fix NoNamenodesAvailableException for a long time, when use observer. ( #6208 )
2023-11-16 09:13:48 -08:00
LiuGuH
342e6caba1
HDFS-17249. Fix TestDFSUtil.testIsValidName() unit test failure ( #6249 )
...
Contributed by liuguanghua.
2023-11-13 18:16:31 +00:00
Tom
f58945d7d1
HDFS-16791. Add getEnclosingRoot() API to filesystem interface and implementations ( #6198 )
...
The enclosing root path is a common ancestor that should be used for temp and staging dirs
as well as within encryption zones and other restricted directories.
Contributed by Tom McCormick
2023-11-08 14:25:21 +00:00
huhaiyang
4ef2322b6d
HDFS-17243. Add the parameter storage type for getBlocks method ( #6238 ). Contributed by Haiyang Hu.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
2023-11-06 11:20:25 +08:00
Gautham B A
cbb153b69e
HDFS-17246. Fix shaded client for building Hadoop on Windows ( #5943 )
2023-11-01 09:10:15 -07:00
PJ Fanning
a079f6261d
HADOOP-18917. Addendum. Fix deprecation issues after commons-io upgrade. ( #6228 ). Contributed by PJ Fanning.
2023-10-30 09:35:02 +05:30
Hiroaki Segawa
93a3c6e2cd
HDFS-17024. Potential data race introduced by HDFS-15865 ( #6223 )
2023-10-26 22:25:00 -07:00
Wei-Chiu Chuang
821ed83873
HDFS-15273. CacheReplicationMonitor hold lock for long time and lead to NN out of service. Contributed by Xiaoqiao He.
2023-10-26 10:35:10 -07:00
huhaiyang
f85ac5b60d
HADOOP-18920. RPC Metrics : Optimize logic for log slow RPCs ( #6146 )
2023-10-25 13:56:39 +08:00
gp1314
a170d58501
HDFS-17231. HA: Safemode should exit when resources are from low to available. ( #6207 ). Contributed by Gu Peng.
...
Reviewed-by: Xing Lin <xinglin@linkedin.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-10-25 11:43:12 +08:00
Stephen O'Donnell
882f08b4bc
HDFS-17237. Remove IPCLoggerChannelMetrics when the logger is closed ( #6217 )
2023-10-24 21:39:03 +01:00
huhaiyang
9d48af8d70
HADOOP-18868. Optimize the configuration and use of callqueue overflow trigger failover ( #5998 )
2023-10-23 14:06:02 -07:00
Zita Dombi
4c04818d3d
HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC ( #6194 )
2023-10-23 11:03:15 -07:00
huhaiyang
5eeab5e1b9
HDFS-17235. Fix javadoc errors in BlockManager ( #6214 ). Contributed by Haiyang Hu.
2023-10-23 20:12:39 +05:30
jianghuazhu
6e13e4addc
HDFS-17228. Improve documentation related to BlockManager. ( #6195 ). Contributed by JiangHua Zhu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-10-23 20:05:33 +05:30
Ayush Saxena
fbd653be9b
Revert "HDFS-17228. Improve documentation related to BlockManager. ( #6195 ). Contributed by JiangHua Zhu."
...
This reverts commit 81ba2e8484
.
2023-10-23 19:35:12 +05:30
GuoPhilipse
615a2a42cf
HDFS-17220. Fix same available space policy in AvailableSpaceVolumeChoosingPolicy ( #6174 )
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-10-18 13:01:32 +08:00
jianghuazhu
81ba2e8484
HDFS-17228. Improve documentation related to BlockManager. ( #6195 ). Contributed by JiangHua Zhu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-10-18 05:05:33 +05:30
GuoPhilipse
c8abca3004
HDFS-17210. Optimize AvailableSpaceBlockPlacementPolicy. ( #6113 ). Contributed by GuoPhilipse.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
2023-10-16 16:34:40 +08:00
Steve Loughran
9bc159f4ac
HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. ( #4996 )
...
Protobuf 2.5 JAR is no longer needed at runtime.
The option common.protobuf.scope defines whether the protobuf 2.5.0
dependency is marked as provided or not.
* New package org.apache.hadoop.ipc.internal for internal only protobuf classes
...with a ShadedProtobufHelper in there which has shaded protobuf refs
only, so guaranteed not to need protobuf-2.5 on the CP
* All uses of org.apache.hadoop.ipc.ProtobufHelper have
been replaced by uses of org.apache.hadoop.ipc.internal.ShadedProtobufHelper
* The scope of protobuf-2.5 is set by the option common.protobuf2.scope
In this patch is it is still "compile"
* There is explicit reference to it in modules where it may be needed.
* The maven scope of the dependency can be set with the common.protobuf2.scope
option. It can be set to "provided" in a build:
-Dcommon.protobuf2.scope=provided
* Add new ipc(callable) method to catch and convert shaded protobuf
exceptions raised during invocation of the supplied lambda expression
* This is adopted in the code where the migration is not traumatically
over-complex. RouterAdminProtocolTranslatorPB is left alone for this
reason.
Contributed by Steve Loughran
2023-10-13 13:48:38 +01:00
huhaiyang
0ed484ac62
HDFS-17208. Add the metrics PendingAsyncDiskOperations in datanode ( #6109 ). Contributed by Haiyang Hu.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-10-12 23:27:15 +08:00
Steve Vaughan
73eccd6d7c
HDFS-16740. Mini cluster test flakiness ( #4835 )
2023-10-10 13:51:46 -07:00
huhaiyang
85af6c3a28
HDFS-17217. Add lifeline RPC start up log when NameNode#startCommonServices ( #6154 ). Contributed by Haiyang Hu.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
2023-10-10 10:20:07 +08:00
hfutatzhanghb
ea3cb12ec8
HDFS-17171. CONGESTION_RATIO should be configurable ( #5996 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-10-08 10:36:09 +08:00
huhaiyang
4c408a557f
HDFS-17205. HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable ( #6112 ). Contributed by Haiyang Hu
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-10-06 12:39:23 +05:30
huhaiyang
5edd21bc85
HDFS-17194. Enhance the log message for striped block recovery ( #6094 )
2023-10-04 11:22:59 -07:00
xiaojunxiang
0cfffb3012
HDFS-17214. RBF: The Quota class' andByStorageType method res has an incorrect initial value. ( #6135 )
...
Co-authored-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
2023-10-03 08:26:03 -07:00
Wang Yu
b87180568b
HDFS-17209. Correct comments to align with the code ( #6110 ). Contributed by Yu Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-10-01 17:30:59 +05:30
ConfX
8931393302
HDFS-17133: TestFsDatasetImpl missing null check when cleaning up ( #6079 ). Contributed by ConfX.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-09-29 12:06:24 +05:30
xiaojunxiang
390cd294f8
HDFS-17211. Fix comments in the RemoteParam class. ( #6124 ). Contributed hellosrc.
...
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-09-29 11:55:59 +05:30
zhangshuyan
26a5f38250
HDFS-17204. EC: Reduce unnecessary log when processing excess redundancy. ( #6107 ). Contributed by Shuyan Zhang.
...
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-09-26 18:49:47 +08:00