big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,234 Commits 2 Branches 2 Tags 264 MiB
f7bb4f1595
Commit Graph

10104 Commits

Author SHA1 Message Date
Jian Zhang
1036544480
HDFS-17302. RBF: ProportionRouterRpcFairnessPolicyController-Sharing and isolation. (#6380) 2024-01-19 14:02:21 -08:00
slfan1989
8444f69511
Preparing for 3.5.0 development (#6411) 
Co-authored-by: slfan1989 <slfan1989@apache.org>
 2024-01-19 15:05:22 +08:00
Xing Lin
27ecc23ae7
HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException (#6446) 
Print a warn log message for read retries and only print the full stack trace for a read request failure.

Contributed by: Xing Lin
 2024-01-18 18:03:28 -08:00
Lei313
cc4c4be1b7
HDFS-17331:Fix Blocks are always -1 and DataNode version are always UNKNOWN in federationhealth.html (#6429). Contributed by lei w. 
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
 2024-01-18 21:10:54 +08:00
hfutatzhanghb
a30681077b
HDFS-17291. DataNode metric bytesWritten is not totally accurate in some situations. (#6360). Contributed by farmmamba. 
Reviewed-by: huangzhaobo <huangzhaobo99@126.com>
Signed-off-by:  Shuyan Zhang <zhangshuyan@apache.org>
 2024-01-13 20:45:00 +08:00
hfutatzhanghb
ead7b7f565
HDFS-17289. Considering the size of non-lastBlocks equals to complete block size can cause append failure. (#6357). Contributed by farmmamba. 
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Reviewed-by: huangzhaobo <huangzhaobo99@126.com>
Signed-off-by:  Shuyan Zhang <zhangshuyan@apache.org>
 2024-01-13 20:34:02 +08:00
hfutatzhanghb
6a053765ee
HDFS-17312. packetsReceived metric should ignore heartbeat packet. (#6394) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2024-01-11 22:08:37 +09:00
xuzifu666
99a59ae9e6
HDFS-17317. Improve the resource release for metaOut in DebugAdmin (#6402). Contributed by xy. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org
 2024-01-07 00:59:31 +05:30
huangzhaobo
08713665c0
HDFS-17315. Optimize the namenode format code logic. (#6400) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2024-01-06 01:47:17 +09:00
LiuGuH
2369f0cddb
HDFS-17309. RBF: Fix Router Safemode check condition error (#6390) Contributed by liuguanghua. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2024-01-05 18:36:28 +08:00
LiuGuH
7d3b6a36b8
HDFS-17306. RBF: Router should not return nameservices that does not enable observer nodes in RpcResponseHeaderProto (#6385) 2024-01-04 14:43:11 -08:00
hfutatzhanghb
d5468d84ba
HDFS-17283. Change the name of variable SECOND in HdfsClientConfigKeys. (#6339). Contributed by farmmamba. 
Reviewed-by: Xing Lin <xinglin@linkedin.com>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
 2024-01-04 19:53:47 +08:00
huhaiyang
7a7db7f0dc
HDFS-17310. DiskBalancer: Enhance the log message for submitPlan (#6391) Contributed by Haiyang Hu. 
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2024-01-04 00:07:51 +08:00
LiuGuH
335587df9e
Add synchronized on lockLeakCheck() because threadCountMap is not thread safe. (#6029) 
Co-authored-by: lgh <liuguanghua@kanzhun.com>
 2024-01-03 21:12:38 +08:00
zzccctv
9f76fba6a4
Delete invalid code logic in namenode format (#6323) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2023-12-29 22:25:05 +09:00
huangzhaobo
e26139beaa
HDFS-17301. Add read and write dataXceiver threads count metrics to datanode. (#6377) 
Reviewed-by: hfutatzhanghb <hfutzhanghb@163.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2023-12-29 12:43:46 +09:00
huhaiyang
3aceec711b
HDFS-17297. The NameNode should remove block from the BlocksMap if the block is marked as deleted (#6369) 
Reviewed-by: ZanderXu <zanderxu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2023-12-28 22:13:16 +09:00
zhtttylz
fc71dc3e94
HDFS-17284. Fix int overflow in calculating numEcReplicatedTasks and numReplicationTasks during block recovery (#6348) 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2023-12-27 10:43:10 +09:00
Lei313
773dd7cc85
HDFS-17275: judge whether the block has been deleted in the block report (#6335). Contributed by lei w. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
 2023-12-26 18:25:02 +08:00
huhaiyang
7e2ebfc17e
HDFS-17298. Fix NPE in DataNode.handleBadBlock and BlockSender (#6374) 2023-12-26 09:41:10 +09:00
xiaojunxiang
e07e445326
HDFS-17215. RBF: Fix some method annotations about @throws . (#6136). Contributed by xiaojunxiang. 2023-12-26 00:37:58 +05:30
huangzhaobo
630ffb280d
HDFS-17056. EC: Fix verifyClusterSetup output in case of an invalid param. (#6379). Contributed by huangzhaobo99. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-25 23:17:25 +05:30
hfutatzhanghb
415e9bdfbd
HDFS-17273. Change the way of computing some local variables duration for better debugging (#6321). Contributed by farmmamba. 
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-12-24 16:57:04 +08:00
Wang Yu
77edca8f0a
HDFS-17240. Fix a typo in DataStorage.java (#6226). Contributed by Yu Wang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
 2023-12-22 01:52:12 +05:30
LiuGuH
b4fed58c6a
HDFS-17285. RBF: Add a safe mode check period configuration (#6347) Contributed by LiuGuH. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-12-21 12:21:59 +08:00
huangzhaobo
95ea31fafb
HDFS-17294. Reconfigure the scheduling cycle of the slowPeerCollectorDaemon thread. (#6366) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2023-12-19 09:32:47 +09:00
huangzhaobo
1498a8685d
HDFS-17282. Reconfig 'SlowIoWarningThreshold' parameters for datanode. (#6338). Contributed by huangzhaobo99 
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-14 11:47:53 +05:30
hfutatzhanghb
562c42c86a
HDFS-17242. Make congestion backoff time configurable. (#6227) 
Reviewed-by: Xing Lin <xinglin@linkedin.com>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-12-13 20:56:53 +08:00
yijut2
81de229cf6
HDFS-17278. Fix order dependent flakiness in TestViewfsWithNfs3.java under hadoop-hdfs-nfs module (#6329). Contributed by Ruby. 
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-12 12:42:26 +05:30
caozhiqiang
37d6cada14
HDFS-17272. NNThroughputBenchmark should support specifying the base directory for multi-client test (#6319). Contributed by caozhiqiang. 
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-10 13:43:04 +05:30
huhaiyang
f96361fd54
HDFS-17279. RBF: Fix link to Fedbalance document (#6333). Contributed by Haiyang Hu. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-09 02:32:04 +05:30
Jian Zhang
29b0a6c28d
HDFS-17265. RBF: Throwing an exception prevents the permit from being released when using FairnessPolicyController (#6298) 2023-12-08 10:18:22 -08:00
Xing Lin
607c981042
HDFS-17262. Fixed the verbose log.warn in DFSUtil.addTransferRateMetric(). (#6290). Contributed by Xing Lin. 
Reviewed-by: Ravindra Dingankar <rdingankar@linkedin.com>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-12-07 10:16:23 +08:00
Thinker313
6a22bead54
HDFS-17270. RBF: Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper client to get token in some case (#6315). Contributed by lei w. 
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-12-06 21:34:46 +08:00
huhaiyang
1fb80ef787
HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted (#6266). Contributed by Haiyang Hu. 
Reviewed-by: ZanderXu <zanderxu@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-12-04 21:19:33 +08:00
huhaiyang
9a6d00aba4
HDFS-17218. NameNode should process time out excess redundancy blocks (#6176). Contributed by Haiyang Hu. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-12-04 21:16:38 +08:00
huangzhaobo
2323ad24a2
HDFS-17260. Fix the logic for reconfigure slow peer enable for Namenode. (#6279). Contributed by huangzhaobo99. 
Reviewed-by: huhaiyang <huhaiyang926@126.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-03 15:09:55 +05:30
Palakur Eshwitha Sai
d0b460f270
HDFS-17233. The conf dfs.datanode.lifeline.interval.seconds is not considering time unit seconds (#6286). Contributed by Palakur Eshwitha Sai. 
Reviewed-by: huhaiyang <huhaiyang926@126.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-02 13:40:43 +05:30
Felix Nguyen
e76477e823
HDFS-17271. Fix dead DN sorting in web UI (#6318). Contributed by Felix N. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-02 13:37:22 +05:30
LiuGuH
071f850841
HDFS-17261. RBF: Fix getFileInfo return wrong path when get mountTable path which is multi-level (#6288). Contributed by liuguanghua. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-12-01 17:05:58 +05:30
huangzhaobo
89e78a76a0
HDFS-17259. Fix typo in TestFsDatasetImpl Class. (#6278). Contributed by huangzhaobo99. 2023-12-01 10:46:31 +05:30
LiuGuH
0417c1c633
HDFS-17263. RBF: Fix client ls trash path cannot get except default nameservices trash path (#6291) Contributed by liuguanghua. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-11-30 22:46:50 +08:00
PJ Fanning
f609460bda
HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-11-20 23:44:48 +05:30
Istvan Fajth
7a55442297
HADOOP-18956. Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and ZKSignerSecretProvider (#6263) 2023-11-17 01:51:43 -08:00
QI Jiale
28068aa320
HDFS-17063. Support to configure different capacity reserved for each disk of DataNode. (#5793). Contributed by QI Jiale. 
Reviewed-by : Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-11-17 11:59:04 +08:00
huhaiyang
469f20a52b
HDFS-17172. Support FSNamesystemLock Parameters reconfigurable. (#6002). Contributed by Haiyang Hu. 
Reviewed-by: ZanderXu <zanderxu@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-11-17 11:50:08 +08:00
Jian Zhang
513e6dcf14
HDFS-17232. RBF: Fix NoNamenodesAvailableException for a long time, when use observer. (#6208) 2023-11-16 09:13:48 -08:00
LiuGuH
342e6caba1
HDFS-17249. Fix TestDFSUtil.testIsValidName() unit test failure (#6249) 
Contributed by liuguanghua.
 2023-11-13 18:16:31 +00:00
Tom
f58945d7d1
HDFS-16791. Add getEnclosingRoot() API to filesystem interface and implementations (#6198) 
The enclosing root path is a common ancestor that should be used for temp and staging dirs
as well as within encryption zones and other restricted directories.

Contributed by Tom McCormick
 2023-11-08 14:25:21 +00:00
huhaiyang
4ef2322b6d
HDFS-17243. Add the parameter storage type for getBlocks method (#6238). Contributed by Haiyang Hu. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
 2023-11-06 11:20:25 +08:00
Gautham B A
cbb153b69e
HDFS-17246. Fix shaded client for building Hadoop on Windows (#5943) 2023-11-01 09:10:15 -07:00
PJ Fanning
a079f6261d
HADOOP-18917. Addendum. Fix deprecation issues after commons-io upgrade. (#6228). Contributed by PJ Fanning. 2023-10-30 09:35:02 +05:30
Hiroaki Segawa
93a3c6e2cd
HDFS-17024. Potential data race introduced by HDFS-15865 (#6223) 2023-10-26 22:25:00 -07:00
Wei-Chiu Chuang
821ed83873
HDFS-15273. CacheReplicationMonitor hold lock for long time and lead to NN out of service. Contributed by Xiaoqiao He. 2023-10-26 10:35:10 -07:00
huhaiyang
f85ac5b60d
HADOOP-18920. RPC Metrics : Optimize logic for log slow RPCs (#6146) 2023-10-25 13:56:39 +08:00
gp1314
a170d58501
HDFS-17231. HA: Safemode should exit when resources are from low to available. (#6207). Contributed by Gu Peng. 
Reviewed-by: Xing Lin <xinglin@linkedin.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-10-25 11:43:12 +08:00
Stephen O'Donnell
882f08b4bc
HDFS-17237. Remove IPCLoggerChannelMetrics when the logger is closed (#6217) 2023-10-24 21:39:03 +01:00
huhaiyang
9d48af8d70
HADOOP-18868. Optimize the configuration and use of callqueue overflow trigger failover (#5998) 2023-10-23 14:06:02 -07:00
Zita Dombi
4c04818d3d
HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC (#6194) 2023-10-23 11:03:15 -07:00
huhaiyang
5eeab5e1b9
HDFS-17235. Fix javadoc errors in BlockManager (#6214). Contributed by Haiyang Hu. 2023-10-23 20:12:39 +05:30
jianghuazhu
6e13e4addc
HDFS-17228. Improve documentation related to BlockManager. (#6195). Contributed by JiangHua Zhu. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-10-23 20:05:33 +05:30
Ayush Saxena
fbd653be9b
Revert "HDFS-17228. Improve documentation related to BlockManager. (#6195). Contributed by JiangHua Zhu." 
This reverts commit 81ba2e8484.
 2023-10-23 19:35:12 +05:30
GuoPhilipse
615a2a42cf
HDFS-17220. Fix same available space policy in AvailableSpaceVolumeChoosingPolicy (#6174) 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-10-18 13:01:32 +08:00
jianghuazhu
81ba2e8484
HDFS-17228. Improve documentation related to BlockManager. (#6195). Contributed by JiangHua Zhu. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-10-18 05:05:33 +05:30
GuoPhilipse
c8abca3004
HDFS-17210. Optimize AvailableSpaceBlockPlacementPolicy. (#6113). Contributed by GuoPhilipse. 
Reviewed-by:  He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
 2023-10-16 16:34:40 +08:00
Steve Loughran
9bc159f4ac
HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996) 
Protobuf 2.5 JAR is no longer needed at runtime. 

The option common.protobuf.scope defines whether the protobuf 2.5.0
dependency is marked as provided or not.

* New package org.apache.hadoop.ipc.internal for internal only protobuf classes
  ...with a ShadedProtobufHelper in there which has shaded protobuf refs
  only, so guaranteed not to need protobuf-2.5 on the CP
* All uses of org.apache.hadoop.ipc.ProtobufHelper have
  been replaced by uses of org.apache.hadoop.ipc.internal.ShadedProtobufHelper
* The scope of protobuf-2.5 is set by the option common.protobuf2.scope
  In this patch is it is still "compile"
* There is explicit reference to it in modules where it may be needed.
*  The maven scope of the dependency can be set with the common.protobuf2.scope
   option. It can be set to "provided" in a build:
       -Dcommon.protobuf2.scope=provided
* Add new ipc(callable) method to catch and convert shaded protobuf
  exceptions raised during invocation of the supplied lambda expression
* This is adopted in the code where the migration is not traumatically
  over-complex. RouterAdminProtocolTranslatorPB is left alone for this
  reason.

Contributed by Steve Loughran
 2023-10-13 13:48:38 +01:00
huhaiyang
0ed484ac62
HDFS-17208. Add the metrics PendingAsyncDiskOperations in datanode (#6109). Contributed by Haiyang Hu. 
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-10-12 23:27:15 +08:00
Steve Vaughan
73eccd6d7c
HDFS-16740. Mini cluster test flakiness (#4835) 2023-10-10 13:51:46 -07:00
huhaiyang
85af6c3a28
HDFS-17217. Add lifeline RPC start up log when NameNode#startCommonServices (#6154). Contributed by Haiyang Hu. 
Reviewed-by:  Shilun Fan <slfan1989@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
 2023-10-10 10:20:07 +08:00
hfutatzhanghb
ea3cb12ec8
HDFS-17171. CONGESTION_RATIO should be configurable (#5996) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-10-08 10:36:09 +08:00
huhaiyang
4c408a557f
HDFS-17205. HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable (#6112). Contributed by Haiyang Hu 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-10-06 12:39:23 +05:30
huhaiyang
5edd21bc85
HDFS-17194. Enhance the log message for striped block recovery (#6094) 2023-10-04 11:22:59 -07:00
xiaojunxiang
0cfffb3012
HDFS-17214. RBF: The Quota class' andByStorageType method res has an incorrect initial value. (#6135) 
Co-authored-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
 2023-10-03 08:26:03 -07:00
Wang Yu
b87180568b
HDFS-17209. Correct comments to align with the code (#6110). Contributed by Yu Wang. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-10-01 17:30:59 +05:30
ConfX
8931393302
HDFS-17133: TestFsDatasetImpl missing null check when cleaning up (#6079). Contributed by ConfX. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-09-29 12:06:24 +05:30
xiaojunxiang
390cd294f8
HDFS-17211. Fix comments in the RemoteParam class. (#6124). Contributed hellosrc. 
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-09-29 11:55:59 +05:30
zhangshuyan
26a5f38250
HDFS-17204. EC: Reduce unnecessary log when processing excess redundancy. (#6107). Contributed by Shuyan Zhang. 
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-26 18:49:47 +08:00
zhangshuyan
ecee022e49
HDFS-17197. Show file replication when listing corrupt files. (#6095). Contributed by Shuyan Zhang. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-25 13:01:25 +08:00
huhaiyang
cc66683b1a
HDFS-17184. Improve BlockReceiver to throws DiskOutOfSpaceException when initialize. (#6044). Contributed by Haiyang Hu. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-21 21:45:30 +08:00
Jian Zhang
d273c13ab5
HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified (#6096) 2023-09-20 10:04:29 -07:00
ConfX
39f36d9071
HDFS-17105. mistakenly purge editLogs even after it is empty in NNStorageRetentionManager. (#6036). Contributed by ConfX. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-20 11:46:57 +08:00
PJ Fanning
c16484ffb2
HADOOP-18890. Remove use of okhttp in runtime code (#6057) 
Contributed by PJ Fanning
 2023-09-19 12:38:36 +01:00
章锡平
60f3a2b101
HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur… (#5921) 2023-09-18 09:40:22 -07:00
hfutatzhanghb
18f9989ff2
HDFS-17192. Add bock info when constructing remote block reader meets IOException. (#6081). Contributed by farmmamba. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-18 20:56:45 +08:00
zhangshuyan
9e489b9ab5
HDFS-17190. EC: Fix bug of OIV processing XAttr. (#6067). Contributed by Shuyan Zhang. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-18 10:35:13 +08:00
zhtttylz
510a7dcae0
HDFS-17180. HttpFS Add Support getTrashRoots API (#6028) Contributed by Hualong Zhang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-09-15 21:01:24 +08:00
dannytbecker
4652d22b91
HDFS-17178: BootstrapStandby needs to handle RollingUpgrade (#6018) 2023-09-12 14:46:08 -07:00
Jian Zhang
e4dd7fe1b7
HDFS-17139. RBF: For the doc of the class RouterAdminProtocolTranslatorPB, it describes the function of the class ClientNamenodeProtocolTranslatorPB (#5917) 2023-09-12 14:44:55 -07:00
huhaiyang
3bd6a751ed
HDFS-17177. ErasureCodingWork should ignore the deleted block while reconstructing blocks (#6024) 2023-09-11 11:57:52 +08:00
dannytbecker
07c8b69f7c
HDFS-17167. Add config to startup NameNode as Observer (#6013) 2023-09-06 09:17:12 -07:00
Liangjun He
c2c6972f25
HDFS-17140. Revisit the BPOfferService.reportBadBlocks() method. (#5924). Contributed by Liangjun He. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-06 20:19:04 +08:00
ZanderXu
7c941e00b4
HDFS-16933. A race in SerialNumberMap will cause wrong owner, group and XATTR. (#5430). Contributed by ZanderXu. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-06 10:27:52 +08:00
Jian Zhang
c40a6bd46a
HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover (#5990) 2023-09-05 17:48:27 -07:00
zhtttylz
9857fbcd61
HDFS-17168. Support getTrashRoots API in WebHDFS (#5992) Contributed by Hualong Zhang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-09-02 15:24:52 +08:00
Steve Loughran
28c533a582 Revert "HADOOP-18860. Upgrade mockito version to 4.11.0 (#5977)" 
This reverts commit 1046f9cf98.
 2023-08-31 14:54:53 +01:00
Anmol Asrani
1046f9cf98
HADOOP-18860. Upgrade mockito version to 4.11.0 (#5977) 
As well as the POM update, this patch moves to the (renamed) verify methods. 
Backporting mockito test changes may now require cherrypicking this patch, otherwise
use the old method names.

Contributed by Anmol Asrani
 2023-08-29 12:12:27 +01:00
yuyanlei
b58885624b
HDFS-17093. Fix block report lease issue to avoid missing some storages report. (#5855). Contributed by Yanlei Yu. 
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-08-28 14:52:38 +08:00
tian bao
11404c57cb
HDFS-17162. RBF: Add missing comments in StateStoreService #5959 
Contributed by tian bao
 2023-08-25 14:03:02 +01:00
Steve Loughran
2dfe947f20
Revert "Add missing comments in StateStoreService (#5959)" 
This reverts commit 439c8653cf.
 2023-08-25 14:02:38 +01:00
tian bao
439c8653cf
Add missing comments in StateStoreService (#5959) 2023-08-23 18:30:09 -07:00