big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,073 Commits 2 Branches 2 Tags 264 MiB
7a7db7f0dc
Commit Graph

10042 Commits

Author SHA1 Message Date
GuoPhilipse
615a2a42cf
HDFS-17220. Fix same available space policy in AvailableSpaceVolumeChoosingPolicy (#6174) 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-10-18 13:01:32 +08:00
jianghuazhu
81ba2e8484
HDFS-17228. Improve documentation related to BlockManager. (#6195). Contributed by JiangHua Zhu. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-10-18 05:05:33 +05:30
GuoPhilipse
c8abca3004
HDFS-17210. Optimize AvailableSpaceBlockPlacementPolicy. (#6113). Contributed by GuoPhilipse. 
Reviewed-by:  He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
 2023-10-16 16:34:40 +08:00
Steve Loughran
9bc159f4ac
HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996) 
Protobuf 2.5 JAR is no longer needed at runtime. 

The option common.protobuf.scope defines whether the protobuf 2.5.0
dependency is marked as provided or not.

* New package org.apache.hadoop.ipc.internal for internal only protobuf classes
  ...with a ShadedProtobufHelper in there which has shaded protobuf refs
  only, so guaranteed not to need protobuf-2.5 on the CP
* All uses of org.apache.hadoop.ipc.ProtobufHelper have
  been replaced by uses of org.apache.hadoop.ipc.internal.ShadedProtobufHelper
* The scope of protobuf-2.5 is set by the option common.protobuf2.scope
  In this patch is it is still "compile"
* There is explicit reference to it in modules where it may be needed.
*  The maven scope of the dependency can be set with the common.protobuf2.scope
   option. It can be set to "provided" in a build:
       -Dcommon.protobuf2.scope=provided
* Add new ipc(callable) method to catch and convert shaded protobuf
  exceptions raised during invocation of the supplied lambda expression
* This is adopted in the code where the migration is not traumatically
  over-complex. RouterAdminProtocolTranslatorPB is left alone for this
  reason.

Contributed by Steve Loughran
 2023-10-13 13:48:38 +01:00
huhaiyang
0ed484ac62
HDFS-17208. Add the metrics PendingAsyncDiskOperations in datanode (#6109). Contributed by Haiyang Hu. 
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-10-12 23:27:15 +08:00
Steve Vaughan
73eccd6d7c
HDFS-16740. Mini cluster test flakiness (#4835) 2023-10-10 13:51:46 -07:00
huhaiyang
85af6c3a28
HDFS-17217. Add lifeline RPC start up log when NameNode#startCommonServices (#6154). Contributed by Haiyang Hu. 
Reviewed-by:  Shilun Fan <slfan1989@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
 2023-10-10 10:20:07 +08:00
hfutatzhanghb
ea3cb12ec8
HDFS-17171. CONGESTION_RATIO should be configurable (#5996) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-10-08 10:36:09 +08:00
huhaiyang
4c408a557f
HDFS-17205. HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable (#6112). Contributed by Haiyang Hu 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-10-06 12:39:23 +05:30
huhaiyang
5edd21bc85
HDFS-17194. Enhance the log message for striped block recovery (#6094) 2023-10-04 11:22:59 -07:00
xiaojunxiang
0cfffb3012
HDFS-17214. RBF: The Quota class' andByStorageType method res has an incorrect initial value. (#6135) 
Co-authored-by: xiaojunxiang <xiaojunxiang@kingsoft.com>
 2023-10-03 08:26:03 -07:00
Wang Yu
b87180568b
HDFS-17209. Correct comments to align with the code (#6110). Contributed by Yu Wang. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-10-01 17:30:59 +05:30
ConfX
8931393302
HDFS-17133: TestFsDatasetImpl missing null check when cleaning up (#6079). Contributed by ConfX. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-09-29 12:06:24 +05:30
xiaojunxiang
390cd294f8
HDFS-17211. Fix comments in the RemoteParam class. (#6124). Contributed hellosrc. 
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-09-29 11:55:59 +05:30
zhangshuyan
26a5f38250
HDFS-17204. EC: Reduce unnecessary log when processing excess redundancy. (#6107). Contributed by Shuyan Zhang. 
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-26 18:49:47 +08:00
zhangshuyan
ecee022e49
HDFS-17197. Show file replication when listing corrupt files. (#6095). Contributed by Shuyan Zhang. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-25 13:01:25 +08:00
huhaiyang
cc66683b1a
HDFS-17184. Improve BlockReceiver to throws DiskOutOfSpaceException when initialize. (#6044). Contributed by Haiyang Hu. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-21 21:45:30 +08:00
Jian Zhang
d273c13ab5
HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified (#6096) 2023-09-20 10:04:29 -07:00
ConfX
39f36d9071
HDFS-17105. mistakenly purge editLogs even after it is empty in NNStorageRetentionManager. (#6036). Contributed by ConfX. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-20 11:46:57 +08:00
PJ Fanning
c16484ffb2
HADOOP-18890. Remove use of okhttp in runtime code (#6057) 
Contributed by PJ Fanning
 2023-09-19 12:38:36 +01:00
章锡平
60f3a2b101
HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur… (#5921) 2023-09-18 09:40:22 -07:00
hfutatzhanghb
18f9989ff2
HDFS-17192. Add bock info when constructing remote block reader meets IOException. (#6081). Contributed by farmmamba. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-18 20:56:45 +08:00
zhangshuyan
9e489b9ab5
HDFS-17190. EC: Fix bug of OIV processing XAttr. (#6067). Contributed by Shuyan Zhang. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-18 10:35:13 +08:00
zhtttylz
510a7dcae0
HDFS-17180. HttpFS Add Support getTrashRoots API (#6028) Contributed by Hualong Zhang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-09-15 21:01:24 +08:00
dannytbecker
4652d22b91
HDFS-17178: BootstrapStandby needs to handle RollingUpgrade (#6018) 2023-09-12 14:46:08 -07:00
Jian Zhang
e4dd7fe1b7
HDFS-17139. RBF: For the doc of the class RouterAdminProtocolTranslatorPB, it describes the function of the class ClientNamenodeProtocolTranslatorPB (#5917) 2023-09-12 14:44:55 -07:00
huhaiyang
3bd6a751ed
HDFS-17177. ErasureCodingWork should ignore the deleted block while reconstructing blocks (#6024) 2023-09-11 11:57:52 +08:00
dannytbecker
07c8b69f7c
HDFS-17167. Add config to startup NameNode as Observer (#6013) 2023-09-06 09:17:12 -07:00
Liangjun He
c2c6972f25
HDFS-17140. Revisit the BPOfferService.reportBadBlocks() method. (#5924). Contributed by Liangjun He. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-06 20:19:04 +08:00
ZanderXu
7c941e00b4
HDFS-16933. A race in SerialNumberMap will cause wrong owner, group and XATTR. (#5430). Contributed by ZanderXu. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-09-06 10:27:52 +08:00
Jian Zhang
c40a6bd46a
HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover (#5990) 2023-09-05 17:48:27 -07:00
zhtttylz
9857fbcd61
HDFS-17168. Support getTrashRoots API in WebHDFS (#5992) Contributed by Hualong Zhang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-09-02 15:24:52 +08:00
Steve Loughran
28c533a582 Revert "HADOOP-18860. Upgrade mockito version to 4.11.0 (#5977)" 
This reverts commit 1046f9cf98.
 2023-08-31 14:54:53 +01:00
Anmol Asrani
1046f9cf98
HADOOP-18860. Upgrade mockito version to 4.11.0 (#5977) 
As well as the POM update, this patch moves to the (renamed) verify methods. 
Backporting mockito test changes may now require cherrypicking this patch, otherwise
use the old method names.

Contributed by Anmol Asrani
 2023-08-29 12:12:27 +01:00
yuyanlei
b58885624b
HDFS-17093. Fix block report lease issue to avoid missing some storages report. (#5855). Contributed by Yanlei Yu. 
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-08-28 14:52:38 +08:00
tian bao
11404c57cb
HDFS-17162. RBF: Add missing comments in StateStoreService #5959 
Contributed by tian bao
 2023-08-25 14:03:02 +01:00
Steve Loughran
2dfe947f20
Revert "Add missing comments in StateStoreService (#5959)" 
This reverts commit 439c8653cf.
 2023-08-25 14:02:38 +01:00
tian bao
439c8653cf
Add missing comments in StateStoreService (#5959) 2023-08-23 18:30:09 -07:00
Benjamin Teke
43c889636a
YARN-11535. Remove jackson-dataformat-yaml dependency. (#5970) 2023-08-22 16:42:49 +02:00
zhangshuyan
60867de422
HDFS-17151. EC: Fix wrong metadata in BlockInfoStriped after recovery. (#5938). Contributed by Shuyan Zhang. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-08-21 10:05:34 +08:00
huhaiyang
65e4a66e25
HDFS-17087. Add Throttler for datanode reading block (#5845) 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-08-17 09:33:50 +08:00
zhangshuyan
10b1d7340b
HDFS-17154. EC: Fix bug in updateBlockForPipeline after failover. (#5941). Contributed by Shuyan Zhang. 
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-08-16 20:29:19 +08:00
zhangshuyan
655c3df050
HDFS-17150. EC: Fix the bug of failed lease recovery. (#5937). Contributed by Shuyan Zhang. 
Reviewed-by: hfutatzhanghb <1036798979@qq.com>
Reviewed-by: Haiyang Hu <haiyang.hu@shopee.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-08-15 19:54:15 +08:00
hchaverri
ad2f45c64f
HDFS-17148. RBF: SQLDelegationTokenSecretManager must cleanup expired tokens in SQL (#5936) 2023-08-11 13:04:32 -07:00
hfutatzhanghb
58314cbbf6
HDFS-17145. Fix description of property dfs.namenode.file.close.num-committed-allowed. (#5933) 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-08-11 13:35:41 +08:00
huhaiyang
5b81caf0cf
HDFS-17137. Standby/Observer NameNode skip to handle redundant replica block logic when set decrease replication. (#5913). Contributed by Haiyang Hu. 
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-08-08 15:42:23 +08:00
hfutatzhanghb
b5b9a81538
HDFS-17144. Remove incorrect comment in method storeAllocatedBlock. (#5932) 
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-08-08 09:26:51 +08:00
hchaverri
bc48e5cbe8
HDFS-17128. Updating SQLDelegationTokenSecretManager to use LoadingCache so tokens are updated frequently. (#5897) Contributed by Hector Sandoval Chaverri. 
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-08-08 07:45:14 +08:00
Xing Lin
55e8301470
HDFS-17118 Fixed a couple checkstyle warnings in TestObserverReadProxyProvider (#5880) 2023-08-05 12:27:52 -07:00
zhtttylz
600dd45e47
HDFS-17122. Rectify the table length discrepancy in the DataNode UI (#5915) 2023-08-04 13:07:19 +08:00
WangYuanben
04c34a2171
HDFS-17113. Reconfig transfer and write bandwidth for datanode. (#5869) 2023-08-03 22:22:06 +08:00
zhangshuyan
5aee0e0c0a
HDFS-17134. RBF: Fix duplicate results of getListing through Router. (#5900). Contributed by Shuyan Zhang. 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-08-01 17:52:54 +08:00
zhiubok
52d66c2c11
HDFS-17136. Fix annotation description and typo in BlockPlacementPolicyDefault Class (#5907) 
Co-authored-by: huangzhaobo <huangzhaobo99@126.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-07-31 09:34:49 +08:00
huhaiyang
6cc56d40f5
HDFS-17135. Update fsck -blockId to display excess state info of blocks (#5904) 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-07-31 09:21:22 +08:00
huhaiyang
87c036efb3
HDFS-17116. RBF: Update invoke millisecond time as monotonicNow() in RouterSafemodeService (#5876). Contributed by Haiyang Hu. 
Reviewed-by: hfutatzhanghb <1036798979@qq.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-07-28 15:15:00 +08:00
huhaiyang
1d09dcc614
HDFS-17117. Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync. (#5877). Contributed by Haiyang Hu. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-07-27 11:28:30 +08:00
zhtttylz
068d8c7e4d
HDFS-17115. HttpFS Add Support getErasureCodeCodecs API (#5875). Contributed by Hualong Zhang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-26 17:40:08 +05:30
Simbarashe Dzinamarira
ad001c93cf
HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled. (#5860). Contributed by Simbarashe Dzinamarira. 2023-07-26 13:22:13 +05:30
Zhaohui Wang
9cfe9ccd26
HDFS-17119. RBF: Logger fix for StateStoreMySQLImpl. (#5882). Contributed by Zhaohui Wang. 
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-24 17:26:23 +05:30
zhangshuyan
49c98da838
HDFS-17112. Show decommission duration in JMX and HTML. (#5866). Contributed by Shuyan Zhang. 
Reviewed-by: hfutatzhanghb <1036798979@qq.com>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-07-24 14:38:00 +08:00
hfutatzhanghb
130bd033d5
HDFS-17074. Remove incorrect comment in TestRedudantBlocks#setup. (#5822). Contributed by farmmamba. 
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-21 00:45:40 +05:30
Xing Lin
80fefd093f
HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy (#5803) 2023-07-20 10:46:41 -07:00
zhangshuyan
c35f31640e
HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled. (#5847) 
Contributed by Shuyan Zhang
 2023-07-20 11:39:13 +01:00
zhangshuyan
7ba2bd6305
HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes. (#5854) 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-07-20 09:21:45 +08:00
huhaiyang
8df1616f53
HDFS-17088. Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md (#5846) 2023-07-19 09:53:51 +08:00
WangYuanben
0fe1863094
HDFS-17091. Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks. (#5849) 2023-07-19 09:51:49 +08:00
huhaiyang
c44823dadb
HDFS-17075. Reconfig disk balancer parameters for datanode (#5823). Contributed by Haiyang Hu. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-16 11:27:31 +05:30
huhaiyang
8dd9c874e1
HDFS-17086. Fix the parameter settings in TestDiskspaceQuotaUpdate#updateCountForQuota (#5842). Contributed by Haiyang Hu. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-15 12:00:07 +05:30
hfutatzhanghb
c457c445b7
HDFS-17068. Datanode should record last directory scan time. (#5809). Contributed by farmmamba. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-14 21:13:47 +05:30
huhaiyang
c1d19c3951
HDFS-17081. EC: Add logic for striped blocks in isSufficientlyReplicated (#5833). Contributed by Haiyang Hu. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-14 11:56:54 +05:30
zhtttylz
dfb351c3a8
HDFS-17083. Support getErasureCodeCodecs API in WebHDFS (#5836). Contributed by Hualong Zhang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-14 11:53:36 +05:30
huhaiyang
4c8d048f25
HDFS-17082. Add documentation for provisionSnapshotTrash command to HDFSCommands.md and HdfsSnapshots.md" (#5834). Contributed by Haiyang Hu. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-14 00:58:43 +05:30
huhaiyang
4e699f0383
HDFS-17076. Remove the unused method isSlownodeByNameserviceId in DataNode (#5824). Contributed by Haiyang Hu. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-11 13:16:06 +05:30
WangYuanben
37b2d36edd
HDFS-17033. Update fsck to display stale state info of blocks accurately. (#5815). Contributed by Wang Yuanben. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Reviewed-by: Hualong Zhang <hualong.z@hotmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-07-10 18:42:37 +08:00
huhaiyang
a84284e974
HDFS-17064. Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter (#5794). Contributed by Haiyang Hu. 
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-10 13:08:47 +05:30
hfutatzhanghb
b673ebfec2
HDFS-17070. Remove unused import in DataNodeMetricHelper.java. (#5812). Contributed by farmmamba. 
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-10 12:58:23 +05:30
huhaiyang
3f983e49e0
HDFS-17073. Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute" (#5817). Contributed by Haiyang Hu. 
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-07-09 23:04:20 +08:00
Zhaohui Wang
9659d0720d
HDFS-17065. Fix typos in hadoop-hdfs-project (#5796) 2023-07-03 08:41:37 -07:00
zhtttylz
750c0fc631
HDFS-17052. Improve BlockPlacementPolicyRackFaultTolerant to avoid choose nodes failed when no enough Rack. (#5759). Contributed by Hualong Zhang and Shuyan Zhang. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-07-02 15:37:19 +08:00
wangzhaohui
8fb611c1f7
HDFS-17062. HA NameNode Web UI should show last HA Transition time (#5791). Contributed by Zhaohui Wang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-30 08:56:34 +05:30
huhaiyang
4bd873b816
HDFS-17044. Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from DataNode. (#5735). Contributed by Haiyang Hu. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-06-28 12:03:15 +08:00
Xing Lin
03902f5ef0
HDFS-17055 Export HAState as a metric from Namenode for monitoring (#5764) 2023-06-26 15:52:59 -07:00
huhaiyang
161f80810f
HDFS-17057. Add DataNode maintenance states to Federation UI (#5775) 2023-06-26 09:38:57 -07:00
Wei-Chiu Chuang
e239d40ab1 Post release update 
* Add jdiff xml files from 3.3.6 release.
* Declare 3.3.6 as the latest stable release.
* Copy release notes.

(cherry picked from commit 7db9895000)
(cherry picked from commit cc121e2124aa01458dc296a060edc5e21a295268)
 2023-06-26 16:08:24 +00:00
hfutatzhanghb
e14c52c25f
HDFS-17053. Optimize method BlockInfoStriped#findSlot to reduce time complexity. (#5757). Contributed by farmmamba. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-19 14:49:23 +05:30
zhtttylz
9a7d1b49e2
HDFS-17043. HttpFS implementation for getAllErasureCodingPolicies (#5734). Contributed by Hualong Zhang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-16 22:36:47 +05:30
wangzhaohui
02027c8dcc
HDFS-17051. Fix wrong time unit in TestFileAppend4#recoverFile (#5749). Contributed by Zhaohui Wang. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-15 18:20:53 +05:30
Tsz-Wo Nicholas Sze
5dd37b2814
HDFS-17045. File renamed from a snapshottable dir to a non-snapshottable dir cannot be deleted. (#5738) 2023-06-15 03:30:04 +08:00
hfutatzhanghb
1a918a6b96
HDFS-17047. BlockManager#addStoredBlock should log storage id when AddBlockResult is REPLACED (#5742). Contributed by farmmamba. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-15 00:24:22 +05:30
Xing Lin
ab47a658bd
HDFS-17030. Limit wait time for getHAServiceState in ObserverReaderProxy (#5700) 2023-06-14 10:53:56 -07:00
Viraj Jasani
f0c4286e3e
HDFS-17041. RBF: Fix putAll impl for mysql and file based state stores (#5723) 2023-06-13 09:55:04 -07:00
NishthaShah
3fbadc5d50
HDFS-16946. Fix getTopTokenRealOwners to return String (#5696). Contributed by Nishtha Shah. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-13 10:01:35 +05:30
Wei-Chiu Chuang
e1bb4acd36
HADOOP-18761. Remove mysql-connector-java (#5731) 2023-06-12 15:31:03 -07:00
Dongjoon Hyun
fb16e00da0
HADOOP-18718. Fix several maven build warnings (#5592). Contributed by Dongjoon Hyun. 
Reviewed-by: Gautham B A <gautham.bangalore@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-11 11:38:13 +05:30
hfutatzhanghb
35158db711
HDFS-17023. RBF: Record proxy time when call invokeConcurrent method. (#5683). Contributed by farmmamba. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-10 00:06:28 +05:30
zhangshuyan
9c989515ba
HDFS-17037. Consider nonDfsUsed when running balancer. (#5715). Contributed by Shuyan Zhang. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-06-09 16:19:08 +08:00
hfutatzhanghb
0e6bd09ae3
HDFS-17003. Erasure Coding: invalidate wrong block after reporting bad blocks from datanode (#5643). Contributed by hfutatzhanghb. 
Reviewed-by: Stephen O'Donnell <sodonnel@apache.org>
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-06-08 18:06:51 +08:00
hfutatzhanghb
ddae78b0ec
HDFS-17035. FsVolumeImpl#getActualNonDfsUsed may return negative value. (#5708). Contributed by farmmamba. 
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-08 14:28:01 +05:30
huhaiyang
0c209961f8
HDFS-17019. Optimize the logic for reconfigure slow peer enable for Namenode" (#5671) 
* HDFS-17019. Optimize the logic for reconfigure slow peer enable for Namenode
 2023-06-08 10:05:49 +08:00
zhtttylz
d9980ab40f
HDFS-17029. Support getECPolices API in WebHDFS (#5698). Contributed by Hualong Zhang. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-05 17:33:37 +05:30