Tsz-Wo Nicholas Sze
b189ef8197
HDFS-17575. SaslDataTransferClient should use SaslParticipant to create messages. ( #6954 )
2024-08-05 10:42:12 -07:00
Takanobu Asanuma
059e996c02
HDFS-17591. RBF: Router should follow X-FRAME-OPTIONS protection setting ( #6963 )
2024-07-30 10:14:33 +09:00
Aswin M Prabhu
e2a0dca43b
HDFS-16690. Automatically format unformatted JNs with JournalNodeSyncer ( #6925 ). Contributed by Aswin M Prabhu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-07-23 20:55:57 +08:00
Viraj Jasani
e000cbf277
HADOOP-19218. Addendum. Update TestFSNamesystemLockReport to exclude hostname resolution from regex. ( #6951 ). Contributed by Viraj Jasani.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-07-23 20:47:36 +08:00
Tsz-Wo Nicholas Sze
e48cd0e987
Revert "HDFS-17575. SaslDataTransferClient should use SaslParticipant to create messages. ( #6933 )"
...
This reverts commit 7638b4727e
2024-07-21 19:07:19 +08:00
Tsz-Wo Nicholas Sze
a5eb5e9611
HDFS-17576. Support user defined auth Callback. ( #6945 )
2024-07-20 15:21:06 +08:00
Tsz-Wo Nicholas Sze
7638b4727e
HDFS-17575. SaslDataTransferClient should use SaslParticipant to create messages. ( #6933 )
2024-07-20 15:16:01 +08:00
fuchaohong
ebbe9628d3
HDFS-17558. RBF: Make maxSizeOfFederatedStateToPropagate work on setResponseHeaderState. ( #6902 )
...
Co-authored-by: fuchaohong <fuchaohong@chinatelecom.cn>
2024-07-17 09:43:00 -07:00
gavin.wang
5730656660
HDFS-17574. Make NNThroughputBenchmark support human-friendly units about blocksize. ( #6931 ). Contributed by wangzhongwei.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-07-16 20:57:50 +08:00
zhengchenyu
8913d379fd
HDFS-17566. Got wrong sorted block order when StorageType is considered. ( #6934 ). Contributed by Chenyu Zheng.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-07-11 17:41:24 +08:00
Yuanbo Liu
f211af30be
HDFS-17534. RBF: Support leader follower mode for multiple subclusters ( #6861 ). Contributed by Yuanbo Liu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2024-07-09 14:01:58 +05:30
gavin.wang
783a852029
HDFS-17555. Fix NumberFormatException of NNThroughputBenchmark when configured dfs.blocksize. ( #6894 ). Contributed by wangzhongwei
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2024-07-09 13:52:15 +05:30
huhaiyang
8ca4627a0d
HDFS-17557. Fix bug for TestRedundancyMonitor#testChooseTargetWhenAllDataNodesStop ( #6897 ). Contributed by Haiyang Hu.
...
website / build (push) Has been cancelled
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2024-07-06 13:18:12 +05:30
huhaiyang
5a8f70a72e
HDFS-17559. Fix the uuid as null in NameNodeMXBean ( #6906 ). Contributed by Haiyang Hu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2024-07-06 13:16:25 +05:30
huhaiyang
ae76e9475c
HDFS-17564. EC: Fix the issue of inaccurate metrics when decommission mark busy DN. ( #6911 ). Contributed by Haiyang Hu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-07-05 20:45:01 +08:00
Yu Zhang
b4ddb2d3bb
HDFS-13603: do not propagate ExecutionException and add maxRetries limit to NameNode edek cache warmup ( #6774 )
2024-06-24 09:34:52 -07:00
Hexiaoqiao
6545b7eeef
HDFS-17098. DatanodeManager does not handle null storage type properly. ( #6840 ). Contributed by ConfX.
...
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2024-06-19 20:58:57 +08:00
Tsz-Wo Nicholas Sze
1e6411c9ec
HDFS-17528. FsImageValidation: set txid when saving a new image ( #6828 )
2024-06-19 11:38:17 +08:00
Fateh Singh
90024d8cb1
HDFS-17439. Support -nonSuperUser for NNThroughputBenchmark: useful for testing auth frameworks such as Ranger ( #6677 )
2024-06-18 13:52:24 +01:00
Heagan A
2fbbfe3cc9
HDFS-17546. Implementing HostsFileReader timeout ( #6873 )
2024-06-14 20:47:21 -07:00
hfutatzhanghb
4b1b16a846
HDFS-17551. Fix unit test failure caused by HDFS-17464. ( #6883 ). Contributed by farmmamba.
2024-06-12 22:21:15 +05:30
Felix Nguyen
776c0a3ab9
HDFS-17539. Make TestFileChecksum fields static ( #6853 )
2024-06-11 15:26:21 +08:00
刘斌
6c08e8e2aa
HADOOP-19156. ZooKeeper based state stores use different ZK address configs. ( #6767 ). Contributed by liu bin.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-05-29 20:44:36 +08:00
Felix Nguyen
74d30a5dce
HDFS-17532. RBF: Allow router state store cache update to overwrite and delete in parallel ( #6839 )
2024-05-28 11:17:08 +08:00
Felix Nguyen
f5c5d35eb0
HDFS-17529. RBF: Improve router state store cache entry deletion ( #6833 )
2024-05-24 09:41:08 +08:00
hfutatzhanghb
fb156e8f05
HDFS-17464. Improve some logs output in class FsDatasetImpl ( #6724 )
2024-05-21 09:46:21 +08:00
Mukund Thakur
47be1ab3b6
HADOOP-18679. Add API for bulk/paged delete of files ( #6726 )
...
Applications can create a BulkDelete instance from a
BulkDeleteSource; the BulkDelete interface provides
the pageSize(): the maximum number of entries which can be
deleted, and a bulkDelete(Collection paths)
method which can take a collection up to pageSize() long.
This is optimized for object stores with bulk delete APIs;
the S3A connector will offer the page size of
fs.s3a.bulk.delete.page.size unless bulk delete has
been disabled.
Even with a page size of 1, the S3A implementation is
more efficient than delete(path)
as there are no safety checks for the path being a directory
or probes for the need to recreate directories.
The interface BulkDeleteSource is implemented by
all FileSystem implementations, with a page size
of 1 and mapped to delete(pathToDelete, false).
This means that callers do not need to have special
case handling for object stores versus classic filesystems.
To aid use through reflection APIs, the class
org.apache.hadoop.io.wrappedio.WrappedIO
has been created with "reflection friendly" methods.
Contributed by Mukund Thakur and Steve Loughran
2024-05-20 17:05:25 +01:00
LiuGuH
8f92cda35c
HDFS-17509. RBF: Fix ClientProtocol.concat will throw NPE if tgr is a empty file. ( #6784 )
2024-05-17 10:37:50 +08:00
ZanderXu
cab0f4c9ec
HDFS-17520. [BugFix] TestDFSAdmin.testAllDatanodesReconfig and TestDFSAdmin.testDecommissionDataNodesReconfig failed ( #6812 ) Contributed by Zengqiang Xu.
...
Reviewed-by: Vinayakumar B <vinayakumarb@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2024-05-15 07:55:24 +08:00
Simbarashe Dzinamarira
6a4f0be854
HDFS-17514: RBF: Routers should unset cached stateID when namenode does not set stateID in RPC response header. ( #6804 )
2024-05-14 08:09:56 -07:00
ConfX
8d9d58dfc8
HDFS-17099. Fix Null Pointer Exception when stop namesystem in HDFS.( #6034 ). Contributed by ConfX.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-05-14 11:14:55 +08:00
zhihui wang
12e0ca6b24
HDFS-17522. JournalNode web interfaces lack configs for X-FRAME-OPTIONS protection ( #6814 ). Contributed by wangzhihui.
...
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-05-13 22:10:08 +08:00
Felix Nguyen
fb0519253d
HDFS-17488. DN can fail IBRs with NPE when a volume is removed ( #6759 )
2024-05-11 15:37:43 +08:00
Zilong Zhu
700b3e4800
HDFS-17503. Unreleased volume references because of OOM. ( #6782 )
2024-05-10 10:34:40 +08:00
kulkabhay
edf985e269
HDFS-17500: Add missing operation name while authorizing some operations ( #6776 ). Contributed by kulkabhay.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-05-06 12:44:30 +08:00
dannytbecker
881034ad45
CachedRecordStore should check if the record state is expired ( #6783 )
2024-05-01 13:56:53 -07:00
fuchaohong
0c9e0b4398
HDFS-17456. Fix the incorrect dfsused statistics of datanode when appending a file. ( #6713 ). Contributed by fuchaohong.
...
Reviewed-by: ZanderXu <zanderxu@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-04-30 12:22:53 +08:00
fuchaohong
ddb805951e
HDFS-17471. Correct the percentage of sample range. ( #6742 ). Contributed by fuchaohong.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-04-30 12:18:47 +08:00
Tsz-Wo Nicholas Sze
78987a71a6
HADOOP-19151. Support configurable SASL mechanism. ( #6740 )
2024-04-29 10:02:23 -07:00
zhtttylz
daafc8a0b8
HDFS-17367. Add PercentUsed for Different StorageTypes in JMX ( #6735 ) Contributed by Hualong Zhang.
...
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2024-04-27 20:36:11 +08:00
dannytbecker
027b4c3259
Remove empty queues from the queueByBlockId map ( #6772 )
2024-04-26 14:25:15 -07:00
cxzl25
23286b0632
HDFS-17469. Audit log for reportBadBlocks RPC ( #6731 )
2024-04-24 09:39:57 +08:00
Jian Zhang
782c501441
HDFS-17451. RBF: fix spotbugs for redundant nullcheck of dns. ( #6697 )
2024-04-23 19:11:51 +08:00
Madhan Neethiraj
e8b2c28dec
HDFS-17478. FSPermissionChecker optimization by initializing AccessControlEnforcer in constructor ( #6749 )
2024-04-18 15:43:31 -07:00
dannytbecker
0c35cf0982
HDFS-17477. IncrementalBlockReport race condition additional edge cases ( #6748 )
2024-04-18 09:04:08 -07:00
章锡平
87cc2f1a1f
HDFS-17465. RBF: Use ProportionRouterRpcFairnessPolicyController get “'ava.Lang. Error: Maximum permit count exceeded' ( #6727 )
2024-04-15 09:28:05 -07:00
Lei313
f49a4df797
HDFS-17383:Datanode current block token should come from active NameNode in HA mode ( #6562 ). Contributed by lei w.
...
Reviewed-by: Shuyan Zhang <zhangshuyan@apache.org>
Signed-off-by: Shuyan Zhang <zhangshuyan@apache.org>
2024-04-15 18:35:53 +08:00
huhaiyang
6ccb223c9c
HDFS-17461. Fix spotbugs in PeerCache#getInternal ( #6721 ). Contributed by Haiyang Hu.
...
Reviewed-by: ZanderXu <zanderxu@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-04-12 19:53:25 +08:00
huhaiyang
81b05977f2
HDFS-17455. Fix Client throw IndexOutOfBoundsException in DFSInputStream#fetchBlockAt ( #6710 ). Contributed by Haiyang Hu.
...
Reviewed-by: ZanderXu <zanderxu@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2024-04-11 18:04:57 +08:00
dannytbecker
05964ad07a
HDFS-17453. IncrementalBlockReport can have race condition with Edit Log Tailer ( #6708 )
2024-04-10 09:30:24 -07:00
