Wei-Chiu Chuang
7db9895000
Post release update
...
* Add jdiff xml files from 3.3.6 release.
* Declare 3.3.6 as the latest stable release.
* Copy release notes.
2023-06-26 15:59:39 +00:00
Wei-Chiu Chuang
8af2d2feb2
Update version to 3.3.6
2023-06-12 15:34:41 -07:00
Dongjoon Hyun
20d073cb2c
HADOOP-18718. Fix several maven build warnings ( #5592 ). Contributed by Dongjoon Hyun.
...
Reviewed-by: Gautham B A <gautham.bangalore@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit fb16e00da0
2023-06-12 10:40:41 -07:00
hfutatzhanghb
0fd2b10bdc
HDFS-16898. Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat. ( #5408 ). Contributed by ZhangHB.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-06-12 13:42:39 +08:00
hfutatzhanghb
a804f37ed5
HDFS-17003. Erasure Coding: invalidate wrong block after reporting bad blocks from datanode ( #5643 ). Contributed by hfutatzhanghb.
...
Reviewed-by: Stephen O'Donnell <sodonnel@apache.org>
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 0e6bd09ae3
2023-06-08 18:22:57 +08:00
huhaiyang
19a6762639
HDFS-17017. Fix the issue of arguments number limit in report command in DFSAdmin ( #5667 ). Contributed by Haiyang Hu.
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-21 09:06:53 +05:30
wangzhaohui
5cafa1fdb9
HDFS-17011. Fix the metric of "HttpPort" at DataNodeInfo ( #5657 ). Contributed by Zhaohui Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 12:13:56 +05:30
Wei-Chiu Chuang
99312bdfdb
HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common ( #5553 ) ( #5619 )
...
* HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common (#5553 )
The HDFS lease APIs have been replicated as interfaces in hadoop-common so other filesystems can
also implement them. Applications which use the leasing APIs should migrate to the new
interface where possible.
Contributed by Stephen Wu
(cherry picked from commit 0e46388474
2023-05-09 06:20:56 +08:00
Ayush Saxena
a226016c52
HADOOP-18662. ListFiles with recursive fails with FNF. ( #5477 ). Contributed by Ayush Saxena.
...
Reviewed-by: Steve Loughran <stevel@apache.org>
2023-05-02 20:12:22 +05:30
fanluoo
408c5c53b1
HDFS-16897. Fix abundant Broken pipe exception in BlockSender ( #5329 ). Contributed by fanluo.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-29 17:29:55 +05:30
cxzl25
4d052a2456
HDFS-16672. Fix lease interval comparison in BlockReportLeaseManager ( #4598 ). Contributed by dzcxzl.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org
2023-04-26 02:12:40 +05:30
wangzhaohui
1ea5fbd192
HDFS-16988. Improve NameServices info at JournalNode web UI ( #5584 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-24 01:14:18 +05:30
Steve Loughran
a505940a2f
HADOOP-18470. Hadoop 3.3.5 release wrap-up ( #5558 )
...
Post-release updates of the branches
* Add jdiff xml files from 3.3.5 release.
* Declare 3.3.5 as the latest stable release.
* Copy release notes.
2023-04-18 10:12:41 +01:00
Stephen O'Donnell
c39696ab1d
HDFS-16942. Addendum. Send error to datanode if FBR is rejected due to bad lease ( #5478 ). Contributed by Stephen O'Donnell/
...
(cherry picked from commit eee2ea075d
2023-03-15 10:17:55 +00:00
Stephen O'Donnell
d7b89d08e1
HDFS-16942. Send error to datanode if FBR is rejected due to bad lease ( #5460 )
...
(cherry picked from commit ca6f5afb6d
2023-03-11 17:04:33 +00:00
zhangshuyan
8cc57f5eb1
HDFS-16939. Fix the thread safety bug in LowRedundancyBlocks. ( #5450 #5471 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-11 16:09:06 +08:00
slfan1989
e12bc4e1d8
HDFS-16934. TestDFSAdmin.testAllDatanodesReconfig regression ( #5434 )
...
Contributed by Shilun Fan
2023-03-06 15:45:42 +00:00
Tom
703158c9c6
HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref… ( #5322 ) ( #5444 )
...
Cherry picked from: 162288bc0a
2023-03-02 00:44:30 +00:00
ZanderXu
a7c1fad0c9
HDFS-16923. [SBN read] getlisting RPC to observer will throw NPE if path does not exist ( #5400 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
(cherry picked from commit 6bd2444815
2023-03-01 16:20:50 -08:00
Viraj Jasani
91ce13ea01
HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks ( #5432 )
...
Contributed by Viraj Jasani
2023-03-01 18:53:52 +00:00
rdingankar
94b3c6dd90
HDFS-16917 Add transfer rate quantile metrics for DataNode reads ( #5397 )
...
Co-authored-by: Ravindra Dingankar <rdingankar@linkedin.com>
2023-02-27 15:49:26 -08:00
Viraj Jasani
e47ff5f3c0
HDFS-16925. Namenode audit log to only include IP address of client ( #5413 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-21 08:33:54 +08:00
Steve Vaughan
d437571fe2
HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown ( #5372 )
...
This is a followup to the original patch, 08f58ecf07
2023-02-13 14:47:00 +00:00
Viraj Jasani
5099077f4d
HDFS-16907. Add LastHeartbeatResponseTime for BP service actor ( #5349 ) ( #5358 )
...
* ADDENDUM Remove unused variables from testDataNodeMXBeanLastHeartbeats (#5373 )
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-12 11:53:59 +05:30
Ayush Saxena
b6d732b96b
HDFS-15368. TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally. Contributed by Xiaoqiao He.
2023-02-12 01:57:34 +05:30
Masatake Iwasaki
d85c12ee69
HDFS-15674. TestBPOfferService#testMissBlocksWhenReregister fails on trunk. ( #2467 )
2023-02-12 01:50:39 +05:30
Ahmed Hussein
c17734b747
HDFS-15654. TestBPOfferService#testMissBlocksWhenReregister fails intermittently ( #2419 )
2023-02-12 01:50:26 +05:30
Steve Vaughan
b6680907f9
HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown ( #5342 )
...
Contributed by Steve Vaughan Jr
2023-02-09 13:29:00 +00:00
Tao Li
10b42e9306
HDFS-16902. Add Namenode status to BPServiceActor metrics and improve logging in offerservice ( #5334 )
...
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-03 10:02:45 +08:00
huhaiyang
80b42625cf
HDFS-16888. BlockManager#maxReplicationStreams, replicationStreamsHardLimit, blocksReplWorkMultiplier and PendingReconstructionBlocks#timeout should be volatile ( #5296 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 88c8ac750d
2023-01-31 17:52:29 +09:00
Viraj Jasani
63443be5f4
HDFS-16891 Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel ( #5300 )
...
Reviewed-by: Stephen O'Donnell <sodonnell@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit 04f3573f6a
2023-01-18 21:14:17 +00:00
Viraj Jasani
c5cf845d78
HDFS-16887 Log start and end of phase/step in startup progress ( #5292 )
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit 1263e024b9
2023-01-12 22:27:35 +00:00
Chengbing Liu
af96e0f5b3
HDFS-16872. Fix log throttling by declaring LogThrottlingHelper as static members ( #5246 )
...
Co-authored-by: Chengbing Liu <liuchengbing@qiyi.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
(cherry picked from commit 4cf304de45
2023-01-10 10:04:05 -08:00
ZanderXu
a49378cb93
HDFS-16764. [SBN Read] ObserverNamenode should throw ObserverRetryOnActiveException instead of FileNotFoundException during processing of addBlock rpc ( #4872 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
(cherry picked from commit 8d221255f2
2023-01-10 10:03:45 -08:00
dingshun3016
ebd2407d48
HDFS-16809. EC striped block is not sufficient when doing in maintenance. ( #5050 )
...
(cherry picked from commit 02afb9ebe1
2022-12-05 16:39:49 +09:00
Ashutosh Gupta
fa2a0a603a
HDFS-16633. Fixing when Reserved Space For Replicas is not released on some cases ( #4452 )
...
* HDFS-16633.Reserved Space For Replicas is not released on some cases
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
(cherry picked from commit b7edc6c60c
2022-12-01 15:00:32 +09:00
Kidd5368
8c7f2ddc10
HDFS-16839 It should consider EC reconstruction work when we determine if a node is busy ( #5128 )
...
Co-authored-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
(cherry picked from commit 72749a4ff8
2022-11-29 17:45:12 -08:00
zhengchenyu
4addf31ef4
HDFS-16832. [SBN READ] Follow-on to HDFS-16732. Fix NPE when check the block location of empty directory ( #5099 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Reviewed-by: Zengqiang Xu <xuzq_zander@163.com>
(cherry picked from commit dc2fba45fe
2022-11-21 08:33:16 -08:00
Lei Yang
b68520d2a5
HDFS-16836: StandbyCheckpointer shouldn't trigger rollback fs image after RU is finalized ( #5135 )
...
Co-authored-by: Lei Yang <leyang@linkedin.com>
2022-11-15 15:10:41 -08:00
huhaiyang
033ceca090
HDFS-16811. Support DecommissionBackoffMonitor parameters reconfigurable ( #5122 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2022-11-10 13:37:09 +08:00
Melissa You
853ffb545a
HADOOP-18515. Backport HADOOP-17612 to branch-3.3(Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0) ( #5097 )
...
* HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (#3241 )
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Co-authored-by: Viraj Jasani <vjasani@apache.org>
Co-authored-by: Melissa You <myou@myou-mn1.linkedin.biz>
2022-11-05 09:28:24 -07:00
M1eyu2018
cbac2c4875
HDFS-16716. Improve appendToFile command: support appending on file with new block ( #4697 )
...
Reviewed-by: xuzq <15040255127@163.com>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-10-27 19:11:51 +08:00
Takanobu Asanuma
53143409a8
HDFS-16822. HostRestrictingAuthorizationFilter should pass through requests if they don't access WebHDFS API. ( #5079 )
...
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Tao Li <tomscut@apache.org>
(cherry picked from commit 545a556883
2022-10-27 14:40:07 +09:00
Viraj Jasani
36a0e818ec
HDFS-16016. BPServiceActor to provide new thread to handle IBR ( #2998 )
...
Contributed by Viraj Jasani
(cherry picked from commit c1bf3cb0da
2022-10-24 15:16:38 +09:00
Takanobu Asanuma
198bc444de
HDFS-16566 Erasure Coding: Recovery may causes excess replicas when busy DN exsits ( #4252 ) ( #5059 )
...
(cherry picked from commit 9376b65989
2022-10-22 13:14:04 +09:00
SevenAddSix
237814a9b3
HDFS-16480. Fix typo: indicies -> indices ( #4020 )
...
(cherry picked from commit 5eab9719cb
2022-10-21 17:32:58 +09:00
Hui Fei
0c2234fd8e
HDFS-15803. EC: Remove unnecessary method (getWeight) in StripedReconstructionInfo. Contributed by huhaiyang
...
(cherry picked from commit 66ecee333e
2022-10-21 17:31:30 +09:00
Steve Loughran
75b04010a2
HDFS-16795. Use secure XML parsers ( #4979 )
...
Move construction of XML parsers in HDFS
modules to using the locked-down parser factory
of HADOOP-18469.
Contributed by P J Fanning
2022-10-20 17:48:58 +01:00
Kidd5368
ceec19e61a
HDFS-16776 Erasure Coding: The length of targets should be checked when DN gets a reconstruction task ( #4901 )
...
(cherry picked from commit 9a29075f91
2022-09-23 12:29:39 +09:00
Steve Vaughan
357c83db94
HDFS-16686. GetJournalEditServlet fails to authorize valid Kerberos request ( #4724 ) ( #4794 )
2022-09-13 10:50:23 -07:00
