big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,320 Commits 2 Branches 2 Tags 264 MiB
3848b44b37
Commit Graph

9009 Commits

Author SHA1 Message Date
Akira Ajisaka
3848b44b37
HDFS-15580. [JDK 12] DFSTestUtil#addDataNodeLayoutVersion fails (#2309) 2020-10-27 19:18:08 +09:00
Ahmed Hussein
afaab3d332
HDFS-15461. TestDFSClientRetries testGetFileChecksum fails (#2404) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
 2020-10-27 16:56:30 +09:00
Hui Fei
12c908c827 HDFS-15641. DataNode could meet deadlock if invoke refreshNameNode. Contributed by Hongbing Wang. 2020-10-26 22:12:27 +08:00
Wei-Chiu Chuang
74634eb002 HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein. 2020-10-23 12:05:53 -07:00
Inigo Goiri
e7aa4da0a2 HDFS-15459. TestBlockTokenWithDFSStriped fails intermittently. Contributed by Ahmed Hussein. 2020-10-23 09:47:10 -07:00
Hui Fei
264c948e60 HDFS-15630. RBF: Fix wrong client IP info in CallerContext when requests mount points with multi-destinations. Contributed by Chengwei Wang 2020-10-23 14:06:22 +08:00
Kihwal Lee
da1b6e3cc2 HDFS-15622. Deleted blocks linger in the replications queue. Contributed by Ahmed Hussein. 2020-10-22 16:59:49 -05:00
Kihwal Lee
88a9f42f32 HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. 2020-10-21 00:59:35 -05:00
Takanobu Asanuma
30f06e0c74
HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client. (#2394) 2020-10-20 19:08:59 +09:00
Ayush Saxena
2e8cafac3b HDFS-14383. Compute datanode load based on StoragePolicy. Contributed by Ayush Saxena. 2020-10-19 10:48:47 +05:30
Ayush Saxena
173310e2f5 HDFS-15629. Add seqno when warning slow mirror/disk in BlockReceiver. Contributed by Haibin Huang. 2020-10-19 10:34:17 +05:30
Ayush Saxena
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. (#2342). Contributed by Ayush Saxena. 2020-10-17 12:01:18 +05:30
Kihwal Lee
740a2c4635 HDFS-15627. Log delete audits before collecting blocks. 2020-10-16 11:13:41 -05:00
Ayush Saxena
cc57eebe45
HDFS-15633. Avoid redundant RPC calls for getDiskStatus. (#2386). Contributed by Ayush Saxena. 2020-10-16 10:36:24 +05:30
Kihwal Lee
e45407128d HDFS-15628. HttpFS server throws NPE if a file is a symlink. Contributed by Ahmed Hussein. 2020-10-14 17:26:04 -05:00
Siyao Meng
a308a1ec22
HDFS-15614. Initialize snapshot trash root during NameNode startup if enabled (#2370) 2020-10-13 10:59:42 -07:00
Gabor Bota
59f01a548e
HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same 
Contributed by Gabor Bota.
 2020-10-13 16:17:44 +01:00
Mukund Thakur
0507c4160f
HDFS-15626. TestWebHDFS.testLargeDirectory failing (#2380) 
Fixes the regression caused by HADOOP-17281, where the WebHDFS client
listStatusIterator (correctly) throws NoSuchElementException when next()
runs out of values.

Contributed by Mukund Thakur.

Change-Id: I6cec41c20467920cf21f169653553535414b2680
 2020-10-13 13:31:00 +01:00
Steve Loughran
d606ddc12c
Revert "HADOOP-17303. TestWebHDFS.testLargeDirectory failing (#2380)" 
(reverting to change the JIRA number)

This reverts commit 3e1b1dfd77.

Change-Id: I8261357f38268b5d7c1c13f6ef366d037be972dd
 2020-10-13 13:30:38 +01:00
Mukund Thakur
3e1b1dfd77
HADOOP-17303. TestWebHDFS.testLargeDirectory failing (#2380) 
Fixes the regression caused by HADOOP-17281, where the WebHDFS client
listStatusIterator (correctly) throws NoSuchElementException when next()
runs out of values.

Contributed by Mukund Thakur.
 2020-10-13 13:27:55 +01:00
Konstantin V Shvachko
b3786d6c3c HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 2020-10-12 17:26:49 -07:00
Uma Maheswara Rao G
b76b36ebbc
HDFS-15625: Namenode trashEmptier should not init ViewFs on startup (#2378). Contributed by Uma Maheswara Rao G. 2020-10-12 14:22:44 -07:00
Akira Ajisaka
69ef9b1ee8
HDFS-15620. RBF: Fix test failures after HADOOP-17281 (#2375) 2020-10-12 09:52:12 +09:00
LeonGao91
de055ec8aa
HDFS-15456. TestExternalStoragePolicySatisfier fails intermittently (#2299) 2020-10-10 22:08:36 +05:30
Hui Fei
518a212cff
HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer (#2363) 2020-10-09 16:12:22 +08:00
Karthik Palanisamy
735e85a1d2
HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6 (#2365) 2020-10-08 12:51:04 +05:30
Karthik Palanisamy
df4006eb81
HDFS-15253 Default checkpoint transfer speed, 50mb per second (#2366) 2020-10-07 22:09:04 +05:30
bshashikant
16aea11c94
HDFS-15611. Add list Snapshot command in WebHDFS. (#2355) 2020-10-07 10:34:32 +05:30
Akira Ajisaka
074f0d46af
HDFS-15613. RBF: Router FSCK fails after HDFS-14442. (#2360) 
* Support getHAServiceState in DFSRouter
 2020-10-07 13:37:45 +09:00
hemanthboyina
921ca1f554 HDFS-15543. RBF: Write Should allow, when a subcluster is unavailable for RANDOM mount points with fault Tolerance enabled. Contributed by Hemanth Boyina. 2020-10-07 09:58:53 +05:30
Doroszlai, Attila
1cfe5916e2 HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-06 14:58:42 -07:00
Fei Hui
6ece6407e1
HDFS-15458. TestNameNodeRetryCacheMetrics fails intermittently (#2356). Contributed by Hui Fei. 2020-10-05 22:11:24 +05:30
Siyao Meng
43b0c0b054
HDFS-15607. Create trash dir when allowing snapshottable dir (#2352) 2020-10-05 05:02:00 -07:00
huangtianhua
a89ca56a1b
HDFS-15600 Fix TestRouterQuota#testStorageTypeQuota (#2347) 2020-09-29 19:00:55 -07:00
Ayush Saxena
d686a655bc HDFS-15603. RBF: Fix getLocationsForPath twice in create operation. Contributed by wangzhaohui. 2020-09-29 00:03:11 +05:30
Ayush Saxena
fc8a6dd8a9 HDFS-15591. RBF: Fix webHdfs file display error. Contributed by wangzhaohui. 2020-09-28 23:55:31 +05:30
Fei Hui
984875c624
HDFS-15604. Fix Typo for HdfsDataNodeAdminGuide doc (#2345) 2020-09-28 09:54:01 -07:00
Inigo Goiri
eae0035a2d HDFS-15530. RBF: Fix typo in DFS_ROUTER_QUOTA_CACHE_UPDATE_INTERVAL var definition. Contributed by Sha Fanghao. 2020-09-28 09:40:39 -07:00
Takanobu Asanuma
d89af79507
HDFS-15577. Refactor TestTracing. (#2302) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2020-09-28 18:27:41 +09:00
Vinayakumar B
82b86e3754
HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng 2020-09-27 19:27:13 +05:30
Uma Maheswara Rao G
899dea2a21
HDFS-15598: ViewHDFS#canonicalizeUri should not be restricted to DFS only API. (#2339). Contributed by Uma Maheswara Rao G. 
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
 2020-09-25 21:21:01 -07:00
Ye Ni
00c4de63cf
HDFS-15594. Lazy calculate live datanodes in safe mode tip (#2332) 2020-09-25 09:47:54 -07:00
Uma Maheswara Rao G
3ccc962b99
HDFS-15596: ViewHDFS#create(f, permission, cflags, bufferSize, replication, blockSize, progress, checksumOpt) should not be restricted to DFS only. (#2333). Contributed by Uma Maheswara Rao G. 
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
 2020-09-24 07:07:48 -07:00
Aryan Gupta
486ddb73f9
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2020-09-24 21:15:55 +09:00
huangtianhua
ff59fbb8b0
HDFS-15025. Applying NVDIMM storage media to HDFS (#2189) 
* HDFS-15025. Applying NVDIMM storage media to HDFS
Co-authored-by: YaYun-Wang <yywangyayun@163.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>
 2020-09-24 14:27:04 +05:30
bshashikant
368f2f637e
HDFS-15590. namenode fails to start when ordered snapshot deletion feature is disabled (#2326) 2020-09-24 14:00:41 +05:30
Ye Ni
ae089f2db7
HDFS-15557. Log the reason why a storage log file can't be deleted (#2274) 2020-09-22 13:23:20 -07:00
Hui Fei
474fa80bfb
HADOOP-17277. Correct spelling errors for separator (#2322) 
Contributed by Hui Fei.
 2020-09-22 17:10:33 +01:00
Kihwal Lee
dfc2682213 HDFS-15581. Access Controlled HttpFS Server. Contributed by Richard Ross. 2020-09-22 10:53:04 -05:00
lfengnan
3e8b1e7426
HDFS-15554. RBF: force router check file existence in destinations before adding/updating mount points (#2266). Contributed by Fengnan Li. 2020-09-21 23:24:13 +05:30