litao
f5e15dfb11
HDFS-16344. Improve DirectoryScanner.Stats#toString ( #3695 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 829af89dc4
)
2021-11-29 17:54:58 +09:00
litao
b428d534fb
HDFS-16339. Show the threshold when mover threads quota is exceeded ( #3689 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 86705eda82
)
2021-11-26 19:53:34 +09:00
Akira Ajisaka
4a032bc88d
Revert "HDFS-16336. Addendum: De-flake TestRollingUpgrade#testRollback ( #3686 )"
...
This reverts commit 3b79e0f794
.
2021-11-25 14:24:30 +09:00
Viraj Jasani
6094e1ec9a
HDFS-16171. De-flake testDecommissionStatus ( #3280 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6342d5e523
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDecommissioningStatus.java
2021-11-25 14:15:10 +09:00
Viraj Jasani
a9125bba24
HDFS-16350. Datanode start time should be set after RPC server starts successfully ( #3711 )
...
(cherry picked from commit cdc13e91b6
)
2021-11-25 10:28:40 +08:00
Ayush Saxena
3b79e0f794
HDFS-16336. Addendum: De-flake TestRollingUpgrade#testRollback ( #3686 )
2021-11-25 01:54:14 +05:30
Viraj Jasani
450efaeabe
HDFS-16336. De-flake TestRollingUpgrade#testRollback ( #3686 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e8566b3812
)
2021-11-24 22:56:23 +09:00
Mukul Kumar Singh
97de402970
HDFS-16343. Add some debug logs when the dfsUsed are not used during Datanode startup. ( #3694 )
...
(cherry picked from commit faa4eeacb1
)
2021-11-23 13:39:16 +00:00
litao
d746e2cdd8
HDFS-16337. Show start time of Datanode on Web ( #3682 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
(cherry picked from commit 0ed817babb
)
2021-11-23 15:43:02 +08:00
litao
90085eb4cb
HDFS-16335. Fix HDFSCommands.md ( #3681 )
...
(cherry picked from commit 20b78c8f97
)
2021-11-22 17:14:17 +09:00
GuoPhilipse
6b78a753e3
HDFS-16334. Correct NameNode ACL description ( #3678 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3d1ca6f6dc
)
2021-11-22 07:45:42 +09:00
litao
166c38bcbf
HDFS-16326. Simplify the code for DiskBalancer ( #3663 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 965e3874e9
)
2021-11-19 11:39:06 +09:00
GuoPhilipse
8166bcc668
HDFS-16328. Correct disk balancer desc ( #3669 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 0c1eb0125b
)
2021-11-19 01:02:31 +09:00
Viraj Jasani
2f587bf15a
HDFS-16330. Fix incorrect placeholder for Exception logs in DiskBalancer ( #3672 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 38614d05e1
)
2021-11-19 00:56:13 +09:00
litao
4036213273
HDFS-16329. Fix log format for BlockManager ( #3670 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit dc751df63b
)
2021-11-19 00:43:25 +09:00
Viraj Jasani
4e77f772fc
HDFS-16323. DatanodeHttpServer doesn't require handler state map while retrieving filter handlers ( #3659 )
...
(cherry picked from commit 646c470e5d
)
2021-11-17 22:19:38 +09:00
litao
026d5860cb
HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode ( #3666 )
2021-11-17 11:06:53 +09:00
Chao Sun
e079fa6577
Preparing for 3.3.3 development
2021-11-16 16:02:34 -08:00
Neil
c182bf65e0
HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. ( #3320 )
...
(cherry picked from commit 5626734a36
)
2021-11-16 22:59:25 +08:00
Ahmed Hussein
25bcc7e3ee
HDFS-15457. TestFsDatasetImpl fails intermittently ( #2407 )
2021-11-16 11:51:35 +09:00
GuoPhilipse
b11e017777
HDFS-16321. Fix invalid config in TestAvailableSpaceRackFaultTolerantBPP ( #3655 ). Contributed by guo.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-11-14 20:00:28 +05:30
Wei-Chiu Chuang
5333e872e2
HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing ( #3340 ) ( #3640 )
...
(cherry picked from commit 356ebbbe80
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
Change-Id: If76fd0d77fafc90fe2f2c19ab1d0c43a58510f6b
Co-authored-by: bshashikant <shashikant@apache.org>
2021-11-12 14:32:02 +08:00
litao
c4e8826194
HDFS-16311. Metric metadataOperationRate calculation error in DataNodeVolumeMetrics ( #3636 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 1c1cf64616
)
2021-11-12 10:25:40 +09:00
litao
808da62cfa
HDFS-16312. Fix typo for DataNodeVolumeMetrics and ProfilingFileIoEvents ( #3637 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 2a1a11c039
)
2021-11-10 19:23:58 +09:00
litao
0db04fab20
HDFS-16298. Improve error msg for BlockMissingException ( #3615 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit ea65fc26d8
)
2021-11-10 19:18:18 +09:00
jianghuazhu
46e69715b5
HDFS-16301. Improve BenchmarkThroughput#SIZE naming standardization. ( #3622 )
...
(cherry picked from commit b307f18a58
)
2021-11-10 11:40:08 +09:00
litao
26b7bc75ed
HDFS-16299. Fix bug for TestDataNodeVolumeMetrics#verifyDataNodeVolumeMetrics ( #3616 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit cbc7fb6bca
)
2021-11-09 17:09:05 +09:00
jianghuazhu
4d04efb900
HDFS-16294.Remove invalid DataNode#CONFIG_PROPERTY_SIMULATED. ( #3605 )
...
(cherry picked from commit 72718a8dc1
)
2021-11-04 16:50:31 +09:00
daimin
2844b98558
HDFS-16286. Add a debug tool to verify the correctness of erasure coding on file ( #3593 )
...
(cherry picked from commit a21895a5b3
)
2021-11-03 20:41:55 +00:00
Stephen O'Donnell
9cf841b1a6
HDFS-16259. Catch and re-throw sub-classes of AccessControlException thrown by any permission provider plugins (eg Ranger) ( #3598 )
...
(cherry picked from commit 2f35cc36cd
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeAttributeProvider.java
2021-11-02 11:17:59 +00:00
jianghuazhu
5fbd9d333e
HDFS-16269. [Fix] Improve NNThroughputBenchmark#blockReport operation. ( #3544 )
...
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 618fea27d2
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNNThroughputBenchmark.java
2021-11-02 01:04:33 +09:00
litao
7e57fc053a
HDFS-16279. Print detail datanode info when process first storage report ( #3564 )
...
(cherry picked from commit 6aa664c624
)
2021-10-29 10:28:40 +09:00
daimin
847cff7d91
HDFS-16282. Remove duplicate generic usage information to hdfs debug command ( #3576 ). Contributed by daimin.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-10-22 21:16:06 +05:30
GuoPhilipse
c4f1db1966
HDFS-16277. improve decision in AvailableSpaceBlockPlacementPolicy ( #3559 ). Contributed by guo.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-10-22 08:02:21 +05:30
Michael Kuchenbecker
78723e045a
HDFS-7612: Fix default cache directory in TestOfflineEditsViewer.testStored. Contributed by Michael Kuchenbecker ( #3571 )
...
(cherry picked from commit a73ff6915a
)
2021-10-21 13:48:38 -07:00
daimin
6019239825
HDFS-16272. Fix int overflow in computing safe length during EC block recovery ( #3548 )
...
(cherry picked from commit 5337bebcc5
)
2021-10-18 12:02:26 +01:00
Ayush Saxena
5b1cf17f88
HDFS-16271. RBF: NullPointerException when setQuota through routers with quota disabled. Contributed by Chengwei Wang.
2021-10-16 14:35:53 +05:30
Masatake Iwasaki
9e2936f8d1
HADOOP-17424. Replace HTrace with No-Op tracer ( #3520 )
...
(cherry picked from commit 1a205cc3ad
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java
Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>
2021-10-12 00:07:09 +09:00
langlaile1221
a35a6ec06b
HDFS-16241. Standby close reconstruction thread ( #3493 )
...
Co-authored-by: zhanghuazong <Zhz@20068867>
(cherry picked from commit 88d8c3d0c8
)
2021-10-11 16:22:46 +08:00
Viraj Jasani
77ee5a4266
HADOOP-17950. Provide replacement for deprecated APIs of commons-io IOUtils ( #3515 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 8071dbb9c6
)
2021-10-07 11:00:19 +09:00
S O'Donnell
f437b3bdaf
HDFS-16252. Correct docs for dfs.http.client.retry.policy.spec. Contributed by Stephen O'Donnell
...
(cherry picked from commit a30231ff8e
)
2021-10-06 15:39:34 +01:00
AngersZhuuuu
4475d8bfe7
HDFS-16235. Fix Deadlock in LeaseRenewer for static remove method ( #3472 )
...
(cherry picked from commit 5f9321a5d4
)
2021-09-29 18:36:36 +08:00
He Xiaoqiao
06954af6f0
HDFS-14575. LeaseRenewer#daemon threads leak in DFSClient. Contributed by Renukaprasad C.
...
Co-authored-by: Tao Yang <taoyang1@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 10b79a26fe
)
2021-09-29 18:36:36 +08:00
Wei-Chiu Chuang
bb08de559a
HDFS-16233. Do not use exception handler to implement copy-on-write for EnumCounters. ( #3468 )
...
(cherry picked from commit 87632bbacf
)
2021-09-24 09:13:15 -07:00
Akira Ajisaka
f1c0dc84cd
HDFS-15977. Call explicit_bzero only if it is available. ( #2914 )
...
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
(cherry picked from commit f0241ec216
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/x-platform/syscall_linux.cc
2021-09-24 11:50:12 +09:00
Chao Sun
ff26a7700d
Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same ( #2383 )"
...
This reverts commit 54c40cbf49
.
2021-09-23 15:04:27 -07:00
Ayush Saxena
80d2bda51c
HDFS-15288. Add Available Space Rack Fault Tolerant BPP. Contributed by Ayush Saxena.(Addendum)
2021-09-23 10:33:16 +05:30
Ayush Saxena
38f529abf7
HDFS-16223. AvailableSpaceRackFaultTolerantBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. ( #3424 ). Contributed by Ayush Saxena.
2021-09-23 08:54:01 +05:30
Ayush Saxena
3355126062
HDFS-15288. Add Available Space Rack Fault Tolerant BPP. Contributed by Ayush Saxena.
2021-09-23 08:52:20 +05:30
wangzhaohui
2f73ac1c14
HDFS-16181. [SBN Read] Fix display of JournalNode metric RpcRequestCacheMissAmount ( #3317 )
...
Co-authored-by: wangzhaohui8 <wangzhaohui8@jd.com>
(cherry picked from commit 232fd7cae170de8c6b52c14841a47dca8735c6d2)
2021-09-15 10:02:13 -07:00