He Xiaoqiao
cb975c3df6
HDFS-16534. Split FsDatasetImpl from block pool locks to volume grain locks. ( #4141 ) Contributed by limingxiang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-04-17 19:21:27 +08:00
Renukaprasad C
f14f305051
HDFS-16526. Add metrics for slow DataNode ( #4162 )
2022-04-15 21:37:05 +05:30
caozhiqiang
cee8c62498
HDFS-16456. EC: Decommission a rack with only on dn will fail when the rack number is equal with replication ( #4126 )
2022-04-14 18:42:39 +09:00
daimin
c65c383b7e
HDFS-16509. Fix decommission UnsupportedOperationException ( #4077 ). Contributed by daimin.
2022-04-14 11:07:06 +08:00
Takanobu Asanuma
2efab92959
HDFS-16479. EC: NameNode should not send a reconstruction work when the source datanodes are insufficient ( #4138 )
2022-04-14 11:23:38 +09:00
Steve Loughran
a7b4e8f03e
HDFS-14478. Add libhdfs APIs for openFile ( #4166 )
...
Contributed by Sahil Takiar
2022-04-13 14:15:27 +01:00
qinyuren
45394433a1
HDFS-16484. [SPS]: Fix an infinite loop bug in SPSPathIdProcessor thread ( #4032 )
2022-04-13 11:27:37 +09:00
Gautham B A
e89fd9645b
HDFS-16474. Make HDFS tail tool cross platform ( #4157 )
...
* The source files for hdfs_tail
uses getopt for parsing the
command line arguments.
* getopt is available only on
Linux and thus, isn't cross
platform.
* We need to replace getopt
with boost::program_options
to make these tools cross
platform.
2022-04-12 20:31:24 +05:30
Gautham B A
d5e97fe4d6
HDFS-16473. Make HDFS stat tool cross platform ( #4145 )
...
* The source files for hdfs_stat
uses getopt for parsing the
command line arguments.
* getopt is available only on
Linux and thus, isn't cross platform.
* We need to replace getopt with
boost::program_options to make
this tool cross platform.
2022-04-08 23:06:39 +05:30
Takanobu Asanuma
37650ced81
HDFS-16497. EC: Add param comment for liveBusyBlockIndices with HDFS-14768. Contributed by caozhiqiang.
2022-04-08 18:38:40 +09:00
litao
5412fbf6d4
HDFS-16460. [SPS]: Handle failure retries for moving tasks ( #4001 )
2022-04-08 12:26:39 +08:00
singer-bin
807a428b55
HDFS-16457.Make fs.getspaceused.classname reconfigurable ( #4069 )
2022-04-08 10:27:35 +09:00
Viraj Jasani
f70935522b
HADOOP-18188. Support touch command for directory ( #4135 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-04-07 17:29:45 +09:00
Viraj Jasani
7c20602b17
HDFS-16522. Set Http and Ipc ports for Datanodes in MiniDFSCluster ( #4108 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-04-06 18:17:02 +09:00
wangzhaohui
61bbdfd3a7
HDFS-16529. Remove unnecessary setObserverRead in TestConsistentReadsObserver ( #4131 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-04-06 17:29:43 +09:00
Stephen O'Donnell
bbfe3500cf
HDFS-16530. setReplication debug log creates a new string even if debug is disabled ( #4142 )
2022-04-06 08:49:52 +01:00
Gautham B A
4ef1d3eef9
HDFS-16472. Make HDFS setrep tool cross platform ( #4130 )
...
* The source files for hdfs_setrep
uses getopt for parsing the
command line arguments.
* getopt is available only on Linux
and thus, isn't cross platform.
* We need to replace getopt
with boost::program_options
to make this tool cross platform.
2022-04-05 22:59:11 +05:30
litao
34b3275bf4
HDFS-16477. [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS ( #4009 ). Contributed by tomscut.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-04-03 01:36:03 +05:30
Xing Lin
15a5ea2c95
HADOOP-18169. getDelegationTokens in ViewFs should also fetch the token from fallback FS ( #4094 )
...
HADOOP-18169. getDelegationTokens in ViewFs should also fetch the token from the fallback FS
2022-03-31 21:59:09 +00:00
litao
9a4dddd640
HDFS-16507. [SBN read] Avoid purging edit log which is in progress ( #4082 )
2022-03-30 23:01:48 -07:00
He Xiaoqiao
2bf78e2416
HDFS-16511. Improve lock type for ReplicaMap under fine-grain lock mode. ( #4085 ). Contributed by limingxiang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-03-31 14:00:38 +08:00
litao
ac50657c37
HDFS-16413. Reconfig dfs usage parameters for datanode ( #3863 )
2022-03-31 00:51:17 +09:00
He Xiaoqiao
6eea28c3f3
HDFS-16498. Fix NPE for checkBlockReportLease #4057 . Contributed by tomscut.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-03-30 14:56:04 +08:00
Lei Yang
a9b43966c0
HDFS-16518: Add shutdownhook to invalidate the KeyProviders in the cache
...
Fixes #4100
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-03-28 13:15:28 -07:00
Masatake Iwasaki
0fbd96a244
Make upstream aware of 3.2.3 release.
2022-03-28 08:07:50 +00:00
GuoPhilipse
046a6204b4
HDFS-16355. Improve the description of dfs.block.scanner.volume.bytes.per.second ( #3724 )
...
Co-authored-by: gf13871 <gf13871@ly.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-03-27 22:23:48 +09:00
litao
565e848d88
HDFS-16434. Add opname to read/write lock for remaining operations ( #3915 )
2022-03-25 20:19:01 +09:00
qinyuren
45ce1cce50
HDFS-16501. Print the exception when reporting a bad block ( #4062 )
...
Reviewed-by: tomscut <litao@bigo.sg>
2022-03-23 15:03:17 +09:00
He Xiaoqiao
ef8bff0df9
HDFS-15987. Improve oiv tool to parse fsimage file in parallel with delimited format. ( #2918 ). Contributed by Hongbing Wang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-03-23 11:32:12 +08:00
He Xiaoqiao
26ba3846cc
Revert "HDFS-14617. Improve oiv tool to parse fsimage file in parallel with delimited format. ( #2918 ). Contributed by Hongbing Wang."
...
This reverts commit 88975496d8
.
2022-03-23 11:01:35 +08:00
Gautham B A
81879eb9cb
HDFS-16471. Make HDFS ls tool cross platform ( #4086 )
...
* The source files for hdfs_ls
uses getopt for parsing the
command line arguments.
* getopt is available only on
Linux and thus, isn't cross
platform.
* Thus, we need to replace
getopt with
boost::program_options to
make this tool cross platform.
2022-03-22 23:12:00 +05:30
He Xiaoqiao
88975496d8
HDFS-14617. Improve oiv tool to parse fsimage file in parallel with delimited format. ( #2918 ). Contributed by Hongbing Wang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-03-22 22:28:09 +08:00
Owen O'Malley
1d5650c4d0
HDFS-13248: Namenode needs to use the actual client IP when going through the
...
RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures
the list of users than can set their client ip address using the client context.
Fixes #4081
2022-03-21 09:27:35 -07:00
litao
e5549a2a68
HDFS-16503. Should verify whether the path name is valid in the WebHDFS ( #4067 ). Contributed by tomscut.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-03-21 12:40:51 +05:30
He Xiaoqiao
62d59e516e
HDFS-16504. Add parameter for NameNode to process getBloks request. ( #4068 ). Contributed by Max Xie.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-03-20 14:01:45 +08:00
Gautham B A
a631f45a99
HDFS-16470. Make HDFS find tool cross platform ( #4076 )
...
* The source files for hdfs_find uses
getopt for parsing the command
line arguments. getopt is available
only on Linux and thus, isn't cross
platform.
* Thus, we need to replace getopt
with boost::program_options to
make hdfs_find cross platform.
2022-03-18 21:41:01 +05:30
litao
7f6a891f03
HDFS-16499. [SPS]: Should not start indefinitely while another SPS process is running ( #4058 )
2022-03-17 12:04:48 +08:00
He Xiaoqiao
a237526988
HDFS-16494.Removed reuse of AvailableSpaceVolumeChoosingPolicy#initLocks(). ( #4048 ). Contributed by JiangHua Zhu.
2022-03-16 20:43:38 +08:00
Viraj Jasani
1c0bc35305
HDFS-16502. Reconfigure Block Invalidate limit ( #4064 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-03-16 09:32:29 +08:00
Xing Lin
8b8158f02d
HADOOP-18144: getTrashRoot in ViewFileSystem should return a path in ViewFS.
...
To get the new behavior, define fs.viewfs.trash.force-inside-mount-point to be true.
If the trash root for path p is in the same mount point as path p,
and one of:
* The mount point isn't at the top of the target fs.
* The resolved path of path is root (eg it is the fallback FS).
* The trash root isn't in user's target fs home directory.
get the corresponding viewFS path for the trash root and return it.
Otherwise, use <mnt>/.Trash/<user>.
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-03-14 11:29:48 -07:00
Owen O'Malley
7b5eac27ff
HDFS-16495: RBF should prepend the client ip rather than append it.
...
Fixes #4054
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-03-14 10:21:35 -07:00
He Xiaoqiao
a32cfc2169
HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks. ( #3941 ). Contributed by limingxiang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: litao <tomleescut@gmail.com>
2022-03-12 18:40:09 +08:00
Stephen O'Donnell
49e4e4d0a5
HDFS-16496. Snapshot diff on snapshotable directory fails with not snapshottable error ( #4051 )
2022-03-08 11:07:11 +00:00
Viraj Jasani
66b72406bd
HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies ( #4000 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-03-08 17:27:04 +09:00
Gautham B A
8f07081789
HDFS-16462. Make HDFS get tool cross platform ( #4003 )
2022-03-05 23:05:14 +05:30
Gautham B A
19561846df
HADOOP-18151. Switch the baseurl for Centos 8 ( #4047 )
...
* Centos 8 has reached its
End-of-Life and thus its
packages are no longer
accessible from
mirror.centos.org.
* This PR switches the baseurl
to vault.centos.org where
the packages are archived.
2022-03-05 07:11:22 +05:30
Viraj Jasani
278568203b
HDFS-16481. Provide support to set Http and Rpc ports in MiniJournalCluster ( #4028 ). Contributed by Viraj Jasani.
2022-03-04 22:17:48 +05:30
Chao Sun
f800b65b40
Make upstream aware of 3.3.2 release
2022-03-02 19:14:50 -08:00
litao
9199787215
HDFS-16458. [SPS]: Fix bug for unit test of reconfiguring SPS mode ( #4041 )
2022-03-02 11:25:50 +08:00
SevenAddSix
5eab9719cb
HDFS-16480. Fix typo: indicies -> indices ( #4020 )
2022-02-28 21:12:18 +08:00