349 Commits

Author SHA1 Message Date
Eric Yin
de41ce8a16
HDFS-16087. Fix stuck issue in rbfbalance tool (). Contributed by Eric Yin. 2021-07-21 00:01:55 +08:00
Viraj Jasani
c488abbc79
HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects ()
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-06-21 10:25:12 +09:00
Viraj Jasani
d9eccb1eb3
HDFS-16080. RBF: Invoking method in all locations should break the loop after successful result (). Contributed by Viraj Jasani.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 17:19:05 +05:30
Viraj Jasani
4ef27a596f
HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom () 2021-06-11 12:15:52 +09:00
Viraj Jasani
eefa664fea
HDFS-16054. Replace Guava Lists usage by Hadoop's own Lists in hadoop-hdfs-project () 2021-06-09 15:12:48 +09:00
litao
c748fce17a
HDFS-16048. RBF: Print network topology on the router web ()
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2021-06-08 16:14:06 +09:00
Ayush Saxena
b8ab19373d Revert "HDFS-15982. Deleted data using HTTP API should be saved to the trash ()"
This reverts commit 041488e8f533a27e9497e86bb779c0b82c2e1d71.
2021-06-01 08:38:13 +05:30
Jinglun
e90c41af34 HDFS-15973. RBF: Add permission check before doing router federation rename. Contributed by Jinglun.
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-05-30 19:28:40 +08:00
Jinglun
2b11e160e2 Revert "HDFS-15973." because the user.name and user.email message is not correct.
This reverts commit b7f69467c19f6114b931ca2377b0ed7a8ada609e.
2021-05-30 19:27:10 +08:00
李经纶
b7f69467c1 HDFS-15973. RBF: Add permission check before doing router federation rename. Contributed by Jinglun.
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-05-30 18:50:03 +08:00
Viraj Jasani
986d0a4f1d
HADOOP-17732. Keep restrict-imports-enforcer-rule for Guava Sets in hadoop-main pom ()
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-05-26 17:14:31 +09:00
zhuxiangyi
e18d806212
HDFS-16024: RBF: Rename data to the Trash should be based on src loca… () 2021-05-26 13:41:49 +08:00
Viraj Jasani
028ec4704b
HADOOP-17720. Replace Guava Sets usage by Hadoop's own Sets in HDFS ()
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-05-25 10:55:58 +09:00
lfengnan
43bf009112
HDFS-15757 RBF: Improving Router Connection Management () 2021-05-19 10:53:42 -07:00
Jinglun
1b6994287a HDFS-15923. RBF: Authentication failed when rename accross sub clusters (). Contributed by zhuobin zheng.
Reviewed-by: Jinglun <jinglun@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-05-08 19:50:17 +08:00
Viraj Jasani
041488e8f5
HDFS-15982. Deleted data using HTTP API should be saved to the trash ()
Reviewed-by: Siyao Meng <siyao@apache.org>
2021-05-04 01:31:23 -07:00
Akira Ajisaka
7c561e2a02
HDFS-15952. TestRouterRpcMultiDestination#testProxyGetTransactionID and testProxyVersionRequest are flaky. () 2021-05-04 00:25:29 +09:00
lfengnan
6e525ab81c
HDFS-15810. RBF: RBFMetrics's TotalCapacity out of bounds ()
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-05-02 19:18:47 +09:00
Wei-Chiu Chuang
f1e1809029
HADOOP-17653. Do not use guava's Files.createTempDir(). ()
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-05-02 11:10:32 +09:00
lfengnan
552e9dcc6c
HDFS-15561. RBF: Remove NPE when local namenode is not configured (). Contributed by Fengnan Li.
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-04-29 21:14:18 +08:00
Wei-Chiu Chuang
b2e54762a4
HDFS-15624. fix the function of setting quota by storage type () ()
1. puts NVDIMM to the end of storage type enum to make sure compatibility.
2. adds check to make sure the software layout version is satisfied

Co-authored-by: su xu <kevinbrandon@163.com>
Co-authored-by: huangtianhua <huangtianhua223@gmail.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>

Change-Id: I3c58beef50730827a09b3c968e9ad637baa57d44
2021-04-28 23:54:39 -07:00
litao
b968fa0957
HDFS-15991. Add location into datanode info for NameNodeMXBean ()
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-26 18:38:43 +09:00
Wei-Chiu Chuang
90c6caf650 Revert "HDFS-15624. fix the function of setting quota by storage type ()"
This reverts commit 394b9f7a5c42ffa38bf7c3a44e7d50a4d19414f9.

Ref: HDFS-15995.
Had to revert this commit, so we can commit HDFS-15566 (a critical bug preventing rolling upgrade to Hadoop 3.3)
Will re-work this fix again later.
2021-04-26 11:27:15 +08:00
zhuxiangyi
dff95c5eca
HDFS-15974. RBF: Unable to display the datanode UI of the router ()
Co-authored-by: zhuxiangyi <zhuxiangyi@360.cn>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-23 10:19:11 +09:00
lfengnan
17be99f9f9
HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters () 2021-04-15 16:11:47 -07:00
Ayush Saxena
7d6f0ca0bd Revert "HDFS-15884. RBF: Remove unused method getCreateLocation in RouterRpcServer (). Contributed by tomscut."
This reverts commit e565b05c80c731898a54f8c1d358c12090ca5bbf.

The removed method needs to be used in HDFS-15423.
2021-04-13 23:28:42 +05:30
Inigo Goiri
82462739f8
Revert "HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters ()" ()
This reverts commit cb3ed32fe0d937980999d590deb35ad97d40f9e6.
2021-04-12 20:44:31 -07:00
lfengnan
cb3ed32fe0
HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters () 2021-04-12 12:42:33 -07:00
Viraj Jasani
4f28738010
HDFS-15931 : Fix non-static inner classes for better memory management (). Contributed by Viraj Jasani
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2021-04-01 16:34:31 -07:00
He Xiaoqiao
da4ceba4aa
HDFS-15863. RBF: Validation message to be corrected in FairnessPolicyController. Contributed by Renukaprasad C. 2021-03-31 13:00:13 +08:00
Harunobu Daikoku
ea6595d3b6
HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode. ()
Co-authored-by: Harunobu Daikoku <hdaikoku@yahoo-corp.jp>
2021-03-29 11:43:29 +09:00
litao
e565b05c80
HDFS-15884. RBF: Remove unused method getCreateLocation in RouterRpcServer (). Contributed by tomscut. 2021-03-14 04:07:31 +05:30
JJiaguodong
8af56de1fa
HADOOP-17560. Fix some spelling errors ()
Co-authored-by: jiaguodong5 <jiaguodong5@jd.com>
2021-03-03 11:41:05 +09:00
Ayush Saxena
2ce5752fa8 HDFS-15839. RBF: Cannot get method setBalancerBandwidth on Router Client. Contributed by Yang Yun. 2021-02-20 19:34:29 +05:30
Akira Ajisaka
0ed9e36282
HDFS-15834. Remove the usage of org.apache.log4j.Level () 2021-02-17 01:59:24 -05:00
Akira Ajisaka
2b3c5b1733
HDFS-15836. RBF: Fix contract tests after HADOOP-13327 ()
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2021-02-16 14:50:50 +09:00
Zehao Chen
07a4220cd2
HDFS-15821. Add metrics for in-service datanodes (). Contributed by Zehao Chen. 2021-02-15 11:14:32 -06:00
YaYun-Wang
394b9f7a5c
HDFS-15624. fix the function of setting quota by storage type ()
1. puts NVDIMM to the end of storage type enum to make sure compatibility.
2. adds check to make sure the software layout version is satisfied

Co-authored-by: su xu <kevinbrandon@163.com>
Co-authored-by: huangtianhua <huangtianhua223@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
2021-02-02 22:44:34 -08:00
Ayush Saxena
e40f99f6d5 HDFS-15767. RBF: Router federation rename of directory. Contributed by Jinglun. 2021-01-26 14:25:27 +05:30
Yiqun Lin
0d7ac54510 HDFS-14558. RBF: Isolation/Fairness documentation. Contributed by Fengnan Li. 2021-01-12 23:38:55 +08:00
Ayush Saxena
2ba7ec2b48 HDFS-15766. RBF: MockResolver.getMountPoints() breaks the semantic of FileSubclusterResolver. Contributed by Jinglun. 2021-01-08 16:27:54 +05:30
Ayush Saxena
77299ae992 HDFS-15748. RBF: Move the router related part from hadoop-federation-balance module to hadoop-hdfs-rbf. Contributed by Jinglun. 2021-01-05 00:05:03 +05:30
Ayush Saxena
5ab1db9e21 HDFS-15380. RBF: Could not fetch real remote IP in RouterWebHdfsMethods. Contributed by tomscut. 2020-12-21 09:58:30 +05:30
Ayush Saxena
2aea43bf4f HDFS-15655. Add option to make balancer prefer to get cold blocks. Contributed by Yang Yun. 2020-12-20 23:06:19 +05:30
Ayush Saxena
7dda804a1a HDFS-14090. RBF: Improved isolation for downstream name nodes. {Static}. Contributed by Fengnan Li and CR Hota. 2020-12-04 22:56:37 +05:30
Masatake Iwasaki
6a1d7d9ed2
HDFS-15677. TestRouterRpcMultiDestination#testGetCachedDatanodeReport fails on trunk. () 2020-12-01 08:06:52 +09:00
Peter Bacsko
cd0490e8c6 Revert "YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak."
This reverts commit b0ab222a6c0073c31c285e89e43508b3e0ca9576.
2020-11-11 17:06:12 +01:00
Peter Bacsko
b0ab222a6c YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak. 2020-11-11 17:03:16 +01:00
Hui Fei
f1d5652359 HDFS-15668. RBF: Fix RouterRPCMetrics annocation and document misplaced error. Contributed by Hongbing Wang. 2020-11-10 13:09:03 +08:00
Akira Ajisaka
b3ba74d72d
HDFS-15657. RBF: TestRouter#testNamenodeHeartBeatEnableDefault fails by BindException () 2020-10-28 19:12:23 +09:00