Owen O'Malley
1175acbb75
HDFS-13248: Namenode needs to use the actual client IP when going through the
...
RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures
the list of users than can set their client ip address using the client context.
Fixes #4081
2022-03-21 10:24:54 -07:00
Owen O'Malley
c52b97d084
HDFS-16495: RBF should prepend the client ip rather than append it.
...
Fixes #4054
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-03-14 10:30:29 -07:00
litao
496657c63f
HDFS-16310. RBF: Add client port to CallerContext for Router ( #3635 )
...
Cherry-picked from 5b05068f by Owen O'Malley
2022-03-14 10:29:30 -07:00
Hui Fei
2479d4ab6c
HDFS-15630. RBF: Fix wrong client IP info in CallerContext when requests mount points with multi-destinations. Contributed by Chengwei Wang
...
Cherry-picked from 264c948e by Owen O'Malley
2022-03-14 10:29:04 -07:00
Hui Fei
8e129e5b8d
HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer ( #2363 )
...
Cherry-picked from 518a212c by Owen O'Malley
2022-03-14 10:28:55 -07:00
luoyuan3471
752a7b6d49
HADOOP-18044. Hadoop - Upgrade to jQuery 3.6.0 ( #3791 )
...
Co-authored-by: luoyuan <luoyuan@shopee.com>
(cherry picked from commit e2d620192aa0b712d05e4092eb63ef2ccdcd8220)
2022-02-11 23:18:25 +08:00
wangzhaohui
cc7b7e174c
HDFS-16391. Avoid evaluation of LOG.debug statement in NameNodeHeartbeatService ( #3820 )
...
(cherry picked from commit 8d251bd629c2db33f03c80e93c7e54fceedbfbe6)
2021-12-21 11:46:23 +00:00
Chao Sun
e079fa6577
Preparing for 3.3.3 development
2021-11-16 16:02:34 -08:00
Ayush Saxena
5b1cf17f88
HDFS-16271. RBF: NullPointerException when setQuota through routers with quota disabled. Contributed by Chengwei Wang.
2021-10-16 14:35:53 +05:30
Symious
c0f32f3cf8
HDFS-16221. RBF: Add usage of refreshCallQueue for Router ( #3421 )
...
(cherry picked from commit 7f6553af7546e512f51ba21cbd6cb4e7b73c1961)
2021-09-13 10:48:09 +08:00
Symious
8affaa6312
HDFS-16210. RBF: Add the option of refreshCallQueue to RouterAdmin ( #3379 )
...
(cherry picked from commit c0890e6d04dda6f2716d07427816721fbdf9c3b4)
2021-09-10 10:01:27 +08:00
Viraj Jasani
b8a98e4f82
HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects ( #3115 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit c488abbc79cc1ad2596cbf509a0cde14acc5ad6b)
2021-06-21 10:28:05 +09:00
Viraj Jasani
ef3a2a3b3b
HDFS-16080. RBF: Invoking method in all locations should break the loop after successful result ( #3121 ). Contributed by Viraj Jasani.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 17:23:52 +05:30
litao
02249171b1
HDFS-16048. RBF: Print network topology on the router web ( #3062 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c748fce17ace8b45ee0f3c3967d87893765eea61)
2021-06-08 16:59:59 +09:00
Wei-Chiu Chuang
86c28f0639
Revert "HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 ( #2959 )"
...
This reverts commit 4ffe5eb1ddb4aa260134005335a003ed6d270685.
2021-05-24 17:37:18 +08:00
Wei-Chiu Chuang
fa4915fdbb
Preparing for 3.3.2 development
2021-05-19 21:52:37 +08:00
Wei-Chiu Chuang
4ffe5eb1dd
HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 ( #2959 )
...
* HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
Change-Id: I0f31409923ece24a82dfba4c4610d8a38c52d9fb
* HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661 ). Contributed by Vinayakumar B.
(cherry picked from commit c4c0683dff577a91ca978939e182ec0fee65b7c3)
Co-authored-by: Xiaoyu Yao <xyao@apache.org>
Co-authored-by: Vinayakumar B <vinayakumarb@apache.org>
2021-05-17 18:57:46 -07:00
Aryan Gupta
28079e9c30
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 ( #2330 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 486ddb73f9999693177787e4abff7c932be9b925)
2021-05-13 12:16:17 +08:00
Akira Ajisaka
25244c4eec
HDFS-15952. TestRouterRpcMultiDestination#testProxyGetTransactionID and testProxyVersionRequest are flaky. ( #2970 )
...
(cherry picked from commit 7c561e2a02526280a2fe7b5a873e80b78cf4900c)
2021-05-04 00:26:34 +09:00
lfengnan
43fac739bb
HDFS-15810. RBF: RBFMetrics's TotalCapacity out of bounds ( #2910 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e525ab81cc2d93eb1e4aeea6b13769635076c06)
2021-05-02 19:19:55 +09:00
Wei-Chiu Chuang
670205c541
HADOOP-17653. Do not use guava's Files.createTempDir(). ( #2945 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f1e180902956fd547ad6a7b440588fdf583870c2)
2021-05-02 11:12:37 +09:00
litao
0e9042fed5
HDFS-15991. Add location into datanode info for NameNodeMXBean ( #2933 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit b968fa0957ddd0f59c9adfd068507722db7207c6)
2021-04-28 18:52:05 +09:00
Ayush Saxena
f809b863fe
HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena.
...
(cherry picked from commit 6e416a83d1e674ecd018d1db74a2d88e738deb40)
2021-04-28 18:51:43 +09:00
Ayush Saxena
3c7433f2a1
HDFS-15247. RBF: Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.
...
(cherry picked from commit 8d49229c3764a205f21750225005a2c9a8124ab9)
2021-04-28 18:51:37 +09:00
zhuxiangyi
2752156e3e
HDFS-15974. RBF: Unable to display the datanode UI of the router ( #2915 )
...
Co-authored-by: zhuxiangyi <zhuxiangyi@360.cn>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit dff95c5eca1c0d9a43b5734f93320fa51aee63da)
2021-04-23 11:22:52 +09:00
Ayush Saxena
2da3356428
HDFS-15591. RBF: Fix webHdfs file display error. Contributed by wangzhaohui.
...
(cherry picked from commit fc8a6dd8a92f8e18436d1a73d6e8ef808c583aee)
2021-04-02 17:13:00 +08:00
Takanobu Asanuma
f5cc1540b4
HDFS-15510. RBF: Quota and Content Summary was not correct in Multiple Destinations. Contributed by Hemanth Boyina.
...
(cherry picked from commit ca8e7a77256003e11ab7e3d079ee4cf9f50080dd)
2021-04-02 17:11:33 +08:00
Ayush Saxena
a549b4a82e
HDFS-15300. RBF: updateActiveNamenode() is invalid when RPC address is IP. Contributed by xuzq.
...
(cherry picked from commit 936bf09c3745cfec26fa9cfa0562f88b1f8be133)
2021-04-02 17:11:16 +08:00
Ayush Saxena
5bfc3a4c3a
HDFS-15198. RBF: Add test for MountTableRefresherService failed to refresh other router MountTableEntries in secure mode. Contributed by zhengchenyu.
...
(cherry picked from commit 8a9a674ef10a951c073ef17ba6db1ff07cff52cd)
2021-04-02 17:10:56 +08:00
Inigo Goiri
327488d688
HDFS-15196. RBF: RouterRpcServer getListing cannot list large dirs correctly. Contributed by Fengnan Li.
...
(cherry picked from commit 80b877a72f52ef0f4acafe15db55b8ed61fbe6d2)
2021-04-02 14:12:17 +08:00
Viraj Jasani
f707d8407a
HDFS-15931 : Fix non-static inner classes for better memory management ( #2830 ). Contributed by Viraj Jasani
...
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2021-04-01 16:35:44 -07:00
Akira Ajisaka
53b024a2b7
HDFS-15508. [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
...
(cherry picked from commit 9062814bab7d6087c584e03b0b16eaa60b31f60a)
2021-03-30 22:29:49 +08:00
Harunobu Daikoku
db8af483d1
HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode. ( #2787 )
...
Co-authored-by: Harunobu Daikoku <hdaikoku@yahoo-corp.jp>
(cherry picked from commit ea6595d3b68ac462aec0d493718d7a10fbda0b6d)
2021-03-29 11:47:16 +09:00
Ayush Saxena
e4217be749
HDFS-15380. RBF: Could not fetch real remote IP in RouterWebHdfsMethods. Contributed by tomscut.
...
(cherry picked from commit 5ab1db9e218e11d00035e234344ef90228d8898d)
2021-03-24 17:44:29 +08:00
Akira Ajisaka
e45149bf70
HDFS-15836. RBF: Fix contract tests after HADOOP-13327 ( #2702 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 2b3c5b173381204bd8d2c984f19c52104a419322)
2021-02-16 14:52:31 +09:00
Zehao Chen
1768c0096f
HDFS-15821. Add metrics for in-service datanodes ( #2690 ). Contributed by Zehao Chen.
...
(cherry picked from commit 07a4220cd27c69b86b837e8da320bad0031f7895)
2021-02-15 17:32:14 +00:00
Ayush Saxena
8378ab9f92
HADOOP-17288. Use shaded guava from thirdparty. Contributed by Ayush Saxena. #2505
2020-12-10 05:50:55 +05:30
Akira Ajisaka
8e0922edb2
HDFS-15657. RBF: TestRouter#testNamenodeHeartBeatEnableDefault fails by BindException ( #2418 )
...
(cherry picked from commit b3ba74d72dfe938ea7ed9e6b0870db6c87aa0a91)
2020-10-28 19:14:19 +09:00
Ayush Saxena
2a043b987c
HDFS-14811. RBF: TestRouterRpc#testErasureCoding is flaky. Contributed by Chen Zhang.
...
(cherry picked from commit 7a6265ac425e7ff1426c015e81f8ac882ff7bcf6)
2020-10-13 18:29:54 +09:00
Akira Ajisaka
2e73871cab
HDFS-15620. RBF: Fix test failures after HADOOP-17281 ( #2375 )
...
(cherry picked from commit 69ef9b1ee8825fbf6f06c2e3b84ca93e22398953)
2020-10-13 17:43:29 +09:00
Steve Loughran
7cf5bdeec0
Revert "HDFS-15620. RBF: Fix test failures after HADOOP-17281 ( #2375 )"
...
This reverts commit 263b7d5dfced0e3d594458c664f14df64589bbc8.
2020-10-12 10:45:18 +01:00
Akira Ajisaka
263b7d5dfc
HDFS-15620. RBF: Fix test failures after HADOOP-17281 ( #2375 )
2020-10-12 10:43:26 +01:00
Akira Ajisaka
89314a7bae
HDFS-15613. RBF: Router FSCK fails after HDFS-14442. ( #2360 )
...
* Support getHAServiceState in DFSRouter
(cherry picked from commit 074f0d46af9993b7d2cc5ac63105a934ccbf6b95)
2020-10-07 13:40:31 +09:00
Hui Fei
ed19f63998
HADOOP-17277. Correct spelling errors for separator ( #2322 )
...
Contributed by Hui Fei.
(cherry picked from commit 474fa80bfb11f20fe21272f793307258315e440f)
2020-09-23 15:39:51 +09:00
Akira Ajisaka
0731c8b5d0
HDFS-15555. RBF: Refresh cacheNS when SocketException occurs. ( #2267 )
...
(cherry picked from commit c78d18023dcc7a905bdd3ee2e89d4c553bb8bebb)
2020-09-14 11:36:26 +09:00
hemanthboyina
ac0a377779
HDFS-15536. RBF: Clear Quota in Router was not consistent.
...
(cherry picked from commit 6e618b6a7e3b7cb6459091945c8eb07fddc0034e)
2020-08-26 13:04:47 +05:30
hemanthboyina
b5d712251c
HDFS-15503. File and directory permissions are not able to be modified from WebUI.
...
(cherry picked from commit 82f3ffcd64d25cf3a2f5e280e07140994e0ba8cb)
2020-08-03 23:43:29 +05:30
Ayush Saxena
e3b8d4eb05
HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.
2020-07-22 18:21:14 +05:30
Vinayakumar B
534b15caf9
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:20:10 +05:30
S O'Donnell
433aaeefa4
HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun.
2020-05-12 15:25:05 +01:00