huhaiyang
87c036efb3
HDFS-17116. RBF: Update invoke millisecond time as monotonicNow() in RouterSafemodeService ( #5876 ). Contributed by Haiyang Hu.
...
Reviewed-by: hfutatzhanghb <1036798979@qq.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-07-28 15:15:00 +08:00
huhaiyang
1d09dcc614
HDFS-17117. Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync. ( #5877 ). Contributed by Haiyang Hu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-07-27 11:28:30 +08:00
Simbarashe Dzinamarira
ad001c93cf
HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled. ( #5860 ). Contributed by Simbarashe Dzinamarira.
2023-07-26 13:22:13 +05:30
Zhaohui Wang
9cfe9ccd26
HDFS-17119. RBF: Logger fix for StateStoreMySQLImpl. ( #5882 ). Contributed by Zhaohui Wang.
...
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-07-24 17:26:23 +05:30
Zhaohui Wang
9659d0720d
HDFS-17065. Fix typos in hadoop-hdfs-project ( #5796 )
2023-07-03 08:41:37 -07:00
huhaiyang
161f80810f
HDFS-17057. Add DataNode maintenance states to Federation UI ( #5775 )
2023-06-26 09:38:57 -07:00
Viraj Jasani
f0c4286e3e
HDFS-17041. RBF: Fix putAll impl for mysql and file based state stores ( #5723 )
2023-06-13 09:55:04 -07:00
NishthaShah
3fbadc5d50
HDFS-16946. Fix getTopTokenRealOwners to return String ( #5696 ). Contributed by Nishtha Shah.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-13 10:01:35 +05:30
Wei-Chiu Chuang
e1bb4acd36
HADOOP-18761. Remove mysql-connector-java ( #5731 )
2023-06-12 15:31:03 -07:00
hfutatzhanghb
35158db711
HDFS-17023. RBF: Record proxy time when call invokeConcurrent method. ( #5683 ). Contributed by farmmamba.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-10 00:06:28 +05:30
hfutatzhanghb
2243cfd225
HDFS-17028. RBF: Optimize debug logs of class ConnectionPool and other related class. ( #5694 ). Contributed by farmmamba.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-05 14:34:46 +05:30
smarthan
9f1e23cc67
HDFS-17031. Reduce some repeated codes in RouterRpcServer. ( #5701 ). Contributed by Chengwei Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-01 08:32:38 +05:30
hchaverri
124313d215
HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency. ( #5691 ). Contributed by hchaverri.
...
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-31 14:26:31 +08:00
Viraj Jasani
3b65b5d68f
HDFS-17020. RBF: mount table addAll should print failed records in std error ( #5674 )
2023-05-22 18:39:52 -07:00
Viraj Jasani
8e17385141
HDFS-17009. RBF: state store putAll should also return failed records ( #5664 )
2023-05-17 09:33:34 -07:00
Viraj Jasani
5d0cc455f5
HDFS-17008. Fix RBF JDK 11 javadoc warnings ( #5648 )
2023-05-12 17:27:13 -07:00
zhtttylz
0c77629849
HDFS-17001. Support getStatus API in WebHDFS ( #5628 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-13 03:49:13 +05:30
Viraj Jasani
fe61d8f073
HDFS-16978. RBF: Admin command to support bulk add of mount points ( #5554 ). Contributed by Viraj Jasani.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-11 08:45:34 +05:30
ZanderXu
4ee92efb73
HDFS-16865. The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC. ( #5200 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-06 08:50:21 +05:30
Viraj Jasani
ceb8878d4f
HDFS-16998. RBF: Add ops metrics for getSlowDatanodeReport in RouterClientActivity ( #5615 )
2023-05-04 09:45:40 -07:00
SevenAddSix
1079890ae3
HDFS-16707. RBF: Expose RouterRpcFairnessPolicyController related request record metrics for each nameservice to Prometheus ( #4665 ). Contributed by Jiale Qi.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-01 21:22:26 +05:30
wangzhaohui
0e63152218
HDFS-16995. Remove unused parameters at NameNodeHttpServer#initWebHdfs ( #5601 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-28 05:25:01 +05:30
zhtttylz
c9e0af9961
HDFS-16981. Support getFileLinkStatus API in WebHDFS ( #5572 ). Contributed by Hualong Zhang.
...
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 03:30:56 +05:30
Neil
1ff7a65b9f
HDFS-16954. RBF: The operation of renaming a multi-subcluster directory to a single-cluster directory should throw ioexception. ( #5483 ). Contributed by Max Xie.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 15:19:18 +05:30
zhtttylz
523ff81624
HDFS-16952. Support getLinkTarget API in WebHDFS ( #5517 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-04-06 19:44:47 +08:00
Simbarashe Dzinamarira
47c22e388e
HDFS-16943. RBF: Implements MySQL based StateStoreDriver. ( #5469 )
2023-04-05 16:44:29 -07:00
Viraj Jasani
422bf3b24c
HDFS-16973. RBF: MountTableResolver cache size lookup should take read lock ( #5533 )
2023-04-05 14:06:38 -07:00
Viraj Jasani
937caf7de9
HDFS-16967. RBF: File based state stores should allow concurrent access to the records ( #5523 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2023-04-04 22:39:53 +09:00
Viraj Jasani
b4bcbb9515
HDFS-16959. RBF: State store cache loading metrics ( #5497 )
2023-03-29 10:43:13 -07:00
Viraj Jasani
f8d0949f7d
HDFS-16953. RBF: Mount table store APIs should update cache only if state store record is successfully updated ( #5482 )
2023-03-18 14:43:25 -07:00
Viraj Jasani
15935fa865
HDFS-16947. RBF NamenodeHeartbeatService to report error for not being able to register namenode in state store ( #5470 )
2023-03-16 00:59:55 +08:00
Simbarashe Dzinamarira
61f369c43e
HDFS-16890: RBF: Ensures router periodically refreshes its record of a namespace's state. ( #5298 )
2023-02-27 17:56:24 +00:00
Simbarashe Dzinamarira
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. ( #5346 )
2023-02-22 21:58:44 +00:00
hchaverr
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL
...
Fixes #1240
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 10:38:50 -08:00
Zita Dombi
4cbe19f3a2
HDFS-16761. Namenode UI for Datanodes page not loading if any data node is down ( #5390 )
2023-02-15 16:16:04 +00:00
hfutatzhanghb
f3c4277576
HDFS-16882. RBF: Add cache hit rate metric in MountTableResolver#getDestinationForPath ( #5276 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-11 08:00:43 +08:00
huhaiyang
113a9e40cb
HADOOP-18625. Fix method name of RPC.Builder#setnumReaders ( #5301 )
...
Changes method name of RPC.Builder#setnumReaders to setNumReaders()
The original method is still there, just marked deprecated.
It is the one which should be used when working with older branches.
Contributed by Haiyang Hu
2023-02-09 13:28:34 +00:00
hchaverri
d310642626
HDFS-16895. [RBF] NamenodeHeartbeatService should use credentials of logged in user
2023-02-07 18:45:05 +00:00
sunhao
0ae075a2af
HDFS-16848. RBF: Improve StateStoreZooKeeperImpl performance ( #5147 )
2023-02-05 09:33:05 +08:00
Simbarashe Dzinamarira
f26d8bc9bd
HDFS-16886: Fixes error in documentation for StateStoreRecordsOperations. ( #5290 )
2023-01-11 19:46:51 +00:00
Simbarashe Dzinamarira
b56d483258
HDFS-16876: Changes cleanup of shared RouterStateIdContext to be driven by namenodeResolver data. ( #5282 )
2023-01-09 23:55:13 +00:00
Simbarashe Dzinamarira
cd19da1309
HDFS-16877: Enables state context for namenode in TestObserverWithRouter ( #5257 )
2023-01-07 00:18:35 +00:00
Neil
d25c1be517
HDFS-16861. RBF. Truncate API always fails when dirs use AllResolver oder on Router ( #5184 )
...
Co-authored-by: xiezhineng <xiezhineng@corp.netease.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-12-26 21:19:33 +08:00
ZanderXu
df093ef9af
HDFS-16831. [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time ( #5098 )
2022-12-23 14:16:42 +08:00
ZanderXu
8a9bdb1edc
HDFS-16837. [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespaces ( #5123 )
2022-12-05 16:15:47 +08:00
Owen O'Malley
03471a736c
HDFS-16851: RBF: Add a utility to dump the StateStore. ( #5155 )
2022-11-29 22:12:35 +00:00
Simbarashe Dzinamarira
ec2856d79c
HDFS-16847: RBF: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException. ( #5145 )
2022-11-29 00:47:01 +00:00
ZanderXu
e0974298ce
HDFS-16826. [RBF SBN] ConnectionManager should advance the client stateId for each request ( #5086 )
2022-11-25 09:23:33 +08:00
Owen O'Malley
c71a68ca80
HDFS-16844: Adds resilancy when StateStore gets exceptions. ( #5138 )
...
Allows the StateStore to stay up when there are errors reading the data.
2022-11-18 17:24:08 +00:00
Simbarashe Dzinamarira
552ee44eba
HDFS-16834: Removes request stateID consistency constraint between clients in different connection pools. ( #5121 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
2022-11-11 15:26:31 +08:00
