This is a followup to the original patch, 08f58ecf07f, which it supercedes
* Switch to org.apache.hadoop.io.IOUtils and closeStream.
* Use cleanupWithLogger to include error logging
Contributed by Steve Vaughan Jr
* ADDENDUM Remove unused variables from testDataNodeMXBeanLastHeartbeats (#5373)
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
Changes method name of RPC.Builder#setnumReaders to setNumReaders()
The original method is still there, just marked deprecated.
It is the one which should be used when working with older branches.
Contributed by Haiyang Hu
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 88c8ac750dc630040a37014dd13391922e625292)
Reviewed-by: Stephen O'Donnell <sodonnell@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit 04f3573f6ad07c64c1e2cf8a82fb3e9263366541)
Co-authored-by: Chengbing Liu <liuchengbing@qiyi.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
(cherry picked from commit 4cf304de4520bac5be265501fdb056114b4154f5)
* HDFS-16633.Reserved Space For Replicas is not released on some cases
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
(cherry picked from commit b7edc6c60ca9e17c4f6a7d07aad5efbd7c2e3b40)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
Co-authored-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
(cherry picked from commit 72749a4ff8a240867a822bb9ae1382d3235083ea)
* HDFS-15383. RBF: Add support for router delegation token without watch (#2047)
Improving router's performance for delegation tokens related operations. It achieves the goal by removing watchers from router on tokens since based on our experience. The huge number of watches inside Zookeeper is degrading Zookeeper's performance pretty hard. The current limit is about 1.2-1.5 million.
* HADOOP-17835. Use CuratorCache implementation instead of PathChildrenCache / TreeCache (#3266)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Co-authored-by: lfengnan <lfengnan@uber.com>
Co-authored-by: Viraj Jasani <vjasani@apache.org>
Co-authored-by: Melissa You <myou@myou-mn1.linkedin.biz>
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Tao Li <tomscut@apache.org>
(cherry picked from commit 545a556883ab8f126f72efeeeed29265974feaf8)
The JournalNodeSyncer will include the local instance in syncing when using a bind host (e.g. 0.0.0.0). There is a mechanism that is supposed to exclude the local instance, but it doesn't recognize the meta-address as a local address.
Running with bind addresses set to 0.0.0.0, the JournalNodeSyncer will log attempts to sync with itself as part of the normal syncing rotation. For an HA configuration running 3 JournalNodes, the "other" list used by the JournalNodeSyncer will include 3 proxies.
Exclude bound local addresses, including the use of a wildcard address in the bound host configurations, while still allowing multiple instances on the same host.
Allow sync attempts with unresolved addresses, so that sync attempts can drive resolution as servers become available.
Backport.
Signed-off-by: stack <stack@apache.org>
