hadoop/hadoop-common-project/hadoop-common/src
Steve Loughran ba55f370a9
HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references (#5144)
This has triggered an OOM in a process which was churning through s3a fs
instances; the increased memory footprint of IOStatistics amplified what
must have been a long-standing issue with FS instances being created
and not closed()

*  Makes sure instrumentation is closed when the FS is closed.
*  Uses a weak reference from metrics to instrumentation, so even
   if the FS wasn't closed (see HADOOP-18478), this back reference
   would not cause the S3AInstrumentation reference to be retained.
*  If S3AFileSystem is configured to log at TRACE it will log the
   calling stack of initialize(), so help identify where the
   instance is being created. This should help track down
   the cause of instance leakage.

Contributed by Steve Loughran.
2022-12-14 18:23:04 +00:00
..
contrib/bash-tab-completion
main HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references (#5144) 2022-12-14 18:23:04 +00:00
site HADOOP-18470. index.md update for 3.3.5 release 2022-12-05 16:22:40 +00:00
test HADOOP-18530. ChecksumFileSystem::readVectored might return byte buffers not positioned at 0 (#5168) 2022-11-29 14:52:25 +00:00
CMakeLists.txt HADOOP-17292. Using lz4-java in Lz4Codec (#2350) 2020-12-29 13:17:26 -08:00
config.h.cmake HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00