hadoop/hadoop-common-project/hadoop-common
Steve Loughran aaf92fe183
HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references (#5144)
This has triggered an OOM in a process which was churning through s3a fs
instances; the increased memory footprint of IOStatistics amplified what
must have been a long-standing issue with FS instances being created
and not closed()

*  Makes sure instrumentation is closed when the FS is closed.
*  Uses a weak reference from metrics to instrumentation, so even
   if the FS wasn't closed (see HADOOP-18478), this back reference
   would not cause the S3AInstrumentation reference to be retained.
*  If S3AFileSystem is configured to log at TRACE it will log the
   calling stack of initialize(), so help identify where the
   instance is being created. This should help track down
   the cause of instance leakage.

Contributed by Steve Loughran.
2022-12-14 18:21:03 +00:00
..
dev-support HADOOP-18442. Remove openstack support (#4855) 2022-10-06 11:49:38 +01:00
src HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references (#5144) 2022-12-14 18:21:03 +00:00
HadoopCommon.cmake HADOOP-17196. Fix C/C++ standard warnings (#2208) 2020-08-11 16:35:08 +09:00
HadoopJNI.cmake HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati 2018-08-17 12:25:36 -05:00
pom.xml HADOOP-18496. Upgrade okhttp3 and dependencies due to kotlin CVEs (#5035) 2022-11-12 14:14:19 +00:00