hadoop/hadoop-common-project
Steve Loughran aaf92fe183
HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references (#5144)
This has triggered an OOM in a process which was churning through s3a fs
instances; the increased memory footprint of IOStatistics amplified what
must have been a long-standing issue with FS instances being created
and not closed()

*  Makes sure instrumentation is closed when the FS is closed.
*  Uses a weak reference from metrics to instrumentation, so even
   if the FS wasn't closed (see HADOOP-18478), this back reference
   would not cause the S3AInstrumentation reference to be retained.
*  If S3AFileSystem is configured to log at TRACE it will log the
   calling stack of initialize(), so help identify where the
   instance is being created. This should help track down
   the cause of instance leakage.

Contributed by Steve Loughran.
2022-12-14 18:21:03 +00:00
..
hadoop-annotations HADOOP-17947. Additional element types for VisibleForTesting (ADDENDUM) (#3521) 2021-10-06 02:17:05 +09:00
hadoop-auth HADOOP-18329. Support for IBM Semeru JVM > 11.0.15.0 Vendor Name Changes (#4537) 2022-12-10 14:27:05 +00:00
hadoop-auth-examples HADOOP-18120. Hadoop auth does not handle HTTP Headers in a case-insensitive way. Contributed by Janos Makai. 2022-05-20 10:43:19 +02:00
hadoop-common HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references (#5144) 2022-12-14 18:21:03 +00:00
hadoop-kms HADOOP-18266. Using HashSet/ TreeSet Constructor for hadoop-common (#4365) 2022-06-20 12:11:04 +05:30
hadoop-minikdc HADOOP-18329. Support for IBM Semeru JVM > 11.0.15.0 Vendor Name Changes (#4537) 2022-12-10 14:27:05 +00:00
hadoop-nfs HADOOP-18569. NFS Gateway may release buffer too early (#5212) 2022-12-14 15:55:44 +01:00
hadoop-registry HDFS-16591. Setup JaasConfiguration in ZKCuratorManager when SASL is enabled 2022-06-28 16:44:02 -07:00
pom.xml HADOOP-11867. Add a high-performance vectored read API. (#3904) 2022-06-22 17:29:32 +01:00