hadoop/hadoop-tools/hadoop-aws
Steve Loughran ee2e5ac4e4
HADOOP-19272. S3A: AWS SDK 2.25.53 warnings logged by transfer manager (#7048)
Disables all logging below error in the AWS SDK Transfer Manager.

This is done in ClientManagerImpl construction so is automatically done
during S3A FS initialization.

ITests verify that
* It is possible to restore the warning log. This verifies the validity of
  the test suite, and will identify when an SDK update fixes this regression.
* Constructing an S3A FS instance will disable the logging.

The log manipulation code is lifted from Cloudstore, where it was used to
dynamically enable logging. It uses reflection to load the Log4J binding;
all uses of the API catch and swallow exceptions.
This is needed to avoid failures when running against different log backends

This is an emergency fix -we could come up with a better design for
the reflection based code using the new DynMethods classes.
But this is based on working code, which is always good.

Contributed by Steve Loughran
2024-09-19 13:50:06 +01:00
..
dev-support HADOOP-19205. S3A: initialization/close slower than with v1 SDK (#6892) 2024-07-05 16:38:37 +01:00
src HADOOP-19272. S3A: AWS SDK 2.25.53 warnings logged by transfer manager (#7048) 2024-09-19 13:50:06 +01:00
pom.xml HADOOP-16928. Make javadoc work on Java 17 (#6976) 2024-09-04 11:50:59 +01:00