hadoop/hadoop-tools
Steve Loughran 464bbd5b7c
HADOOP-17511. Add audit/telemetry logging to S3A connector (#2807)
The S3A connector supports
"an auditor", a plugin which is invoked
at the start of every filesystem API call,
and whose issued "audit span" provides a context
for all REST operations against the S3 object store.

The standard auditor sets the HTTP Referrer header
on the requests with information about the API call,
such as process ID, operation name, path,
and even job ID.

If the S3 bucket is configured to log requests, this
information will be preserved there and so can be used
to analyze and troubleshoot storage IO.

Contributed by Steve Loughran.

Change-Id: Ic0a105c194342ed2d529833ecc42608e8ba2f258
2021-05-25 12:55:38 +01:00
..
hadoop-aliyun Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-archive-logs Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-archives Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-aws HADOOP-17511. Add audit/telemetry logging to S3A connector (#2807) 2021-05-25 12:55:38 +01:00
hadoop-azure HADOOP-17670. S3AFS and ABFS to log IOStats at DEBUG mode or optionally at INFO level in close() (#2963) 2021-05-24 13:04:20 +01:00
hadoop-azure-datalake Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-datajoin Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-distcp Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-dynamometer Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-extras Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-fs2img Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-gridmix Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-kafka Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-openstack Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-pipes Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-resourceestimator Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-rumen Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-sls Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-streaming Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-tools-dist Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
pom.xml Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00