hadoop/hadoop-common-project
Steve Loughran 8bd1f65efc
HADOOP-18948. S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete (#6218)
S3A directory delete and rename will optionally abort all pending multipart uploads
in their under their to-be-deleted paths when.

fs.s3a.directory.operations.purge.upload is true

It is off by default.

The filesystems hasPathCapability("fs.s3a.directory.operations.purge.upload")
probe will return true when this feature is enabled.

Multipart uploads may accrue from interrupted data writes, uncommitted 
staging/magic committer jobs and other operations/applications. On AWS S3
lifecycle rules are the recommended way to clean these; this change improves
support for stores which lack these rules.

Contributed by Steve Loughran
2023-10-25 17:39:16 +01:00
..
hadoop-annotations HADOOP-17947. Additional element types for VisibleForTesting (ADDENDUM) (#3521) 2021-10-06 02:17:05 +09:00
hadoop-auth Revert "HADOOP-18207. Introduce hadoop-logging module (#5503)" 2023-06-05 09:34:40 +05:30
hadoop-auth-examples Revert "HADOOP-18207. Introduce hadoop-logging module (#5503)" 2023-06-05 09:34:40 +05:30
hadoop-common HADOOP-18948. S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete (#6218) 2023-10-25 17:39:16 +01:00
hadoop-kms Revert "HADOOP-18207. Introduce hadoop-logging module (#5503)" 2023-06-05 09:34:40 +05:30
hadoop-minikdc Revert "HADOOP-18207. Introduce hadoop-logging module (#5503)" 2023-06-05 09:34:40 +05:30
hadoop-nfs HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future (#5315) 2023-02-14 03:24:06 +08:00
hadoop-registry HADOOP-18359. Update commons-cli from 1.2 to 1.5. (#5095). Contributed by Shilun Fan. 2023-05-10 01:42:12 +05:30
pom.xml Revert "HADOOP-18207. Introduce hadoop-logging module (#5503)" 2023-06-05 09:34:40 +05:30