hadoop/hadoop-common-project
Steve Loughran 511df1e837 HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions
Contributed by Steve Loughran.

This overlaps the scanning for directory entries with batched calls to S3 DELETE and updates of the S3Guard tables.
It also uses S3Guard to list the files to delete, so find newly created files even when S3 listings are not use consistent.

For path which the client considers S3Guard to be authoritative, we also do a recursive LIST of the store and delete files; this is to find unindexed files and do guarantee that the delete(path, true) call really does delete everything underneath.

Change-Id: Ice2f6e940c506e0b3a78fa534a99721b1698708e
2019-09-05 14:25:15 +01:00
..
hadoop-annotations HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-auth HADOOP-16527. Add a whitelist of endpoints to skip Kerberos authentication (#1336) Contributed by Akira Ajisaka. 2019-08-28 14:28:41 +09:00
hadoop-auth-examples HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-common HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions 2019-09-05 14:25:15 +01:00
hadoop-kms HADOOP-14784. [KMS] Improve KeyAuthorizationKeyProvider#toString(). Contributed by Yeliang Cang. 2019-08-19 11:12:09 -07:00
hadoop-minikdc HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-nfs HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-registry YARN-9783. Remove low-level zookeeper test to be able to build Hadoop against zookeeper 3.5.5. Contributed by Mate Szalay-Beko. 2019-08-30 10:13:10 +09:00
pom.xml HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00