Go to file
Steve Loughran 511df1e837 HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions
Contributed by Steve Loughran.

This overlaps the scanning for directory entries with batched calls to S3 DELETE and updates of the S3Guard tables.
It also uses S3Guard to list the files to delete, so find newly created files even when S3 listings are not use consistent.

For path which the client considers S3Guard to be authoritative, we also do a recursive LIST of the store and delete files; this is to find unindexed files and do guarantee that the delete(path, true) call really does delete everything underneath.

Change-Id: Ice2f6e940c506e0b3a78fa534a99721b1698708e
2019-09-05 14:25:15 +01:00
dev-support HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
hadoop-assemblies HADOOP-16534. Exclude submarine from hadoop source build. (#1356) 2019-09-03 17:40:38 +05:30
hadoop-build-tools HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-client-modules HADOOP-15998. Ensure jar validation works on Windows. 2019-08-29 23:09:04 -05:00
hadoop-cloud-storage-project HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-common-project HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions 2019-09-05 14:25:15 +01:00
hadoop-dist HDFS-14639. [Dynamometer] Remove unnecessary duplicate directory from the distribution. Contributed by Erik Krogen. 2019-07-29 13:50:14 -07:00
hadoop-hdds HDDS-1898. GrpcReplicationService#download cannot replicate the container. (#1326) 2019-09-05 18:12:36 +05:30
hadoop-hdfs-project HDFS-14812. RBF: MountTableRefresherService should load cache when refresh. Contributed by xuzq. 2019-09-05 08:29:56 +05:30
hadoop-mapreduce-project MAPREDUCE-7230. TestHSWebApp.testLogsViewSingle fails. Contributed by Prabhu Joseph 2019-08-15 17:07:43 +02:00
hadoop-maven-plugins HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-minicluster HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-ozone HDDS-1898. GrpcReplicationService#download cannot replicate the container. (#1326) 2019-09-05 18:12:36 +05:30
hadoop-project HADOOP-16533. Upgrade jackson-databind to 2.9.9.3. (#1354). Contributed by Akira Ajisaka. 2019-08-27 15:54:35 -07:00
hadoop-project-dist HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-submarine SUBMARINE-45. Can't specify queue by using the parameter --queue. Contributed by Ayush Saxena, Zac Zhou. 2019-08-15 13:18:29 +08:00
hadoop-tools HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions 2019-09-05 14:25:15 +01:00
hadoop-yarn-project YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. Contributed by zhuqi. 2019-09-05 16:20:05 +08:00
licenses HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
licenses-binary HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore HDDS-1115. Provide ozone specific top-level pom.xml. 2019-02-24 14:40:52 -08:00
BUILDING.txt HADOOP-16263. Update BUILDING.txt with macOS native build instructions. Contributed by Siyao Meng. 2019-06-11 15:04:59 -07:00
Jenkinsfile HADOOP-16183. Use latest Yetus to support ozone specific build process 2019-05-02 16:48:30 +02:00
LICENSE-binary HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
LICENSE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
NOTICE-binary HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.ozone.xml HDDS-2077. Add maven-gpg-plugin.version to pom.ozone.xml. (#1396) 2019-09-04 15:28:59 +05:30
pom.xml HADOOP-16534. Exclude submarine from hadoop source build. (#1356) 2019-09-03 17:40:38 +05:30
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
start-build-env.sh HADOOP-16240. start-build-env.sh can consume all disk space during image creation. 2019-04-10 08:48:11 -07:00

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/