hadoop/hadoop-common-project/hadoop-common/src
Steve Loughran 9960c01a25
HADOOP-17244. S3A directory delete tombstones dir markers prematurely. (#2280)
This changes directory tree deletion so that only files are incrementally deleted
from S3Guard after the objects are deleted; the directories are left alone
until metadataStore.deleteSubtree(path) is invoke.

This avoids directory tombstones being added above files/child directories,
which stop the treewalk and delete phase from working.

Also:

* Callback to delete objects splits files and dirs so that
any problems deleting the dirs doesn't trigger s3guard updates
* New statistic to measure #of objects deleted, alongside request count.
* Callback listFilesAndEmptyDirectories renamed listFilesAndDirectoryMarkers
  to clarify behavior.
* Test enhancements to replicate the failure and verify the fix

Contributed by Steve Loughran
2020-09-10 17:03:52 +01:00
..
contrib/bash-tab-completion HADOOP-10998. Fix bash tab completion code to work (Jim Hester via aw) 2014-08-22 18:25:09 +00:00
main HADOOP-17244. S3A directory delete tombstones dir markers prematurely. (#2280) 2020-09-10 17:03:52 +01:00
site HADOOP-17165. Implement service-user feature in DecayRPCScheduler. (#2240) 2020-09-09 09:56:58 -07:00
test HADOOP-17244. S3A directory delete tombstones dir markers prematurely. (#2280) 2020-09-10 17:03:52 +01:00
CMakeLists.txt HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He. 2019-09-22 22:02:54 +05:30
config.h.cmake HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00