hadoop/hadoop-tools/hadoop-aws
Steve Loughran 6a3433bffd
HADOOP-16357. TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists.
Contributed by Steve Loughran.

This patch

* changes the default for the staging committer to append, as we get for the classic FileOutputFormat committer
* adds a check for the dest path being a file not a dir
* adds tests for this
* Changes AbstractCommitTerasortIT. to not use the simple parser, so fails if the file is present.

Change-Id: Id53742958ed1cf321ff96c9063505d64f3254f53
2019-07-11 18:15:34 +01:00
..
dev-support HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile(); 2019-02-05 11:51:02 +00:00
src HADOOP-16357. TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists. 2019-07-11 18:15:34 +01:00
pom.xml HADOOP-15183. S3Guard store becomes inconsistent after partial failure of rename. 2019-06-20 09:56:40 +01:00