hadoop/hadoop-tools/hadoop-aws
Mukund Thakur 29b19cd592
HADOOP-16900. Very large files can be truncated when written through the S3A FileSystem.
Contributed by Mukund Thakur and Steve Loughran.

This patch ensures that writes to S3A fail when more than 10,000 blocks are
written. That upper bound still exists. To write massive files, make sure
that the value of fs.s3a.multipart.size is set to a size which is large
enough to upload the files in fewer than 10,000 blocks.

Change-Id: Icec604e2a357ffd38d7ae7bc3f887ff55f2d721a
2020-05-20 13:42:25 +01:00
..
dev-support HADOOP-16697. Tune/audit S3A authoritative mode. 2020-01-10 11:11:56 +00:00
src HADOOP-16900. Very large files can be truncated when written through the S3A FileSystem. 2020-05-20 13:42:25 +01:00
pom.xml Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30