big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
b0c9e4f1b5
hadoop/hadoop-tools
History
Mukund Thakur b0c9e4f1b5
HADOOP-16900. Very large files can be truncated when written through the S3A FileSystem. 
Contributed by Mukund Thakur and Steve Loughran.

This patch ensures that writes to S3A fail when more than 10,000 blocks are
written. That upper bound still exists. To write massive files, make sure
that the value of fs.s3a.multipart.size is set to a size which is large
enough to upload the files in fewer than 10,000 blocks.

Change-Id: Icec604e2a357ffd38d7ae7bc3f887ff55f2d721a
2020-06-01 12:01:13 +01:00
..
hadoop-aliyun Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-archive-logs Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-archives Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-aws HADOOP-16900. Very large files can be truncated when written through the S3A FileSystem. 2020-06-01 12:01:13 +01:00
hadoop-azure Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-azure-datalake Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-datajoin Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-distcp Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)." 2020-05-14 19:20:34 +01:00
hadoop-dynamometer Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-extras Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-fs2img Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-gridmix Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-kafka Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-openstack Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-pipes Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-resourceestimator Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-rumen Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-sls Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-streaming Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
hadoop-tools-dist Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
pom.xml Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00