big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
29b19cd592
hadoop/hadoop-tools
History
Mukund Thakur 29b19cd592
HADOOP-16900. Very large files can be truncated when written through the S3A FileSystem. 
Contributed by Mukund Thakur and Steve Loughran.

This patch ensures that writes to S3A fail when more than 10,000 blocks are
written. That upper bound still exists. To write massive files, make sure
that the value of fs.s3a.multipart.size is set to a size which is large
enough to upload the files in fewer than 10,000 blocks.

Change-Id: Icec604e2a357ffd38d7ae7bc3f887ff55f2d721a
2020-05-20 13:42:25 +01:00
..
hadoop-aliyun Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-archive-logs Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-archives Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-aws HADOOP-16900. Very large files can be truncated when written through the S3A FileSystem. 2020-05-20 13:42:25 +01:00
hadoop-azure Hadoop-17015. ABFS: Handling Rename and Delete idempotency 2020-05-19 12:30:07 -07:00
hadoop-azure-datalake Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-datajoin Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-distcp Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)." 2020-05-14 19:04:36 +01:00
hadoop-dynamometer Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-extras Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-fs2img Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-gridmix HADOOP-17011. Tolerate leading and trailing spaces in fs.defaultFS. Contributed by Ctest 2020-04-30 14:15:28 -07:00
hadoop-kafka Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-openstack Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-pipes Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-resourceestimator Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-rumen Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-sls YARN-9301. Too many InvalidStateTransitionException with SLS. Contributed by Bilwa S T. 2020-05-12 08:24:34 -07:00
hadoop-streaming Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-tools-dist Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
pom.xml Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30