big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5bfca65692
hadoop/hadoop-tools
History
Syed Shameerur Rahman 032796a0fb
HADOOP-19047: S3A: Support in-memory tracking of Magic Commit data (#6468) 
If the option fs.s3a.committer.magic.track.commits.in.memory.enabled
is set to true, then rather than save data about in-progress uploads
to S3, this information is cached in memory.

If the number of files being committed is low, this will save network IO
in both the generation of .pending and marker files, and in the scanning
of task attempt directory trees during task commit.

Contributed by Syed Shameerur Rahman
2024-03-26 15:29:35 +00:00
..
hadoop-aliyun Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-archive-logs Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-archives Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-aws HADOOP-19047: S3A: Support in-memory tracking of Magic Commit data (#6468) 2024-03-26 15:29:35 +00:00
hadoop-azure HADOOP-19089: [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path (#6592) 2024-03-25 14:13:24 +00:00
hadoop-azure-datalake Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-benchmark Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-compat-bench HADOOP-19085. Compatibility Benchmark over HCFS Implementations 2024-03-17 16:48:29 +08:00
hadoop-datajoin Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-distcp HDFS-17376. Distcp creates Factor 1 replication file on target if Source is EC. (#6540) 2024-02-09 17:26:32 +00:00
hadoop-dynamometer Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-extras Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-federation-balance Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-fs2img Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-gridmix Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-kafka Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-openstack Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-pipes Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-resourceestimator Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-rumen Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-sls Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-streaming Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-tools-dist Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
pom.xml HADOOP-19085. Compatibility Benchmark over HCFS Implementations 2024-03-17 16:48:29 +08:00