hadoop/hadoop-tools/hadoop-distcp
Steve Loughran b6dc00f481
HADOOP-16775. DistCp reuses the same temp file within the task for different files.
Contributed by Amir Shenavandeh.

This avoids overwrite consistency issues with S3 and other stores -though
given S3's copy operation is O(data), you are still best of using -direct
when distcp-ing to it.

Change-Id: I8dc9f048ad0cc57ff01543b849da1ce4eaadf8c3
2020-01-02 15:36:33 +00:00
..
src HADOOP-16775. DistCp reuses the same temp file within the task for different files. 2020-01-02 15:36:33 +00:00
pom.xml HADOOP-16512. [hadoop-tools] Fix order of actual and expected expression in assert statements 2019-10-07 16:38:08 +09:00
README HADOOP-11437. Remove the version and author information from distcp's README file (Brahma Reddy Battula via aw) 2015-02-11 15:47:36 -08:00

DistCp (distributed copy) is a tool used for large inter/intra-cluster copying. 
It uses Map/Reduce to effect its distribution, error handling and recovery, 
and reporting. It expands a list of files and directories into input to map tasks, 
each of which will copy a partition of the files specified in the source list.