big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-17932. Distcp file length comparison have no effect (#3519)

Browse Source 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 280ae1c0a9e156b832fb4e3ddd90faafa6477802)
This commit is contained in:
adol001 2021-10-18 18:07:53 +08:00 committed by Akira Ajisaka
parent 4024afec28
commit 21bd015df2
No known key found for this signature in database
GPG Key ID: C1EDBB9CA400FD50
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
View File

@ -145,7 +145,7 @@ public class RetriableFileCopyCommand extends RetriableCommand {
if (!source.isSplit()) {
DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
sourcePath, sourceChecksum, targetFS,
targetPath, skipCrc, source.getLen());
targetPath, skipCrc, offset + bytesRead);
}
// it's not append or direct write (preferred for s3a) case, thus we first
// write to a temporary file, then rename it to the target path.