big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade.

Browse Source
This commit is contained in:
cnauroth 2015-10-22 14:29:57 -07:00
parent 0fce5f9a49
commit cb282d5b89
2 changed files with 30 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-common-project/hadoop-common/CHANGES.txt
View File

@ -1338,6 +1338,9 @@ Release 2.8.0 - UNRELEASED
HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass
Azure Storage Throttling after retries. (Gaurav Kanade via cnauroth) Azure Storage Throttling after retries. (Gaurav Kanade via cnauroth)
HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race
Condition Scenarios. (Gaurav Kanade via cnauroth)
Release 2.7.2 - UNRELEASED Release 2.7.2 - UNRELEASED
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

32
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
View File

@ -545,10 +545,32 @@ private void finishSingleFileRename(String fileName)
// Get a lease on source to block write access. // Get a lease on source to block write access.
String srcName = fs.pathToKey(srcFile); String srcName = fs.pathToKey(srcFile);
SelfRenewingLease lease = fs.acquireLease(srcFile); SelfRenewingLease lease = null;
try {
// Delete the file. This will free the lease too. lease = fs.acquireLease(srcFile);
fs.getStoreInterface().delete(srcName, lease); // Delete the file. This will free the lease too.
fs.getStoreInterface().delete(srcName, lease);
} catch(AzureException e) {
String errorCode = "";
try {
StorageException e2 = (StorageException) e.getCause();
errorCode = e2.getErrorCode();
} catch(Exception e3) {
// do nothing if cast fails
}
// If the rename already finished do nothing
if(!errorCode.equals("BlobNotFound")){
throw e;
}
} finally {
try {
if(lease != null){
lease.free();
}
} catch(StorageException e) {
LOG.warn("Unable to free lease because: " + e.getMessage());
}
}
} else if (!srcExists && dstExists) { } else if (!srcExists && dstExists) {
// The rename already finished, so do nothing. // The rename already finished, so do nothing.
@ -2442,4 +2464,4 @@ private static void cleanup(Logger log, java.io.Closeable closeable) {
} }
} }
} }
} }