big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
90b1e737d3
hadoop/hadoop-tools
History
Mehakmeet Singh 90b1e737d3
HADOOP-18242. ABFS Rename Failure when tracking metadata is in an incomplete state (#4517) 
ABFS rename fails intermittently when the Storage-blob tracking
metadata is in an incomplete state. This surfaces as the error code
404 and an error message of "RenameDestinationParentPathNotFound"

To mitigate this issue, when a request fails with this response.
the ABFS client issues a HEAD call on the source file
and then retries the rename operation again

ABFS filesystem statistics track when this occurs with new counters
  rename_recovery
  metadata_incomplete_rename_failures
  rename_path_attempts

This is very rare occurrence and appears to be triggered under certain
heavy load conditions, just as with HADOOP-18163.

Contributed by Mehakmeet Singh.
2022-07-02 01:49:14 +05:30
..
hadoop-aliyun HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-archive-logs HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-archives HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-aws HADOOP-18106: Handle memory fragmentation in S3A Vectored IO. (#4445) 2022-06-23 17:34:29 -05:00
hadoop-azure HADOOP-18242. ABFS Rename Failure when tracking metadata is in an incomplete state (#4517) 2022-07-02 01:49:14 +05:30
hadoop-azure-datalake HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-benchmark HADOOP-18322. Yetus build failure in branch-3.3. 2022-06-30 15:05:38 -05:00
hadoop-datajoin HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-distcp HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-dynamometer HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-extras HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-fs2img HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-gridmix HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-kafka HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-openstack HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-pipes HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-resourceestimator HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-rumen HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-sls HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
hadoop-streaming MAPREDUCE-7371. DistributedCache alternative APIs should not use DistributedCache APIs internally (#3855) 2022-06-22 13:13:05 +01:00
hadoop-tools-dist HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 2022-06-22 13:09:50 +01:00
pom.xml HADOOP-11867. Add a high-performance vectored read API. (#3904) 2022-06-23 17:09:16 -05:00