big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-16767 Handle non-IO exceptions in reopen()

Browse Source 
Contributed by Sergei Poganshev.

Catches Exception instead of IOException in closeStream() 
and so handle exceptions such as SdkClientException by 
aborting the wrapped stream. This will increase resilience
to failures, as any which occuring during stream closure
will be caught. Furthermore, because the
underlying HTTP connection is aborted, rather than closed,
it will not be recycled to cause problems on subsequent
operations.
This commit is contained in:
spoganshev 2020-03-02 19:17:54 +02:00 committed by GitHub
parent e9eecedf69
commit e553eda9cd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java
View File

@ -576,7 +576,7 @@ private void closeStream(String reason, long length, boolean forceAbort) {
// and an abort is triggered, the initial attempt's statistics // and an abort is triggered, the initial attempt's statistics
// aren't collected. // aren't collected.
streamStatistics.streamClose(false, drained); streamStatistics.streamClose(false, drained);
} catch (IOException e) { } catch (Exception e) {
// exception escalates to an abort // exception escalates to an abort
LOG.debug("When closing {} stream for {}", uri, reason, e); LOG.debug("When closing {} stream for {}", uri, reason, e);
shouldAbort = true; shouldAbort = true;