big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-19015. Increase fs.s3a.connection.maximum to 500 to minimize risk of Timeout waiting for connection from pool

Browse Source
This commit is contained in:
Mukund Thakur 2023-12-19 14:04:07 -06:00
parent f52c7d3e9a
commit e28f83a1eb
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
View File

@ -1530,7 +1530,7 @@
<property>
<name>fs.s3a.connection.maximum</name>
<value>200</value>
<value>500</value>
<description>Controls the maximum number of simultaneous connections to S3.
This must be bigger than the value of fs.s3a.threads.max so as to stop
threads being blocked waiting for new HTTPS connections.

2
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java
View File

@ -173,7 +173,7 @@ private Constants() {
* Future releases are likely to increase this value.
* Keep in sync with the value in {@code core-default.xml}
*/
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 200;
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 500;
/**
* Configuration option to configure expiration time of