HADOOP-19015. Increase fs.s3a.connection.maximum to 500 to minimize risk of Timeout waiting for connection from pool. (#6372)
HADOOP-19015. Increase fs.s3a.connection.maximum to 500 to minimize the risk of Timeout waiting for connection from the pool Contributed By: Mukund Thakur
This commit is contained in:
parent
d378853790
commit
7b1570e2f1
@ -1530,7 +1530,7 @@
|
||||
|
||||
<property>
|
||||
<name>fs.s3a.connection.maximum</name>
|
||||
<value>200</value>
|
||||
<value>500</value>
|
||||
<description>Controls the maximum number of simultaneous connections to S3.
|
||||
This must be bigger than the value of fs.s3a.threads.max so as to stop
|
||||
threads being blocked waiting for new HTTPS connections.
|
||||
@ -1538,6 +1538,7 @@
|
||||
</description>
|
||||
</property>
|
||||
|
||||
|
||||
<property>
|
||||
<name>fs.s3a.connection.ssl.enabled</name>
|
||||
<value>true</value>
|
||||
|
@ -173,7 +173,7 @@ private Constants() {
|
||||
* Future releases are likely to increase this value.
|
||||
* Keep in sync with the value in {@code core-default.xml}
|
||||
*/
|
||||
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 200;
|
||||
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 500;
|
||||
|
||||
/**
|
||||
* Configuration option to configure expiration time of
|
||||
|
@ -226,7 +226,7 @@ The choice is yours. Generally recovery is better, but sometimes fail-fast is mo
|
||||
|
||||
| Property | Default | V2 | Meaning |
|
||||
|-----------------------------------------|---------|:----|-------------------------------------------------------|
|
||||
| `fs.s3a.connection.maximum` | `200` | | Connection pool size |
|
||||
| `fs.s3a.connection.maximum` | `500` | | Connection pool size |
|
||||
| `fs.s3a.connection.keepalive` | `false` | `*` | Use TCP keepalive on open channels |
|
||||
| `fs.s3a.connection.acquisition.timeout` | `60s` | `*` | Timeout for waiting for a connection from the pool. |
|
||||
| `fs.s3a.connection.establish.timeout` | `30s` | | Time to establish the TCP/TLS connection |
|
||||
|
Loading…
Reference in New Issue
Block a user