96d7ceb39a
Contributed by Mukund Thakur. If you set the log org.apache.hadoop.fs.s3a.impl.NetworkBinding to DEBUG, then when the S3A bucket probe is made -the DNS address of the S3 endpoint is calculated and printed. This is useful to see if a large set of processes are all using the same IP address from the pool of load balancers to which AWS directs clients when an AWS S3 endpoint is resolved. This can have implications for performance: if all clients access the same load balancer performance may be suboptimal. Note: if bucket probes are disabled, fs.s3a.bucket.probe = 0, the DNS logging does not take place. Change-Id: I21b3ac429dc0b543f03e357fdeb94c2d2a328dd8 |
||
|---|---|---|
| .. | ||
| dev-support | ||
| src | ||
| pom.xml | ||