hadoop/hadoop-tools
Mukund Thakur 96d7ceb39a
HADOOP-13873. log DNS addresses on s3a initialization.
Contributed by Mukund Thakur.

If you set the log org.apache.hadoop.fs.s3a.impl.NetworkBinding
to DEBUG, then when the S3A bucket probe is made -the DNS address
of the S3 endpoint is calculated and printed.

This is useful to see if a large set of processes are all using
the same IP address from the pool of load balancers to which AWS
directs clients when an AWS S3 endpoint is resolved.

This can have implications for performance: if all clients
access the same load balancer performance may be suboptimal.

Note: if bucket probes are disabled, fs.s3a.bucket.probe = 0,
the DNS logging does not take place.

Change-Id: I21b3ac429dc0b543f03e357fdeb94c2d2a328dd8
2020-04-17 14:20:54 +01:00
..
hadoop-aliyun HADOOP-16840. AliyunOSS: getFileStatus throws FileNotFoundException in versioning bucket. Contributed by wujinhu. 2020-03-08 21:01:34 -07:00
hadoop-archive-logs HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-archives HADOOP-16512. [hadoop-tools] Fix order of actual and expected expression in assert statements 2019-10-07 16:38:08 +09:00
hadoop-aws HADOOP-13873. log DNS addresses on s3a initialization. 2020-04-17 14:20:54 +01:00
hadoop-azure HADOOP-16855. Changing wildfly dependency scope in hadoop-azure to compile 2020-04-14 19:18:15 +01:00
hadoop-azure-datalake HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden 2020-03-09 14:44:28 +00:00
hadoop-datajoin HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-distcp HADOOP-16932. distcp copy calls getFileStatus() needlessly and can fail against S3 (#1936) 2020-04-09 18:23:47 +01:00
hadoop-dynamometer YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal 2020-01-22 16:25:07 +01:00
hadoop-extras HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-fs2img HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
hadoop-gridmix HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-kafka HADOOP-16512. [hadoop-tools] Fix order of actual and expected expression in assert statements 2019-10-07 16:38:08 +09:00
hadoop-openstack HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden 2020-03-09 14:44:28 +00:00
hadoop-pipes HADOOP-16739. Fix native build failure of hadoop-pipes on CentOS 8. 2020-02-10 13:13:11 +09:00
hadoop-resourceestimator HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-rumen HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
hadoop-sls YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak. 2020-03-25 16:50:19 +05:30
hadoop-streaming YARN-8767. TestStreamingStatus fails. Contributed by Andras Bokor 2020-02-25 21:48:16 +01:00
hadoop-tools-dist HADOOP-16331. Fix ASF License check in pom.xml 2019-05-29 17:25:13 +09:00
pom.xml HDFS-12345 Add Dynamometer to hadoop-tools, a tool for scale testing the HDFS NameNode with real metadata and workloads. Contributed by Erik Krogen. 2019-06-25 08:07:39 -07:00