hadoop/hadoop-common-project/hadoop-common/src
Steve Loughran e0563fed50
HADOOP-18908. Improve S3A region handling. (#6187)
S3A region logic improved for better inference and
to be compatible with previous releases

1. If you are using an AWS S3 AccessPoint, its region is determined
   from the ARN itself.
2. If fs.s3a.endpoint.region is set and non-empty, it is used.
3. If fs.s3a.endpoint is an s3.*.amazonaws.com url, 
   the region is determined by by parsing the URL 
   Note: vpce endpoints are not handled by this.
4. If fs.s3a.endpoint.region==null, and none could be determined
   from the endpoint, use us-east-2 as default.
5. If fs.s3a.endpoint.region=="" then it is handed off to
   The default AWS SDK resolution process.

Consult the AWS SDK documentation for the details on its resolution
process, knowing that it is complicated and may use environment variables,
entries in ~/.aws/config, IAM instance information within
EC2 deployments and possibly even JSON resources on the classpath.
Put differently: it is somewhat brittle across deployments.

Contributed by Ahmar Suhail
2023-10-17 15:37:36 +01:00
..
contrib/bash-tab-completion
main HADOOP-18908. Improve S3A region handling. (#6187) 2023-10-17 15:37:36 +01:00
site HDFS-17200. Add some datanode related metrics to Metrics.md. (#6099). Contributed by huangzhaobo99 2023-10-06 12:40:44 +05:30
test HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996) 2023-10-13 13:48:38 +01:00
CMakeLists.txt HADOOP-17292. Using lz4-java in Lz4Codec (#2350) 2020-11-18 12:03:25 -08:00
config.h.cmake HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00