Go to file
Steve Loughran e0563fed50
HADOOP-18908. Improve S3A region handling. (#6187)
S3A region logic improved for better inference and
to be compatible with previous releases

1. If you are using an AWS S3 AccessPoint, its region is determined
   from the ARN itself.
2. If fs.s3a.endpoint.region is set and non-empty, it is used.
3. If fs.s3a.endpoint is an s3.*.amazonaws.com url, 
   the region is determined by by parsing the URL 
   Note: vpce endpoints are not handled by this.
4. If fs.s3a.endpoint.region==null, and none could be determined
   from the endpoint, use us-east-2 as default.
5. If fs.s3a.endpoint.region=="" then it is handed off to
   The default AWS SDK resolution process.

Consult the AWS SDK documentation for the details on its resolution
process, knowing that it is complicated and may use environment variables,
entries in ~/.aws/config, IAM instance information within
EC2 deployments and possibly even JSON resources on the classpath.
Put differently: it is somewhat brittle across deployments.

Contributed by Ahmar Suhail
2023-10-17 15:37:36 +01:00
.github HADOOP-18823. Add Labeler Github Action. (#5874). Contributed by Ayush Saxena. 2023-07-25 03:04:49 +05:30
.yetus Add .yetus/excludes.txt (#4984) 2022-10-11 09:23:34 -07:00
dev-support HADOOP-18789. Remove ozone from hadoop dev support. (#5800). Contributed by Xiaoqiao He. 2023-07-02 15:23:32 +08:00
hadoop-assemblies HDFS-15346. FedBalance tool implementation. Contributed by Jinglun. 2020-06-18 13:33:25 +08:00
hadoop-build-tools HADOOP-17968 Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports (#3584) 2021-10-28 15:57:15 +09:00
hadoop-client-modules HADOOP-18916. Exclude all module-info classes from uber jars (#6131) 2023-10-13 20:01:44 +01:00
hadoop-cloud-storage-project HADOOP-18890. Remove use of okhttp in runtime code (#6057) 2023-09-19 12:38:36 +01:00
hadoop-common-project HADOOP-18908. Improve S3A region handling. (#6187) 2023-10-17 15:37:36 +01:00
hadoop-dist HADOOP-18718. Fix several maven build warnings (#5592). Contributed by Dongjoon Hyun. 2023-06-11 11:38:13 +05:30
hadoop-hdfs-project HDFS-17210. Optimize AvailableSpaceBlockPlacementPolicy. (#6113). Contributed by GuoPhilipse. 2023-10-16 16:34:40 +08:00
hadoop-mapreduce-project HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996) 2023-10-13 13:48:38 +01:00
hadoop-maven-plugins HADOOP-18441. Remove hadoop custom ServicesResourceTransformer (#4850). Contributed by PJ Fanning. 2022-09-07 17:11:12 +05:30
hadoop-minicluster HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies (#4000) 2022-03-08 17:27:04 +09:00
hadoop-project HADOOP-18932. S3A. upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565 (#6178) 2023-10-17 12:59:50 +01:00
hadoop-project-dist HADOOP-18751. Fix incorrect output path in javadoc build phase (#5688) 2023-06-26 15:52:17 -07:00
hadoop-tools HADOOP-18908. Improve S3A region handling. (#6187) 2023-10-17 15:37:36 +01:00
hadoop-yarn-project YARN.11590. RM process stuck after calling confStore.format() when ZK SSL/TLS is enabled, as netty thread waits indefinitely. Contributed by Ferenc Erdelyi 2023-10-16 15:17:58 -04:00
licenses HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina. 2020-10-18 18:37:46 +05:30
licenses-binary HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. (#1796) 2020-01-09 16:24:58 +09:00
.asf.yaml HADOOP-18630. Add gh-pages in asf.yaml to deploy the current trunk doc (#5393). Contributed by Simhadri Govindappa. 2023-02-14 18:13:29 +05:30
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore HADOOP-18774. Add .vscode to gitignore. (#5756). Contributed by Xiaoqiao He. 2023-06-18 14:08:38 +08:00
BUILDING.txt HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996) 2023-10-13 13:48:38 +01:00
LICENSE-binary HADOOP-18932. S3A. upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565 (#6178) 2023-10-17 12:59:50 +01:00
LICENSE.txt YARN-11356. Upgrade DataTables to 1.11.5 to fix CVEs. Contributed by Bence Kosztolnik. 2022-10-26 22:29:01 +02:00
NOTICE-binary HADOOP-18890. Remove use of okhttp in runtime code (#6057) 2023-09-19 12:38:36 +01:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml HADOOP-18923. Switch to SPDX identifier for license name (#6149). Contributed by Colm O hEigeartaigh. 2023-10-07 22:50:38 +05:30
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
start-build-env.sh HADOOP-18052. Support Apple Silicon in start-build-env.sh (#3817) 2021-12-23 18:13:18 +09:00

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/