hadoop/hadoop-tools
Steve Loughran d378853790
HADOOP-18975 S3A: Add option fs.s3a.endpoint.fips to use AWS FIPS endpoints (#6277)
Adds a new option `fs.s3a.endpoint.fips` to switch the SDK client to use
FIPS endpoints, as an alternative to explicitly declaring them.


* The option is available as a path capability for probes.
* SDK v2 itself doesn't know that some regions don't have FIPS endpoints
* SDK only fails with endpoint + fips flag as a retried exception; wit this
  change the S3A client should fail fast.
  PR fails fast.
* Adds a new "connecting.md" doc; moves existing docs there and restructures.
* New Tests in ITestS3AEndpointRegion

bucket-info command support:

* added to list of path capabilities
* added -fips flag and test for explicit probe
* also now prints bucket region
* and removed some of the obsolete s3guard options
* updated docs

Contributed by Steve Loughran
2024-01-16 14:16:12 +00:00
..
hadoop-aliyun HADOOP-18458: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS (#4912) 2023-03-28 14:27:01 +08:00
hadoop-archive-logs HADOOP-18917. Addendum. Fix deprecation issues after commons-io upgrade. (#6228). Contributed by PJ Fanning. 2023-10-30 09:35:02 +05:30
hadoop-archives HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 2023-11-20 23:44:48 +05:30
hadoop-aws HADOOP-18975 S3A: Add option fs.s3a.endpoint.fips to use AWS FIPS endpoints (#6277) 2024-01-16 14:16:12 +00:00
hadoop-azure HADOOP-18971. [ABFS] Read and cache file footer with fs.azure.footer.read.request.size (#6270) 2024-01-03 12:49:52 +00:00
hadoop-azure-datalake HADOOP-18641. Cloud connector dependency and LICENSE fixup. (#5429) 2023-02-28 10:48:54 +00:00
hadoop-benchmark HADOOP-18718. Fix several maven build warnings (#5592). Contributed by Dongjoon Hyun. 2023-06-11 11:38:13 +05:30
hadoop-datajoin HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies (#4000) 2022-03-08 17:27:04 +09:00
hadoop-distcp HADOOP-18996. S3A to provide full support for S3 Express One Zone (#6308) 2023-12-01 14:16:33 +00:00
hadoop-dynamometer HADOOP-18359. Update commons-cli from 1.2 to 1.5. (#5095). Contributed by Shilun Fan. 2023-05-10 01:42:12 +05:30
hadoop-extras HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 2023-11-20 23:44:48 +05:30
hadoop-federation-balance HADOOP-18718. Fix several maven build warnings (#5592). Contributed by Dongjoon Hyun. 2023-06-11 11:38:13 +05:30
hadoop-fs2img HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 2023-11-20 23:44:48 +05:30
hadoop-gridmix HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 2023-11-20 23:44:48 +05:30
hadoop-kafka HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 2023-11-20 23:44:48 +05:30
hadoop-openstack HADOOP-18442. Remove openstack support (#4855) 2022-10-06 11:49:38 +01:00
hadoop-pipes Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-resourceestimator HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 2023-11-20 23:44:48 +05:30
hadoop-rumen HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 2023-11-20 23:44:48 +05:30
hadoop-sls YARN-11599. Incorrect log4j properties file in SLS sample conf (#6220) Contributed by Junfan Zhang. 2023-11-05 13:57:48 +08:00
hadoop-streaming HADOOP-18957. Use StandardCharsets.UTF_8 (#6231). Contributed by PJ Fanning. 2023-11-20 23:44:48 +05:30
hadoop-tools-dist HADOOP-18442. Remove openstack support (#4855) 2022-10-06 11:49:38 +01:00
pom.xml HADOOP-11867. Add a high-performance vectored read API. (#3904) 2022-06-22 17:29:32 +01:00