big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-18328. Add documentation for S3A support on S3 Outposts (#5976)

Browse Source 
Contributed by Yuting Chen
This commit is contained in:
Yuting Chen 2023-08-24 02:16:10 -07:00 committed by GitHub
parent 439c8653cf
commit ce5bc4891f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
View File

@ -1709,6 +1709,24 @@ the storage class you want.
Please note that S3A does not support reading from archive storage classes at the moment.
`AccessDeniedException` with InvalidObjectState will be thrown if you're trying to do so.
## <a name="upload"></a>Configuring S3A for S3 on Outposts
S3A now supports [S3 on Outposts](https://docs.aws.amazon.com/AmazonS3/latest/userguide/S3onOutposts.html).
Accessing data through an access point is done by using its Amazon Resource Name (ARN), as opposed to just the bucket name.
The only supported storage class on Outposts is **OUTPOSTS**, and by default objects are encrypted with [SSE-S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-outposts-data-encryption.html).
You can set the Access Point ARN property using the following per bucket configuration property:
```xml
<property>
<name>fs.s3a.bucket.sample-outpost-bucket.accesspoint.arn</name>
<value>arn:aws:s3-outposts:region:account-id:outpost/outpost-id/accesspoint/accesspoint-name</value>
<description>Configure S3a traffic to use this S3 on Outposts Access Point ARN</description>
</property>
```
This configures access to the `sample-outpost-bucket` for S3A to go through the new Access Point ARN. So, for example `s3a://sample-outpost-bucket/key` will now use your configured ARN when getting data from S3 on Outpost instead of your bucket.
## <a name="upload"></a>How S3A writes data to S3
The original S3A client implemented file writes by