big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-18328. Add documentation for S3A support on S3 Outposts (#5976)

Browse Source 
Contributed by Yuting Chen
This commit is contained in:
Yuting Chen 2023-08-24 02:16:10 -07:00 committed by GitHub
parent 439c8653cf
commit ce5bc4891f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
View File

@ -1709,6 +1709,24 @@ the storage class you want.
Please note that S3A does not support reading from archive storage classes at the moment. Please note that S3A does not support reading from archive storage classes at the moment.
`AccessDeniedException` with InvalidObjectState will be thrown if you're trying to do so. `AccessDeniedException` with InvalidObjectState will be thrown if you're trying to do so.
## <a name="upload"></a>Configuring S3A for S3 on Outposts
S3A now supports [S3 on Outposts](https://docs.aws.amazon.com/AmazonS3/latest/userguide/S3onOutposts.html).
Accessing data through an access point is done by using its Amazon Resource Name (ARN), as opposed to just the bucket name.
The only supported storage class on Outposts is **OUTPOSTS**, and by default objects are encrypted with [SSE-S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-outposts-data-encryption.html).
You can set the Access Point ARN property using the following per bucket configuration property:
```xml
<property>
<name>fs.s3a.bucket.sample-outpost-bucket.accesspoint.arn</name>
<value>arn:aws:s3-outposts:region:account-id:outpost/outpost-id/accesspoint/accesspoint-name</value>
<description>Configure S3a traffic to use this S3 on Outposts Access Point ARN</description>
</property>
```
This configures access to the `sample-outpost-bucket` for S3A to go through the new Access Point ARN. So, for example `s3a://sample-outpost-bucket/key` will now use your configured ARN when getting data from S3 on Outpost instead of your bucket.
## <a name="upload"></a>How S3A writes data to S3 ## <a name="upload"></a>How S3A writes data to S3
The original S3A client implemented file writes by The original S3A client implemented file writes by