Sean Mackrory
|
1a25bbe9ec
|
HADOOP-15845. Require explicit URI on CLI for s3guard init and destroy. Contributed by Gabor Bota.
|
2018-12-11 08:33:13 -07:00 |
|
Sean Mackrory
|
c35de95a22
|
HADOOP-15987. ITestDynamoDBMetadataStore should check if table configured properly. Contributed by Gabor Bota.
|
2018-12-11 08:29:39 -07:00 |
|
Sean Mackrory
|
3ff8580f22
|
HADOOP-15428. s3guard bucket-info will create s3guard table if FS is set to do this automatically. (Contributed by Gabor Bota)
|
2018-12-10 14:03:08 -07:00 |
|
Ewan Higgs
|
c1d24f8483
|
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests.
Contributed by Ewan Higgs and Steve Loughran.
|
2018-11-29 15:12:17 +00:00 |
|
Sean Mackrory
|
085f10e75d
|
HADOOP-15947. Fix ITestDynamoDBMetadataStore test error issues. Contributed by Gabor Bota.
|
2018-11-28 10:45:09 -07:00 |
|
Sean Mackrory
|
e148c3ff09
|
HADOOP-15798. LocalMetadataStore put() does not retain isDeleted in parent listing. Contributed by Gabor Bota.
|
2018-11-28 10:45:09 -07:00 |
|
Steve Loughran
|
4c106fca0c
|
HADOOP-15932. Oozie unable to create sharelib in s3a filesystem.
Contributed by Steve Loughran.
|
2018-11-27 20:39:54 +00:00 |
|
Steve Loughran
|
ee816f1fd7
|
HADOOP-15837. DynamoDB table Update can fail S3A FS init.
Contributed by Steve Loughran.
|
2018-10-11 14:57:38 +01:00 |
|
Aaron Fabbri
|
4f752d442b
|
HADOOP-15621 2/2 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota
|
2018-10-03 00:24:29 -07:00 |
|
Aaron Fabbri
|
046b8768af
|
HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota
|
2018-10-02 21:22:49 -07:00 |
|
Mingliang Liu
|
c07715e378
|
HADOOP-15781 S3A assumed role tests failing due to changed error text in AWS exceptions. Contributed by Steve Loughran
|
2018-09-24 12:53:21 -07:00 |
|
Steve Loughran
|
26d0c63a1e
|
HADOOP-15754. s3guard: testDynamoTableTagging should clear existing config.
Contributed by Gabor Bota.
|
2018-09-17 22:40:08 +01:00 |
|
Steve Loughran
|
d7c0a08a1c
|
HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran)
|
2018-09-12 21:04:49 -07:00 |
|
Aaron Fabbri
|
d32a8d5d58
|
HADOOP-14734 add option to tag DDB table(s) created. (Contributed by Gabor Bota and Abe Fine)
|
2018-09-12 16:36:01 -07:00 |
|
Mingliang Liu
|
1f6c4545cf
|
HADOOP-15750. Remove obsolete S3A test ITestS3ACredentialsInURL. Contributed by Steve Loughran
|
2018-09-12 10:58:39 -07:00 |
|
Sean Mackrory
|
47b72c87eb
|
HADOOP-15635. s3guard set-capacity command to fail fast if bucket is unguarded.
Contributed by Gabor Bota.
|
2018-09-12 09:12:38 -06:00 |
|
Mingliang Liu
|
87f63b6479
|
HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran
|
2018-09-11 17:18:42 -07:00 |
|
Steve Loughran
|
5a0babf765
|
HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs.
Contributed by Steve Loughran.
|
2018-08-30 14:49:53 +01:00 |
|
Aaron Fabbri
|
d7232857d8
|
HADOOP-14154 Persist isAuthoritative bit in DynamoDBMetaStore (Contributed by Gabor Bota)
|
2018-08-17 10:15:39 -07:00 |
|
Akira Ajisaka
|
3e3963b035
|
HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering.
|
2018-08-16 00:31:59 +09:00 |
|
Ewan Higgs
|
a13929ddcb
|
HADOOP-15645. ITestS3GuardToolLocal.testDiffCommand fails if bucket has per-bucket binding to DDB. Contributed by Steve Loughran.
|
2018-08-13 12:57:45 +02:00 |
|
Steve Loughran
|
da9a39eed1
|
HADOOP-15583. Stabilize S3A Assumed Role support.
Contributed by Steve Loughran.
|
2018-08-08 22:57:24 -07:00 |
|
Ewan Higgs
|
2ec97abb2e
|
HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran.
|
2018-08-08 13:50:23 +02:00 |
|
Steve Loughran
|
48673bc2a8
|
HADOOP-15626. FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append.
Contributed by Steve Loughran.
|
2018-08-03 16:06:00 -07:00 |
|
Sean Mackrory
|
59adeb8d7f
|
HADOOP-15636. Follow-up from HADOOP-14918; restoring test under new name. Contributed by Gabor Bota.
|
2018-07-27 18:23:29 -06:00 |
|
Aaron Fabbri
|
93ac01cb59
|
HADOOP-15215 s3guard set-capacity command to fail on read/write of 0 (Gabor Bota)
|
2018-07-03 13:50:11 -07:00 |
|
Akira Ajisaka
|
2b2399d623
|
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma.
|
2018-06-28 14:37:22 +09:00 |
|
Sean Mackrory
|
c687a6617d
|
HADOOP-15423. Merge fileCache and dirCache into ine single cache in LocalMetadataStore. Contributed by Gabor Bota.
|
2018-06-25 14:59:41 -06:00 |
|
Sean Mackrory
|
b089a06793
|
HADOOP-14918. Remove the Local Dynamo DB test option. Contributed by Gabor Bota.
|
2018-06-20 16:45:08 -06:00 |
|
Aaron Fabbri
|
5f6769f796
|
HADOOP-15480 AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo (Gabor Bota)
|
2018-05-29 19:20:22 -07:00 |
|
Aaron Fabbri
|
30284d020d
|
HADOOP-14946 S3Guard testPruneCommandCLI can fail. Contributed by Gabor Bota.
|
2018-05-29 13:38:15 -07:00 |
|
Steve Loughran
|
cc3600aabd
|
HADOOP-15469. S3A directory committer commit job fails if _temporary directory created under dest.
Contributed by Steve Loughran.
(cherry picked from commit 170f1040d46f9b1a084e6637def91e9864446acc)
|
2018-05-17 19:08:55 +01:00 |
|
Steve Loughran
|
b670837469
|
HADOOP-15442. ITestS3AMetrics.testMetricsRegister can't know metrics source's name.
Contributed by Sean Mackrory.
|
2018-05-15 16:13:56 +01:00 |
|
Aaron Fabbri
|
8981674bbc
|
HADOOP-15420 s3guard ITestS3GuardToolLocal failures in diff tests (Gabor Bota)
|
2018-05-08 18:58:42 -07:00 |
|
Aaron Fabbri
|
69aac696d9
|
HADOOP-13649 s3guard: implement time-based (TTL) expiry for LocalMetadataStore (Gabor Bota)
|
2018-05-08 15:29:54 -07:00 |
|
Aaron Fabbri
|
96c843f64b
|
HADOOP-15444 ITestS3GuardToolDynamo should only run with -Ddynamo (Aaron Fabbri)
|
2018-05-04 11:34:45 -07:00 |
|
Aaron Fabbri
|
919865a34b
|
HADOOP-15239 S3ABlockOutputStream.flush() be no-op when stream closed. Contributed by Gabor Bota.
|
2018-04-30 16:02:57 -07:00 |
|
Aaron Fabbri
|
7d8bcf534a
|
HADOOP-13756 LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash. (Gabor Bota)
|
2018-04-26 20:41:29 -07:00 |
|
Aaron Fabbri
|
989a3929a9
|
HADOOP-14756 S3Guard: expose capability query in MetadataStore and add tests of authoritative mode (Gabor Bota)
|
2018-04-23 15:26:53 -07:00 |
|
Aaron Fabbri
|
b0aff8a962
|
HADOOP-15346 S3ARetryPolicy for 400/BadArgument to be "fail". Contributed by Steve Loughran.
|
2018-04-10 23:55:38 -07:00 |
|
Aaron Fabbri
|
ea3849f0cc
|
HADOOP-14759 S3GuardTool prune to prune specific bucket entries. Contributed by Gabor Bota.
|
2018-04-05 20:23:17 -07:00 |
|
Steve Loughran
|
1976e0066e
|
HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories.
Contributed by Steve Loughran.
|
2018-03-15 18:05:14 +00:00 |
|
Steve Loughran
|
dd05871b8b
|
HADOOP-15297. Make S3A etag => checksum feature optional.
Contributed by Steve Loughran.
|
2018-03-12 14:01:42 +00:00 |
|
Steve Loughran
|
e0307e53e2
|
HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled.
Contributed by Anis Elleuch.
(cherry picked from commit 1dedc68f9d8d8544d715e67ee77cd3f017c21699)
|
2018-03-07 18:51:10 +00:00 |
|
Steve Loughran
|
8110d6a0d5
|
HADOOP-13761. S3Guard: implement retries for DDB failures and throttling; translate exceptions.
Contributed by Aaron Fabbri.
|
2018-03-05 14:06:20 +00:00 |
|
Steve Loughran
|
7ac88244c5
|
HADOOP-14507. Extend per-bucket secret key config with explicit getPassword() on fs.s3a.$bucket.secret.key.
Contributed by Steve Loughran.
|
2018-02-16 16:37:06 +00:00 |
|
Steve Loughran
|
9a013b255f
|
HADOOP-15176. Enhance IAM Assumed Role support in S3A client.
Contributed by Steve Loughran
(cherry picked from commit 96c047fbb98c2378eed9693a724d4cbbd03c00fd)
|
2018-02-15 15:57:10 +00:00 |
|
Steve Loughran
|
1093a73689
|
HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits.
Contributed by Aaron Fabbri
|
2018-01-18 13:13:58 +00:00 |
|
Steve Loughran
|
f274fe33ea
|
Revert "HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits."
This reverts commit 35ad9b1dd2.
|
2018-01-18 12:35:57 +00:00 |
|
Aaron Fabbri
|
268ab4e027
|
HADOOP-15141 Support IAM Assumed roles in S3A. Contributed by Steve Loughran.
|
2018-01-17 00:05:24 -08:00 |
|
Steve Loughran
|
a0c71dcc33
|
HADOOP-15079. ITestS3AFileOperationCost#testFakeDirectoryDeletion failing
after OutputCommitter patch.
Contributed by Steve Loughran
|
2018-01-15 11:33:32 +00:00 |
|
Steve Loughran
|
b62a5ece95
|
HADOOP-15161. s3a: Stream and common statistics missing from metrics
Contributed by Sean Mackrory
|
2018-01-09 18:46:52 +00:00 |
|
Steve Loughran
|
c8ff0cc304
|
HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls.
Contributed by Steve Loughran
|
2017-12-21 14:58:58 +00:00 |
|
Steve Loughran
|
ef450df443
|
HADOOP-15113. NPE in S3A getFileStatus: null instrumentation on using closed instance.
Contributed by Steve Loughran.
|
2017-12-21 14:15:53 +00:00 |
|
Steve Loughran
|
1ba491ff90
|
HADOOP-14965. S3a input stream "normal" fadvise mode to be adaptive
|
2017-12-20 18:25:33 +00:00 |
|
Steve Loughran
|
35ad9b1dd2
|
HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits.
Contributed by Aaron Fabbri
|
2017-12-18 21:19:06 +00:00 |
|
Kai Zheng
|
f86c81d923
|
HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu
|
2017-12-14 11:14:52 +08:00 |
|
Kai Zheng
|
28792b6b7f
|
Revert "HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu"
This reverts commit 479d6a5792262c977025c26fd4960574b0db6847
|
2017-12-14 11:05:11 +08:00 |
|
Kai Zheng
|
73b86979d6
|
HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu
|
2017-12-06 12:01:36 +08:00 |
|
Aaron Fabbri
|
6555af81a2
|
HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed by Younger and Sean Mackrory.
|
2017-12-05 11:06:32 -08:00 |
|
Steve Loughran
|
de8b6ca5ef
|
HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints.
Contributed by Steve Loughran and Ryan Blue.
|
2017-11-22 15:28:12 +00:00 |
|
Aaron Fabbri
|
47011d7dd3
|
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran
|
2017-09-25 15:59:38 -07:00 |
|
Aaron Fabbri
|
49467165a5
|
HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran.
|
2017-09-14 14:10:48 -07:00 |
|
Steve Loughran
|
5bbca80428
|
HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A.
Contributed by Aaron Fabbri
|
2017-09-08 12:07:02 +01:00 |
|
Steve Loughran
|
621b43e254
|
HADOOP-13345 HS3Guard: Improved Consistency for S3A.
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
|
2017-09-01 14:13:41 +01:00 |
|
Steve Loughran
|
ee243e5289
|
HADOOP-14190. Add more on S3 regions to the s3a documentation.
Contributed by Steve Loughran
|
2017-06-28 10:22:13 +01:00 |
|
Mingliang Liu
|
bc4dfe9c9c
|
HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu
|
2017-06-27 13:19:14 -07:00 |
|
Mingliang Liu
|
f214a9961f
|
HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran
|
2017-06-15 11:16:16 -07:00 |
|
Akira Ajisaka
|
6a52b5e144
|
HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu.
|
2017-05-25 15:36:44 +09:00 |
|
Mingliang Liu
|
6672810eea
|
HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran
|
2017-05-19 11:51:43 -07:00 |
|
Steve Loughran
|
ba70225cf6
|
HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries.
Contributed by Steve Loughran.
(cherry picked from commit 2ac5aab8d725f761a9f9723471a4426f6b5d78c4)
|
2017-05-18 15:44:39 +01:00 |
|
Steve Loughran
|
5f934f8386
|
HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus.
Contributed by Steve Moist.
|
2017-04-24 20:33:19 +01:00 |
|
Mingliang Liu
|
667966c13c
|
HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran
|
2017-04-20 17:13:36 -07:00 |
|
Chris Nauroth
|
b8305e6d06
|
HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. Contributed by Mingliang Liu.
|
2017-04-12 10:02:13 -07:00 |
|
Andrew Wang
|
4966a6e26e
|
HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy.
|
2017-03-29 14:37:21 -07:00 |
|
Andrew Wang
|
52b00600df
|
HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy.
|
2017-03-24 11:44:46 -07:00 |
|
Mingliang Liu
|
2e30aa72e0
|
HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu
|
2017-03-23 11:33:29 -07:00 |
|
Mingliang Liu
|
fa67a96d7b
|
HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu
|
2017-03-14 14:38:21 -07:00 |
|
Mingliang Liu
|
e6cda5819b
|
Revert "HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu"
This reverts commit b8c69557b7.
|
2017-03-14 13:03:42 -07:00 |
|
Mingliang Liu
|
b8c69557b7
|
HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu
|
2017-03-13 14:15:02 -07:00 |
|
Steve Loughran
|
092ec39fb9
|
HADOOP-14111 cut some obsolete, ignored s3 tests in TestS3Credentials.
Contributed by Yuanbo Liu
|
2017-03-10 17:43:22 +00:00 |
|
Steve Loughran
|
dab00da19f
|
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures.
Contributed by Steve Loughran.
(cherry picked from commit 29fe5af017b945d8750c074ca39031b5b777eddd)
|
2017-02-25 15:35:19 +00:00 |
|
Steve Loughran
|
9c22a91662
|
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory.
(cherry picked from commit ff87ca84418a710c6dc884fe8c70947fcc6489d5)
|
2017-02-24 10:41:36 +00:00 |
|
Mingliang Liu
|
0013090fb4
|
HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu
|
2017-02-22 13:34:20 -08:00 |
|
Steve Loughran
|
2158496f6b
|
HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory.
(cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)
|
2017-02-21 18:28:49 +00:00 |
|
Akira Ajisaka
|
b9f8491252
|
HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin.
|
2017-02-15 01:45:56 +09:00 |
|
Lei Xu
|
839b690ed5
|
HADOOP-13075. Add support for SSE-KMS and SSE-C in s3a filesystem. (Steve Moist via lei)
|
2017-02-11 13:59:03 -08:00 |
|
Steve Loughran
|
e648b6e138
|
HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran
|
2017-01-11 17:25:15 +00:00 |
|
Mingliang Liu
|
c6a3923245
|
HADOOP-13871. ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful. Contributed by Steve Loughran
|
2016-12-12 14:55:34 -08:00 |
|
Mingliang Liu
|
2ff84a0040
|
HADOOP-13857. S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs. Contributed by Steve Loughran
|
2016-12-02 13:36:04 -08:00 |
|
Mingliang Liu
|
d60a60be8a
|
HADOOP-13823. s3a rename: fail if dest file exists. Contributed by Steve Loughran
|
2016-11-28 16:30:29 -08:00 |
|
Steve Loughran
|
0de0c32ddd
|
HADOOP-13801 regression: ITestS3AMiniYarnCluster failing. Contributed by Steve Loughran
|
2016-11-23 21:37:39 +00:00 |
|
Mingliang Liu
|
af791b774b
|
HADOOP-13050. Upgrade to AWS SDK 10.11+. Contributed by Chris Nauroth and Steve Loughran
|
2016-11-22 14:00:35 -08:00 |
|
Steve Loughran
|
a1761a841e
|
HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi.
|
2016-10-31 20:54:46 +00:00 |
|
Chris Nauroth
|
1f8490a5ba
|
HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth.
|
2016-10-26 08:55:42 -07:00 |
|
Chris Nauroth
|
9cad3e2350
|
HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran.
|
2016-10-26 08:27:26 -07:00 |
|
Chris Nauroth
|
3372e94030
|
HADOOP-12774. s3a should use UGI.getCurrentUser.getShortname() for username. Contributed by Steve Loughran.
|
2016-10-24 21:54:06 -07:00 |
|
Chris Nauroth
|
d8fa1cfa67
|
HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth.
|
2016-10-24 21:22:34 -07:00 |
|
Anu Engineer
|
3fbf4cd5da
|
HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran.
|
2016-10-20 12:33:58 -07:00 |
|
Steve Loughran
|
6c348c5691
|
HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran
|
2016-10-18 21:16:02 +01:00 |
|