Commit Graph

1048 Commits

Author SHA1 Message Date
Sammi Chen
3fade865ce HADOOP-15917. AliyunOSS: fix incorrect ReadOps and WriteOps in statistics. Contributed by Jinhu Wu. 2018-11-14 12:58:57 +08:00
Da Zhou
a13be203b7
HADOOP-15876. Use keySet().removeAll() to remove multiple keys from Map in AzureBlobFileSystemStore
Contributed by Da Zhou.
2018-11-13 21:46:18 +00:00
Da Zhou
9dbb2b67c6
HADOOP-15812. ABFS: Improve AbfsRestOperationException format to ensure full msg can be displayed on console.
Author:    Da Zhou <da.zhou@microsoft.com>
2018-11-09 11:06:23 +00:00
Junhua Gu
724c15007b
HADOOP-15846. ABFS: fix mask related bugs in setAcl, modifyAclEntries and removeAclEntries.
Contributed by Junhua Gu.

(cherry picked from commit 66715005f9e8f4f25faa352a06d142b75a029f0e)
2018-11-08 17:21:40 +00:00
Sammi Chen
7574d18538 HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu. 2018-10-26 15:19:56 +08:00
Daniel Templeton
e374584479 HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI
(Contributed by Da Zhou via Daniel Templeton)

Change-Id: I546ab3a1df1efec635c08c388148e718dc4a9843
2018-10-24 14:18:35 -07:00
Robert Kanter
5e294e950b YARN-8919. Some tests fail due to NoClassDefFoundError for OperatorCreationException (tasanuma0829 via rkanter) 2018-10-23 13:37:17 -07:00
Ted Yu
e2cecb681e HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-19 13:21:06 -07:00
Giovanni Matteo Fumarola
cc09b2b0c7 HDFS-14007. Incompatible layout when generating FSImage. Contributed by Inigo Goiri. 2018-10-18 17:24:23 -07:00
Steve Loughran
d59ca43bff
HADOOP-15826. @Retries annotation of putObject() call & uses wrong.
Contributed by Steve Loughran and Ewan Higgs.
2018-10-16 20:02:54 +01:00
Steve Loughran
ef9dc6c44c
HADOOP-15851. Disable wildfly logs to the console.
Contributed by Vishwajeet Dusane.
2018-10-15 22:53:55 +01:00
bibinchundatt
b4a38e7b3e YARN-8830. SLS tool fix node addition. Contributed by Bibin A Chundatt. 2018-10-15 16:10:25 +05:30
Steve Loughran
e36ae9639f
HADOOP-15831. Include modificationTime in the toString method of CopyListingFileStatus.
Contributed by Ted Yu.
2018-10-12 09:59:19 +01:00
Steve Loughran
ee816f1fd7
HADOOP-15837. DynamoDB table Update can fail S3A FS init.
Contributed by Steve Loughran.
2018-10-11 14:57:38 +01:00
Steve Loughran
bd50fa956b
HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth.
Contributed by Da Zhou.
2018-10-09 20:02:12 +01:00
Jason Lowe
4b5b1ac3d1 MAPREDUCE-7130. Rumen crashes trying to handle MRAppMaster recovery events. Contributed by Peter Bacsko 2018-10-09 13:27:03 -05:00
Steve Loughran
7ba1cfdea7
HADOOP-15827. NPE in DynamoDBMetadataStore.lambda$listChildren for root + auth S3Guard.
Contributed by Gabor Bota
2018-10-09 10:46:41 +01:00
Steve Loughran
273cc2d4e9
HADOOP-15809. ABFS: better exception handling when making getAccessToken call.
Contributed by Da Zhou
2018-10-05 11:29:03 +01:00
Steve Loughran
7051bd78b1
HADOOP-15795. Make HTTPS the default protocol for ABFS.
Contributed by Da Zhou.
2018-10-03 12:52:53 +01:00
Aaron Fabbri
4f752d442b
HADOOP-15621 2/2 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota 2018-10-03 00:24:29 -07:00
Aaron Fabbri
046b8768af
HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota 2018-10-02 21:22:49 -07:00
Steve Loughran
e8b8604314
HADOOP-15792. typo in AzureBlobFileSystem.getIsNamespaceEnabeld.
Contributed by Abhishek Modi.
2018-10-02 18:57:20 +01:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Steve Loughran
a383ac47ca
HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore.
Contributed by Da Zhou
2018-10-02 11:42:52 +01:00
Steve Loughran
43bc984891
HADOOP-15793. ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization
Contributed by Yuan Gao.
2018-10-02 11:37:28 +01:00
Steve Loughran
a4abf02028
HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies.
Contributed by Da Zhou.
2018-09-25 20:58:32 +01:00
Steve Loughran
d5da9928c9 HADOOP-15723. ABFS: Ranger Support.
Contributed by Yuan Gao.
2018-09-25 19:13:10 +01:00
Sammi Chen
2b635125fb HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 2018-09-25 19:48:30 +08:00
Mingliang Liu
c07715e378 HADOOP-15781 S3A assumed role tests failing due to changed error text in AWS exceptions. Contributed by Steve Loughran 2018-09-24 12:53:21 -07:00
Sunil G
d060cbea48 HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran. 2018-09-24 21:19:47 +05:30
Sean Mackrory
0def61482b Merge branch 'HADOOP-15407' into trunk 2018-09-22 21:19:12 -06:00
Steve Loughran
d0b4624c88
HADOOP-15778. ABFS: Fix client side throttling for read.
Contributed by Sneha Varma.
2018-09-21 11:06:24 +01:00
Steve Loughran
a5692c2da5 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable.
Contributed by Steve Loughran.
2018-09-20 17:36:18 +01:00
Sean Mackrory
8e831ba458 HADOOP-15773. Fixing checkstyle and other issues raised by Yetus. 2018-09-19 16:56:33 -06:00
Steve Loughran
a55d26b23e
HADOOP-15769. ABFS: distcp tests are always skipped.
Contributed by Steve Loughran
2018-09-19 13:57:39 +01:00
Steve Loughran
df2166a643
HADOOP-15719. Fail-fast when using OAuth over http.
Contributed by Da Zhou.
2018-09-18 12:20:52 +01:00
Steve Loughran
51d368982b
HADOOP-15714. Tune abfs/wasb parallel and sequential test execution.
Contributed by Da Zhou.
2018-09-18 12:09:25 +01:00
Steve Loughran
524776625d
HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min.
Contributed by Da Zhou
2018-09-18 11:48:46 +01:00
Steve Loughran
1cf38a38da
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch.
Contributed by Steve Loughran.
2018-09-18 10:56:56 +01:00
Steve Loughran
26d0c63a1e
HADOOP-15754. s3guard: testDynamoTableTagging should clear existing config.
Contributed by Gabor Bota.
2018-09-17 22:40:08 +01:00
Thomas Marquardt
b4c23043d3 HADOOP-15757. ABFS: remove dependency on common-codec Base64.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
26211019c8 HADOOP-15753. ABFS: support path "abfs://mycluster/file/path"
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
e5593cbd83 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts.
Contributed by Sean Mackrory.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
13c70e9ba3 HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept.
Contributed by Sneha Varma.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
6801b30733 HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation
for storage account with no namespace feature.

Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
347a52a866 Fixing findbugs and license issues related to:
HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
97f06b3fc7 HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
4410eacba7 HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
81dc4a995c HADOOP-15663. ABFS: Simplify configuration.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt
df57c6c3b1 HADOOP-15692. ABFS: extensible support for custom oauth.
Contributed by Junhua Gu and Rajeev Bansal.
2018-09-17 19:54:01 +00:00