big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,647 Commits 2 Branches 2 Tags 264 MiB
a8302e398c
Commit Graph

1053 Commits

Author SHA1 Message Date
Da Zhou
a8302e398c
HADOOP-15872. ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2. 
Contributed by Junhua Gu and Da Zhou.
 2018-11-23 14:17:36 +00:00
Weiwei Yang
5ff0cf86a9 HADOOP-15943. AliyunOSS: add missing owner & group attributes for oss FileStatus. Contributed by wujinhu. 2018-11-23 13:56:54 +08:00
Weiwei Yang
be0708c6eb HADOOP-15919. AliyunOSS: Enable Yarn to use OSS. Contributed by wujinhu. 2018-11-19 14:00:54 +08:00
Arpit Agarwal
914b0cf15f HADOOP-12558. distcp documentation is woefully out of date. Contributed by Dinesh Chitlangia. 2018-11-15 13:58:13 -08:00
Akira Ajisaka
66b1335bb3
HADOOP-15926. Document upgrading the section in NOTICE.txt when upgrading the version of AWS SDK. Contributed by Dinesh Chitlangia. 2018-11-15 16:30:24 +09:00
Sammi Chen
3fade865ce HADOOP-15917. AliyunOSS: fix incorrect ReadOps and WriteOps in statistics. Contributed by Jinhu Wu. 2018-11-14 12:58:57 +08:00
Da Zhou
a13be203b7
HADOOP-15876. Use keySet().removeAll() to remove multiple keys from Map in AzureBlobFileSystemStore 
Contributed by Da Zhou.
 2018-11-13 21:46:18 +00:00
Da Zhou
9dbb2b67c6
HADOOP-15812. ABFS: Improve AbfsRestOperationException format to ensure full msg can be displayed on console. 
Author:    Da Zhou <da.zhou@microsoft.com>
 2018-11-09 11:06:23 +00:00
Junhua Gu
724c15007b
HADOOP-15846. ABFS: fix mask related bugs in setAcl, modifyAclEntries and removeAclEntries. 
Contributed by Junhua Gu.

(cherry picked from commit 66715005f9e8f4f25faa352a06d142b75a029f0e)
 2018-11-08 17:21:40 +00:00
Sammi Chen
7574d18538 HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu. 2018-10-26 15:19:56 +08:00
Daniel Templeton
e374584479 HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI 
(Contributed by Da Zhou via Daniel Templeton)

Change-Id: I546ab3a1df1efec635c08c388148e718dc4a9843
 2018-10-24 14:18:35 -07:00
Robert Kanter
5e294e950b YARN-8919. Some tests fail due to NoClassDefFoundError for OperatorCreationException (tasanuma0829 via rkanter) 2018-10-23 13:37:17 -07:00
Ted Yu
e2cecb681e HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-19 13:21:06 -07:00
Giovanni Matteo Fumarola
cc09b2b0c7 HDFS-14007. Incompatible layout when generating FSImage. Contributed by Inigo Goiri. 2018-10-18 17:24:23 -07:00
Steve Loughran
d59ca43bff
HADOOP-15826. @Retries annotation of putObject() call & uses wrong. 
Contributed by Steve Loughran and Ewan Higgs.
 2018-10-16 20:02:54 +01:00
Steve Loughran
ef9dc6c44c
HADOOP-15851. Disable wildfly logs to the console. 
Contributed by Vishwajeet Dusane.
 2018-10-15 22:53:55 +01:00
bibinchundatt
b4a38e7b3e YARN-8830. SLS tool fix node addition. Contributed by Bibin A Chundatt. 2018-10-15 16:10:25 +05:30
Steve Loughran
e36ae9639f
HADOOP-15831. Include modificationTime in the toString method of CopyListingFileStatus. 
Contributed by Ted Yu.
 2018-10-12 09:59:19 +01:00
Steve Loughran
ee816f1fd7
HADOOP-15837. DynamoDB table Update can fail S3A FS init. 
Contributed by Steve Loughran.
 2018-10-11 14:57:38 +01:00
Steve Loughran
bd50fa956b
HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. 
Contributed by Da Zhou.
 2018-10-09 20:02:12 +01:00
Jason Lowe
4b5b1ac3d1 MAPREDUCE-7130. Rumen crashes trying to handle MRAppMaster recovery events. Contributed by Peter Bacsko 2018-10-09 13:27:03 -05:00
Steve Loughran
7ba1cfdea7
HADOOP-15827. NPE in DynamoDBMetadataStore.lambda$listChildren for root + auth S3Guard. 
Contributed by Gabor Bota
 2018-10-09 10:46:41 +01:00
Steve Loughran
273cc2d4e9
HADOOP-15809. ABFS: better exception handling when making getAccessToken call. 
Contributed by Da Zhou
 2018-10-05 11:29:03 +01:00
Steve Loughran
7051bd78b1
HADOOP-15795. Make HTTPS the default protocol for ABFS. 
Contributed by Da Zhou.
 2018-10-03 12:52:53 +01:00
Aaron Fabbri
4f752d442b
HADOOP-15621 2/2 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota 2018-10-03 00:24:29 -07:00
Aaron Fabbri
046b8768af
HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota 2018-10-02 21:22:49 -07:00
Steve Loughran
e8b8604314
HADOOP-15792. typo in AzureBlobFileSystem.getIsNamespaceEnabeld. 
Contributed by Abhishek Modi.
 2018-10-02 18:57:20 +01:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Steve Loughran
a383ac47ca
HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore. 
Contributed by Da Zhou
 2018-10-02 11:42:52 +01:00
Steve Loughran
43bc984891
HADOOP-15793. ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization 
Contributed by Yuan Gao.
 2018-10-02 11:37:28 +01:00
Steve Loughran
a4abf02028
HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies. 
Contributed by Da Zhou.
 2018-09-25 20:58:32 +01:00
Steve Loughran
d5da9928c9 HADOOP-15723. ABFS: Ranger Support. 
Contributed by Yuan Gao.
 2018-09-25 19:13:10 +01:00
Sammi Chen
2b635125fb HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 2018-09-25 19:48:30 +08:00
Mingliang Liu
c07715e378 HADOOP-15781 S3A assumed role tests failing due to changed error text in AWS exceptions. Contributed by Steve Loughran 2018-09-24 12:53:21 -07:00
Sunil G
d060cbea48 HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran. 2018-09-24 21:19:47 +05:30
Sean Mackrory
0def61482b Merge branch 'HADOOP-15407' into trunk 2018-09-22 21:19:12 -06:00
Steve Loughran
d0b4624c88
HADOOP-15778. ABFS: Fix client side throttling for read. 
Contributed by Sneha Varma.
 2018-09-21 11:06:24 +01:00
Steve Loughran
a5692c2da5 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable. 
Contributed by Steve Loughran.
 2018-09-20 17:36:18 +01:00
Sean Mackrory
8e831ba458 HADOOP-15773. Fixing checkstyle and other issues raised by Yetus. 2018-09-19 16:56:33 -06:00
Steve Loughran
a55d26b23e
HADOOP-15769. ABFS: distcp tests are always skipped. 
Contributed by Steve Loughran
 2018-09-19 13:57:39 +01:00
Steve Loughran
df2166a643
HADOOP-15719. Fail-fast when using OAuth over http. 
Contributed by Da Zhou.
 2018-09-18 12:20:52 +01:00
Steve Loughran
51d368982b
HADOOP-15714. Tune abfs/wasb parallel and sequential test execution. 
Contributed by Da Zhou.
 2018-09-18 12:09:25 +01:00
Steve Loughran
524776625d
HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min. 
Contributed by Da Zhou
 2018-09-18 11:48:46 +01:00
Steve Loughran
1cf38a38da
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. 
Contributed by Steve Loughran.
 2018-09-18 10:56:56 +01:00
Steve Loughran
26d0c63a1e
HADOOP-15754. s3guard: testDynamoTableTagging should clear existing config. 
Contributed by Gabor Bota.
 2018-09-17 22:40:08 +01:00
Thomas Marquardt
b4c23043d3 HADOOP-15757. ABFS: remove dependency on common-codec Base64. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt
26211019c8 HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt
e5593cbd83 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. 
Contributed by Sean Mackrory.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt
13c70e9ba3 HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. 
Contributed by Sneha Varma.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt
6801b30733 HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation 
for storage account with no namespace feature.

Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00