big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,650 Commits 2 Branches 2 Tags 264 MiB
253e3e78ab
Commit Graph

645 Commits

Author SHA1 Message Date
John Zhuge
253e3e78ab HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge. 2017-03-27 23:05:57 -07:00
John Zhuge
d1b7439b48 HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge. 2017-03-24 13:37:04 -07:00
Andrew Wang
52b00600df HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy. 2017-03-24 11:44:46 -07:00
Chris Douglas
c55195588f HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge 2017-03-23 12:17:27 -07:00
Chris Douglas
03955612b7 HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria 2017-03-23 12:13:23 -07:00
Mingliang Liu
2e30aa72e0 HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu 2017-03-23 11:33:29 -07:00
Steve Loughran
a5a4867f3b
HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject(). 
Contributed by Yuanbo Liu

(cherry picked from commit 20878d052cebc715c2494a97362fdff08885a77f)
 2017-03-23 12:54:54 +00:00
Robert Kanter
cc938e99ec YARN-6326. Shouldn't use AppAttemptIds to fetch applications while AM Simulator tracks app in SLS (yufeigu via rkanter) 2017-03-21 15:21:11 -07:00
Steve Loughran
2841666f1f
HADOOP-14204 S3A multipart commit failing, 
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".

Contributed by Steve Loughran

(cherry picked from commit 595501c9ddacd5fdb740ac1bf9bfeb0ffbbd6c0e)
 2017-03-21 13:15:15 +00:00
Mingliang Liu
8e15e24059 HADOOP-13945. Azure: Add Kerberos and Delegation token support to WASB client. Contributed by Santhosh G Nayak 
Change-Id: I39fb0a3b0491bf2160571366939a0502a0045429
 2017-03-20 16:31:24 -07:00
Arpit Agarwal
6c399a88e9 HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran. 2017-03-20 11:29:23 -07:00
Mingliang Liu
86035c1644 HADOOP-14192. Aliyun OSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu 2017-03-17 11:01:00 -07:00
Mingliang Liu
fa67a96d7b HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu 2017-03-14 14:38:21 -07:00
Mingliang Liu
e6cda5819b Revert "HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu" 
This reverts commit b8c69557b7.
 2017-03-14 13:03:42 -07:00
Mingliang Liu
b8c69557b7 HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu 2017-03-13 14:15:02 -07:00
John Zhuge
5a40bafdae HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge. 
Change-Id: I5dc6f885816b8834f718874542dfa373458b0333
 2017-03-13 08:11:25 -07:00
Steve Loughran
092ec39fb9
HADOOP-14111 cut some obsolete, ignored s3 tests in TestS3Credentials. 
Contributed by Yuanbo Liu
 2017-03-10 17:43:22 +00:00
Mingliang Liu
881ec4d97b HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu 2017-03-10 00:16:09 -08:00
John Zhuge
c5ee7fded4 HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge. 
Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e
 2017-03-09 18:30:17 -08:00
John Zhuge
98142d2f72 Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" 
This reverts commit 53a12fa721.
 2017-03-08 09:54:22 -08:00
Mingliang Liu
28daaf0eb2 HADOOP-14150. Implement getHomeDirectory() method in NativeAzureFileSystem. Contributed by Santhosh G Nayak 2017-03-07 14:55:52 -08:00
Robert Kanter
1598fd3b79 YARN-6275. Fail to show real-time tracking charts in SLS (yufeigu via rkanter) 2017-03-07 13:47:52 -08:00
Akira Ajisaka
f597f4c43e
HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri. 2017-03-07 15:14:55 +09:00
Mingliang Liu
686823529b HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth 2017-03-06 17:16:36 -08:00
Mingliang Liu
52d7d5aa1a Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth" 
This reverts commit 6b7cd62b8c.
 2017-03-06 17:10:11 -08:00
Mingliang Liu
c571cda5c7 HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA 2017-03-06 17:00:13 -08:00
Mingliang Liu
6b7cd62b8c HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth 2017-03-03 17:09:12 -08:00
Akira Ajisaka
490abfb10f HADOOP-14057. Fix package.html to compile with Java 9. 2017-03-04 00:25:22 +09:00
Steve Loughran
a97833e0ed
HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. 
Contributed by Steve Loughran

(cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)
 2017-03-02 21:44:30 +00:00
Allen Wittenauer
989bd56b9f HADOOP-12556. KafkaSink jar files are created but not copied to target dist (Babak Behzad via aw) 2017-02-28 16:15:40 -08:00
Wangda Tan
eac6b4c35c YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda) 2017-02-28 10:35:50 -08:00
Akira Ajisaka
258342e76c HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR. 2017-02-28 13:47:44 +09:00
Yongjun Zhang
d235dcdf0b HADOOP-14127. Add log4j configuration to enable logging in hadoop-distcp's tests. (Xiao Chen via Yongjun Zhang) 2017-02-27 20:42:13 -08:00
Steve Loughran
dab00da19f
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures. 
Contributed by Steve Loughran.

(cherry picked from commit 29fe5af017b945d8750c074ca39031b5b777eddd)
 2017-02-25 15:35:19 +00:00
Sunil G
b32ffa2753 YARN-4779. Fix AM container allocation logic in SLS. Contributed by Wangda Tan. 2017-02-24 21:39:25 +05:30
Steve Loughran
e60c6543d5
HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran 2017-02-24 13:24:59 +00:00
Steve Loughran
9c22a91662
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory. 
(cherry picked from commit ff87ca84418a710c6dc884fe8c70947fcc6489d5)
 2017-02-24 10:41:36 +00:00
Mingliang Liu
82607fce39 HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari. 2017-02-23 13:48:44 -08:00
Mingliang Liu
a4d4a23785 HADOOP-13321. Deprecate FileSystem APIs that promote inefficient call patterns. Contributed by Chris Nauroth and Mingliang Liu 2017-02-23 12:55:40 -08:00
Mingliang Liu
0013090fb4 HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu 2017-02-22 13:34:20 -08:00
Steve Loughran
4f4250fbcc HADOOP-14099 Split S3 testing documentation out into its own file. Contributed by Steve Loughran. 2017-02-22 11:43:48 +00:00
Mingliang Liu
924def7854 HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane 2017-02-21 13:44:42 -08:00
Steve Loughran
2158496f6b HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory. 
(cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)
 2017-02-21 18:28:49 +00:00
Steve Loughran
8035749c26 HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan 2017-02-20 16:21:46 +00:00
Steve Loughran
3a2e30fa9f HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge 
(cherry picked from commit b1c1f05b1dc997906390d653dfafb4f0d7e193c4)
 2017-02-18 18:17:11 +00:00
Mingliang Liu
658702efff HADOOP-14040. Use shaded aws-sdk uber-JAR 1.11.86. Contributed by Steve Loughran and Sean Mackrory 2017-02-16 16:51:03 -08:00
Mingliang Liu
bdad8b7b97 HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran 2017-02-16 16:41:31 -08:00
Mingliang Liu
f432999025 HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane 2017-02-16 15:14:25 -08:00
Kai Zheng
cd3e59a3dc HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu 2017-02-15 16:34:30 +08:00
Akira Ajisaka
b9f8491252 HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin. 2017-02-15 01:45:56 +09:00