Commit Graph

731 Commits

Author SHA1 Message Date
Jitendra Pandey
f2921e51f0 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-24 23:01:01 -07:00
Ravi Prakash
94ca52ae9e HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 2017-07-24 16:01:45 -07:00
Wei-Chiu Chuang
44350fdf49 HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default). Contributed by Bharat Viswanadham. 2017-07-20 18:23:13 -07:00
Haibo Chen
0ba8cda135 YARN-6685. Add job count in to SLS JSON input format. (Yufei Gu via Haibo Chen) 2017-07-20 08:15:46 -07:00
Jitendra Pandey
2843c688bc HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 00:13:06 -07:00
Jitendra Pandey
b0e78ae085 HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak. 2017-07-17 02:27:55 -07:00
Steve Loughran
7d272ea124
HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb.
Contributed by Varada Hemeswari

(cherry picked from commit 1e69e5260351effc8077d1bdc397cec57cf1ff1b)
2017-07-12 10:37:39 +01:00
Steve Loughran
d670c3a4da
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs.
Contributed by Thomas Marquardt
2017-07-11 21:34:27 +01:00
Haibo Chen
147df300bf YARN-5067 Support specifying resources for AM containers in SLS. (Yufei Gu via Haibo Chen) 2017-06-30 17:03:44 -07:00
Mingliang Liu
38996fdcf0 HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak 2017-06-30 16:53:48 -07:00
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Mingliang Liu
72993b33b7 HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran
Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc
2017-06-29 17:07:52 -07:00
Mingliang Liu
b08cc97396 HADOOP-14601. Azure: Reuse ObjectMapper. Contributed by Mingliang Liu 2017-06-29 09:22:00 -07:00
Allen Wittenauer
20ba86d66a MAPREDUCE-6536. hadoop-pipes doesn't use maven properties for openssl
Signed-off-by: Ravi Prakash <raviprak@apache.org>
2017-06-28 17:33:44 -07:00
Mingliang Liu
990aa34de2 HADOOP-14609. NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code. Contributed by Steve Loughran 2017-06-28 14:18:59 -07:00
Steve Loughran
ee243e5289
HADOOP-14190. Add more on S3 regions to the s3a documentation.
Contributed by Steve Loughran
2017-06-28 10:22:13 +01:00
Mingliang Liu
7e031c2c18 HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas 2017-06-27 17:32:07 -07:00
Mingliang Liu
686a634f01 HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran 2017-06-27 16:48:47 -07:00
Mingliang Liu
bc4dfe9c9c HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu 2017-06-27 13:19:14 -07:00
Steve Loughran
48f4a229a9
HADOOP-14461 Azure: handle failure gracefully in case of missing account access key.
Contributed by Mingliang Liu.
2017-06-26 13:39:47 +01:00
Akira Ajisaka
a594f96fb1
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he. 2017-06-26 16:41:00 +09:00
Steve Loughran
c6bd73c6c5
HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations.
Contributed by Thomas.
2017-06-23 21:38:41 +01:00
Jason Lowe
dd65eea74b HADOOP-8143. Change distcp to have -pb on by default. Contributed by Mithun Radhakrishnan 2017-06-20 09:53:47 -05:00
Akira Ajisaka
3369540653
HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools. 2017-06-20 13:18:26 +09:00
Mingliang Liu
f214a9961f HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran 2017-06-15 11:16:16 -07:00
Mingliang Liu
325163f23f HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu 2017-06-09 13:07:11 -07:00
Mingliang Liu
df35e4cc3b HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan 2017-06-07 13:54:52 -07:00
Carlo Curino
b65100c14b YARN-6547. Enhance SLS-based tests leveraging invariant checker. 2017-06-06 17:19:15 -07:00
Mingliang Liu
536f057158 HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu 2017-06-06 11:09:28 -07:00
Mingliang Liu
6b5285bbcb HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu 2017-06-06 11:06:49 -07:00
Mingliang Liu
5fd9742c83 HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan 2017-06-05 15:56:43 -07:00
Mingliang Liu
6aeda55bb8 HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 2017-06-05 11:26:56 -07:00
Mingliang Liu
ece33208b8 HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu 2017-06-01 11:52:11 -07:00
Akira Ajisaka
6e6e96583f
HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang. 2017-06-01 13:08:01 +09:00
Akira Ajisaka
07e60f85d8
HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu. 2017-05-30 15:11:10 +09:00
John Zhuge
6c6a7a5962 HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge. 2017-05-28 22:25:00 -07:00
Mingliang Liu
89bb8bfe58 HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari 2017-05-26 17:54:00 -07:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Akira Ajisaka
6a52b5e144
HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu. 2017-05-25 15:36:44 +09:00
Sunil G
b6f66b0da1 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-22 14:10:06 +05:30
Yufei Gu
fcbdecc54d YARN-6111. Rumen input does't work in SLS. Contributed by Yufei Gu. 2017-05-21 21:04:06 -07:00
Mingliang Liu
6672810eea HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran 2017-05-19 11:51:43 -07:00
Yongjun Zhang
b4adc8392c HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang) 2017-05-18 15:35:22 -07:00
Sunil G
40e6a85d25 YARN-6560. SLS doesn't honor node total resource specified in sls-runner.xml. Contributed by Wangda Tan. 2017-05-18 22:49:42 +05:30
Steve Loughran
ba70225cf6
HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries.
Contributed by Steve Loughran.

(cherry picked from commit 2ac5aab8d725f761a9f9723471a4426f6b5d78c4)
2017-05-18 15:44:39 +01:00
Akira Ajisaka
d4aa9e3c69
HADOOP-14415. Use java.lang.AssertionError instead of junit.framework.AssertionFailedError. Contributed by Chen Liang. 2017-05-17 15:38:28 -04:00
Robert Kanter
101852ca11 YARN-6535. Program needs to exit when SLS finishes. (yufeigu via rkanter) 2017-05-16 17:52:17 -07:00
Mingliang Liu
b415c6fe74 HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger. Contributed by Sivaguru Sankaridurg 2017-05-16 11:22:32 -07:00
Akira Ajisaka
0d5c8ed8e0
HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor. 2017-05-11 16:37:32 -05:00
Mingliang Liu
eed731496f HADOOP-14361. Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently. Contributed by Santhosh G Nayak 2017-05-10 15:05:11 -07:00