Steve Loughran
ee243e5289
HADOOP-14190. Add more on S3 regions to the s3a documentation.
...
Contributed by Steve Loughran
2017-06-28 10:22:13 +01:00
Mingliang Liu
7e031c2c18
HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas
2017-06-27 17:32:07 -07:00
Mingliang Liu
686a634f01
HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran
2017-06-27 16:48:47 -07:00
Mingliang Liu
bc4dfe9c9c
HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu
2017-06-27 13:19:14 -07:00
Steve Loughran
48f4a229a9
HADOOP-14461 Azure: handle failure gracefully in case of missing account access key.
...
Contributed by Mingliang Liu.
2017-06-26 13:39:47 +01:00
Akira Ajisaka
a594f96fb1
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he.
2017-06-26 16:41:00 +09:00
Steve Loughran
c6bd73c6c5
HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations.
...
Contributed by Thomas.
2017-06-23 21:38:41 +01:00
Jason Lowe
dd65eea74b
HADOOP-8143. Change distcp to have -pb on by default. Contributed by Mithun Radhakrishnan
2017-06-20 09:53:47 -05:00
Akira Ajisaka
3369540653
HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools.
2017-06-20 13:18:26 +09:00
Mingliang Liu
f214a9961f
HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran
2017-06-15 11:16:16 -07:00
Mingliang Liu
325163f23f
HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu
2017-06-09 13:07:11 -07:00
Mingliang Liu
df35e4cc3b
HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan
2017-06-07 13:54:52 -07:00
Carlo Curino
b65100c14b
YARN-6547. Enhance SLS-based tests leveraging invariant checker.
2017-06-06 17:19:15 -07:00
Mingliang Liu
536f057158
HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu
2017-06-06 11:09:28 -07:00
Mingliang Liu
6b5285bbcb
HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu
2017-06-06 11:06:49 -07:00
Mingliang Liu
5fd9742c83
HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan
2017-06-05 15:56:43 -07:00
Mingliang Liu
6aeda55bb8
HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu
2017-06-05 11:26:56 -07:00
Mingliang Liu
ece33208b8
HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu
2017-06-01 11:52:11 -07:00
Akira Ajisaka
6e6e96583f
HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.
2017-06-01 13:08:01 +09:00
Akira Ajisaka
07e60f85d8
HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.
2017-05-30 15:11:10 +09:00
John Zhuge
6c6a7a5962
HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge.
2017-05-28 22:25:00 -07:00
Mingliang Liu
89bb8bfe58
HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari
2017-05-26 17:54:00 -07:00
Andrew Wang
16ad896d5c
Update maven version for 3.0.0-alpha4 development
2017-05-26 14:09:44 -07:00
Akira Ajisaka
6a52b5e144
HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu.
2017-05-25 15:36:44 +09:00
Sunil G
b6f66b0da1
YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.
2017-05-22 14:10:06 +05:30
Yufei Gu
fcbdecc54d
YARN-6111. Rumen input does't work in SLS. Contributed by Yufei Gu.
2017-05-21 21:04:06 -07:00
Mingliang Liu
6672810eea
HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran
2017-05-19 11:51:43 -07:00
Yongjun Zhang
b4adc8392c
HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang)
2017-05-18 15:35:22 -07:00
Sunil G
40e6a85d25
YARN-6560. SLS doesn't honor node total resource specified in sls-runner.xml. Contributed by Wangda Tan.
2017-05-18 22:49:42 +05:30
Steve Loughran
ba70225cf6
HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries.
...
Contributed by Steve Loughran.
(cherry picked from commit 2ac5aab8d725f761a9f9723471a4426f6b5d78c4)
2017-05-18 15:44:39 +01:00
Akira Ajisaka
d4aa9e3c69
HADOOP-14415. Use java.lang.AssertionError instead of junit.framework.AssertionFailedError. Contributed by Chen Liang.
2017-05-17 15:38:28 -04:00
Robert Kanter
101852ca11
YARN-6535. Program needs to exit when SLS finishes. (yufeigu via rkanter)
2017-05-16 17:52:17 -07:00
Mingliang Liu
b415c6fe74
HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger. Contributed by Sivaguru Sankaridurg
2017-05-16 11:22:32 -07:00
Akira Ajisaka
0d5c8ed8e0
HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor.
2017-05-11 16:37:32 -05:00
Mingliang Liu
eed731496f
HADOOP-14361. Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently. Contributed by Santhosh G Nayak
2017-05-10 15:05:11 -07:00
Carlo Curino
5cb6e3e082
YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations.
2017-05-10 14:49:25 -07:00
Akira Ajisaka
2ba9903932
HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang.
2017-05-10 05:57:12 -05:00
Akira Ajisaka
c60164fb60
HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong.
2017-05-09 05:37:43 -05:00
Andrew Wang
54fd0e44b7
HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy.
2017-05-08 21:59:49 -07:00
Andrew Wang
8065129d87
HADOOP-14298. TestHadoopArchiveLogsRunner fails. Contribute dby Akira Ajisaka.
2017-05-07 13:59:15 -07:00
Steve Loughran
4e6bbd049d
HADOOP-14382 Remove usages of MoreObjects.toStringHelper.
...
Contributed by Andrew Wang
2017-05-05 13:03:09 +01:00
Robert Kanter
3082552b3b
YARN-6522. Make SLS JSON input file format simple and scalable (yufeigu via rkanter)
2017-05-04 17:21:46 -07:00
Mingliang Liu
83dded556d
HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran
2017-05-03 11:22:44 -07:00
Andrew Wang
b062b323b7
HADOOP-14281. Fix TestKafkaMetrics#testPutMetrics. Contributed by Alison Yu.
2017-05-02 11:49:19 -07:00
Akira Ajisaka
62579b69a0
HADOOP-11614. Remove httpclient dependency from hadoop-openstack. Contributed by Akira Ajisaka, Brahma Reddy Battula, and Steve Loughran.
2017-04-27 16:57:08 +09:00
Mingliang Liu
8b5f2c372e
HADOOP-14351. Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache. Contributed by Santhosh G Nayak
2017-04-26 13:47:18 -07:00
Mingliang Liu
e4c32bccde
HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu
2017-04-25 20:54:56 -07:00
Robert Kanter
475f933b41
YARN-6423. Queue metrics doesn't work for Fair Scheduler in SLS (yufeigu via rkanter)
2017-04-25 16:26:09 -07:00
Steve Loughran
5f934f8386
HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus.
...
Contributed by Steve Moist.
2017-04-24 20:33:19 +01:00
Wangda Tan
de69d6e811
YARN-6363. Extending SLS: Synthetic Load Generator. (Carlo Curino via wangda)
2017-04-20 21:54:30 -07:00
Mingliang Liu
667966c13c
HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran
2017-04-20 17:13:36 -07:00
Andrew Wang
7e075a50e3
YARN-6365. Get static SLS html resources from classpath. Contributed by Yufei Gu.
2017-04-19 11:08:59 -07:00
Steve Loughran
6b015d00c9
HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs.
...
Contributed by Steve Loughran
2017-04-19 10:21:44 +01:00
Mingliang Liu
0cab57223e
HADOOP-14274. Azure: Simplify Ranger-WASB policy model. Contributed by Sivaguru Sankaridurg
2017-04-12 16:07:10 -07:00
Mingliang Liu
b053fdc547
HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu
2017-04-12 14:30:34 -07:00
Chris Nauroth
b8305e6d06
HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. Contributed by Mingliang Liu.
2017-04-12 10:02:13 -07:00
John Zhuge
56e81f2a20
HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge.
...
Signed-off-by: John Zhuge <jzhuge@apache.org>
2017-04-07 12:04:44 -07:00
Andrew Wang
7eea8d83fa
HADOOP-14280. Fix compilation of TestKafkaMetrics.
2017-04-05 15:12:03 -07:00
Mingliang Liu
56ab02eed9
HADOOP-14273. Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag. Contributed by Santhosh G Nayak
2017-04-04 11:10:28 -07:00
Mingliang Liu
27a44b60c1
HADOOP-14272. Azure: WasbRemoteCallHelper should use String equals for comparison. Contributed by Santhosh G Nayak
2017-04-04 11:03:59 -07:00
Chris Douglas
6eba79232f
HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang
2017-04-03 20:13:14 -07:00
Mingliang Liu
5faa949b78
HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka
2017-04-03 11:07:14 -07:00
Mingliang Liu
26172a94d6
HADOOP-14267. Make DistCpOptions immutable. Contributed by Mingliang Liu
2017-03-31 20:04:26 -07:00
Yongjun Zhang
bf3fb585aa
HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li.
2017-03-30 17:38:56 -07:00
Yongjun Zhang
144f1cf765
Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen."
...
This reverts commit 064c8b25ec
2017-03-30 17:38:18 -07:00
Yongjun Zhang
064c8b25ec
HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen.
2017-03-30 17:01:15 -07:00
Wei-Chiu Chuang
8c591b8d19
HDFS-10974. Document replication factor for EC files. Contributed by Yiqun Lin.
2017-03-30 11:16:05 -07:00
Akira Ajisaka
0d053eeb30
HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula.
2017-03-30 18:11:50 +09:00
Robert Kanter
6a5516c238
YARN-5654. Not be able to run SLS with FairScheduler (yufeigu via rkanter)
2017-03-29 16:18:13 -07:00
Andrew Wang
4966a6e26e
HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy.
2017-03-29 14:37:21 -07:00
Andrew Wang
0e6f8e4bc6
HDFS-10971. Distcp should not copy replication factor if source file is erasure coded. Contributed by Manoj Govindassamy.
2017-03-28 22:14:03 -07:00
John Zhuge
253e3e78ab
HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge.
2017-03-27 23:05:57 -07:00
John Zhuge
d1b7439b48
HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge.
2017-03-24 13:37:04 -07:00
Andrew Wang
52b00600df
HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy.
2017-03-24 11:44:46 -07:00
Chris Douglas
c55195588f
HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge
2017-03-23 12:17:27 -07:00
Chris Douglas
03955612b7
HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria
2017-03-23 12:13:23 -07:00
Mingliang Liu
2e30aa72e0
HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu
2017-03-23 11:33:29 -07:00
Steve Loughran
a5a4867f3b
HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject().
...
Contributed by Yuanbo Liu
(cherry picked from commit 20878d052cebc715c2494a97362fdff08885a77f)
2017-03-23 12:54:54 +00:00
Robert Kanter
cc938e99ec
YARN-6326. Shouldn't use AppAttemptIds to fetch applications while AM Simulator tracks app in SLS (yufeigu via rkanter)
2017-03-21 15:21:11 -07:00
Steve Loughran
2841666f1f
HADOOP-14204 S3A multipart commit failing,
...
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".
Contributed by Steve Loughran
(cherry picked from commit 595501c9ddacd5fdb740ac1bf9bfeb0ffbbd6c0e)
2017-03-21 13:15:15 +00:00
Mingliang Liu
8e15e24059
HADOOP-13945. Azure: Add Kerberos and Delegation token support to WASB client. Contributed by Santhosh G Nayak
...
Change-Id: I39fb0a3b0491bf2160571366939a0502a0045429
2017-03-20 16:31:24 -07:00
Arpit Agarwal
6c399a88e9
HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran.
2017-03-20 11:29:23 -07:00
Mingliang Liu
86035c1644
HADOOP-14192. Aliyun OSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu
2017-03-17 11:01:00 -07:00
Mingliang Liu
fa67a96d7b
HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu
2017-03-14 14:38:21 -07:00
Mingliang Liu
e6cda5819b
Revert "HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu"
...
This reverts commit b8c69557b7
2017-03-14 13:03:42 -07:00
Mingliang Liu
b8c69557b7
HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu
2017-03-13 14:15:02 -07:00
John Zhuge
5a40bafdae
HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge.
...
Change-Id: I5dc6f885816b8834f718874542dfa373458b0333
2017-03-13 08:11:25 -07:00
Steve Loughran
092ec39fb9
HADOOP-14111 cut some obsolete, ignored s3 tests in TestS3Credentials.
...
Contributed by Yuanbo Liu
2017-03-10 17:43:22 +00:00
Mingliang Liu
881ec4d97b
HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu
2017-03-10 00:16:09 -08:00
John Zhuge
c5ee7fded4
HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge.
...
Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e
2017-03-09 18:30:17 -08:00
John Zhuge
98142d2f72
Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran"
...
This reverts commit 53a12fa721
2017-03-08 09:54:22 -08:00
Mingliang Liu
28daaf0eb2
HADOOP-14150. Implement getHomeDirectory() method in NativeAzureFileSystem. Contributed by Santhosh G Nayak
2017-03-07 14:55:52 -08:00
Robert Kanter
1598fd3b79
YARN-6275. Fail to show real-time tracking charts in SLS (yufeigu via rkanter)
2017-03-07 13:47:52 -08:00
Akira Ajisaka
f597f4c43e
HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri.
2017-03-07 15:14:55 +09:00
Mingliang Liu
686823529b
HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth
2017-03-06 17:16:36 -08:00
Mingliang Liu
52d7d5aa1a
Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth"
...
This reverts commit 6b7cd62b8c
2017-03-06 17:10:11 -08:00
Mingliang Liu
c571cda5c7
HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA
2017-03-06 17:00:13 -08:00
Mingliang Liu
6b7cd62b8c
HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth
2017-03-03 17:09:12 -08:00
Akira Ajisaka
490abfb10f
HADOOP-14057. Fix package.html to compile with Java 9.
2017-03-04 00:25:22 +09:00
Steve Loughran
a97833e0ed
HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry.
...
Contributed by Steve Loughran
(cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)
2017-03-02 21:44:30 +00:00
