Kai Zheng
|
28792b6b7f
|
Revert "HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu"
This reverts commit 479d6a5792262c977025c26fd4960574b0db6847
|
2017-12-14 11:05:11 +08:00 |
|
Surendra Singh Lilhore
|
00129c5314
|
HDFS-12833. Distcp : Update the usage of delete option for dependency with update and overwrite option. Contributed by usharani.
|
2017-12-12 00:28:02 +05:30 |
|
Kai Zheng
|
ce04340ec7
|
HADOOP-15104. AliyunOSS: change the default value of max error retry. Contributed by Jinhu Wu
|
2017-12-08 23:03:54 +08:00 |
|
Kai Zheng
|
73b86979d6
|
HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu
|
2017-12-06 12:01:36 +08:00 |
|
Aaron Fabbri
|
6555af81a2
|
HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed by Younger and Sean Mackrory.
|
2017-12-05 11:06:32 -08:00 |
|
Steve Loughran
|
3150c019ae
|
HADOOP-15071 S3a troubleshooting docs to add a couple more failure modes.
Contributed by Steve Loughran
|
2017-12-05 15:05:41 +00:00 |
|
Steve Loughran
|
0ea182d0fa
|
HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0.
Contributed by Rajesh Balamohan
|
2017-11-28 11:52:59 +00:00 |
|
Rohith Sharma K S
|
3cd75845da
|
MAPREDUCE-7014. Fix java doc errors in jdk1.8. Contributed by Steve Loughran.
|
2017-11-27 22:01:00 +05:30 |
|
Steve Loughran
|
de8b6ca5ef
|
HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints.
Contributed by Steve Loughran and Ryan Blue.
|
2017-11-22 15:28:12 +00:00 |
|
Steve Loughran
|
c326fc89b0
|
HADOOP-15024 Support user agent configuration and include that & Hadoop version information to oss server.
Contributed by Sammi Chen.
|
2017-11-20 18:56:42 +00:00 |
|
Akira Ajisaka
|
6903cf096e
|
HADOOP-13514. Upgrade maven surefire plugin to 2.20.1
Signed-off-by: Allen Wittenauer <aw@apache.org>
|
2017-11-19 12:39:37 -08:00 |
|
Kai Zheng
|
18621af7ae
|
HADOOP-14993. AliyunOSS: Override listFiles and listLocatedStatus. Contributed Genmao Yu
|
2017-11-14 17:58:37 +08:00 |
|
Daniel Templeton
|
ba8136615a
|
YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton)
Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b
|
2017-11-09 12:09:48 -08:00 |
|
John Zhuge
|
bf6a660232
|
HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge.
|
2017-11-09 10:16:12 -08:00 |
|
Subru Krishnan
|
f2df6b8983
|
HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru).
|
2017-11-08 18:07:12 -08:00 |
|
Arun Suresh
|
53c0fb7efe
|
HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh)
|
2017-11-02 17:15:33 -07:00 |
|
Akira Ajisaka
|
cc3f3eca40
|
MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi.
|
2017-11-02 18:32:24 +09:00 |
|
Chris Douglas
|
139cc752d2
|
HADOOP-14991. Add missing figures to Resource Estimator tool. Contributed by Rui Li
|
2017-10-27 21:59:27 -07:00 |
|
Yufei Gu
|
99880d0a16
|
YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak)
|
2017-10-27 14:41:37 -07:00 |
|
Subru Krishnan
|
625039ef20
|
HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru).
|
2017-10-25 15:51:27 -07:00 |
|
Wangda Tan
|
281d83604d
|
YARN-7318. Fix shell check warnings of SLS. (Gergely Novák via wangda)
|
2017-10-20 15:54:15 -07:00 |
|
Steve Loughran
|
f36cbc8475
|
HADOOP-14942. DistCp#cleanup() should check whether jobFS is null.
Contributed by Andras Bokor.
|
2017-10-20 22:27:04 +01:00 |
|
Haibo Chen
|
46eb1033a8
|
YARN-6546. SLS is slow while loading 10k queues. (Yufei Gu via Haibo Chen)
|
2017-10-17 16:04:19 -07:00 |
|
Steve Loughran
|
9fcc3a1fc8
|
HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled.
Contributed by Santhosh G Nayak
|
2017-10-16 16:01:47 +01:00 |
|
Steve Loughran
|
3de574413c
|
HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB.
Contributed by Varada Hemeswari.
|
2017-10-11 18:06:43 +01:00 |
|
Steve Loughran
|
572cdb5463
|
HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB
Contributed by Kannapiran Srinivasan.
|
2017-10-06 17:43:38 +01:00 |
|
Haibo Chen
|
c071aad5da
|
YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen)
|
2017-10-05 15:50:13 -07:00 |
|
Steve Loughran
|
9288206cb3
|
HADOOP-14845. Azure wasb: getFileStatus not making any auth check.
Contributed by Sivaguru Sankaridurg
|
2017-10-05 15:05:23 +01:00 |
|
Yufei Gu
|
79e37dc6db
|
YARN-7211. AMSimulator in SLS does't work due to responseId refactoring.(Contributed by Botong Huang via Yufei Gu)
|
2017-10-03 15:10:50 -07:00 |
|
Akira Ajisaka
|
c9b525bd94
|
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari.
|
2017-10-04 07:01:54 +09:00 |
|
John Zhuge
|
8aca46e53f
|
HADOOP-14651. Update okhttp version to 2.7.5. Contributed by Ray Chiang and John Zhuge.
|
2017-09-29 12:15:06 -07:00 |
|
Steve Loughran
|
a530e7ab3b
|
HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB
Contributed by Varada Hemeswari
|
2017-09-28 19:52:56 +01:00 |
|
Aaron Fabbri
|
47011d7dd3
|
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran
|
2017-09-25 15:59:38 -07:00 |
|
Jason Lowe
|
b133dc5700
|
MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari
|
2017-09-22 15:21:42 -05:00 |
|
Steve Loughran
|
2d2d97fa7d
|
HADOOP-14553. Add (parallelized) integration tests to hadoop-azure
Contributed by Steve Loughran
|
2017-09-15 17:03:01 +01:00 |
|
Aaron Fabbri
|
49467165a5
|
HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran.
|
2017-09-14 14:10:48 -07:00 |
|
ChenSammi
|
e0b3c644e1
|
HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen
|
2017-09-14 09:15:29 +08:00 |
|
John Zhuge
|
a4661850c1
|
HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria.
|
2017-09-08 11:51:03 -07:00 |
|
Steve Loughran
|
5bbca80428
|
HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A.
Contributed by Aaron Fabbri
|
2017-09-08 12:07:02 +01:00 |
|
Steve Loughran
|
83449abb40
|
HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter.
Contributed by Yonger.
|
2017-09-07 19:55:56 +01:00 |
|
Steve Loughran
|
13eda50003
|
HADOOP-14520. WASB: Block compaction for Azure Block Blobs.
Contributed by Georgi Chalakov
|
2017-09-07 18:35:03 +01:00 |
|
Xiaoyu Yao
|
63720ef574
|
HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin.
|
2017-09-05 23:34:55 -07:00 |
|
John Zhuge
|
50506e90a8
|
HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge.
|
2017-09-05 23:26:57 -07:00 |
|
Steve Loughran
|
792eff9ea7
|
HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS.
Contributed by Sivaguru Sankaridurg
|
2017-09-05 14:16:57 +01:00 |
|
Andrew Wang
|
0d419c984f
|
Preparing for 3.1.0 development
|
2017-09-01 11:53:48 -07:00 |
|
Allen Wittenauer
|
a3fee475f7
|
HADOOP-14414. Calling maven-site-plugin directly for docs profile is unnecessary (Andras Bokor via aw)
|
2017-09-01 08:38:30 -07:00 |
|
Steve Loughran
|
621b43e254
|
HADOOP-13345 HS3Guard: Improved Consistency for S3A.
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
|
2017-09-01 14:13:41 +01:00 |
|
John Zhuge
|
7ecc6dbed6
|
HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria.
|
2017-08-31 21:13:22 -07:00 |
|
Allen Wittenauer
|
71bbb86d69
|
HADOOP-14670. Increase minimum cmake version for all platforms
Signed-off-by: Chris Douglas <cdouglas@apache.org>
|
2017-08-30 21:47:24 -07:00 |
|
Haibo Chen
|
d04f85f387
|
MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen)
|
2017-08-30 10:07:48 -07:00 |
|
Yufei Gu
|
26fafc3597
|
YARN-7077. TestAMSimulator and TestNMSimulator fail. (Contributed by Akira Ajisaka via Yufei Gu)
|
2017-08-29 16:33:51 -07:00 |
|
Steve Loughran
|
9374f38820
|
HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory
Contributed by Esfandiar Manii and Thomas Marquardt.
|
2017-08-29 19:03:44 +01:00 |
|
Steve Loughran
|
021974f4cb
|
HADOOP-14802. Add support for using container saskeys for all accesses.
Contributed by Sivaguru Sankaridurg
|
2017-08-29 19:02:43 +01:00 |
|
Andrew Wang
|
f29a0fc288
|
HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou.
|
2017-08-25 14:14:23 -07:00 |
|
Arpit Agarwal
|
8b7cbe3840
|
HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar.
|
2017-08-25 09:55:46 -07:00 |
|
Steve Loughran
|
27ab5f7385
|
HADOOP-14787. AliyunOSS: Implement the createNonRecursive operator.
Contributed by Genmao Yu
|
2017-08-22 11:55:48 +01:00 |
|
Kai Zheng
|
267e19a09f
|
HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu
|
2017-08-21 13:36:28 +08:00 |
|
Arun Suresh
|
8410d862d3
|
YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh)
|
2017-08-20 07:54:09 -07:00 |
|
Steve Loughran
|
c6b4e656b7
|
HADOOP-14769. WASB: delete recursive should not fail if a file is deleted.
Contributed by Thomas Marquardt
|
2017-08-18 14:13:40 +01:00 |
|
Andrew Wang
|
dd7916d3cd
|
HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas.
|
2017-08-17 09:35:36 -07:00 |
|
Chris Douglas
|
645a8f2a4d
|
HADOOP-14726. Mark FileStatus::isDir as final
|
2017-08-14 21:57:20 -07:00 |
|
John Zhuge
|
7769e96149
|
HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria.
|
2017-08-13 00:22:34 -07:00 |
|
Allen Wittenauer
|
4222c97108
|
HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw)
|
2017-08-11 09:25:56 -07:00 |
|
Steve Loughran
|
54356b1e83
|
HADOOP-14183. Remove service loader config file for wasb fs.
Contributed by Esfandiar Manii.
|
2017-08-10 16:46:33 +01:00 |
|
Steve Loughran
|
f4e1aa0508
|
HADOOP-14715. TestWasbRemoteCallHelper failing.
Contributed by Esfandiar Manii.
|
2017-08-08 23:37:47 +01:00 |
|
Chris Douglas
|
55a181f845
|
HADOOP-14730. Support protobuf FileStatus in AdlFileSystem.
|
2017-08-07 21:31:28 -07:00 |
|
Steve Loughran
|
d91b7a8451
|
HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek.
Contributed by Thomas Marquardt
|
2017-08-06 20:19:23 +01:00 |
|
Steve Loughran
|
7fc324aabd
|
HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws.
Contributed by Steve Loughran
(cherry picked from commit ced547d5f0dbea571cbc472c5f55fe89d5900a6f)
|
2017-08-04 11:09:08 +01:00 |
|
Sean Mackrory
|
1a1bf6b7d0
|
HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer.
|
2017-08-02 12:25:05 -06:00 |
|
Steve Loughran
|
778d4edd9a
|
HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded.
Contributed by Thomas Marquardt.
|
2017-08-01 21:36:28 +01:00 |
|
John Zhuge
|
f735ad1b67
|
HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge.
|
2017-07-28 09:46:54 -07:00 |
|
Jitendra Pandey
|
a92bf39e23
|
HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt.
|
2017-07-25 16:26:48 -07:00 |
|
Jitendra Pandey
|
f2921e51f0
|
HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov.
|
2017-07-24 23:01:01 -07:00 |
|
Ravi Prakash
|
94ca52ae9e
|
HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash.
|
2017-07-24 16:01:45 -07:00 |
|
Wei-Chiu Chuang
|
44350fdf49
|
HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default). Contributed by Bharat Viswanadham.
|
2017-07-20 18:23:13 -07:00 |
|
Haibo Chen
|
0ba8cda135
|
YARN-6685. Add job count in to SLS JSON input format. (Yufei Gu via Haibo Chen)
|
2017-07-20 08:15:46 -07:00 |
|
Jitendra Pandey
|
2843c688bc
|
HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg.
|
2017-07-19 00:13:06 -07:00 |
|
Jitendra Pandey
|
b0e78ae085
|
HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak.
|
2017-07-17 02:27:55 -07:00 |
|
Steve Loughran
|
7d272ea124
|
HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb.
Contributed by Varada Hemeswari
(cherry picked from commit 1e69e5260351effc8077d1bdc397cec57cf1ff1b)
|
2017-07-12 10:37:39 +01:00 |
|
Steve Loughran
|
d670c3a4da
|
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs.
Contributed by Thomas Marquardt
|
2017-07-11 21:34:27 +01:00 |
|
Haibo Chen
|
147df300bf
|
YARN-5067 Support specifying resources for AM containers in SLS. (Yufei Gu via Haibo Chen)
|
2017-06-30 17:03:44 -07:00 |
|
Mingliang Liu
|
38996fdcf0
|
HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak
|
2017-06-30 16:53:48 -07:00 |
|
Andrew Wang
|
af2773f609
|
Updating version for 3.0.0-beta1 development
|
2017-06-29 17:57:40 -07:00 |
|
Mingliang Liu
|
72993b33b7
|
HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran
Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc
|
2017-06-29 17:07:52 -07:00 |
|
Mingliang Liu
|
b08cc97396
|
HADOOP-14601. Azure: Reuse ObjectMapper. Contributed by Mingliang Liu
|
2017-06-29 09:22:00 -07:00 |
|
Allen Wittenauer
|
20ba86d66a
|
MAPREDUCE-6536. hadoop-pipes doesn't use maven properties for openssl
Signed-off-by: Ravi Prakash <raviprak@apache.org>
|
2017-06-28 17:33:44 -07:00 |
|
Mingliang Liu
|
990aa34de2
|
HADOOP-14609. NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code. Contributed by Steve Loughran
|
2017-06-28 14:18:59 -07:00 |
|
Steve Loughran
|
ee243e5289
|
HADOOP-14190. Add more on S3 regions to the s3a documentation.
Contributed by Steve Loughran
|
2017-06-28 10:22:13 +01:00 |
|
Mingliang Liu
|
7e031c2c18
|
HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas
|
2017-06-27 17:32:07 -07:00 |
|
Mingliang Liu
|
686a634f01
|
HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran
|
2017-06-27 16:48:47 -07:00 |
|
Mingliang Liu
|
bc4dfe9c9c
|
HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu
|
2017-06-27 13:19:14 -07:00 |
|
Steve Loughran
|
48f4a229a9
|
HADOOP-14461 Azure: handle failure gracefully in case of missing account access key.
Contributed by Mingliang Liu.
|
2017-06-26 13:39:47 +01:00 |
|
Akira Ajisaka
|
a594f96fb1
|
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he.
|
2017-06-26 16:41:00 +09:00 |
|
Steve Loughran
|
c6bd73c6c5
|
HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations.
Contributed by Thomas.
|
2017-06-23 21:38:41 +01:00 |
|
Jason Lowe
|
dd65eea74b
|
HADOOP-8143. Change distcp to have -pb on by default. Contributed by Mithun Radhakrishnan
|
2017-06-20 09:53:47 -05:00 |
|
Akira Ajisaka
|
3369540653
|
HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools.
|
2017-06-20 13:18:26 +09:00 |
|
Mingliang Liu
|
f214a9961f
|
HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran
|
2017-06-15 11:16:16 -07:00 |
|
Mingliang Liu
|
325163f23f
|
HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu
|
2017-06-09 13:07:11 -07:00 |
|
Mingliang Liu
|
df35e4cc3b
|
HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan
|
2017-06-07 13:54:52 -07:00 |
|
Carlo Curino
|
b65100c14b
|
YARN-6547. Enhance SLS-based tests leveraging invariant checker.
|
2017-06-06 17:19:15 -07:00 |
|