hadoop

Author	SHA1	Message	Date
Steve Loughran	52babbb4a0	HADOOP-15086. NativeAzureFileSystem file rename is not atomic. Contributed by Thomas Marquardt	2017-12-22 11:39:55 +00:00
Steve Loughran	c8ff0cc304	HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls. Contributed by Steve Loughran	2017-12-21 14:58:58 +00:00
Steve Loughran	ef450df443	HADOOP-15113. NPE in S3A getFileStatus: null instrumentation on using closed instance. Contributed by Steve Loughran.	2017-12-21 14:15:53 +00:00
Akira Ajisaka	5ab632baf5	HDFS-12949. Fix findbugs warning in ImageWriter.java.	2017-12-21 10:04:34 +09:00
Steve Loughran	1ba491ff90	HADOOP-14965. S3a input stream "normal" fadvise mode to be adaptive	2017-12-20 18:25:33 +00:00
Steve Loughran	35ad9b1dd2	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. Contributed by Aaron Fabbri	2017-12-18 21:19:06 +00:00
Chris Douglas	4b3a785914	HDFS-12903. [READ] Fix closing streams in ImageWriter. Contributed by Virajith Jalaparti	2017-12-15 17:51:42 -08:00
Chris Douglas	e515103a83	Revert "HDFS-12903. [READ] Fix closing streams in ImageWriter" This reverts commit c1bf2654b0e9118985b8518b0254eac4dd302a2f.	2017-12-15 17:51:42 -08:00
Virajith Jalaparti	8239e3afb3	HDFS-12712. [9806] Code style cleanup	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	80c3fec3a1	HDFS-12912. [READ] Fix configuration and implementation of LevelDB-based alias maps	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	c89b29bd42	HDFS-12893. [READ] Support replication of Provided blocks with non-default topologies.	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	0f6aa9564c	HDFS-12905. [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	962b5e722b	HDFS-12903. [READ] Fix closing streams in ImageWriter	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	4531588a94	HDFS-11640. [READ] Datanodes should use a unique identifier when reading from external stores	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	fb996a32a9	HDFS-12894. [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	9c35be86e1	HDFS-12713. [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata. Contributed by Ewan Higgs	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	a027055dd2	HDFS-12885. Add visibility/stability annotations. Contributed by Chris Douglas	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	352f994b64	HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs.	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	4d59dabb7f	HDFS-12809. [READ] Fix the randomized selection of locations in {{ProvidedBlocksBuilder}}.	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	3d3be87e30	HDFS-12778. [READ] Report multiple locations for PROVIDED blocks	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	3b1d30301b	HDFS-12775. [READ] Fix reporting of Provided volumes	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	6cd80b2521	HDFS-12779. [READ] Allow cluster id to be specified to the Image generation tool	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	90d1b47a2a	HDFS-12776. [READ] Increasing replication for PROVIDED files should create local replicas	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	87dc026bee	HDFS-12789. [READ] Image generation tool does not close an opened stream	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	c293cc8e9b	HDFS-12671. [READ] Test NameNode restarts when PROVIDED is configured	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	71d0a82571	HDFS-12607. [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	98f5ed5aa3	HDFS-11902. [READ] Merge BlockFormatProvider and FileRegionProvider.	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	d6a9a89973	HDFS-12605. [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	17052c4aff	HDFS-12584. [READ] Fix errors in image generation tool from latest rebase	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	aca023b72c	HDFS-12289. [READ] HDFS-12091 breaks the tests for provided block reads	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	546b95f484	HDFS-11673. [READ] Handle failures of Datanode with PROVIDED storage	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	4851f06bc2	HDFS-11791. [READ] Test for increasing replication of provided files.	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	aa5ec85f7f	HDFS-11663. [READ] Fix NullPointerException in ProvidedBlocksBuilder	2017-12-15 17:51:37 -08:00
Virajith Jalaparti	d65df0f273	HDFS-11190. [READ] Namenode support for data stored in external stores.	2017-12-15 17:51:37 -08:00
Virajith Jalaparti	8da3a6e314	HDFS-10706. [READ] Add tool generating FSImage from external store	2017-12-15 17:51:37 -08:00
Kai Zheng	f86c81d923	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-14 11:14:52 +08:00
Kai Zheng	28792b6b7f	Revert "HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu" This reverts commit 479d6a5792262c977025c26fd4960574b0db6847	2017-12-14 11:05:11 +08:00
Surendra Singh Lilhore	00129c5314	HDFS-12833. Distcp : Update the usage of delete option for dependency with update and overwrite option. Contributed by usharani.	2017-12-12 00:28:02 +05:30
Kai Zheng	ce04340ec7	HADOOP-15104. AliyunOSS: change the default value of max error retry. Contributed by Jinhu Wu	2017-12-08 23:03:54 +08:00
Kai Zheng	73b86979d6	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-06 12:01:36 +08:00
Aaron Fabbri	6555af81a2	HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed by Younger and Sean Mackrory.	2017-12-05 11:06:32 -08:00
Steve Loughran	3150c019ae	HADOOP-15071 S3a troubleshooting docs to add a couple more failure modes. Contributed by Steve Loughran	2017-12-05 15:05:41 +00:00
Steve Loughran	0ea182d0fa	HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. Contributed by Rajesh Balamohan	2017-11-28 11:52:59 +00:00
Rohith Sharma K S	3cd75845da	MAPREDUCE-7014. Fix java doc errors in jdk1.8. Contributed by Steve Loughran.	2017-11-27 22:01:00 +05:30
Steve Loughran	de8b6ca5ef	HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints. Contributed by Steve Loughran and Ryan Blue.	2017-11-22 15:28:12 +00:00
Steve Loughran	c326fc89b0	HADOOP-15024 Support user agent configuration and include that & Hadoop version information to oss server. Contributed by Sammi Chen.	2017-11-20 18:56:42 +00:00
Akira Ajisaka	6903cf096e	HADOOP-13514. Upgrade maven surefire plugin to 2.20.1 Signed-off-by: Allen Wittenauer <aw@apache.org>	2017-11-19 12:39:37 -08:00
Kai Zheng	18621af7ae	HADOOP-14993. AliyunOSS: Override listFiles and listLocatedStatus. Contributed Genmao Yu	2017-11-14 17:58:37 +08:00
Daniel Templeton	ba8136615a	YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton) Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b	2017-11-09 12:09:48 -08:00
John Zhuge	bf6a660232	HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge.	2017-11-09 10:16:12 -08:00
Subru Krishnan	f2df6b8983	HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru).	2017-11-08 18:07:12 -08:00
Arun Suresh	53c0fb7efe	HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh)	2017-11-02 17:15:33 -07:00
Akira Ajisaka	cc3f3eca40	MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi.	2017-11-02 18:32:24 +09:00
Chris Douglas	139cc752d2	HADOOP-14991. Add missing figures to Resource Estimator tool. Contributed by Rui Li	2017-10-27 21:59:27 -07:00
Yufei Gu	99880d0a16	YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak)	2017-10-27 14:41:37 -07:00
Subru Krishnan	625039ef20	HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru).	2017-10-25 15:51:27 -07:00
Wangda Tan	281d83604d	YARN-7318. Fix shell check warnings of SLS. (Gergely Novák via wangda)	2017-10-20 15:54:15 -07:00
Steve Loughran	f36cbc8475	HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. Contributed by Andras Bokor.	2017-10-20 22:27:04 +01:00
Haibo Chen	46eb1033a8	YARN-6546. SLS is slow while loading 10k queues. (Yufei Gu via Haibo Chen)	2017-10-17 16:04:19 -07:00
Steve Loughran	9fcc3a1fc8	HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled. Contributed by Santhosh G Nayak	2017-10-16 16:01:47 +01:00
Steve Loughran	3de574413c	HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. Contributed by Varada Hemeswari.	2017-10-11 18:06:43 +01:00
Steve Loughran	572cdb5463	HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB Contributed by Kannapiran Srinivasan.	2017-10-06 17:43:38 +01:00
Haibo Chen	c071aad5da	YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen)	2017-10-05 15:50:13 -07:00
Steve Loughran	9288206cb3	HADOOP-14845. Azure wasb: getFileStatus not making any auth check. Contributed by Sivaguru Sankaridurg	2017-10-05 15:05:23 +01:00
Yufei Gu	79e37dc6db	YARN-7211. AMSimulator in SLS does't work due to responseId refactoring.(Contributed by Botong Huang via Yufei Gu)	2017-10-03 15:10:50 -07:00
Akira Ajisaka	c9b525bd94	YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari.	2017-10-04 07:01:54 +09:00
John Zhuge	8aca46e53f	HADOOP-14651. Update okhttp version to 2.7.5. Contributed by Ray Chiang and John Zhuge.	2017-09-29 12:15:06 -07:00
Steve Loughran	a530e7ab3b	HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB Contributed by Varada Hemeswari	2017-09-28 19:52:56 +01:00
Aaron Fabbri	47011d7dd3	HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran	2017-09-25 15:59:38 -07:00
Jason Lowe	b133dc5700	MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari	2017-09-22 15:21:42 -05:00
Steve Loughran	2d2d97fa7d	HADOOP-14553. Add (parallelized) integration tests to hadoop-azure Contributed by Steve Loughran	2017-09-15 17:03:01 +01:00
Aaron Fabbri	49467165a5	HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran.	2017-09-14 14:10:48 -07:00
ChenSammi	e0b3c644e1	HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen	2017-09-14 09:15:29 +08:00
John Zhuge	a4661850c1	HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria.	2017-09-08 11:51:03 -07:00
Steve Loughran	5bbca80428	HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A. Contributed by Aaron Fabbri	2017-09-08 12:07:02 +01:00
Steve Loughran	83449abb40	HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. Contributed by Yonger.	2017-09-07 19:55:56 +01:00
Steve Loughran	13eda50003	HADOOP-14520. WASB: Block compaction for Azure Block Blobs. Contributed by Georgi Chalakov	2017-09-07 18:35:03 +01:00
Xiaoyu Yao	63720ef574	HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin.	2017-09-05 23:34:55 -07:00
John Zhuge	50506e90a8	HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge.	2017-09-05 23:26:57 -07:00
Steve Loughran	792eff9ea7	HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. Contributed by Sivaguru Sankaridurg	2017-09-05 14:16:57 +01:00
Andrew Wang	0d419c984f	Preparing for 3.1.0 development	2017-09-01 11:53:48 -07:00
Allen Wittenauer	a3fee475f7	HADOOP-14414. Calling maven-site-plugin directly for docs profile is unnecessary (Andras Bokor via aw)	2017-09-01 08:38:30 -07:00
Steve Loughran	621b43e254	HADOOP-13345 HS3Guard: Improved Consistency for S3A. Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu, Sean Mackrory, Steve Loughran and others.	2017-09-01 14:13:41 +01:00
John Zhuge	7ecc6dbed6	HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria.	2017-08-31 21:13:22 -07:00
Allen Wittenauer	71bbb86d69	HADOOP-14670. Increase minimum cmake version for all platforms Signed-off-by: Chris Douglas <cdouglas@apache.org>	2017-08-30 21:47:24 -07:00
Haibo Chen	d04f85f387	MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen)	2017-08-30 10:07:48 -07:00
Yufei Gu	26fafc3597	YARN-7077. TestAMSimulator and TestNMSimulator fail. (Contributed by Akira Ajisaka via Yufei Gu)	2017-08-29 16:33:51 -07:00
Steve Loughran	9374f38820	HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory Contributed by Esfandiar Manii and Thomas Marquardt.	2017-08-29 19:03:44 +01:00
Steve Loughran	021974f4cb	HADOOP-14802. Add support for using container saskeys for all accesses. Contributed by Sivaguru Sankaridurg	2017-08-29 19:02:43 +01:00
Andrew Wang	f29a0fc288	HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou.	2017-08-25 14:14:23 -07:00
Arpit Agarwal	8b7cbe3840	HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar.	2017-08-25 09:55:46 -07:00
Steve Loughran	27ab5f7385	HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator. Contributed by Genmao Yu	2017-08-22 11:55:48 +01:00
Kai Zheng	267e19a09f	HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu	2017-08-21 13:36:28 +08:00
Arun Suresh	8410d862d3	YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh)	2017-08-20 07:54:09 -07:00
Steve Loughran	c6b4e656b7	HADOOP-14769. WASB: delete recursive should not fail if a file is deleted. Contributed by Thomas Marquardt	2017-08-18 14:13:40 +01:00
Andrew Wang	dd7916d3cd	HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas.	2017-08-17 09:35:36 -07:00
Chris Douglas	645a8f2a4d	HADOOP-14726. Mark FileStatus::isDir as final	2017-08-14 21:57:20 -07:00
John Zhuge	7769e96149	HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria.	2017-08-13 00:22:34 -07:00
Allen Wittenauer	4222c97108	HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw)	2017-08-11 09:25:56 -07:00
Steve Loughran	54356b1e83	HADOOP-14183. Remove service loader config file for wasb fs. Contributed by Esfandiar Manii.	2017-08-10 16:46:33 +01:00
Steve Loughran	f4e1aa0508	HADOOP-14715. TestWasbRemoteCallHelper failing. Contributed by Esfandiar Manii.	2017-08-08 23:37:47 +01:00
Chris Douglas	55a181f845	HADOOP-14730. Support protobuf FileStatus in AdlFileSystem.	2017-08-07 21:31:28 -07:00
Steve Loughran	d91b7a8451	HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek. Contributed by Thomas Marquardt	2017-08-06 20:19:23 +01:00
Steve Loughran	7fc324aabd	HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws. Contributed by Steve Loughran (cherry picked from commit ced547d5f0dbea571cbc472c5f55fe89d5900a6f)	2017-08-04 11:09:08 +01:00
Sean Mackrory	1a1bf6b7d0	HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer.	2017-08-02 12:25:05 -06:00
Steve Loughran	778d4edd9a	HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded. Contributed by Thomas Marquardt.	2017-08-01 21:36:28 +01:00
John Zhuge	f735ad1b67	HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge.	2017-07-28 09:46:54 -07:00
Jitendra Pandey	a92bf39e23	HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt.	2017-07-25 16:26:48 -07:00
Jitendra Pandey	f2921e51f0	HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov.	2017-07-24 23:01:01 -07:00
Ravi Prakash	94ca52ae9e	HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash.	2017-07-24 16:01:45 -07:00
Wei-Chiu Chuang	44350fdf49	HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default). Contributed by Bharat Viswanadham.	2017-07-20 18:23:13 -07:00
Haibo Chen	0ba8cda135	YARN-6685. Add job count in to SLS JSON input format. (Yufei Gu via Haibo Chen)	2017-07-20 08:15:46 -07:00
Jitendra Pandey	2843c688bc	HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg.	2017-07-19 00:13:06 -07:00
Jitendra Pandey	b0e78ae085	HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak.	2017-07-17 02:27:55 -07:00
Steve Loughran	7d272ea124	HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb. Contributed by Varada Hemeswari (cherry picked from commit 1e69e5260351effc8077d1bdc397cec57cf1ff1b)	2017-07-12 10:37:39 +01:00
Steve Loughran	d670c3a4da	HADOOP-14535 wasb: implement high-performance random access and seek of block blobs. Contributed by Thomas Marquardt	2017-07-11 21:34:27 +01:00
Haibo Chen	147df300bf	YARN-5067 Support specifying resources for AM containers in SLS. (Yufei Gu via Haibo Chen)	2017-06-30 17:03:44 -07:00
Mingliang Liu	38996fdcf0	HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak	2017-06-30 16:53:48 -07:00
Andrew Wang	af2773f609	Updating version for 3.0.0-beta1 development	2017-06-29 17:57:40 -07:00
Mingliang Liu	72993b33b7	HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc	2017-06-29 17:07:52 -07:00
Mingliang Liu	b08cc97396	HADOOP-14601. Azure: Reuse ObjectMapper. Contributed by Mingliang Liu	2017-06-29 09:22:00 -07:00
Allen Wittenauer	20ba86d66a	MAPREDUCE-6536. hadoop-pipes doesn't use maven properties for openssl Signed-off-by: Ravi Prakash <raviprak@apache.org>	2017-06-28 17:33:44 -07:00
Mingliang Liu	990aa34de2	HADOOP-14609. NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code. Contributed by Steve Loughran	2017-06-28 14:18:59 -07:00
Steve Loughran	ee243e5289	HADOOP-14190. Add more on S3 regions to the s3a documentation. Contributed by Steve Loughran	2017-06-28 10:22:13 +01:00
Mingliang Liu	7e031c2c18	HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas	2017-06-27 17:32:07 -07:00
Mingliang Liu	686a634f01	HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran	2017-06-27 16:48:47 -07:00
Mingliang Liu	bc4dfe9c9c	HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu	2017-06-27 13:19:14 -07:00
Steve Loughran	48f4a229a9	HADOOP-14461 Azure: handle failure gracefully in case of missing account access key. Contributed by Mingliang Liu.	2017-06-26 13:39:47 +01:00
Akira Ajisaka	a594f96fb1	HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he.	2017-06-26 16:41:00 +09:00
Steve Loughran	c6bd73c6c5	HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations. Contributed by Thomas.	2017-06-23 21:38:41 +01:00
Jason Lowe	dd65eea74b	HADOOP-8143. Change distcp to have -pb on by default. Contributed by Mithun Radhakrishnan	2017-06-20 09:53:47 -05:00
Akira Ajisaka	3369540653	HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools.	2017-06-20 13:18:26 +09:00
Mingliang Liu	f214a9961f	HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran	2017-06-15 11:16:16 -07:00
Mingliang Liu	325163f23f	HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu	2017-06-09 13:07:11 -07:00
Mingliang Liu	df35e4cc3b	HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan	2017-06-07 13:54:52 -07:00
Carlo Curino	b65100c14b	YARN-6547. Enhance SLS-based tests leveraging invariant checker.	2017-06-06 17:19:15 -07:00
Mingliang Liu	536f057158	HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu	2017-06-06 11:09:28 -07:00
Mingliang Liu	6b5285bbcb	HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu	2017-06-06 11:06:49 -07:00
Mingliang Liu	5fd9742c83	HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan	2017-06-05 15:56:43 -07:00
Mingliang Liu	6aeda55bb8	HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu	2017-06-05 11:26:56 -07:00
Mingliang Liu	ece33208b8	HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu	2017-06-01 11:52:11 -07:00
Akira Ajisaka	6e6e96583f	HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.	2017-06-01 13:08:01 +09:00
Akira Ajisaka	07e60f85d8	HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.	2017-05-30 15:11:10 +09:00
John Zhuge	6c6a7a5962	HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge.	2017-05-28 22:25:00 -07:00
Mingliang Liu	89bb8bfe58	HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari	2017-05-26 17:54:00 -07:00
Andrew Wang	16ad896d5c	Update maven version for 3.0.0-alpha4 development	2017-05-26 14:09:44 -07:00
Akira Ajisaka	6a52b5e144	HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu.	2017-05-25 15:36:44 +09:00
Sunil G	b6f66b0da1	YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.	2017-05-22 14:10:06 +05:30
Yufei Gu	fcbdecc54d	YARN-6111. Rumen input does't work in SLS. Contributed by Yufei Gu.	2017-05-21 21:04:06 -07:00
Mingliang Liu	6672810eea	HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran	2017-05-19 11:51:43 -07:00
Yongjun Zhang	b4adc8392c	HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang)	2017-05-18 15:35:22 -07:00
Sunil G	40e6a85d25	YARN-6560. SLS doesn't honor node total resource specified in sls-runner.xml. Contributed by Wangda Tan.	2017-05-18 22:49:42 +05:30
Steve Loughran	ba70225cf6	HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries. Contributed by Steve Loughran. (cherry picked from commit 2ac5aab8d725f761a9f9723471a4426f6b5d78c4)	2017-05-18 15:44:39 +01:00
Akira Ajisaka	d4aa9e3c69	HADOOP-14415. Use java.lang.AssertionError instead of junit.framework.AssertionFailedError. Contributed by Chen Liang.	2017-05-17 15:38:28 -04:00
Robert Kanter	101852ca11	YARN-6535. Program needs to exit when SLS finishes. (yufeigu via rkanter)	2017-05-16 17:52:17 -07:00
Mingliang Liu	b415c6fe74	HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger. Contributed by Sivaguru Sankaridurg	2017-05-16 11:22:32 -07:00
Akira Ajisaka	0d5c8ed8e0	HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor.	2017-05-11 16:37:32 -05:00
Mingliang Liu	eed731496f	HADOOP-14361. Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently. Contributed by Santhosh G Nayak	2017-05-10 15:05:11 -07:00
Carlo Curino	5cb6e3e082	YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations.	2017-05-10 14:49:25 -07:00
Akira Ajisaka	2ba9903932	HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang.	2017-05-10 05:57:12 -05:00
Akira Ajisaka	c60164fb60	HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong.	2017-05-09 05:37:43 -05:00
Andrew Wang	54fd0e44b7	HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy.	2017-05-08 21:59:49 -07:00
Andrew Wang	8065129d87	HADOOP-14298. TestHadoopArchiveLogsRunner fails. Contribute dby Akira Ajisaka.	2017-05-07 13:59:15 -07:00
Steve Loughran	4e6bbd049d	HADOOP-14382 Remove usages of MoreObjects.toStringHelper. Contributed by Andrew Wang	2017-05-05 13:03:09 +01:00
Robert Kanter	3082552b3b	YARN-6522. Make SLS JSON input file format simple and scalable (yufeigu via rkanter)	2017-05-04 17:21:46 -07:00
Mingliang Liu	83dded556d	HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran	2017-05-03 11:22:44 -07:00
Andrew Wang	b062b323b7	HADOOP-14281. Fix TestKafkaMetrics#testPutMetrics. Contributed by Alison Yu.	2017-05-02 11:49:19 -07:00
Akira Ajisaka	62579b69a0	HADOOP-11614. Remove httpclient dependency from hadoop-openstack. Contributed by Akira Ajisaka, Brahma Reddy Battula, and Steve Loughran.	2017-04-27 16:57:08 +09:00
Mingliang Liu	8b5f2c372e	HADOOP-14351. Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache. Contributed by Santhosh G Nayak	2017-04-26 13:47:18 -07:00
Mingliang Liu	e4c32bccde	HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu	2017-04-25 20:54:56 -07:00
Robert Kanter	475f933b41	YARN-6423. Queue metrics doesn't work for Fair Scheduler in SLS (yufeigu via rkanter)	2017-04-25 16:26:09 -07:00
Steve Loughran	5f934f8386	HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus. Contributed by Steve Moist.	2017-04-24 20:33:19 +01:00
Wangda Tan	de69d6e811	YARN-6363. Extending SLS: Synthetic Load Generator. (Carlo Curino via wangda)	2017-04-20 21:54:30 -07:00
Mingliang Liu	667966c13c	HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran	2017-04-20 17:13:36 -07:00
Andrew Wang	7e075a50e3	YARN-6365. Get static SLS html resources from classpath. Contributed by Yufei Gu.	2017-04-19 11:08:59 -07:00
Steve Loughran	6b015d00c9	HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. Contributed by Steve Loughran	2017-04-19 10:21:44 +01:00
Mingliang Liu	0cab57223e	HADOOP-14274. Azure: Simplify Ranger-WASB policy model. Contributed by Sivaguru Sankaridurg	2017-04-12 16:07:10 -07:00
Mingliang Liu	b053fdc547	HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu	2017-04-12 14:30:34 -07:00
Chris Nauroth	b8305e6d06	HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. Contributed by Mingliang Liu.	2017-04-12 10:02:13 -07:00
John Zhuge	56e81f2a20	HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge. Signed-off-by: John Zhuge <jzhuge@apache.org>	2017-04-07 12:04:44 -07:00
Andrew Wang	7eea8d83fa	HADOOP-14280. Fix compilation of TestKafkaMetrics.	2017-04-05 15:12:03 -07:00
Mingliang Liu	56ab02eed9	HADOOP-14273. Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag. Contributed by Santhosh G Nayak	2017-04-04 11:10:28 -07:00
Mingliang Liu	27a44b60c1	HADOOP-14272. Azure: WasbRemoteCallHelper should use String equals for comparison. Contributed by Santhosh G Nayak	2017-04-04 11:03:59 -07:00
Chris Douglas	6eba79232f	HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang	2017-04-03 20:13:14 -07:00
Mingliang Liu	5faa949b78	HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka	2017-04-03 11:07:14 -07:00
Mingliang Liu	26172a94d6	HADOOP-14267. Make DistCpOptions immutable. Contributed by Mingliang Liu	2017-03-31 20:04:26 -07:00
Yongjun Zhang	bf3fb585aa	HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li.	2017-03-30 17:38:56 -07:00
Yongjun Zhang	144f1cf765	Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen." This reverts commit `064c8b25ec`.	2017-03-30 17:38:18 -07:00
Yongjun Zhang	064c8b25ec	HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen.	2017-03-30 17:01:15 -07:00
Wei-Chiu Chuang	8c591b8d19	HDFS-10974. Document replication factor for EC files. Contributed by Yiqun Lin.	2017-03-30 11:16:05 -07:00
Akira Ajisaka	0d053eeb30	HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula.	2017-03-30 18:11:50 +09:00
Robert Kanter	6a5516c238	YARN-5654. Not be able to run SLS with FairScheduler (yufeigu via rkanter)	2017-03-29 16:18:13 -07:00
Andrew Wang	4966a6e26e	HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy.	2017-03-29 14:37:21 -07:00
Andrew Wang	0e6f8e4bc6	HDFS-10971. Distcp should not copy replication factor if source file is erasure coded. Contributed by Manoj Govindassamy.	2017-03-28 22:14:03 -07:00
John Zhuge	253e3e78ab	HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge.	2017-03-27 23:05:57 -07:00
John Zhuge	d1b7439b48	HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge.	2017-03-24 13:37:04 -07:00
Andrew Wang	52b00600df	HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy.	2017-03-24 11:44:46 -07:00
Chris Douglas	c55195588f	HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge	2017-03-23 12:17:27 -07:00
Chris Douglas	03955612b7	HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria	2017-03-23 12:13:23 -07:00
Mingliang Liu	2e30aa72e0	HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu	2017-03-23 11:33:29 -07:00
Steve Loughran	a5a4867f3b	HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject(). Contributed by Yuanbo Liu (cherry picked from commit 20878d052cebc715c2494a97362fdff08885a77f)	2017-03-23 12:54:54 +00:00
Robert Kanter	cc938e99ec	YARN-6326. Shouldn't use AppAttemptIds to fetch applications while AM Simulator tracks app in SLS (yufeigu via rkanter)	2017-03-21 15:21:11 -07:00
Steve Loughran	2841666f1f	HADOOP-14204 S3A multipart commit failing, "UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort". Contributed by Steve Loughran (cherry picked from commit 595501c9ddacd5fdb740ac1bf9bfeb0ffbbd6c0e)	2017-03-21 13:15:15 +00:00
Mingliang Liu	8e15e24059	HADOOP-13945. Azure: Add Kerberos and Delegation token support to WASB client. Contributed by Santhosh G Nayak Change-Id: I39fb0a3b0491bf2160571366939a0502a0045429	2017-03-20 16:31:24 -07:00
Arpit Agarwal	6c399a88e9	HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran.	2017-03-20 11:29:23 -07:00
Mingliang Liu	86035c1644	HADOOP-14192. Aliyun OSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu	2017-03-17 11:01:00 -07:00
Mingliang Liu	fa67a96d7b	HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu	2017-03-14 14:38:21 -07:00
Mingliang Liu	e6cda5819b	Revert "HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu" This reverts commit `b8c69557b7`.	2017-03-14 13:03:42 -07:00
Mingliang Liu	b8c69557b7	HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu	2017-03-13 14:15:02 -07:00
John Zhuge	5a40bafdae	HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge. Change-Id: I5dc6f885816b8834f718874542dfa373458b0333	2017-03-13 08:11:25 -07:00
Steve Loughran	092ec39fb9	HADOOP-14111 cut some obsolete, ignored s3 tests in TestS3Credentials. Contributed by Yuanbo Liu	2017-03-10 17:43:22 +00:00
Mingliang Liu	881ec4d97b	HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu	2017-03-10 00:16:09 -08:00
John Zhuge	c5ee7fded4	HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge. Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e	2017-03-09 18:30:17 -08:00
John Zhuge	98142d2f72	Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" This reverts commit `53a12fa721`.	2017-03-08 09:54:22 -08:00
Mingliang Liu	28daaf0eb2	HADOOP-14150. Implement getHomeDirectory() method in NativeAzureFileSystem. Contributed by Santhosh G Nayak	2017-03-07 14:55:52 -08:00
Robert Kanter	1598fd3b79	YARN-6275. Fail to show real-time tracking charts in SLS (yufeigu via rkanter)	2017-03-07 13:47:52 -08:00
Akira Ajisaka	f597f4c43e	HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri.	2017-03-07 15:14:55 +09:00
Mingliang Liu	686823529b	HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth	2017-03-06 17:16:36 -08:00
Mingliang Liu	52d7d5aa1a	Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth" This reverts commit `6b7cd62b8c`.	2017-03-06 17:10:11 -08:00
Mingliang Liu	c571cda5c7	HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA	2017-03-06 17:00:13 -08:00
Mingliang Liu	6b7cd62b8c	HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth	2017-03-03 17:09:12 -08:00
Akira Ajisaka	490abfb10f	HADOOP-14057. Fix package.html to compile with Java 9.	2017-03-04 00:25:22 +09:00
Steve Loughran	a97833e0ed	HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. Contributed by Steve Loughran (cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)	2017-03-02 21:44:30 +00:00
Allen Wittenauer	989bd56b9f	HADOOP-12556. KafkaSink jar files are created but not copied to target dist (Babak Behzad via aw)	2017-02-28 16:15:40 -08:00
Wangda Tan	eac6b4c35c	YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda)	2017-02-28 10:35:50 -08:00
Akira Ajisaka	258342e76c	HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR.	2017-02-28 13:47:44 +09:00
Yongjun Zhang	d235dcdf0b	HADOOP-14127. Add log4j configuration to enable logging in hadoop-distcp's tests. (Xiao Chen via Yongjun Zhang)	2017-02-27 20:42:13 -08:00
Steve Loughran	dab00da19f	HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures. Contributed by Steve Loughran. (cherry picked from commit 29fe5af017b945d8750c074ca39031b5b777eddd)	2017-02-25 15:35:19 +00:00
Sunil G	b32ffa2753	YARN-4779. Fix AM container allocation logic in SLS. Contributed by Wangda Tan.	2017-02-24 21:39:25 +05:30
Steve Loughran	e60c6543d5	HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran	2017-02-24 13:24:59 +00:00
Steve Loughran	9c22a91662	HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory. (cherry picked from commit ff87ca84418a710c6dc884fe8c70947fcc6489d5)	2017-02-24 10:41:36 +00:00
Mingliang Liu	82607fce39	HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari.	2017-02-23 13:48:44 -08:00
Mingliang Liu	a4d4a23785	HADOOP-13321. Deprecate FileSystem APIs that promote inefficient call patterns. Contributed by Chris Nauroth and Mingliang Liu	2017-02-23 12:55:40 -08:00
Mingliang Liu	0013090fb4	HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu	2017-02-22 13:34:20 -08:00
Steve Loughran	4f4250fbcc	HADOOP-14099 Split S3 testing documentation out into its own file. Contributed by Steve Loughran.	2017-02-22 11:43:48 +00:00
Mingliang Liu	924def7854	HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane	2017-02-21 13:44:42 -08:00
Steve Loughran	2158496f6b	HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory. (cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)	2017-02-21 18:28:49 +00:00
Steve Loughran	8035749c26	HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan	2017-02-20 16:21:46 +00:00
Steve Loughran	3a2e30fa9f	HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge (cherry picked from commit b1c1f05b1dc997906390d653dfafb4f0d7e193c4)	2017-02-18 18:17:11 +00:00
Mingliang Liu	658702efff	HADOOP-14040. Use shaded aws-sdk uber-JAR 1.11.86. Contributed by Steve Loughran and Sean Mackrory	2017-02-16 16:51:03 -08:00
Mingliang Liu	bdad8b7b97	HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran	2017-02-16 16:41:31 -08:00
Mingliang Liu	f432999025	HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane	2017-02-16 15:14:25 -08:00
Kai Zheng	cd3e59a3dc	HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu	2017-02-15 16:34:30 +08:00
Akira Ajisaka	b9f8491252	HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin.	2017-02-15 01:45:56 +09:00
Lei Xu	71c23c9fc9	HADOOP-13929. ADLS connector should not check in contract-test-options.xml. (John Zhuge via lei)	2017-02-13 13:33:13 -08:00
Kai Zheng	01be4503c3	HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui	2017-02-13 13:18:20 +08:00
Lei Xu	839b690ed5	HADOOP-13075. Add support for SSE-KMS and SSE-C in s3a filesystem. (Steve Moist via lei)	2017-02-11 13:59:03 -08:00
Arpit Agarwal	2b7a7bbe0f	HADOOP-14055. SwiftRestClient includes pass length in exception if auth fails. Contributed by Marcell Hegedus.	2017-02-10 10:09:31 -08:00
Kai Zheng	5b151290ae	HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation `deleteDirs`. Contributed by Genmao Yu	2017-02-10 14:53:18 +08:00
Kai Zheng	a8a594b4c8	HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui	2017-02-09 18:00:22 +08:00
Akira Ajisaka	98c35bb1a4	HADOOP-13858. TestGridmixMemoryEmulation and TestResourceUsageEmulators fail on the environment other than Linux or Windows.	2017-01-21 17:31:20 +09:00
Lei Xu	ccf2d66244	HADOOP-13962. Update ADLS SDK to 2.1.4. (John Zhuge via lei)	2017-01-21 10:52:40 +08:00
Lei Xu	e015b56319	HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei)	2017-01-20 14:48:04 +08:00
Andrew Wang	5d8b80ea9b	Preparing for 3.0.0-alpha3 development	2017-01-19 15:50:07 -08:00
Steve Loughran	ed33ce11dd	HADOOP-13496. Include file lengths in Mismatch in length error for distcp. Contributed by Ted Yu (cherry picked from commit 77401bd5fcca5127c9908156971eeec468371f47)	2017-01-19 11:25:40 +00:00
Steve Loughran	e648b6e138	HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran	2017-01-11 17:25:15 +00:00
Wangda Tan	287d3d6804	YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)	2017-01-09 08:40:39 -08:00
Chris Nauroth	4c8f9e1302	HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore.	2017-01-05 15:04:47 -08:00
Wangda Tan	0a55bd841e	YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)	2017-01-05 10:31:05 -08:00
Wangda Tan	5ed63e3e9d	YARN-4899. Queue metrics of SLS capacity scheduler only activated after app submit to the queue. (Jonathan Hung via wangda)	2017-01-04 19:17:11 -08:00
Akira Ajisaka	01d31fe938	HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang.	2017-01-04 14:10:36 +09:00
Mingliang Liu	c6a5b689db	HADOOP-13942. Build failure due to errors of javadoc build in hadoop-azure. Contributed by Kai Sasaki	2016-12-29 16:24:06 -08:00
Mingliang Liu	e92a77099b	HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by Dushyanth	2016-12-22 20:15:18 -08:00
Akira Ajisaka	ae401539ea	YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor.	2016-12-22 14:32:24 +09:00
Wangda Tan	3bcfe3a9de	YARN-5706. Fail to launch SLSRunner due to NPE. (Kai Sasaki via wangda)	2016-12-20 11:03:29 -08:00
Mingliang Liu	f92913c35b	HADOOP-13831. Correct check for error code to detect Azure Storage Throttling and provide retries. Contributed by Gaurav Kanade	2016-12-15 12:35:08 -08:00
Mingliang Liu	ef34bf2bb9	HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane	2016-12-13 10:34:32 -08:00
Mingliang Liu	c6a3923245	HADOOP-13871. ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful. Contributed by Steve Loughran	2016-12-12 14:55:34 -08:00
Mingliang Liu	15dd1f3381	HADOOP-13675. Bug in return value for delete() calls in WASB. Contributed by Dushyanth	2016-12-05 12:04:07 -08:00
Mingliang Liu	4113ec5fa5	HADOOP-13257. Improve Azure Data Lake contract tests. Contributed by Vishwajeet Dusane	2016-12-02 15:54:57 -08:00
Mingliang Liu	51211a7d7a	HADOOP-13855. Fix a couple of the s3a statistic names to be consistent with the rest. Contributed by Steve Loughran	2016-12-02 14:01:42 -08:00
Mingliang Liu	2ff84a0040	HADOOP-13857. S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs. Contributed by Steve Loughran	2016-12-02 13:36:04 -08:00
Mingliang Liu	3fd844b99f	HADOOP-13830. Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string". Contributed by Steve Loughran	2016-11-30 13:01:19 -08:00
Mingliang Liu	d60a60be8a	HADOOP-13823. s3a rename: fail if dest file exists. Contributed by Steve Loughran	2016-11-28 16:30:29 -08:00
Akira Ajisaka	209e805430	HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.	2016-11-28 14:34:57 +09:00
Akira Ajisaka	9e173d2bac	HADOOP-13816. Ambiguous plugin version warning from maven build. Contributed by Kai Sasaki.	2016-11-28 13:10:31 +09:00
Steve Loughran	0de0c32ddd	HADOOP-13801 regression: ITestS3AMiniYarnCluster failing. Contributed by Steve Loughran	2016-11-23 21:37:39 +00:00
Mingliang Liu	af791b774b	HADOOP-13050. Upgrade to AWS SDK 10.11+. Contributed by Chris Nauroth and Steve Loughran	2016-11-22 14:00:35 -08:00
Mingliang Liu	beb70fed4f	HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran This closes #131	2016-11-22 13:12:23 -08:00
Wei-Chiu Chuang	c0b1a44f6c	HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory.	2016-11-17 22:48:35 -06:00
Mingliang Liu	5af572b644	HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu	2016-11-15 10:57:00 -08:00
Chris Douglas	5c61ad2488	HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane	2016-11-11 11:15:07 -08:00
Mingliang Liu	89354f0475	HADOOP-13687. Provide a unified dependency artifact that transitively includes the cloud storage modules shipped with Hadoop. Contributed by Chris Nauroth	2016-11-10 08:58:37 -08:00
Jason Lowe	3f93ac0733	YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri	2016-11-08 22:01:26 +00:00
Xiao Chen	acd509dc57	HADOOP-13795. Skip testGlobStatusThrowsExceptionForUnreadableDir in TestFSMainOperationsSwift. Contributed by John Zhuge.	2016-11-07 09:21:01 -08:00
Akira Ajisaka	b970446b2c	HADOOP-13798. TestHadoopArchives times out.	2016-11-07 19:53:43 +09:00
Akira Ajisaka	de0132748d	MAPREDUCE-6796. Remove unused properties from JTConfig.java. Contributed by Haibo Chen.	2016-11-05 03:49:19 +09:00
Arun Suresh	0aafc122d4	YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh)	2016-11-04 07:31:54 -07:00
Xiao Chen	20c4d8efab	HADOOP-13787. Azure testGlobStatusThrowsExceptionForUnreadableDir fails. Contributed by John Zhuge.	2016-11-03 11:16:07 -07:00
Steve Loughran	a1761a841e	HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi.	2016-10-31 20:54:46 +00:00
Robert Kanter	5877f20f9c	HADOOP-10075. Update jetty dependency to version 9 (rkanter)	2016-10-27 16:09:00 -07:00
Masatake Iwasaki	0bdd263d82	HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran.	2016-10-27 15:46:59 +09:00
Chris Nauroth	1f8490a5ba	HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth.	2016-10-26 08:55:42 -07:00
Chris Nauroth	9cad3e2350	HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran.	2016-10-26 08:27:26 -07:00
Akira Ajisaka	d3bb69a667	YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton.	2016-10-26 15:32:07 +09:00
Yongjun Zhang	0f0c15f7a5	HDFS-11040. Add documentation for HDFS-9820 distcp improvement. Contributed by Yongjun Zhang.	2016-10-25 12:25:40 -07:00
Yongjun Zhang	3a60573039	Revert "Fix HDFS-11040" This reverts commit `54c1815790`.	2016-10-25 12:25:02 -07:00
Yongjun Zhang	54c1815790	Fix HDFS-11040	2016-10-25 12:19:34 -07:00
Chris Nauroth	309a43925c	HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth.	2016-10-25 09:03:03 -07:00
Chris Nauroth	3372e94030	HADOOP-12774. s3a should use UGI.getCurrentUser.getShortname() for username. Contributed by Steve Loughran.	2016-10-24 21:54:06 -07:00
Chris Nauroth	d8fa1cfa67	HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth.	2016-10-24 21:22:34 -07:00
Chris Douglas	a1a0281e12	HADOOP-13626. Remove distcp dependency on FileStatus serialization	2016-10-24 12:46:54 -07:00
Anu Engineer	3fbf4cd5da	HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran.	2016-10-20 12:33:58 -07:00
Steve Loughran	9ae270af02	HADOOP-13735 ITestS3AFileContextStatistics.testStatistics() failing. Contributed by Pieter Reuse	2016-10-20 14:50:59 +01:00
Yongjun Zhang	8650cc84f2	HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.	2016-10-19 17:37:54 -07:00
Xiao Chen	efdf810cf9	HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge.	2016-10-18 18:18:43 -07:00
Steve Loughran	6c348c5691	HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran	2016-10-18 21:16:02 +01:00
Yongjun Zhang	0bc6d37f3c	Revert "HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang." This reverts commit `412c4c9a34`.	2016-10-17 22:47:37 -07:00
Andrew Wang	987ee51141	HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei.	2016-10-17 13:25:58 -07:00
Yongjun Zhang	412c4c9a34	HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.	2016-10-17 11:04:42 -07:00
Mingliang Liu	d9f73f1b7c	HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu	2016-10-13 22:33:55 -07:00
Jing Zhao	0a85d07983	HADOOP-13024. Distcp with -delete feature on raw data not implemented. Contributed by Mavin Martin.	2016-10-13 13:24:54 -07:00
Kai Zheng	c31b5e61b1	HADOOP-13701. AbstractContractRootDirectoryTest can fail when handling delete "/". Contributed by Genmao Yu	2016-10-08 17:35:59 +06:00
Kai Zheng	a57bba470b	Merge branch 'trunk' into HADOOP-12756	2016-10-09 10:29:40 +08:00
Chris Nauroth	69620f9559	HADOOP-13692. hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts. Contributed by Chris Nauroth.	2016-10-07 11:41:19 -07:00
Steve Loughran	ebd4f39a39	HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran.	2016-10-07 12:51:40 +01:00
Steve Loughran	2cc841f16e	HADOOP-13678 Update jackson from 1.9.13 to 2.x in hadoop-tools. Contributed by Akira Ajisaka.	2016-10-06 16:31:00 +01:00
Steve Loughran	d6be1e75d8	HADOOP-12667 s3a to support createNonRecursive API. Contributed by Sean Mackrory	2016-10-05 15:02:11 +01:00
Akira Ajisaka	6cfe649375	HADOOP-13681. Reduce Kafka dependencies in hadoop-kafka module. Contributed by Grant Henke. This closes #133	2016-10-05 20:44:39 +09:00
Chris Nauroth	88b9444a81	HADOOP-13674. S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint. Contributed by Chris Nauroth.	2016-10-04 10:36:58 -07:00
Steve Loughran	ee0c722dc8	HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan.	2016-09-29 17:01:00 +01:00
Chris Nauroth	47f80922dc	HADOOP-13599. s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown. Contributed by Steve Loughran.	2016-09-28 16:19:32 -07:00
Kai Zheng	d1443988f8	Revert "HADOOP-13584. hdoop-aliyun: merge HADOOP-12756 branch back" This reverts commit `5707f88d85`	2016-09-29 09:18:27 +08:00
Brahma Reddy Battula	e17a4970be	HDFS-9885. Correct the distcp counters name while displaying counters. Contributed by Surendra Singh Lilhore	2016-09-27 10:45:12 +05:30
Kai Zheng	5707f88d85	HADOOP-13584. hdoop-aliyun: merge HADOOP-12756 branch back. HADOOP-12756 branch: Incorporate Aliyun OSS file system implementation. Contributors: Mingfei Shi (mingfei.shi@intel.com) Genmao Yu (genmao.ygm@alibaba-inc.com)	2016-09-26 20:42:22 +08:00
Steve Loughran	e5ef51e717	HADOOP-13643. Math error in AbstractContractDistCpTest. Contributed by Aaron Fabbri.	2016-09-23 10:01:30 +01:00
Kai Zheng	26d5df390c	HADOOP-13634. Some configuration in doc has been outdated. Contributed by Genmao Yu	2016-09-23 08:44:28 +08:00
Kai Zheng	a49b3be38e	Merge branch 'trunk' into HADOOP-12756	2016-09-21 14:03:21 +08:00
Kai Zheng	22af6f8db3	HADOOP-13624. Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu	2016-09-21 14:02:44 +08:00
Kai Zheng	08b37603d9	HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu	2016-09-20 15:12:02 +08:00
Kai Zheng	9cd4760257	HADOOP-13609. Refine credential provider related codes for AliyunOss integration. Contributed by Genmao Yu	2016-09-20 11:39:02 +08:00
Kai Zheng	005f4528c7	Merge branch 'trunk' into HADOOP-12756	2016-09-20 08:09:29 +08:00
Chris Nauroth	98bdb51397	HADOOP-13169. Randomize file list in SimpleCopyListing. Contributed by Rajesh Balamohan.	2016-09-19 15:16:47 -07:00
Kai Zheng	a1940464a4	HADOOP-13610. Clean up AliyunOss integration tests. Contributed by Genmao Yu	2016-09-18 19:10:48 +08:00
Kai Zheng	dbb28eb1f2	Merge branch 'trunk' into HADOOP-12756	2016-09-18 19:03:39 +08:00
Mingliang Liu	96142efa2d	HADOOP-13621. s3:// should have been fully cut off from trunk. Contributed by Mingliang Liu.	2016-09-17 22:07:46 -07:00
Steve Loughran	53a12fa721	HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran	2016-09-14 15:45:04 +01:00
Kai Zheng	60f66a9306	Merge branch 'trunk' into HADOOP-12756	2016-09-14 16:12:40 +08:00
Allen Wittenauer	58ed4fa544	HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS This commit includes the following changes: HADOOP-13356. Add a function to handle command_subcommand_OPTS HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts HADOOP-13357. Modify common to use hadoop_subcommand_opts HADOOP-13359. Modify YARN to use hadoop_subcommand_opts HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity) HADOOP-13564. modify mapred to use hadoop_subcommand_opts HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug HADOOP-13360. Documentation for HADOOP_subcommand_OPTS This closes apache/hadoop#126	2016-09-12 11:10:00 -07:00
Ravi Prakash	9faccd1046	HADOOP-13587. distcp.map.bandwidth.mb is overwritten even when -bandwidth flag isn't set. Contributed by Zoran Dimitrijevic	2016-09-12 08:26:08 -07:00
Steve Loughran	4b6d795f28	HADOOP-13540 improve section on troubleshooting s3a auth problems. Contributed by Steve Loughran	2016-09-09 18:55:32 +01:00
Steve Loughran	7fdfcd8a6c	HADOOP-13541 explicitly declare the Joda time version S3A depends on. Contributed by Stevel Loughran	2016-09-07 12:25:23 +01:00
Mingfei	e671a0f52b	HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.	2016-09-07 11:17:43 +08:00
Mingfei	d33e928fbe	HADOOP-13529. Do some code refactoring. Contributed by Genmao Yu.	2016-09-07 11:15:48 +08:00
Mingfei	cdb77110e7	HADOOP-13498. The number of multi-part upload part should not bigger than 10000. Contributed by Genmao Yu.	2016-09-07 11:15:48 +08:00
Mingfei	6bb741b9f8	HADOOP-13499. Support session credentials for authenticating with Aliyun. Contributed by Genmao Yu.	2016-09-07 11:15:48 +08:00
Mingfei	bd2d97adee	HADOOP-13483. File create should throw error rather than overwrite directories. Contributed by Genmao Yu.	2016-09-07 11:15:47 +08:00
Mingfei	4d84c814fc	HADOOP-13491. Fix several warnings from findbugs. Contributed by Genmao Yu.	2016-09-07 11:15:47 +08:00
Kai Zheng	a5d5342228	HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou	2016-09-07 11:15:47 +08:00
Chris Nauroth	d152557cf7	HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth.	2016-09-06 09:36:21 -07:00
Arun Suresh	d6d9cff21b	YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh)	2016-08-30 15:52:29 -07:00
Xiaoyu Yao	8b57be108f	HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth. (cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41) (cherry picked from commit a65e159c59de41d26e12212dd1b7c28c28265cc3)	2016-08-29 15:30:49 -07:00
Chris Douglas	c37346d0e3	HADOOP-12726. Unsupported FS operations should throw UnsupportedOperationException. Contributed by Daniel Templeton.	2016-08-23 14:12:52 -07:00
Chris Nauroth	6f9c346e57	HADOOP-13446. Support running isolated unit tests separate from AWS integration tests. Contributed by Chris Nauroth.	2016-08-23 07:18:49 -07:00
Chris Nauroth	763f0497bb	HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran.	2016-08-19 10:48:10 -07:00
Chris Nauroth	ae4db25443	HADOOP-13513. Java 1.7 support for org.apache.hadoop.fs.azure testcases. Contributed by Tibor Kiss.	2016-08-18 10:44:36 -07:00
Junping Du	0da69c324d	YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. (cherry picked from commit d464483bf7f0b3e3be3ba32cd6c3eee546747ab5) Conflicts: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java	2016-08-18 07:27:23 -07:00
Steve Loughran	040c185d62	HADOOP-13405 doc for fs.s3a.acl.default indicates incorrect values. Contributed by Shen Yinjie	2016-08-18 14:36:55 +01:00
Chris Nauroth	822d661b8f	HADOOP-13208. S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories. Contributed by Steve Loughran.	2016-08-17 14:54:54 -07:00
Chris Nauroth	3808876c73	HADOOP-13324. s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints). Contributed by Steve Loughran.	2016-08-16 17:05:52 -07:00
Chris Nauroth	2ed58c40e5	HADOOP-13403. AzureNativeFileSystem rename/delete performance improvements. Contributed by Subramanyam Pattipaka.	2016-08-08 12:28:40 -07:00
Chris Nauroth	4ef1bc0bdb	HADOOP-13459. hadoop-azure runs several test cases repeatedly, causing unnecessarily long running time. Contributed by Chris Nauroth.	2016-08-02 22:32:21 -07:00
Chris Nauroth	b43de80031	HADOOP-13354. Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients. Contributed by Sivaguru Sankaridurg.	2016-07-27 15:50:38 -07:00
Akira Ajisaka	54fe17a607	HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.	2016-07-27 19:41:09 +09:00
Steve Loughran	86ae218893	HADOOP-13188 S3A file-create should throw error rather than overwrite directories. Contributed by Steve Loughran	2016-07-25 15:50:23 +01:00
Steve Loughran	be9e46b42d	HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and mkdirs(). Contributed by Rajesh Balamohan	2016-07-25 14:45:03 +01:00
Steve Loughran	7052ca8804	HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials. Contributed by Steven K. Wong	2016-07-23 18:00:08 +01:00
Chris Nauroth	c60a68ce1d	HADOOP-13392. [Azure Data Lake] OAuth2 configuration should be default set to true when AdlFileSystem is configured. Contributed by Vishwajeet Dusane and Chris Douglas.	2016-07-22 09:16:38 -07:00
Chris Douglas	dca6335dbc	HADOOP-13393. Omit unsupported fs.defaultFS setting in ADLS documentation	2016-07-21 23:21:47 -07:00
mattf-horton	12aa184479	HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley.	2016-07-21 14:43:57 -07:00
Steve Loughran	37362c2f92	HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan)	2016-07-20 13:42:51 +01:00
Andrew Wang	da456ffd62	Preparing for 3.0.0-alpha2 development	2016-07-15 19:04:17 -07:00
Jitendra Pandey	c2f9cd584c	HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu.	2016-07-15 14:28:53 -07:00
Jian He	819224dcf9	YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan	2016-07-11 22:36:20 -07:00
Andrew Wang	f292624bd8	HDFS-10300. TestDistCpSystem should share MiniDFSCluster. Contributed by John Zhuge.	2016-07-11 18:06:28 -07:00
Jitendra Pandey	6e597600f7	HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu.	2016-07-07 14:58:19 -07:00
Jitendra Pandey	aa42c7a6dd	HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu.	2016-07-01 15:34:03 -07:00
Steve Loughran	96fa0f848b	HADOOP-12709 Cut s3:// from trunk. Contributed by Mingliang Liu.	2016-06-29 16:04:50 +01:00
Akira Ajisaka	8d202f1258	HADOOP-13235. Use Date and Time API in KafkaSink.	2016-06-29 07:40:12 -07:00
Yongjun Zhang	8113855b3a	HDFS-10396. Using -diff option with DistCp may get "Comparison method violates its general contract" exception. Contributed by Yongjun Zhang.	2016-06-28 23:15:13 -07:00
Allen Wittenauer	422c73a865	HADOOP-13034. Log message about input options in distcp lacks some items (Takashi Ohnishi via aw)	2016-06-28 07:21:04 -07:00
Steve Loughran	4ee3543625	HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel.	2016-06-22 15:45:25 +01:00
Chris Nauroth	b2c596cdda	HADOOP-13287. TestS3ACredentials#testInstantiateFromURL fails if AWS secret key contains +. Contributed by Chris Nauroth.	2016-06-21 11:28:52 -07:00
Jitendra Pandey	b7c4cf7129	HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu.	2016-06-20 16:25:30 -07:00
Chris Nauroth	51d16e7b38	HADOOP-13242. Authenticate to Azure Data Lake using client ID and keys. Contributed by Atul Sikaria.	2016-06-16 23:35:20 -07:00
Chris Nauroth	127d2c7281	HADOOP-13241. document s3a better. Contributed by Steve Loughran.	2016-06-16 11:18:02 -07:00
Ravi Prakash	4aefe119a0	HADOOP-3733. "s3x:" URLs break when Secret Key contains a slash, even if encoded. Contributed by Steve Loughran.	2016-06-16 11:13:35 -07:00
Chris Nauroth	c9e71382a5	HADOOP-12875. [Azure Data Lake] Support for contract test and unit test cases. Contributed by Vishwajeet Dusane.	2016-06-16 09:46:05 -07:00
Wangda Tan	c77a1095dc	YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)	2016-06-14 15:06:38 -07:00
Chris Nauroth	9581fb715c	HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane.	2016-06-09 14:33:31 -07:00
Andrew Wang	58c3486850	HADOOP-13175. Remove hadoop-ant from hadoop-tools. Contributed by Chris Douglas.	2016-06-09 13:49:52 -07:00
Steve Loughran	31ffaf76f2	HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory.	2016-06-09 21:00:47 +01:00
Steve Loughran	656c460c0e	HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth	2016-06-09 17:28:49 +01:00
Steve Loughran	a3f78d8fa8	HADOOP-12807 S3AFileSystem should read AWS credentials from environment variables. Contributed by Tobin Baker.	2016-06-06 23:42:36 +02:00
Chris Nauroth	c58a59f708	HADOOP-13171. Add StorageStatistics to S3A; instrument some more operations. Contributed by Steve Loughran.	2016-06-03 08:55:33 -07:00
Chris Nauroth	16b1cc7af9	HADOOP-13131. Add tests to verify that S3A supports SSE-S3 encryption. Contributed by Steve Loughran.	2016-06-01 14:49:22 -07:00
Steve Loughran	5870611030	HADOOP-13162. Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs. (Rajesh Balamohan via stevel)	2016-06-01 14:18:20 +01:00
Varun Vasudev	42f90ab885	YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.	2016-05-29 21:24:16 +05:30
Yongjun Zhang	cfb860dee7	HADOOP-13199. Add doc for distcp -filters. (John Zhuge via Yongjun Zhang)	2016-05-26 23:30:31 -07:00
Steve Loughran	39ec1515a2	HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran)	2016-05-21 16:39:31 +01:00
Steve Loughran	757050ff35	HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong.	2016-05-20 13:52:15 +01:00
Steve Loughran	c918286b17	HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth.	2016-05-20 12:21:59 +01:00
Chris Nauroth	93258459fa	HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. Contributed by Steve Loughran.	2016-05-19 22:00:21 -07:00
Jing Zhao	03788d3015	HDFS-10397. Distcp should ignore -delete option if -diff option is provided instead of exiting. Contributed by Mingliang Liu.	2016-05-17 15:46:30 -07:00
Steve Loughran	08ea07f1b8	HADOOP-13158 S3AFileSystem#toString might throw NullPointerException due to null cannedACL. (Chris Nauroth via stevel)	2016-05-17 13:19:29 +01:00
Steve Loughran	c69a649257	HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel)	2016-05-17 13:00:18 +01:00
Allen Wittenauer	730bc746f9	HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw) This commit contains the following JIRA issues: HADOOP-12931. bin/hadoop work for dynamic subcommands HADOOP-12932. bin/yarn work for dynamic subcommands HADOOP-12933. bin/hdfs work for dynamic subcommands HADOOP-12934. bin/mapred work for dynamic subcommands HADOOP-12935. API documentation for dynamic subcommands HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands HADOOP-13086. enable daemonization of dynamic commands HADOOP-13087. env var doc update for dynamic commands HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement HADOOP-13089. hadoop distcp adds client opts twice when dynamic HADOOP-13094. hadoop-common unit tests for dynamic commands HADOOP-13095. hadoop-hdfs unit tests for dynamic commands HADOOP-13107. clean up how rumen is executed HADOOP-13108. dynamic subcommands need a way to manipulate arguments HADOOP-13110. add a streaming subcommand to mapred HADOOP-13111. convert hadoop gridmix to be dynamic HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow HADOOP-13117. clarify daemonization and security vars for dynamic commands HADOOP-13120. add a --debug message when dynamic commands have been used HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop HADOOP-13129. fix typo in dynamic subcommand docs HADOOP-13151. Underscores should be escaped in dynamic subcommands document HADOOP-13153. fix typo in debug statement for dynamic subcommands	2016-05-16 17:54:45 -07:00
Chris Nauroth	b9685e85d5	HADOOP-13148. TestDistCpViewFs to include IOExceptions in test error reports. Contributed by Steve Loughran.	2016-05-16 11:53:17 -07:00
Steve Loughran	dd3a8bed0a	HADOOP-13113 Enable parallel test execution for hadoop-aws. Chris Nauroth via stevel	2016-05-13 10:47:12 +01:00
Andrew Wang	fa440a39b7	HADOOP-12868. Fix hadoop-openstack undeclared and unused dependencies. Contributed by Masatake Iwasaki.	2016-05-12 23:58:18 -07:00
Andrew Wang	3c5c57af28	HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.	2016-05-12 18:27:28 -07:00
Andrew Wang	ca5613af91	Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." This reverts commit `6b53802cba`.	2016-05-12 15:32:45 -07:00
Steve Loughran	27c4e90efc	HADOOP-13028 add low level counter metrics for S3A; use in read performance tests. contributed by: stevel patch includes HADOOP-12844 Recover when S3A fails on IOException in read() HADOOP-13058 S3A FS fails during init against a read-only FS if multipart purge HADOOP-13047 S3a Forward seek in stream length to be configurable	2016-05-12 19:24:20 +01:00
Andrew Wang	6b53802cba	Update project version to 3.0.0-alpha1-SNAPSHOT.	2016-05-12 11:05:05 -07:00
Steve Loughran	3f2816ab5a	HADOOP-13116 Jets3tNativeS3FileSystemContractTest does not run.	2016-05-12 16:35:57 +01:00
Steve Loughran	def2a6d385	HADOOP-13122 Customize User-Agent header sent in HTTP requests by S3A. Chris Nauroth via stevel.	2016-05-12 13:57:35 +01:00
Chris Nauroth	27242f211e	HADOOP-12801. Suppress obsolete S3FileSystem tests. Contributed by Chris Nauroth.	2016-05-10 15:21:06 -07:00
Steve Loughran	025219b12f	HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel)	2016-05-10 21:37:22 +01:00
Jian He	996a210ab0	YARN-4778. Support specifying resources for task containers in SLS. Contributed by Wangda Tan	2016-05-09 14:49:41 -07:00
Allen Wittenauer	1c5bbf6499	HADOOP-12866. add a subcommand for gridmix (Kai Sasaki via aw)	2016-05-06 14:07:02 -07:00
Jason Lowe	d0da13229c	YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla	2016-05-05 14:07:54 +00:00
Jing Zhao	af942585a1	HADOOP-12469. distcp should not ignore the ignoreFailures option. Contributed by Mingliang Liu.	2016-05-04 10:23:04 -07:00
Masatake Iwasaki	ef0870ad03	HADOOP-12378. Fix findbugs warnings in hadoop-tools module. Contributed by Akira AJISAKA.	2016-04-29 20:25:52 +09:00
Arun Suresh	341888a0aa	YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh)	2016-04-26 20:12:12 -07:00
Yongjun Zhang	959a28dd12	HDFS-10313. Distcp need to enforce the order of snapshot names passed to -diff. (Lin Yiqun via Yongjun Zhang)	2016-04-26 16:08:03 -07:00
Akira Ajisaka	02c51c27d9	HDFS-10298. Document the usage of distcp -diff option. Contributed by Takashi Ohnishi.	2016-04-25 22:33:09 +09:00
Jing Zhao	63e5412f1a	HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.	2016-04-22 15:14:40 -07:00
Steve Loughran	19f0f9608e	HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel)	2016-04-22 11:25:03 +01:00
Yongjun Zhang	a749ba0cea	HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via Yongjun Zhang)	2016-04-21 12:17:17 -07:00
Jing Zhao	404f57f328	HDFS-10216. Distcp -diff throws exception when handling relative path. Contributed by Takashi Ohnishi.	2016-04-14 10:35:22 -07:00
Steve Loughran	df18b6e984	HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel)	2016-04-14 12:44:55 +01:00
Jason Lowe	2a5da97f81	Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" This reverts commit `1cbcd4a491`.	2016-04-11 15:51:01 +00:00
Steve Loughran	b9e3eff62a	HADOOP-12444 Support lazy seek in S3AInputStream. Rajesh Balamohan via stevel	2016-04-09 11:25:31 +01:00
Chris Nauroth	843ee8d59d	HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran.	2016-04-08 13:36:58 -07:00
Steve Loughran	ce5b4812f0	HADOOP-12753 S3A JUnit tests failing if using HTTP proxy. (Zoran Rajic via stevel)	2016-04-08 17:10:00 +01:00
Akira Ajisaka	9c32f8785e	HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.	2016-04-08 15:32:13 +09:00
Jason Lowe	1cbcd4a491	YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla	2016-04-05 13:40:19 +00:00
Steve Loughran	89c9347511	HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse. - omitted new S3A subclass	2016-04-04 17:02:04 +01:00
Steve Loughran	0ecdd4cffa	HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse.	2016-04-03 16:40:19 +01:00
Harsh J	256c82fe29	HADOOP-11687. Ignore x-* and response headers when copying an Amazon S3 object. Contributed by Aaron Peterson and harsh.	2016-04-01 14:18:10 +05:30
Allen Wittenauer	0a74610d1c	HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)	2016-03-31 07:51:05 -07:00
Akira Ajisaka	0050fa5f1c	MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.	2016-03-29 18:17:52 +09:00
Akira Ajisaka	115be193df	Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote." This reverts commit `2c268cc936`.	2016-03-28 16:05:35 +09:00
Akira Ajisaka	18c7e58283	HDFS-9640. Remove hsftp from DistCp in trunk. Contributed by Wei-Chiu Chuang.	2016-03-28 15:32:38 +09:00
Akira Ajisaka	2c268cc936	MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.	2016-03-25 15:17:27 +09:00
Allen Wittenauer	738155063e	HADOOP-12857. rework hadoop-tools (aw)	2016-03-23 13:46:38 -07:00
Chris Nauroth	c50aad0f85	HADOOP-12717. NPE when trying to rename a directory in Windows Azure Storage FileSystem. Contributed by Robert Yokota and Gaurav Kanade.	2016-03-04 22:57:43 -08:00
Masatake Iwasaki	cbd31328a6	HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)	2016-03-04 14:11:36 +09:00
Masatake Iwasaki	33a412e8a4	HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)	2016-03-03 18:57:23 +09:00
Chris Nauroth	f98dff329b	HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty.	2016-03-01 13:18:42 -08:00
Chris Nauroth	d251e55415	HADOOP-12851. S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders. Contributed by Larry McCay.	2016-02-29 21:59:52 -08:00
Masatake Iwasaki	809ebc0b14	HADOOP-12552. Fix undeclared/unused dependency to httpclient (iwasakims)	2016-02-29 18:20:14 +09:00
Chris Nauroth	7634d404b7	HADOOP-12846. Credential Provider Recursive Dependencies. Contributed by Larry McCay.	2016-02-28 11:22:55 -08:00
cnauroth	3369a4f691	HADOOP-12535. Run FileSystem contract tests with hadoop-azure. Contributed by Madhumita Chakraborty.	2016-02-23 21:37:50 -08:00
cnauroth	27b77751c1	HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay.	2016-02-22 17:30:13 -08:00
cnauroth	d4f5fc23b2	HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki.	2016-02-22 13:39:38 -08:00
Yongjun Zhang	ba1c9d484a	HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang)	2016-02-19 10:17:37 -08:00
cnauroth	8ab7658025	HADOOP-11031. Design Document for Credential Provider API. Contributed by Larry McCay.	2016-02-18 14:06:38 -08:00
cnauroth	76fab26c5c	HADOOP-12548. Read s3a creds from a Credential Provider. Contributed by Larry McCay.	2016-02-17 12:19:59 -08:00
cnauroth	91a96eaa53	HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty.	2016-02-12 15:50:10 -08:00
Steve Loughran	29ae258013	HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel)	2016-02-06 15:05:16 +00:00
Akira Ajisaka	700a176fa7	HADOOP-12757. Findbug compilation fails for 'Kafka Library support'. (aajisaka)	2016-02-02 23:15:39 +09:00
Yongjun Zhang	eddd823cd6	HDFS-9638. Improve DistCp Help and documentation. (Wei-Chiu Chuang via Yongjun Zhang)	2016-01-29 12:11:55 -08:00
cnauroth	41da9a0773	MAPREDUCE-6563. Streaming documentation contains a stray % character. Contributed by Chris Nauroth.	2016-01-28 14:46:01 -08:00
Xuan	4efdf3a979	YARN-4612. Fix rumen and scheduler load simulator handle killed tasks properly. Contributed by Ming Ma.	2016-01-26 18:17:12 -08:00
Akira Ajisaka	736eb17a79	HADOOP-12731. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.	2016-01-25 13:47:29 +09:00
Akira Ajisaka	0bae506c22	HADOOP-12730. Hadoop streaming -mapper and -reducer options are wrongly documented as required. Contributed by Kengo Seki.	2016-01-22 21:08:31 +09:00
Xuan	1708a4cd23	YARN-4611. Fix scheduler load simulator to support multi-layer network location. Contributed by Ming Ma.	2016-01-20 15:41:50 -08:00
Ravi Prakash	1acc509b45	HADOOP-12696. Add tests for S3FileSystem Contract. Contributed by Matt Paduano	2016-01-19 13:57:08 -08:00
Wangda Tan	89d1fd5dac	HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda)	2016-01-19 21:27:38 +08:00
Wangda Tan	a44ce3f14f	YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)	2016-01-19 09:30:04 +08:00
Wangda Tan	150f5ae034	Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" This reverts commit `3fe5728563`. Conflicts: hadoop-yarn-project/CHANGES.txt	2016-01-19 09:27:36 +08:00
cnauroth	8bc93db2e7	HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth.	2016-01-18 09:08:58 -08:00
Wangda Tan	3fe5728563	YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) (cherry picked from commit `805a9ed85e`)	2016-01-18 17:06:05 +08:00
Wangda Tan	adf260a728	Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" This reverts commit `805a9ed85e`.	2016-01-18 16:50:45 +08:00
Wangda Tan	805a9ed85e	YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)	2016-01-18 11:04:25 +08:00
Yongjun Zhang	a9c69ebeb7	HDFS-9612. DistCp worker threads are not terminated after jobs are done. (Wei-Chiu Chuang via Yongjun Zhang)	2016-01-15 10:03:09 -08:00
Colin Patrick Mccabe	b1ed28fa77	HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe)	2016-01-14 11:02:34 -08:00
Colin Patrick Mccabe	8315582c4f	HDFS-9517. Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes (Wei-Chiu Chuang via cmccabe)	2016-01-13 16:28:06 -08:00
Lei Xu	126705f67e	HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei)	2016-01-12 12:19:53 -08:00
Xiaoyu Yao	c2e2e13455	HDFS-8584. NPE in distcp when ssl configuration file does not exist in class path. Contributed by Surendra Singh Lilhore.	2016-01-11 17:08:26 -08:00
Zhe Zhang	95f32015ad	HDFS-9630. DistCp minor refactoring and clean up. Contributed by Kai Zheng. Change-Id: I363c4ffcac32116ddcdc0a22fac3db92f14a0db0	2016-01-11 09:46:56 -08:00
cnauroth	0e76f1fcea	HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API. Contributed by Dushyanth.	2016-01-09 22:18:11 -08:00
cnauroth	f0fa6d869b	HADOOP-12678. Handle empty rename pending metadata file during atomic rename in redo path. Contributed by Madhumita Chakraborty.	2016-01-08 20:18:59 -08:00
cnauroth	978bbdfeb2	HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy. Contributed by Gaurav Kanade.	2016-01-06 11:15:59 -08:00
Ravi Prakash	2d16f40dab	HADOOP-12689. S3 filesystem operations stopped working correctly	2016-01-05 23:26:03 -08:00
Wangda Tan	a9594c61bb	YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda) (cherry picked from commit 6eefae1b33928486e7a8572e14eaf00b83f2acfc)	2015-12-30 15:39:27 -08:00
Wangda Tan	8310b2e9ff	YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda)	2015-12-30 15:30:12 -08:00
Wangda Tan	79c41b1d83	YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda)	2015-12-16 13:18:19 -08:00
Robert Kanter	6d84cc16b3	MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor (rkanter)	2015-11-25 17:12:40 -08:00
Akira Ajisaka	bab03f3ee6	HADOOP-12598. Add XML namespace declarations for some hadoop/tools modules. Contributed by Xin Wang.	2015-11-26 01:21:20 +09:00
Karthik Kambatla	52948bb20b	YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha)	2015-11-24 13:47:17 +05:30
Haohui Mai	dc46c46b91	HADOOP-10465. Fix use of generics within SortedMapWritable. Contributed by Bertrand Dechoux.	2015-11-22 18:10:08 -08:00
Haohui Mai	0d27917042	HADOOP-12098. Remove redundant test dependencies in Hadoop Archives. Contributed by Varun Saxena.	2015-11-22 17:25:22 -08:00
Haohui Mai	176ff5ce90	HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su.	2015-11-22 15:54:41 -08:00
Akira Ajisaka	f94d892704	HADOOP-12545. Hadoop javadoc has broken links for AccessControlList, ImpersonationProvider, DefaultImpersonationProvider, and DistCp. Contributed by Arshad Mohammad.	2015-11-13 21:52:57 +09:00
Akira Ajisaka	3ae82863f0	HADOOP-12553. [JDK8] Fix javadoc error caused by illegal tag. (aajisaka)	2015-11-08 19:17:57 +09:00
Lei Xu	bff7c90a56	HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei)	2015-11-05 18:35:15 -08:00
Allen Wittenauer	19a0c2660c	MAPREDUCE-1125. SerialUtils.cc: deserializeFloat is out of sync with SerialUtils.hh (Simone Leo via aw)	2015-11-05 16:57:22 -08:00
Allen Wittenauer	fc7cd46faf	MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (Joe Mudd via aw)	2015-11-05 16:53:32 -08:00
cnauroth	0fb1867fd6	HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. Contributed by Gaurav Kanade.	2015-11-04 10:28:44 -08:00
Varun Vasudev	73b9c7b82b	HADOOP-10787. Rename/remove non-HADOOP_*, etc from the shell scripts. Contributed by Allen Wittenauer.	2015-11-04 15:56:17 +05:30
cnauroth	9e7dcab185	HADOOP-12508. delete fails with exception when lease is held on blob. Contributed by Gaurav Kanade.	2015-11-02 10:21:39 -08:00
cnauroth	3ce0a6502e	HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API. Contributed by Dushyanth.	2015-11-02 10:17:41 -08:00
Kihwal Lee	b24fe06483	Addendum to MAPREDUCE-6451	2015-10-30 16:05:23 -05:00
Kihwal Lee	2868ca0328	MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla.	2015-10-30 14:56:41 -05:00
Gábor Lipták	18727c63da	HADOOP-12133 Add schemas to Maven Assembly XMLs Signed-off-by: Allen Wittenauer <aw@apache.org>	2015-10-30 11:36:52 -07:00
cnauroth	2d10cb8e00	HADOOP-12519. hadoop-azure tests should avoid creating a metrics configuration fiale in the module root directory. Contributed by Chris Nauroth.	2015-10-28 09:44:10 -07:00
cnauroth	1f7ecb0c84	HADOOP-11685. StorageException complaining " no lease ID" during HBase distributed log splitting. Contributed by Duo Xu.	2015-10-27 22:56:22 -07:00
cnauroth	73822de7c3	HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. Contributed by Chris Nauroth.	2015-10-27 22:48:56 -07:00
Varun Vasudev	c28e16b40c	YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt.	2015-10-27 20:56:00 +05:30
Akira Ajisaka	bcb2386e39	HADOOP-12515. Mockito dependency is missing in hadoop-kafka module. Contributed by Kai Zheng.	2015-10-27 20:17:16 +09:00
cnauroth	cb282d5b89	HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade.	2015-10-22 14:29:57 -07:00
cnauroth	47641fcbc9	HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. Contributed by Gaurav Kanade.	2015-10-22 12:21:32 -07:00
Robert Kanter	0c4af0f998	MAPREDUCE-6495. Docs for archive-logs tool (rkanter)	2015-10-20 17:34:34 -07:00
Haohui Mai	da8441d0fe	HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu.	2015-10-13 15:24:24 -07:00
Haohui Mai	7e2c971fed	Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu." This reverts commit `fa5a54cac7`.	2015-10-10 03:24:59 -07:00
Haohui Mai	fa5a54cac7	HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu.	2015-10-09 22:38:06 -07:00
Robert Kanter	1bca1bb517	MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead of HADOOP_HOME (rkanter)	2015-10-06 16:49:50 -07:00
cnauroth	5f6edb30c2	HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and writes. Contributed by Dushyanth.	2015-10-05 20:11:41 -07:00
Allen Wittenauer	b59e434116	HADOOP-10949. metrics2 sink plugin for Apache Kafka (Babak Behzad via aw)	2015-10-05 14:02:07 -07:00
Akira Ajisaka	ecbfd68974	HADOOP-10296. Incorrect null check in SwiftRestClient#buildException(). Contributed by Rahul Palamuttam and Kanaka Kumar Avvaru.	2015-10-01 18:25:31 +02:00
Robert Kanter	5db371f52f	MAPREDUCE-6494. Permission issue when running archive-logs tool as different users (rkanter)	2015-09-30 17:33:53 -07:00
Kihwal Lee	3abbdc929b	HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu.	2015-09-28 17:31:51 -05:00
Harsh J	66dad854c0	MAPREDUCE-6471. Document distcp incremental copy. Contributed by Neelesh Srinivas Salian.	2015-09-28 13:12:43 +05:30
cnauroth	7fe521b1dd	HADOOP-11918. Listing an empty s3a root directory throws FileNotFound. Contributed by Lei (Eddy) Xu.	2015-09-25 22:33:53 -07:00
Robert Kanter	d3c49e7662	MAPREDUCE-6480. archive-logs tool may miss applications (rkanter)	2015-09-25 15:02:42 -07:00
Jian He	89cab1ba5f	YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan	2015-09-23 13:29:38 -07:00
Harsh J	cc2b473990	MAPREDUCE-5045. UtilTest#isCygwin method appears to be unused. Contributed by Neelesh Srinivas Salian.	2015-09-22 21:38:06 +05:30
Tsuyoshi Ozawa	dfd807afab	HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.	2015-09-22 12:54:29 +09:00
Steve Loughran	7269906254	HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel).	2015-09-13 14:25:26 +01:00
Karthik Kambatla	119cc75e7e	MAPREDUCE-6415. Create a tool to combine aggregated logs into HAR files. (Robert Kanter via kasha)	2015-09-09 17:45:19 -07:00
Tsuyoshi Ozawa	355eaaa33d	HADOOP-10318. Incorrect reference to nodeFile in RumenToSLSConverter error message. Contributed by Wei Yan.	2015-09-03 22:48:53 +09:00
Lei Xu	6ab2d19f5c	HADOOP-12346. Increase some default timeouts / retries for S3a connector. (Sean Mackrory via Lei (Eddy) Xu)	2015-08-29 09:59:30 -07:00
Rohith Sharma K S	feaf034994	YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks)	2015-08-24 11:25:07 +05:30
Yongjun Zhang	0bc15cb6e6	HDFS-8828. Utilize Snapshot diff report to build diff copy list in distcp. (Yufei Gu via Yongjun Zhang)	2015-08-20 08:02:54 -07:00
cnauroth	3da0bedaec	HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth.	2015-08-05 14:56:41 -07:00
Lei Xu	d5403747b5	HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu)	2015-08-04 18:51:52 -07:00
cnauroth	efa97243ec	HADOOP-12239. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.	2015-07-22 11:16:49 -07:00
Vinayakumar B	94c6a4aa85	HADOOP-12017. Hadoop archives command should use configurable replication factor when closing (Contributed by Bibin A Chundatt)	2015-07-22 10:25:49 +05:30
Allen Wittenauer	68d1f4bfe8	HADOOP-11762. Enable swift distcp to secure HDFS (Chen He via aw)	2015-07-21 11:19:29 -07:00
Wangda Tan	c39ca541f4	YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda)	2015-07-21 09:57:23 -07:00
Vinayakumar B	87f29c6b8a	HADOOP-12214. Parse 'HadoopArchive' commandline using cli Options. (Contributed by Vinayakumar B)	2015-07-21 13:12:46 +05:30
Steve Loughran	05130e94c5	HADOOP-12235 hadoop-openstack junit & mockito dependencies should be "provided". (Ted Yu via stevel)	2015-07-20 11:22:39 +01:00
Chris Douglas	a431ed9075	HADOOP-12211. Collect disk usage on the node. Contributed by Robert Grandl	2015-07-13 15:36:11 -07:00
Ming Ma	08244264c0	YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma)	2015-07-10 08:30:10 -07:00
Chris Douglas	1a0752d85a	HADOOP-12210. Collect network usage on the node. Contributed by Robert Grandl	2015-07-09 17:48:43 -07:00
Robert Kanter	aaafa0b2ee	HADOOP-12159. Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces (rchiang via rkanter)	2015-06-30 16:49:27 -07:00
cnauroth	460e98f7b3	HADOOP-12089. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.	2015-06-29 13:48:02 -07:00
Colin Patrick Mccabe	5a27c3fd76	HADOOP-12104. Migrate Hadoop Pipes native build to new CMake framework (Alan Burlison via Colin P. McCabe)	2015-06-29 12:31:26 -07:00
Amareshwari Sriramadasu	8ef07f767f	HADOOP-11203. Allow ditscp to accept bandwitdh in fraction MegaBytes. Contributed by Raju Bairishetti	2015-06-26 09:52:06 +05:30
cnauroth	c45784bc90	HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on EOF. Contributed by Ivan Mitic.	2015-06-08 22:42:14 -07:00
Andrew Wang	2dbc40e608	HADOOP-12059. S3Credentials should support use of CredentialProvider. Contributed by Sean Busbey.	2015-06-05 13:11:01 -07:00
Robert Kanter	cc70df98e7	MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter)	2015-06-03 18:41:45 -07:00
cnauroth	94e7d49a6d	HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic.	2015-05-28 12:31:06 -07:00
Colin Patrick Mccabe	7dba7005b7	HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe)	2015-05-26 12:15:46 -07:00
Vinod Kumar Vavilapalli	500a1d9c76	YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev.	2015-05-26 11:38:35 -07:00
Colin Patrick Mccabe	f34638399a	HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe)	2015-05-22 12:51:03 -07:00
Akira Ajisaka	4aa730ce85	HADOOP-11698. Remove DistCpV1 and Logalyzer. Contributed by Brahma Reddy Battula.	2015-05-20 19:25:45 +09:00
Jing Zhao	0790275f05	HADOOP-1540. Support file exclusion list in distcp. Contributed by Rich Haase.	2015-05-18 13:24:35 -07:00
Akira Ajisaka	f5c48238d5	HADOOP-11939. Deprecate DistCpV1 and Logalyzer. Contributed by Brahma Reddy Battula.	2015-05-18 16:13:16 +09:00
cnauroth	cb8e69a80c	HADOOP-11960. Enable Azure-Storage Client Side logging. Contributed by Dushyanth.	2015-05-14 22:22:24 -07:00
Akira Ajisaka	92c38e41e1	HADOOP-9723. Improve error message when hadoop archive output path already exists. Contributed by Jean-Baptiste Onofré and Yongjun Zhang.	2015-05-13 17:28:57 +09:00
Allen Wittenauer	ed0f4db0dd	HADOOP-11775. Fix Javadoc typos in hadoop-openstack module (Yanjun Wang via aw)	2015-05-08 17:01:08 -07:00
Akira Ajisaka	c5afe444df	HDFS-8349. Remove .xml and documentation references to dfs.webhdfs.enabled. Contributed by Ray Chiang.	2015-05-08 15:57:13 +09:00
Vinayakumar B	f33efbc827	HADOOP-11922. Misspelling of threshold in log4j.properties for tests in hadoop-tools (Contributed by Gabor Liptak)	2015-05-08 10:22:11 +05:30
Jonathan Eagles	2d7363b273	HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles)	2015-05-01 13:11:50 -05:00
Akira Ajisaka	f384a063a6	HADOOP-11821. Fix findbugs warnings in hadoop-sls. Contributed by Brahma Reddy Battula.	2015-04-30 19:34:41 +09:00
Allen Wittenauer	618ba707f0	MAPREDUCE-6057. Remove obsolete entries from mapred-default.xml (Ray Chiang via aw)	2015-04-26 20:31:40 -07:00
Ravi Prakash	a287d2fb77	HADOOP-11876. Refactor code to make it more readable, minor maybePrintStats bug (Zoran Dimitrijevic via raviprak)	2015-04-24 13:39:07 -07:00
Steve Loughran	19262d99eb	HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel)	2015-04-23 21:41:51 +01:00
Ravi Prakash	cfba355052	HADOOP-11827. Speed-up distcp buildListing() using threadpool (Zoran Dimitrijevic via raviprak)	2015-04-21 16:49:37 -07:00
Jing Zhao	4c097e473b	HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao.	2015-04-15 12:37:20 -07:00
Akira Ajisaka	83979e61a4	MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina.	2015-04-10 20:21:00 +09:00
Colin Patrick Mccabe	932730df7d	HADOOP-11785. Reduce the number of listStatus operation in distcp buildListing (Zoran Dimitrijevic via Colin P. McCabe)	2015-04-03 14:08:25 -07:00
Haohui Mai	75cb1d42ab	HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao.	2015-04-01 16:50:59 -07:00
Harsh J	27d49e6714	MAPREDUCE-6291. Correct mapred queue usage command. Contributed by Brahma Reddy Battula.	2015-03-28 11:58:17 +05:30
Tsuyoshi Ozawa	e074952bd6	HADOOP-11760. Fix typo of javadoc in DistCp. Contributed by Brahma Reddy Battula.	2015-03-27 23:15:51 +09:00
Yongjun Zhang	44809b8081	HADOOP-11724. DistCp throws NPE when the target directory is root. (Lei Eddy Xu via Yongjun Zhang)	2015-03-25 15:45:45 -07:00
Harsh J	b6dea9776b	MAPREDUCE-579. Streaming slowmatch documentation.	2015-03-25 14:39:00 +05:30
Harsh J	e5370477c2	MAPREDUCE-4653. TestRandomAlgorithm has an unused import statement. Contributed by Amir Sanjar.	2015-03-17 14:11:54 +05:30
Tsuyoshi Ozawa	ef9946cd52	HADOOP-11720. [JDK8] Fix javadoc errors caused by incorrect or illegal tags in hadoop-tools. Contributed by Akira AJISAKA.	2015-03-17 16:09:21 +09:00
Tsuyoshi Ozawa	79426f3334	HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto.	2015-03-15 14:29:49 +09:00
Tsuyoshi Ozawa	b308a8d181	Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki." This reverts commit `7da136ecca`.	2015-03-15 14:27:38 +09:00
Tsuyoshi Ozawa	7da136ecca	HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki.	2015-03-15 14:17:35 +09:00
cnauroth	7a346bcb4f	HADOOP-11693. Azure Storage FileSystem rename operations are throttled too aggressively to complete HBase WAL archiving. Contributed by Duo Xu.	2015-03-11 14:36:51 -07:00
Steve Loughran	64443490d7	HADOOP-11670. Regression: s3a auth setup broken. (Adam Budde via stevel)	2015-03-08 11:22:16 -07:00
cnauroth	608ebd52ba	HADOOP-11642. Upgrade azure sdk version from 0.6.0 to 2.0.0. Contributed by Shashank Khandelwal and Ivan Mitic.	2015-03-06 15:25:01 -08:00
Jing Zhao	ed70fa142c	HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao.	2015-03-04 10:30:53 -08:00
Vinod Kumar Vavilapalli	5af693fde2	MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public API in DistCp. Contributed by Jing Zhao.	2015-03-03 16:28:41 -08:00
Steve Loughran	15b7076ad5	HADOOP-11183. Memory-based S3AOutputstream. (Thomas Demoor via stevel)	2015-03-03 16:18:51 -08:00
Tsuyoshi Ozawa	9ae7f9eb7b	MAPREDUCE-5657. Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA.	2015-03-03 18:06:26 +09:00
Tsuyoshi Ozawa	d1c6accb6f	HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)	2015-03-03 14:17:52 +09:00
Allen Wittenauer	039366e3b4	MAPREDUCE-5653. DistCp does not honour config-overrides for mapreduce.[map,reduce].memory.mb (Ratandeep Ratti via aw)	2015-02-28 22:53:38 -08:00
Allen Wittenauer	dce8b9c4d0	HADOOP-11637. bash location hard-coded in shell scripts (aw)	2015-02-26 09:29:16 -08:00
cnauroth	5731c0e0d0	HADOOP-11629. WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true. Contributed by Shanyu Zhao.	2015-02-25 09:08:55 -08:00
Akira Ajisaka	1a625b8158	HADOOP-11480. Typo in hadoop-aws/index.md uses wrong scheme for test.fs.s3.name. Contributed by Ted Yu.	2015-02-24 17:11:46 -08:00
Tsuyoshi Ozawa	9cedad11d8	Revert "HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)" This reverts commit `946456c6d8`. Conflicts: hadoop-common-project/hadoop-common/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaByStorageTypeEntry.java	2015-02-25 00:32:04 +09:00
Steve Loughran	709ff99cff	HADOOP-11584 s3a file block size set to 0 in getFileStatus. (Brahma Reddy Battula via stevel)	2015-02-21 12:03:03 +00:00
Steve Loughran	aa1c437b6a	HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel)	2015-02-20 20:51:24 +00:00
Tsuyoshi Ozawa	946456c6d8	HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)	2015-02-19 13:06:53 +09:00
Allen Wittenauer	b6fc1f3e43	HADOOP-11593. Convert site documentation from apt to markdown (stragglers) (Masatake Iwasaki via aw)	2015-02-17 21:30:24 -10:00
Karthik Kambatla	409113d8f9	MAPREDUCE-6234. TestHighRamJob fails due to the change in MAPREDUCE-5785. (Masatake Iwasaki via kasha)	2015-02-17 14:38:00 -08:00
Steve Loughran	00b80958d8	HADOOP-11521. Make connection timeout configurable in s3a. (Thomas Demoor via stevel)	2015-02-17 20:06:27 +00:00
Steve Loughran	78a7e8d3a6	HADOOP-11522. Update S3A Documentation. (Thomas Demoor via stevel)	2015-02-17 18:15:02 +00:00
Steve Loughran	826267f789	HADOOP-11570. S3AInputStream.close() downloads the remaining bytes of the object from S3. (Dan Hecht via stevel).	2015-02-17 16:36:44 +00:00
Allen Wittenauer	7776847a59	HADOOP-11437. Remove the version and author information from distcp's README file (Brahma Reddy Battula via aw)	2015-02-11 15:47:36 -08:00
Steve Loughran	17165d3df9	HADOOP-11520. Clean incomplete multi-part uploads in S3A tests. (Thomas Demoor via stevel)	2015-02-06 21:01:41 +00:00
Steve Loughran	4e7ad4f0a8	HADOOP-11463 Replace method-local TransferManager object with S3AFileSystem#transfers. (Ted Yu via stevel)	2015-02-05 12:20:23 +00:00
Allen Wittenauer	9112f093cd	HADOOP-10976. moving the source code of hadoop-tools docs to the directory under hadoop-tools (Masatake Iwasaki via aw)	2015-02-04 17:57:34 -08:00
Allen Wittenauer	f990e9d229	HADOOP-11346. Rewrite sls/rumen to use new shell framework (John Smith via aw)	2015-02-04 17:27:45 -08:00
Akira Ajisaka	34fe11c987	MAPREDUCE-6243. Fix findbugs warnings in hadoop-rumen. Contributed by Masatake Iwasaki.	2015-02-04 09:25:44 -08:00
Tsuyoshi Ozawa	ba58da29f4	HADOOP-11546. Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck. (ozawa)	2015-02-05 00:42:14 +09:00
Akira Ajisaka	bd69fb2d44	MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka)	2015-02-03 14:30:09 -08:00
Karthik Kambatla	c559df2219	YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)	2015-02-03 13:20:41 -08:00
Haohui Mai	c89977f89c	HADOOP-11529. Fix findbugs warnings in hadoop-archives. Contributed by Masatake Iwasaki.	2015-02-03 10:53:17 -08:00
cnauroth	1c09ca2ba4	HADOOP-11442. hadoop-azure: Create test jar. Contributed by Shashank Khandelwal.	2015-02-02 11:50:43 -08:00
Harsh J	ffc75d6ebe	HADOOP-11488. Difference in default connection timeout for S3A FS. Contributed by Daisuke Kobayashi.	2015-02-01 00:17:04 +05:30
Allen Wittenauer	12e883007c	MAPREDUCE-6151. Update document of GridMix (Masatake Iwasaki via aw)	2015-01-30 11:08:20 -08:00
cnauroth	f2c91098c4	HADOOP-11523. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.	2015-01-29 17:02:38 -08:00
Allen Wittenauer	3f982c5c26	MAPREDUCE-6150. Update document of Rumen (Masatake Iwasaki via aw)	2015-01-29 14:17:44 -08:00
cnauroth	ad55083f75	HADOOP-11441. Hadoop-azure: Change few methods scope to public. Contributed by Shashank Khandelwal.	2015-01-29 13:07:51 -08:00
Allen Wittenauer	0a05ae1782	HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw)	2015-01-27 14:31:27 -08:00
Tsuyoshi Ozawa	7b82c4ab4e	HADOOP-11450. Cleanup DistCpV1 not to use deprecated methods and fix javadocs. Contributed by Varun Saxena.	2015-01-26 12:58:38 +09:00
Allen Wittenauer	42715547f0	HADOOP-11256. Some site docs have inconsistent appearance (Masatake Iwasaki via aw)	2015-01-21 08:44:22 -08:00
Steve Loughran	2908fe4ec5	HADOOP-11171 Enable using a proxy server to connect to S3a. (Thomas Demoor via stevel)	2015-01-17 18:40:50 +00:00
Steve Loughran	c6c0f4eb25	HADOOP-10542 Potential null pointer dereference in Jets3tFileSystemStore retrieveBlock(). (Ted Yu via stevel)	2015-01-17 18:26:26 +00:00
Steve Loughran	000ca83ea3	HADOOP-11261 Set custom endpoint for S3A. (Thomas Demoor via stevel)	2015-01-16 10:15:51 +00:00
Arpit Agarwal	c4cba6165a	HADOOP-11465. Fix findbugs warnings in hadoop-gridmix. (Contributed by Varun Saxena)	2015-01-12 21:22:58 -08:00
Steve Loughran	27d8395867	HADOOP-11446 S3AOutputStream should use shared thread pool to avoid OutOfMemoryError	2015-01-05 13:00:01 +00:00
Karthik Kambatla	e7257acd8a	YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha)	2014-12-30 19:55:24 -08:00
Tsuyoshi Ozawa	a164ce2ac9	HADOOP-11283. SequenceFile.Writer can leak file descriptors in DistCpV1#setup(). (Contributed by Varun Saxena via ozawa)	2014-12-25 23:30:45 +09:00
Jian He	fdf042dfff	YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan	2014-12-22 16:51:15 -08:00
Haohui Mai	02b21b7131	HADOOP-11429. Findbugs warnings in hadoop extras. Contributed by Varun Saxena.	2014-12-21 14:57:34 -08:00
arp	c1f857b0b4	HADOOP-11395. Add site documentation for Azure Storage FileSystem integration. (Contributed by Chris Nauroth)	2014-12-19 18:54:22 -08:00
Haohui Mai	efe6357748	HADOOP-11431. Clean up redundant maven-site-plugin configuration. Contributed by Herve Boutemy.	2014-12-18 16:23:35 -08:00
Steve Loughran	07bb0b0bbb	HADOOP-11412 POMs mention "The Apache Software License" rather than "Apache License". (Herve Boutemy via stevel)	2014-12-16 11:04:36 +00:00
cnauroth	9458cd5bce	HADOOP-11394. hadoop-aws documentation missing. Contributed by Chris Nauroth.	2014-12-12 23:29:11 -08:00
Haohui Mai	2e98ad34ce	HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack. Contributed by Li Lu.	2014-12-09 20:45:21 -08:00
Haohui Mai	7bceb13ba9	HADOOP-11367. Fix warnings from findbugs 3.0 in hadoop-streaming. Contributed by Li Lu.	2014-12-09 10:41:35 -08:00
Jing Zhao	57cb43be50	HADOOP-11354. ThrottledInputStream doesn't perform effective throttling. Contributed by Ted Yu.	2014-12-08 11:08:39 -08:00
cnauroth	79301e80d7	HADOOP-11201. Hadoop Archives should support globs resolving to files. Contributed by Gera Shegalov.	2014-11-18 17:05:48 -08:00
Aaron T. Myers	6ba52d88ec	HADOOP-10714. AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call. Contributed by Juan Yu.	2014-11-05 17:17:04 -08:00
cnauroth	734eeb4f35	HADOOP-11248. Add hadoop configuration to disable Azure Filesystem metrics collection. Contributed by Shanyu Zhao.	2014-11-03 14:29:18 -08:00
Zhijie Shen	260ab6d5f4	HADOOP-11241. Fixed intermittent TestNMSimulator failure due to timing issue. Contributed by Varun Vasudev.	2014-10-31 18:02:39 -07:00
Haohui Mai	f1a149e916	HADOOP-11253. Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows. Contributed by Varun Vasudev.	2014-10-31 11:54:33 -07:00
Kihwal Lee	b056048114	MAPREDUCE-6022. map_input_file is missing from streaming job environment. Contributed by Jason Lowe.	2014-10-29 12:29:07 -05:00
Jian He	b8f7966c7a	MAPREDUCE-6126. Fixed Rumen JobBuilder to ignore NormalizedResourceEvent. Contributed by Junping Du	2014-10-22 10:43:20 -07:00
Vinod Kumar Vavilapalli	f2ea555ac6	YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan. YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.	2014-10-15 18:33:06 -07:00
cnauroth	f4b7e99f4e	HADOOP-11188. hadoop-azure: automatically expand page blobs when they become full. Contributed by Eric Hanson.	2014-10-10 15:05:52 -07:00
cnauroth	2217e2f8ff	HADOOP-10809. hadoop-azure: page blob support. Contributed by Dexter Bradshaw, Mostafa Elhemali, Eric Hanson, and Mike Liddell.	2014-10-08 14:20:23 -07:00
Kihwal Lee	c8617ff136	Missing files from the previous commit of HADOOP-11007	2014-10-07 16:30:53 -05:00
Kihwal Lee	99d7a452be	HADOOP-11007. Reinstate building of ant tasks support. Contributed by Jason Lowe.	2014-10-07 16:20:16 -05:00
Allen Wittenauer	83264cf457	HADOOP-10954. Adding site documents of hadoop-tools (Masatake Iwasaki via aw)	2014-09-26 11:59:45 -07:00
Andrew Wang	4ea77efa3a	HADOOP-11140. hadoop-aws only need test-scoped dependency on hadoop-common's tests jar. Contributed by Juan Yu.	2014-09-25 21:56:26 -07:00
Allen Wittenauer	f614cb71d4	MAPREDUCE-6109. Fix minor typo in distcp -p usage text (Charles Lamb via aw)	2014-09-25 11:27:33 -07:00
Allen Wittenauer	3cde37c991	HADOOP-11009. Add Timestamp Preservation to DistCp (Gary Steelman via aw)	2014-09-24 15:38:30 -07:00
Allen Wittenauer	aa1052c34b	YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw)	2014-09-19 14:07:38 -07:00
Allen Wittenauer	9f03a7c018	HADOOP-10946. Fix a bunch of typos in log messages (Ray Chiang via aw)	2014-09-19 11:33:07 -07:00
Colin Patrick Mccabe	0ac760a58d	HADOOP-11091. Eliminate old configuration parameter names from s3a (dsw via cmccabe)	2014-09-15 16:49:15 -07:00
Aaron T. Myers	24d920b80e	HADOOP-10400. Incorporate new S3A FileSystem implementation. Contributed by Jordan Mendelson and Dave Wang.	2014-09-15 08:27:07 -07:00
Colin Patrick Mccabe	5ec7fcd9dd	HADOOP-11074. Move s3-related FS connector code to hadoop-aws. (David S. Wang via Colin Patrick McCabe)	2014-09-10 16:14:53 -07:00
Karthik Kambatla	28d99db992	YARN-2526. SLS can deadlock when all the threads are taken by AMSimulators. (Wei Yan via kasha)	2014-09-09 15:13:16 -07:00
Allen Wittenauer	d989ac0444	MAPREDUCE-5972. Fix typo 'programatically' in job.xml (and a few other places) (Akira AJISAKA via aw)	2014-09-08 13:15:46 -07:00
Steve Loughran	59384dfb71	HADOOP-10373 create tools/hadoop-amazon for aws/EMR support	2014-09-02 20:11:13 +01:00
Jian He	5c14bc426b	YARN-1506. Changed RMNode/SchedulerNode to update resource with event notification. Contributed by Junping Du	2014-08-29 23:05:51 -07:00
Andrew Wang	ea1c6f31c2	HADOOP-11021. Configurable replication factor in the hadoop archive command. Contributed by Zhe Zhang.	2014-08-29 14:44:37 -07:00
Sanford Ryza	5109157ed1	MAPREDUCE-5130. Add missing job config options to mapred-default.xml (Ray Chiang via Sandy Ryza) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619626 13f79535-47bb-0310-9956-ffa450edef68	2014-08-21 23:28:44 +00:00
Andrew Wang	d2a39b61aa	Merge from trunk to branch. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1619018 13f79535-47bb-0310-9956-ffa450edef68	2014-08-20 01:22:42 +00:00
Charles Lamb	14b01dd046	HDFS-6872. Fix TestOptionsParser. (clamb) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618974 13f79535-47bb-0310-9956-ffa450edef68	2014-08-19 20:41:15 +00:00
Zhijie Shen	f6a778c372	YARN-2249. Avoided AM release requests being lost on work preserving RM restart. Contributed by Jian He. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618972 13f79535-47bb-0310-9956-ffa450edef68	2014-08-19 20:33:49 +00:00
Andrew Wang	0cc08f6da4	Merge from trunk to branch. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618700 13f79535-47bb-0310-9956-ffa450edef68	2014-08-18 18:41:31 +00:00
Allen Wittenauer	399e428deb	HADOOP-10121. Fix javadoc spelling for HadoopArchives#writeTopLevelDirs (Akira AJISAKA via aw) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618044 13f79535-47bb-0310-9956-ffa450edef68	2014-08-14 20:28:36 +00:00
Colin McCabe	e90d940699	HADOOP-10843. TestGridmixRecord unit tests failure on PowerPC (Jinghui Wang via Colin Patrick McCabe) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617860 13f79535-47bb-0310-9956-ffa450edef68	2014-08-14 01:36:12 +00:00
Karthik Kambatla	486e718fc1	YARN-2399. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617600 13f79535-47bb-0310-9956-ffa450edef68	2014-08-12 21:43:27 +00:00
Charles Lamb	041b8326a1	MAPREDUCE-6007. Add support to distcp to preserve raw.* namespace extended attributes. (clamb) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616657 13f79535-47bb-0310-9956-ffa450edef68	2014-08-08 01:32:18 +00:00
Steve Loughran	96fc3b3ecd	HADOOP-10931 compile error on tools/hadoop-openstack git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616482 13f79535-47bb-0310-9956-ffa450edef68	2014-08-07 12:38:35 +00:00
Karthik Kambatla	4625792c20	YARN-1726. Add missing files. ResourceSchedulerWrapper broken due to AbstractYarnScheduler. (Wei Yan via kasha) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613552 13f79535-47bb-0310-9956-ffa450edef68	2014-07-26 02:03:18 +00:00
Karthik Kambatla	b0c2c85619	YARN-1726. ResourceSchedulerWrapper broken due to AbstractYarnScheduler. (Wei Yan via kasha) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613547 13f79535-47bb-0310-9956-ffa450edef68	2014-07-26 01:29:25 +00:00
Karthik Kambatla	77363b9d83	YARN-2335. Annotate all hadoop-sls APIs as @Private. (Wei Yan via kasha) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613478 13f79535-47bb-0310-9956-ffa450edef68	2014-07-25 17:12:22 +00:00
Chris Nauroth	0a02b5a19b	HADOOP-10840. Fix OutOfMemoryError caused by metrics system in Azure File System. Contributed by Shanyu Zhao. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611247 13f79535-47bb-0310-9956-ffa450edef68	2014-07-17 05:04:53 +00:00
Andrew Wang	e8a1ff7efe	MAPREDUCE-5971. Move the default options for distcp -p to DistCpOptionSwitch. Contributed by Charles Lamb. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611217 13f79535-47bb-0310-9956-ffa450edef68	2014-07-16 23:42:34 +00:00
Steve Loughran	a10a4f72ab	HADOOP-9361: Strictly define FileSystem APIs - OpenStack portion git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607600 13f79535-47bb-0310-9956-ffa450edef68	2014-07-03 12:12:52 +00:00
Steve Loughran	becc23fb65	HADOOP-9361: Strictly define FileSystem APIs - OpenStack portion git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607599 13f79535-47bb-0310-9956-ffa450edef68	2014-07-03 12:10:52 +00:00
Chris Nauroth	0d91576ec3	HADOOP-10728. Metrics system for Windows Azure Storage Filesystem. Contributed by Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao, Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys, Alexander Stojanovic, Brian Swan, and Min Wei. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605187 13f79535-47bb-0310-9956-ffa450edef68	2014-06-24 20:52:44 +00:00
Chris Nauroth	61bf9f7799	HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604236 13f79535-47bb-0310-9956-ffa450edef68	2014-06-20 18:36:11 +00:00
Chris Nauroth	8300b9fb38	HADOOP-10689. InputStream is not closed in AzureNativeFileSystemStore#retrieve(). Contributed by Chen He. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604233 13f79535-47bb-0310-9956-ffa450edef68	2014-06-20 18:30:49 +00:00
Haohui Mai	22cbcd16cb	HADOOP-10717. Revert r1603571. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603620 13f79535-47bb-0310-9956-ffa450edef68	2014-06-18 20:37:25 +00:00
Steve Loughran	bae5837d3b	HADOOP-10717. Missing JSP support in Jetty git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603534 13f79535-47bb-0310-9956-ffa450edef68	2014-06-18 17:08:45 +00:00
Andrew Wang	50cd2a6bed	HADOOP-10666. Remove Copyright /d/d/d/d Apache Software Foundation from the source files license header. Contributed by Henry Saputra. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603025 13f79535-47bb-0310-9956-ffa450edef68	2014-06-16 23:23:33 +00:00
Chris Nauroth	81bc395deb	HADOOP-9629. Support Windows Azure Storage - Blob as a file system in Hadoop. Contributed by Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao, Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys, Alexander Stojanovic, Brian Swan, and Min Wei. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601781 13f79535-47bb-0310-9956-ffa450edef68	2014-06-10 22:26:45 +00:00
Vinod Kumar Vavilapalli	6480fdfd33	MAPREDUCE-5834. Increased test-timeouts in TestGridMixClasses to avoid occassional failures. Contributed by Mit Desai. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601764 13f79535-47bb-0310-9956-ffa450edef68	2014-06-10 20:24:57 +00:00
Steve Loughran	3d077726d0	HADOOP-10647. String Format Exception in SwiftNativeFileSystemStore.java. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600971 13f79535-47bb-0310-9956-ffa450edef68	2014-06-06 18:00:52 +00:00
Uma Maheswara Rao G	f81c7b0252	MAPREDUCE-5898. distcp to support preserving HDFS extended attributes(XAttrs). Contributed by Yi Liu. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600900 13f79535-47bb-0310-9956-ffa450edef68	2014-06-06 14:45:39 +00:00
Karthik Kambatla	a4ba451802	YARN-1474. Make schedulers services. (Tsuyoshi Ozawa via kasha) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598908 13f79535-47bb-0310-9956-ffa450edef68	2014-05-31 19:33:09 +00:00
Chris Nauroth	806f4d11f2	HADOOP-10602. Documentation has broken "Go Back" hyperlinks. Contributed by Akira AJISAKA. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598337 13f79535-47bb-0310-9956-ffa450edef68	2014-05-29 17:26:50 +00:00
Jing Zhao	3671a5e16f	MAPREDUCE-5899. Support incremental data copy in DistCp. Contributed by Jing Zhao. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596931 13f79535-47bb-0310-9956-ffa450edef68	2014-05-22 18:17:11 +00:00
Vinod Kumar Vavilapalli	82f3454f5a	YARN-2017. Merged some of the common scheduler code. Contributed by Jian He. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596753 13f79535-47bb-0310-9956-ffa450edef68	2014-05-22 05:32:26 +00:00
Chris Nauroth	11be7334c4	MAPREDUCE-5809. Enhance distcp to support preserving HDFS ACLs. Contributed by Chris Nauroth. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595283 13f79535-47bb-0310-9956-ffa450edef68	2014-05-16 18:25:23 +00:00
Tsz-wo Sze	03db13206f	MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE, MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable. Contributed by Tsuyoshi OZAWA git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592703 13f79535-47bb-0310-9956-ffa450edef68	2014-05-06 10:24:10 +00:00

... 12 13 14 15 16 ...

1489 Commits