1607 Commits

Author	SHA1	Message	Date
Thomas Marquardt	26211019c8	HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" ... Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	e5593cbd83	HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. ... Contributed by Sean Mackrory.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	13c70e9ba3	HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. ... Contributed by Sneha Varma.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	6801b30733	HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation ... for storage account with no namespace feature. Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	347a52a866	Fixing findbugs and license issues related to: ... HADOOP-15703. ABFS - Implement client-side throttling. Contributed by Sneha Varma and Thomas Marquardt.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	97f06b3fc7	HADOOP-15703. ABFS - Implement client-side throttling. ... Contributed by Sneha Varma and Thomas Marquardt.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	4410eacba7	HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. ... Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	81dc4a995c	HADOOP-15663. ABFS: Simplify configuration. ... Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	df57c6c3b1	HADOOP-15692. ABFS: extensible support for custom oauth. ... Contributed by Junhua Gu and Rajeev Bansal.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	dd2b22fa31	HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. ... Contributed by Thomas Marquardt.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	6b6f8cc2be	HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. ... Contributed by Sean Mackrory.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	9c1e4e8139	HADOOP-15661. ABFS: Add support for ACL. ... Contributed by Junhua Gu and Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	9149b9703e	HADOOP-15660. ABFS: Add support for OAuth ... Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	d6a4f39bd5	HADOOP-15669. ABFS: Improve HTTPS Performance. ... Contributed by Vishwajeet Dusane.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	cc5cc60c41	Fixing issue due to commit 2b2399d6 after rebase onto trunk.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	b54b0c1b67	HADOOP-15659. Code changes for bug fix and new tests. ... Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	ce03a93f78	HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. ... Contributed by Steve Loughran and Da Zhou.	2018-09-17 19:54:01 +00:00
Steve Loughran	a271fd0eca	HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. ... Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Steve Loughran	f044deedbb	HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. ... Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran	2018-09-17 19:54:01 +00:00
Steve Loughran	d7c0a08a1c	HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran)	2018-09-12 21:04:49 -07:00
Aaron Fabbri	d32a8d5d58	HADOOP-14734 add option to tag DDB table(s) created. (Contributed by Gabor Bota and Abe Fine)	2018-09-12 16:36:01 -07:00
Mingliang Liu	1f6c4545cf	HADOOP-15750. Remove obsolete S3A test ITestS3ACredentialsInURL. Contributed by Steve Loughran	2018-09-12 10:58:39 -07:00
Sean Mackrory	47b72c87eb	HADOOP-15635. s3guard set-capacity command to fail fast if bucket is unguarded. ... Contributed by Gabor Bota.	2018-09-12 09:12:38 -06:00
bibinchundatt	c44088ac19	YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan.	2018-09-12 16:01:01 +05:30
Naganarasimha	eb08543c7a	YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt.	2018-09-12 16:01:01 +05:30
Sunil G	89b3ebd11e	YARN-8092. Expose Node Attributes info via RM nodes REST API. Contributed by Weiwei Yang.	2018-09-12 16:01:00 +05:30
Mingliang Liu	87f63b6479	HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran	2018-09-11 17:18:42 -07:00
Steve Loughran	1eef0acfe5	HADOOP-15677. WASB: Add support for StreamCapabilities. ... Contributed by Thomas Marquardt.	2018-09-10 14:24:41 +01:00
Gabor Bota	36c7c78260	HADOOP-15709 Move S3Guard LocalMetadataStore constants to org.apache.hadoop.fs.s3a.Constants (Contributed by Gabor Bota)	2018-09-07 10:25:20 -07:00
Steve Loughran	5a0babf765	HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs. ... Contributed by Steve Loughran.	2018-08-30 14:49:53 +01:00
Steve Loughran	e8d138ca7c	HADOOP-15680. ITestNativeAzureFileSystemConcurrencyLive times out. ... Contributed by Andras Bokor.	2018-08-30 14:36:00 +01:00
Steve Loughran	2e6c1109dc	HADOOP-15667. FileSystemMultipartUploader should verify that UploadHandle has non-0 length. ... Contributed by Ewan Higgs	2018-08-30 14:33:16 +01:00
Yufei Gu	8563fd67be	YARN-8632. Threads in SLS quit without logging exception. Contributed by Xianghao Lu.	2018-08-24 11:23:17 -07:00
Surendra Singh Lilhore	96c4575d73	HDFS-13805. Journal Nodes should allow to format non-empty directories with -force option. Contributed by Surendra Singh Lilhore.	2018-08-24 08:14:57 +05:30
Aaron Fabbri	d7232857d8	HADOOP-14154 Persist isAuthoritative bit in DynamoDBMetaStore (Contributed by Gabor Bota)	2018-08-17 10:15:39 -07:00
Steve Loughran	0e832e7a74	HADOOP-15642. Update aws-sdk version to 1.11.375. ... Contributed by Steve Loughran.	2018-08-16 09:58:46 -07:00
Akira Ajisaka	3e3963b035	HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering.	2018-08-16 00:31:59 +09:00
Ewan Higgs	a13929ddcb	HADOOP-15645. ITestS3GuardToolLocal.testDiffCommand fails if bucket has per-bucket binding to DDB. Contributed by Steve Loughran.	2018-08-13 12:57:45 +02:00
Steve Loughran	da9a39eed1	HADOOP-15583. Stabilize S3A Assumed Role support. ... Contributed by Steve Loughran.	2018-08-08 22:57:24 -07:00
Ewan Higgs	2ec97abb2e	HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran.	2018-08-08 13:50:23 +02:00
Sean Mackrory	7862f1523f	HADOOP-15400. Improve S3Guard documentation on Authoritative Mode implementation. (Contributed by Gabor Bota)	2018-08-07 20:13:09 -06:00
Steve Loughran	48673bc2a8	HADOOP-15626. FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append. ... Contributed by Steve Loughran.	2018-08-03 16:06:00 -07:00
Inigo Goiri	9fea5c9ee7	YARN-8175. Add support for Node Labels in SLS. Contributed by Abhishek Modi.	2018-07-31 09:36:34 -07:00
Sammi Chen	0857f116b7	HADOOP-15607. AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream. Contributed by Jinhu Wu.	2018-07-30 10:53:44 +08:00
Sean Mackrory	59adeb8d7f	HADOOP-15636. Follow-up from HADOOP-14918; restoring test under new name. Contributed by Gabor Bota.	2018-07-27 18:23:29 -06:00
bibinchundatt	3d3158cea4	YARN-8577. Fix the broken anchor in SLS site-doc. Contributed by Weiwei Yang.	2018-07-25 18:59:05 +05:30
Steve Loughran	45d9568aaa	HADOOP-15547/ WASB: improve listStatus performance. ... Contributed by Thomas Marquardt. (cherry picked from commit 749fff577ed9afb4ef8a54b8948f74be083cc620)	2018-07-19 12:31:19 -07:00
Wangda Tan	752dcce5f4	YARN-8511. When AM releases a container, RM removes allocation tags before it is released by NM. (Weiwei Yang via wangda) ... Change-Id: I6f9f409f2ef685b405cbff547dea9623bf3322d9	2018-07-16 10:54:41 -07:00
Sean Mackrory	a08812a1b1	HADOOP-15349. S3Guard DDB retryBackoff to be more informative on limits exceeded. Contributed by Gabor Bota.	2018-07-12 17:24:01 +02:00
Sean Mackrory	d503f65b66	HADOOP-15541. [s3a] Shouldn't try to drain stream before aborting ... connection in case of timeout.	2018-07-10 17:52:57 +02:00
Steve Loughran	ca8b80bf59	HADOOP-15384. distcp numListstatusThreads option doesn't get to -delete scan. ... Contributed by Steve Loughran.	2018-07-10 10:43:59 +01:00
Aaron Fabbri	93ac01cb59	HADOOP-15215 s3guard set-capacity command to fail on read/write of 0 (Gabor Bota)	2018-07-03 13:50:11 -07:00
Inigo Goiri	1804a31515	HDFS-13536. [PROVIDED Storage] HA for InMemoryAliasMap. Contributed by Virajith Jalaparti.	2018-07-02 10:48:20 -07:00
Inigo Goiri	73746c5da7	HDFS-13707. [PROVIDED Storage] Fix failing integration tests in ITestProvidedImplementation. Contributed by Virajith Jalaparti.	2018-06-29 09:56:13 -07:00
Akira Ajisaka	2b2399d623	HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma.	2018-06-28 14:37:22 +09:00
Todd Lipcon	7a3c6e9c3c	HADOOP-15550. Avoid static initialization of ObjectMappers	2018-06-25 15:36:45 -07:00
Sean Mackrory	c687a6617d	HADOOP-15423. Merge fileCache and dirCache into ine single cache in LocalMetadataStore. Contributed by Gabor Bota.	2018-06-25 14:59:41 -06:00
Anu Engineer	e16e5b307d	HDDS-177. Create a releasable ozonefs artifact ... Contributed by Marton, Elek.	2018-06-24 01:05:04 -07:00
Anu Engineer	ca14fec02f	HDDS-184. Upgrade common-langs version to 3.7 in hadoop-tools/hadoop-ozone. ... Contributed by Takanobu Asanuma.	2018-06-23 15:49:44 -07:00
Sean Mackrory	55fad6a3de	HADOOP-15416. Clear error message in S3Guard diff if source not found. Contributed by Gabor Bota.	2018-06-22 11:36:56 -06:00
Sean Mackrory	b089a06793	HADOOP-14918. Remove the Local Dynamo DB test option. Contributed by Gabor Bota.	2018-06-20 16:45:08 -06:00
Chris Douglas	f34744603e	HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii	2018-06-17 23:12:18 -07:00
Chris Douglas	980031bb04	HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs	2018-06-17 11:54:26 -07:00
Jitendra Pandey	f4c7c91123	HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan.	2018-06-13 00:36:02 -07:00
Inigo Goiri	c35481594f	YARN-8422. TestAMSimulator failing with NPE. Contributed by Giovanni Matteo Fumarola.	2018-06-12 10:59:50 -07:00
Steve Loughran	d901be6795	HADOOP-15506. Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks. ... Contributed by Esfandiar Manii.	2018-06-06 18:28:14 +01:00
Anu Engineer	3b34148c4f	HDDS-88. Create separate message structure to represent ports in DatanodeDetails. ... Contributed by Nanda Kumar.	2018-05-30 08:52:07 -07:00
Aaron Fabbri	5f6769f796	HADOOP-15480 AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo (Gabor Bota)	2018-05-29 19:20:22 -07:00
Aaron Fabbri	30284d020d	HADOOP-14946 S3Guard testPruneCommandCLI can fail. Contributed by Gabor Bota.	2018-05-29 13:38:15 -07:00
Inigo Goiri	8fdc993a99	HADOOP-15498. TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows. Contributed by Anbang Hu.	2018-05-28 16:45:42 -07:00
Johan Gustavsson	d14e26b31f	HADOOP-15477. Make unjar in RunJar overrideable ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-05-28 17:29:59 +09:00
Steve Loughran	ba842847c9	HADOOP-15478. WASB: hflush() and hsync() regression. ... Contributed by Thomas Marquardt.	2018-05-21 11:12:34 +01:00
Steve Loughran	cc3600aabd	HADOOP-15469. S3A directory committer commit job fails if _temporary directory created under dest. ... Contributed by Steve Loughran. (cherry picked from commit 170f1040d46f9b1a084e6637def91e9864446acc)	2018-05-17 19:08:55 +01:00
Steve Loughran	b670837469	HADOOP-15442. ITestS3AMetrics.testMetricsRegister can't know metrics source's name. ... Contributed by Sean Mackrory.	2018-05-15 16:13:56 +01:00
Steve Loughran	ba051b0686	HADOOP-15354. hadoop-aliyun & hadoop-azure modules to mark hadoop-common as provided ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-05-10 18:38:08 +09:00
Anu Engineer	46e0f27862	HDDS-18. Ozone Shell should use RestClient and RpcClient. Contributed by Lokesh Jain.	2018-05-09 23:00:13 -07:00
Sean Mackrory	1cfe7506f7	HADOOP-15356. Make HTTP timeout configurable in ADLS connector. Contributed by Atul Sikaria and Sean Mackrory.	2018-05-09 15:15:51 -06:00
Mukul Kumar Singh	343b51dcf4	HDDS-28. Ozone:Duplicate declaration in hadoop-tools/hadoop-ozone/pom.xml. Contributed by Tsz Wo Nicholas Sze.	2018-05-09 20:20:21 +05:30
Mukul Kumar Singh	208b97e969	HDDS-20. Ozone: Add support for rename key within a bucket for rpc client. Contributed by Lokesh Jain.	2018-05-09 19:06:07 +05:30
Aaron Fabbri	8981674bbc	HADOOP-15420 s3guard ITestS3GuardToolLocal failures in diff tests (Gabor Bota)	2018-05-08 18:58:42 -07:00
Aaron Fabbri	69aac696d9	HADOOP-13649 s3guard: implement time-based (TTL) expiry for LocalMetadataStore (Gabor Bota)	2018-05-08 15:29:54 -07:00
Steve Loughran	5b11b9fd41	HADOOP-15446. WASB: PageBlobInputStream.skip breaks HBASE replication. ... Contributed by Thomas Marquardt	2018-05-07 11:54:08 +01:00
Aaron Fabbri	96c843f64b	HADOOP-15444 ITestS3GuardToolDynamo should only run with -Ddynamo (Aaron Fabbri)	2018-05-04 11:34:45 -07:00
Sean Mackrory	85381c7b60	HADOOP-15434. Upgrade to ADLS SDK that exposes current timeout.	2018-05-02 21:30:31 -06:00
Aaron Fabbri	919865a34b	HADOOP-15239 S3ABlockOutputStream.flush() be no-op when stream closed. Contributed by Gabor Bota.	2018-04-30 16:02:57 -07:00
Weiwei Yang	ba2db39c65	YARN-8152. Add chart in SLS to illustrate the throughput of the scheduler. Contributed by Tao Yang.	2018-04-27 15:36:21 +08:00
Aaron Fabbri	7d8bcf534a	HADOOP-13756 LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash. (Gabor Bota)	2018-04-26 20:41:29 -07:00
Owen O'Malley	2adda92de1	HDFS-7240 git commit fixup ... Signed-off-by: Owen O'Malley <omalley@apache.org>	2018-04-26 13:52:17 -07:00
Mukul Kumar Singh	ae8ac7f082	HDFS-13424. Ozone: Refactor MiniOzoneClassicCluster. Contributed by Nanda Kumar.	2018-04-26 05:36:04 -07:00
Nanda kumar	025058f251	HDFS-13446. Ozone: Fix OzoneFileSystem contract test failures. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	8b832f3c35	HDFS-13405. Ozone: Rename HDSL to HDDS. ... Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.	2018-04-26 05:36:04 -07:00
Nanda kumar	7ae9543a57	HDFS-13391. Ozone: Make dependency of internal sub-module scope as provided in maven. Contributed by Nanda kumar.	2018-04-26 05:36:04 -07:00
Anu Engineer	04e1287771	HDFS-13298. Ozone: Make ozone/hdsl/cblock modules turned off by default. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Anu Engineer	4e61bc431e	HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. ... Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.	2018-04-26 05:36:04 -07:00
Anu Engineer	5e7164c614	HDFS-13137. Ozone: Ozonefs read fails because ChunkGroupInputStream#read does not iterate through all the blocks in the key. ... Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	61651dcf5c	HDFS-13108. Ozone: OzoneFileSystem: Simplified url schema for Ozone File System. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Mukul Kumar Singh	cd807c9890	HDFS-13133. Ozone: OzoneFileSystem: Calling delete with non-existing path shouldn't be logged on ERROR level. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Mukul Kumar Singh	9272e1021d	HDFS-12636. Ozone: OzoneFileSystem: Implement seek functionality for rpc client. Contributed by Lokesh Jain.	2018-04-26 05:36:04 -07:00
Anu Engineer	4460ac9c57	HDFS-12756. Ozone: Add datanodeID to heartbeat responses and container protocol. Contributed by Anu Engineer.	2018-04-26 05:36:04 -07:00
Anu Engineer	740a06cdd7	HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Chen Liang	e395244e95	HDFS-12758. Ozone: Correcting assertEquals argument order in test cases. Contributed by Bharat Viswanadham.	2018-04-26 05:36:04 -07:00
Xiaoyu Yao	2ec24a7542	HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar.	2018-04-26 05:36:04 -07:00
Chen Liang	895a4cc60a	HDFS-12632. Ozone: OzoneFileSystem: Add contract tests to OzoneFileSystem. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	d90ab3365d	HDFS-12608. Ozone: Remove Warnings when building. Contributed by Bharat Viswanadham.	2018-04-26 05:36:04 -07:00
Xiaoyu Yao	9445a9267f	HDFS-12572. Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	0b3270a8de	HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar.	2018-04-26 05:36:04 -07:00
Xiaoyu Yao	5e4a6b686c	HDFS-12425. Ozone: OzoneFileSystem: read/write/create/open/getFileInfo APIs. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Xiaoyu Yao	f2d9360b9d	HDFS-12199. Ozone: OzoneFileSystem: OzoneFileystem initialization code. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Akira Ajisaka	3e2ca23a53	HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang.	2018-04-26 05:36:04 -07:00
Aaron Fabbri	989a3929a9	HADOOP-14756 S3Guard: expose capability query in MetadataStore and add tests of authoritative mode (Gabor Bota)	2018-04-23 15:26:53 -07:00
Inigo Goiri	fd24fd0ff7	YARN-8137. Parallelize node addition in SLS. Contributed by Abhishek Modi.	2018-04-20 09:09:12 -07:00
Yiqun Lin	1312f9ae4c	HADOOP-15391. Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules.	2018-04-18 16:04:00 +08:00
Inigo Goiri	78860372bd	YARN-8134. Support specifying node resources in SLS. Contributed by Abhishek Modi.	2018-04-17 13:11:34 -07:00
Aaron Fabbri	b0aff8a962	HADOOP-15346 S3ARetryPolicy for 400/BadArgument to be "fail". Contributed by Steve Loughran.	2018-04-10 23:55:38 -07:00
Xiao Chen	7c9cdad6d0	HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo.	2018-04-10 21:31:48 -07:00
Aaron Fabbri	ea3849f0cc	HADOOP-14759 S3GuardTool prune to prune specific bucket entries. Contributed by Gabor Bota.	2018-04-05 20:23:17 -07:00
Steve Loughran	5a174f8ac6	HADOOP-14758. S3GuardTool.prune to handle UnsupportedOperationException. ... Contributed by Gabor Bota.	2018-04-03 16:31:34 +01:00
Sammi Chen	6542d17ea4	HADOOP-14999. AliyunOSS: provide one asynchronous multi-part based uploading mechanism. Contributed by Genmao Yu.	2018-03-30 20:23:05 +08:00
Sean Mackrory	9d7a9031a5	HADOOP-15342. Updating ADLS connector to use the current SDK version (2.2.7). Contributed by Atul Sikaria.	2018-03-29 10:13:40 -06:00
Chris Douglas	081c350188	HADOOP-15320. Remove customized getFileBlockLocations for hadoop-azure and hadoop-azure-datalake. Contributed by Shanyu Zhao	2018-03-28 11:58:59 -07:00
Haibo Chen	fcea5a4d79	YARN-7794. SLSRunner is not loading timeline service jars, causing failure. (Yufei Gu via Haibo Chen)	2018-03-26 14:30:11 -07:00
Sean Mackrory	82665a7887	HADOOP-15299. Bump Jackson 2 version to Jackson 2.9.x.	2018-03-26 14:05:15 -06:00
Sean Mackrory	7ce6b41509	HADOOP-15332. Fix typos in hadoop-aws markdown docs. Contributed by Gabor Bota.	2018-03-20 21:12:20 -07:00
Sammi Chen	d67a5e2dec	HADOOP-15262. AliyunOSS: move files under a directory in parallel when rename a directory. Contributed by Jinhu Wu.	2018-03-19 15:02:37 +08:00
Wangda Tan	f47659fb97	MAPREDUCE-7047. Make HAR tool support IndexedLogAggregtionController. (Xuan Gong via wangda) ... Change-Id: Ice5ae1c706f2476361997dcbb29f2c33c77d4f0c	2018-03-15 13:26:01 -07:00
Steve Loughran	1976e0066e	HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories. ... Contributed by Steve Loughran.	2018-03-15 18:05:14 +00:00
fang zhenyi	5e013d50d1	HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-03-15 16:12:07 +09:00
Chris Douglas	45cccadd2e	HDFS-12780. Fix spelling mistake in DistCpUtils.java. Contributed by Jianfei Jiang	2018-03-13 11:08:11 -07:00
Chris Douglas	45d1b0fdcc	HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer	2018-03-12 20:05:39 -07:00
Steve Loughran	dd05871b8b	HADOOP-15297. Make S3A etag => checksum feature optional. ... Contributed by Steve Loughran.	2018-03-12 14:01:42 +00:00
Steve Loughran	7ef4d942dd	HADOOP-15273.distcp can't handle remote stores with different checksum algorithms. ... Contributed by Steve Loughran.	2018-03-08 11:24:06 +00:00
Steve Loughran	3bd6b1fd85	HADOOP-15292. Distcp's use of pread is slowing it down. ... Contributed by Virajith Jalaparti.	2018-03-08 11:15:46 +00:00
Steve Loughran	e0307e53e2	HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled. ... Contributed by Anis Elleuch. (cherry picked from commit 1dedc68f9d8d8544d715e67ee77cd3f017c21699)	2018-03-07 18:51:10 +00:00
Steve Loughran	2e1e049bd0	HADOOP-15288. TestSwiftFileSystemBlockLocation doesn't compile. ... Contributed by Steve Loughran.	2018-03-05 18:12:38 +00:00
Steve Loughran	8110d6a0d5	HADOOP-13761. S3Guard: implement retries for DDB failures and throttling; translate exceptions. ... Contributed by Aaron Fabbri.	2018-03-05 14:06:20 +00:00
Gergely Novák	e99a1e7767	MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-03-02 16:15:04 -08:00
Ray Chiang	923e1773e1	HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang)	2018-03-01 15:36:10 -08:00
fang zhenyi	29233c3c4a	HADOOP-15274. Move hadoop-openstack to slf4j ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-03-01 15:08:30 -08:00
Steve Loughran	edc9f1451b	HADOOP-15263. hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake. ... Contributed by Steve Loughran	2018-02-28 14:42:25 +00:00
Weiwei Yang	7af4f34de5	YARN-7929. Support to set container execution type in SLS. (Jiandan Yang via Weiwei Yang)	2018-02-28 17:58:09 +08:00
Arpit Agarwal	2fa7963c3d	HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi.	2018-02-24 14:41:55 -08:00
Xuan Gong	68ce193efc	MAPREDUCE-7027: HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails. Contributed by Gergely Novák	2018-02-23 14:37:26 -08:00
Carlo Curino	84cea0011f	YARN-7732. Support Generic AM Simulator from SynthGenerator. (Contributed by Young Chen via curino)	2018-02-20 17:00:34 -08:00
fang zhenyi	4d4dde5112	HADOOP-15223. Replace Collections.EMPTY* with empty* when available ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-02-18 22:19:39 +09:00
Steve Loughran	7ac88244c5	HADOOP-14507. Extend per-bucket secret key config with explicit getPassword() on fs.s3a.$bucket.secret.key. ... Contributed by Steve Loughran.	2018-02-16 16:37:06 +00:00
Steve Loughran	481d79fedc	HADOOP-13972. ADLS to support per-store configuration. ... Contributed by Sharad Sonker. (cherry picked from commit 050f5287b79324b7f6231b879c0bfc608203b980)	2018-02-15 16:27:31 +00:00
Steve Loughran	9a013b255f	HADOOP-15176. Enhance IAM Assumed Role support in S3A client. ... Contributed by Steve Loughran (cherry picked from commit 96c047fbb98c2378eed9693a724d4cbbd03c00fd)	2018-02-15 15:57:10 +00:00
Steve Loughran	b27ab7dd81	HADOOP-15076. Enhance S3A troubleshooting documents and add a performance document. ... Contributed by Steve Loughran. (cherry picked from commit c761e658f6594c4e519ed39ef36669de2c5cee15)	2018-02-15 14:57:56 +00:00
Steve Loughran	c9a373fb14	HADOOP-15090. Add ADL troubleshooting doc. ... Contributed by Steve Loughran. (cherry picked from commit 58a2120e8a31307f19551f87be4e81d4fb626de1)	2018-02-15 14:26:00 +00:00
Steve Loughran	f20dc0d577	HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. ... Contributed by Andras Bokor.	2018-02-14 16:20:14 +00:00
Steve Loughran	8cf88fcd1f	HADOOP-15187. Remove ADL mock test dependency on REST call invoked from Java SDK. ... Contributed by Vishwajeet Dusane. (cherry picked from commit bd5ab5912564d2d687651b01f552b8e4ca8c145a)	2018-02-12 15:13:00 +00:00
Wangda Tan	60f9e60b3b	Preparing for 3.2.0 development ... Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938	2018-02-11 11:17:38 +08:00
Anu Engineer	4304fcd5bd	HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen.	2018-02-06 13:43:45 -08:00
Arpit Agarwal	d4e13a4647	HADOOP-15198. Correct the spelling in CopyFilter.java. Contributed by Mukul Kumar Singh.	2018-02-02 11:37:51 -08:00
Sunil G	9b81cb0537	YARN-7779. Display allocation tags in RM web UI and expose same through REST API. Contributed by Weiwei Yang.	2018-01-31 01:30:17 -08:00
Arun Suresh	38af237969	YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh)	2018-01-31 01:30:17 -08:00
Steve Loughran	7fd287b4af	HADOOP-15186. Allow Azure Data Lake SDK dependency version to be set on the command line. ... Contributed by Vishwajeet Dusane. (cherry picked from commit 95a96b13e2a54e01ea6c6933045d912998477da3)	2018-01-29 09:48:14 -08:00
Jason Lowe	ff8378eb1b	YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang	2018-01-25 17:47:19 -06:00
Yufei Gu	16be42d309	YARN-7798. Refactor SLS Reservation Creation. Contributed by Young Chen.	2018-01-25 13:06:01 -08:00
Steve Loughran	f63d13f10d	HADOOP-15185. Update adls connector to use the current version of ADLS SDK. ... Contributed by Atul Sikaria.	2018-01-23 09:09:11 -08:00
Steve Loughran	1093a73689	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. ... Contributed by Aaron Fabbri	2018-01-18 13:13:58 +00:00
Steve Loughran	f274fe33ea	Revert "HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits." ... This reverts commit 35ad9b1dd2.	2018-01-18 12:35:57 +00:00
Aaron Fabbri	268ab4e027	HADOOP-15141 Support IAM Assumed roles in S3A. Contributed by Steve Loughran.	2018-01-17 00:05:24 -08:00
Sammi Chen	9195a6e302	HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)	2018-01-17 15:55:59 +08:00
Steve Loughran	a0c71dcc33	HADOOP-15079. ITestS3AFileOperationCost#testFakeDirectoryDeletion failing ... after OutputCommitter patch. Contributed by Steve Loughran	2018-01-15 11:33:32 +00:00
Mukul Kumar Singh	2e1e9017aa	HADOOP-15172. Fix the javadoc warning in WriteOperationHelper.java ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-01-15 15:45:51 +09:00
Steve Loughran	1a09da7400	HADOOP-15163. Fix S3ACommitter documentation ... Contributed by Alessandro Andrioni. (cherry picked from commit 100e8a1ae1d930dde084af7d1281e491c7f124ec)	2018-01-10 15:37:07 +00:00
Steve Loughran	b62a5ece95	HADOOP-15161. s3a: Stream and common statistics missing from metrics ... Contributed by Sean Mackrory	2018-01-09 18:46:52 +00:00
Steve Loughran	52babbb4a0	HADOOP-15086. NativeAzureFileSystem file rename is not atomic. ... Contributed by Thomas Marquardt	2017-12-22 11:39:55 +00:00
Steve Loughran	c8ff0cc304	HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls. ... Contributed by Steve Loughran	2017-12-21 14:58:58 +00:00
Steve Loughran	ef450df443	HADOOP-15113. NPE in S3A getFileStatus: null instrumentation on using closed instance. ... Contributed by Steve Loughran.	2017-12-21 14:15:53 +00:00
Akira Ajisaka	5ab632baf5	HDFS-12949. Fix findbugs warning in ImageWriter.java.	2017-12-21 10:04:34 +09:00
Steve Loughran	1ba491ff90	HADOOP-14965. S3a input stream "normal" fadvise mode to be adaptive	2017-12-20 18:25:33 +00:00
Steve Loughran	35ad9b1dd2	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. ... Contributed by Aaron Fabbri	2017-12-18 21:19:06 +00:00
Chris Douglas	4b3a785914	HDFS-12903. [READ] Fix closing streams in ImageWriter. Contributed by Virajith Jalaparti	2017-12-15 17:51:42 -08:00
Chris Douglas	e515103a83	Revert "HDFS-12903. [READ] Fix closing streams in ImageWriter" ... This reverts commit c1bf2654b0e9118985b8518b0254eac4dd302a2f.	2017-12-15 17:51:42 -08:00
Virajith Jalaparti	8239e3afb3	HDFS-12712. [9806] Code style cleanup	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	80c3fec3a1	HDFS-12912. [READ] Fix configuration and implementation of LevelDB-based alias maps	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	c89b29bd42	HDFS-12893. [READ] Support replication of Provided blocks with non-default topologies.	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	0f6aa9564c	HDFS-12905. [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	962b5e722b	HDFS-12903. [READ] Fix closing streams in ImageWriter	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	4531588a94	HDFS-11640. [READ] Datanodes should use a unique identifier when reading from external stores	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	fb996a32a9	HDFS-12894. [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	9c35be86e1	HDFS-12713. [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata. Contributed by Ewan Higgs	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	a027055dd2	HDFS-12885. Add visibility/stability annotations. Contributed by Chris Douglas	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	352f994b64	HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs.	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	4d59dabb7f	HDFS-12809. [READ] Fix the randomized selection of locations in {{ProvidedBlocksBuilder}}.	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	3d3be87e30	HDFS-12778. [READ] Report multiple locations for PROVIDED blocks	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	3b1d30301b	HDFS-12775. [READ] Fix reporting of Provided volumes	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	6cd80b2521	HDFS-12779. [READ] Allow cluster id to be specified to the Image generation tool	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	90d1b47a2a	HDFS-12776. [READ] Increasing replication for PROVIDED files should create local replicas	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	87dc026bee	HDFS-12789. [READ] Image generation tool does not close an opened stream	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	c293cc8e9b	HDFS-12671. [READ] Test NameNode restarts when PROVIDED is configured	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	71d0a82571	HDFS-12607. [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	98f5ed5aa3	HDFS-11902. [READ] Merge BlockFormatProvider and FileRegionProvider.	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	d6a9a89973	HDFS-12605. [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	17052c4aff	HDFS-12584. [READ] Fix errors in image generation tool from latest rebase	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	aca023b72c	HDFS-12289. [READ] HDFS-12091 breaks the tests for provided block reads	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	546b95f484	HDFS-11673. [READ] Handle failures of Datanode with PROVIDED storage	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	4851f06bc2	HDFS-11791. [READ] Test for increasing replication of provided files.	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	aa5ec85f7f	HDFS-11663. [READ] Fix NullPointerException in ProvidedBlocksBuilder	2017-12-15 17:51:37 -08:00
Virajith Jalaparti	d65df0f273	HDFS-11190. [READ] Namenode support for data stored in external stores.	2017-12-15 17:51:37 -08:00
Virajith Jalaparti	8da3a6e314	HDFS-10706. [READ] Add tool generating FSImage from external store	2017-12-15 17:51:37 -08:00
Kai Zheng	f86c81d923	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-14 11:14:52 +08:00
Kai Zheng	28792b6b7f	Revert "HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu" ... This reverts commit 479d6a5792262c977025c26fd4960574b0db6847	2017-12-14 11:05:11 +08:00
Surendra Singh Lilhore	00129c5314	HDFS-12833. Distcp : Update the usage of delete option for dependency with update and overwrite option. Contributed by usharani.	2017-12-12 00:28:02 +05:30
Kai Zheng	ce04340ec7	HADOOP-15104. AliyunOSS: change the default value of max error retry. Contributed by Jinhu Wu	2017-12-08 23:03:54 +08:00
Kai Zheng	73b86979d6	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-06 12:01:36 +08:00
Aaron Fabbri	6555af81a2	HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed by Younger and Sean Mackrory.	2017-12-05 11:06:32 -08:00
Steve Loughran	3150c019ae	HADOOP-15071 S3a troubleshooting docs to add a couple more failure modes. ... Contributed by Steve Loughran	2017-12-05 15:05:41 +00:00
Steve Loughran	0ea182d0fa	HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. ... Contributed by Rajesh Balamohan	2017-11-28 11:52:59 +00:00
Rohith Sharma K S	3cd75845da	MAPREDUCE-7014. Fix java doc errors in jdk1.8. Contributed by Steve Loughran.	2017-11-27 22:01:00 +05:30
Steve Loughran	de8b6ca5ef	HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints. ... Contributed by Steve Loughran and Ryan Blue.	2017-11-22 15:28:12 +00:00
Steve Loughran	c326fc89b0	HADOOP-15024 Support user agent configuration and include that & Hadoop version information to oss server. ... Contributed by Sammi Chen.	2017-11-20 18:56:42 +00:00
Akira Ajisaka	6903cf096e	HADOOP-13514. Upgrade maven surefire plugin to 2.20.1 ... Signed-off-by: Allen Wittenauer <aw@apache.org>	2017-11-19 12:39:37 -08:00
Kai Zheng	18621af7ae	HADOOP-14993. AliyunOSS: Override listFiles and listLocatedStatus. Contributed Genmao Yu	2017-11-14 17:58:37 +08:00
Daniel Templeton	ba8136615a	YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton) ... Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b	2017-11-09 12:09:48 -08:00
John Zhuge	bf6a660232	HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge.	2017-11-09 10:16:12 -08:00
Subru Krishnan	f2df6b8983	HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru).	2017-11-08 18:07:12 -08:00
Arun Suresh	53c0fb7efe	HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh)	2017-11-02 17:15:33 -07:00
Akira Ajisaka	cc3f3eca40	MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi.	2017-11-02 18:32:24 +09:00
Chris Douglas	139cc752d2	HADOOP-14991. Add missing figures to Resource Estimator tool. Contributed by Rui Li	2017-10-27 21:59:27 -07:00
Yufei Gu	99880d0a16	YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak)	2017-10-27 14:41:37 -07:00
Subru Krishnan	625039ef20	HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru).	2017-10-25 15:51:27 -07:00
Wangda Tan	281d83604d	YARN-7318. Fix shell check warnings of SLS. (Gergely Novák via wangda)	2017-10-20 15:54:15 -07:00
Steve Loughran	f36cbc8475	HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. ... Contributed by Andras Bokor.	2017-10-20 22:27:04 +01:00
Haibo Chen	46eb1033a8	YARN-6546. SLS is slow while loading 10k queues. (Yufei Gu via Haibo Chen)	2017-10-17 16:04:19 -07:00
Steve Loughran	9fcc3a1fc8	HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled. ... Contributed by Santhosh G Nayak	2017-10-16 16:01:47 +01:00
Steve Loughran	3de574413c	HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. ... Contributed by Varada Hemeswari.	2017-10-11 18:06:43 +01:00
Steve Loughran	572cdb5463	HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB ... Contributed by Kannapiran Srinivasan.	2017-10-06 17:43:38 +01:00
Haibo Chen	c071aad5da	YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen)	2017-10-05 15:50:13 -07:00
Steve Loughran	9288206cb3	HADOOP-14845. Azure wasb: getFileStatus not making any auth check. ... Contributed by Sivaguru Sankaridurg	2017-10-05 15:05:23 +01:00
Yufei Gu	79e37dc6db	YARN-7211. AMSimulator in SLS does't work due to responseId refactoring.(Contributed by Botong Huang via Yufei Gu)	2017-10-03 15:10:50 -07:00
Akira Ajisaka	c9b525bd94	YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari.	2017-10-04 07:01:54 +09:00
John Zhuge	8aca46e53f	HADOOP-14651. Update okhttp version to 2.7.5. Contributed by Ray Chiang and John Zhuge.	2017-09-29 12:15:06 -07:00
Steve Loughran	a530e7ab3b	HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB ... Contributed by Varada Hemeswari	2017-09-28 19:52:56 +01:00
Aaron Fabbri	47011d7dd3	HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran	2017-09-25 15:59:38 -07:00
Jason Lowe	b133dc5700	MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari	2017-09-22 15:21:42 -05:00
Steve Loughran	2d2d97fa7d	HADOOP-14553. Add (parallelized) integration tests to hadoop-azure ... Contributed by Steve Loughran	2017-09-15 17:03:01 +01:00
Aaron Fabbri	49467165a5	HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran.	2017-09-14 14:10:48 -07:00
ChenSammi	e0b3c644e1	HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen	2017-09-14 09:15:29 +08:00
John Zhuge	a4661850c1	HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria.	2017-09-08 11:51:03 -07:00
Steve Loughran	5bbca80428	HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A. ... Contributed by Aaron Fabbri	2017-09-08 12:07:02 +01:00
Steve Loughran	83449abb40	HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. ... Contributed by Yonger.	2017-09-07 19:55:56 +01:00
Steve Loughran	13eda50003	HADOOP-14520. WASB: Block compaction for Azure Block Blobs. ... Contributed by Georgi Chalakov	2017-09-07 18:35:03 +01:00
Xiaoyu Yao	63720ef574	HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin.	2017-09-05 23:34:55 -07:00
John Zhuge	50506e90a8	HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge.	2017-09-05 23:26:57 -07:00
Steve Loughran	792eff9ea7	HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. ... Contributed by Sivaguru Sankaridurg	2017-09-05 14:16:57 +01:00
Andrew Wang	0d419c984f	Preparing for 3.1.0 development	2017-09-01 11:53:48 -07:00
Allen Wittenauer	a3fee475f7	HADOOP-14414. Calling maven-site-plugin directly for docs profile is unnecessary (Andras Bokor via aw)	2017-09-01 08:38:30 -07:00
Steve Loughran	621b43e254	HADOOP-13345 HS3Guard: Improved Consistency for S3A. ... Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu, Sean Mackrory, Steve Loughran and others.	2017-09-01 14:13:41 +01:00
John Zhuge	7ecc6dbed6	HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria.	2017-08-31 21:13:22 -07:00
Allen Wittenauer	71bbb86d69	HADOOP-14670. Increase minimum cmake version for all platforms ... Signed-off-by: Chris Douglas <cdouglas@apache.org>	2017-08-30 21:47:24 -07:00
Haibo Chen	d04f85f387	MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen)	2017-08-30 10:07:48 -07:00
Yufei Gu	26fafc3597	YARN-7077. TestAMSimulator and TestNMSimulator fail. (Contributed by Akira Ajisaka via Yufei Gu)	2017-08-29 16:33:51 -07:00
Steve Loughran	9374f38820	HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory ... Contributed by Esfandiar Manii and Thomas Marquardt.	2017-08-29 19:03:44 +01:00
Steve Loughran	021974f4cb	HADOOP-14802. Add support for using container saskeys for all accesses. ... Contributed by Sivaguru Sankaridurg	2017-08-29 19:02:43 +01:00
Andrew Wang	f29a0fc288	HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou.	2017-08-25 14:14:23 -07:00
Arpit Agarwal	8b7cbe3840	HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar.	2017-08-25 09:55:46 -07:00
Steve Loughran	27ab5f7385	HADOOP-14787. AliyunOSS: Implement the createNonRecursive operator. ... Contributed by Genmao Yu	2017-08-22 11:55:48 +01:00
Kai Zheng	267e19a09f	HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu	2017-08-21 13:36:28 +08:00
Arun Suresh	8410d862d3	YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh)	2017-08-20 07:54:09 -07:00
Steve Loughran	c6b4e656b7	HADOOP-14769. WASB: delete recursive should not fail if a file is deleted. ... Contributed by Thomas Marquardt	2017-08-18 14:13:40 +01:00
Andrew Wang	dd7916d3cd	HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas.	2017-08-17 09:35:36 -07:00
Chris Douglas	645a8f2a4d	HADOOP-14726. Mark FileStatus::isDir as final	2017-08-14 21:57:20 -07:00
John Zhuge	7769e96149	HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria.	2017-08-13 00:22:34 -07:00
Allen Wittenauer	4222c97108	HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw)	2017-08-11 09:25:56 -07:00
Steve Loughran	54356b1e83	HADOOP-14183. Remove service loader config file for wasb fs. ... Contributed by Esfandiar Manii.	2017-08-10 16:46:33 +01:00
Steve Loughran	f4e1aa0508	HADOOP-14715. TestWasbRemoteCallHelper failing. ... Contributed by Esfandiar Manii.	2017-08-08 23:37:47 +01:00
Chris Douglas	55a181f845	HADOOP-14730. Support protobuf FileStatus in AdlFileSystem.	2017-08-07 21:31:28 -07:00
Steve Loughran	d91b7a8451	HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek. ... Contributed by Thomas Marquardt	2017-08-06 20:19:23 +01:00
Steve Loughran	7fc324aabd	HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws. ... Contributed by Steve Loughran (cherry picked from commit ced547d5f0dbea571cbc472c5f55fe89d5900a6f)	2017-08-04 11:09:08 +01:00
Sean Mackrory	1a1bf6b7d0	HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer.	2017-08-02 12:25:05 -06:00
Steve Loughran	778d4edd9a	HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded. ... Contributed by Thomas Marquardt.	2017-08-01 21:36:28 +01:00
John Zhuge	f735ad1b67	HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge.	2017-07-28 09:46:54 -07:00
Jitendra Pandey	a92bf39e23	HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt.	2017-07-25 16:26:48 -07:00
Jitendra Pandey	f2921e51f0	HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov.	2017-07-24 23:01:01 -07:00
Ravi Prakash	94ca52ae9e	HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash.	2017-07-24 16:01:45 -07:00
Wei-Chiu Chuang	44350fdf49	HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default). Contributed by Bharat Viswanadham.	2017-07-20 18:23:13 -07:00
Haibo Chen	0ba8cda135	YARN-6685. Add job count in to SLS JSON input format. (Yufei Gu via Haibo Chen)	2017-07-20 08:15:46 -07:00
Jitendra Pandey	2843c688bc	HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg.	2017-07-19 00:13:06 -07:00
Jitendra Pandey	b0e78ae085	HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak.	2017-07-17 02:27:55 -07:00
Steve Loughran	7d272ea124	HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb. ... Contributed by Varada Hemeswari (cherry picked from commit 1e69e5260351effc8077d1bdc397cec57cf1ff1b)	2017-07-12 10:37:39 +01:00
Steve Loughran	d670c3a4da	HADOOP-14535 wasb: implement high-performance random access and seek of block blobs. ... Contributed by Thomas Marquardt	2017-07-11 21:34:27 +01:00
Haibo Chen	147df300bf	YARN-5067 Support specifying resources for AM containers in SLS. (Yufei Gu via Haibo Chen)	2017-06-30 17:03:44 -07:00
Mingliang Liu	38996fdcf0	HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak	2017-06-30 16:53:48 -07:00
Andrew Wang	af2773f609	Updating version for 3.0.0-beta1 development	2017-06-29 17:57:40 -07:00
Mingliang Liu	72993b33b7	HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran ... Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc	2017-06-29 17:07:52 -07:00
Mingliang Liu	b08cc97396	HADOOP-14601. Azure: Reuse ObjectMapper. Contributed by Mingliang Liu	2017-06-29 09:22:00 -07:00
Allen Wittenauer	20ba86d66a	MAPREDUCE-6536. hadoop-pipes doesn't use maven properties for openssl ... Signed-off-by: Ravi Prakash <raviprak@apache.org>	2017-06-28 17:33:44 -07:00
Mingliang Liu	990aa34de2	HADOOP-14609. NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code. Contributed by Steve Loughran	2017-06-28 14:18:59 -07:00
Steve Loughran	ee243e5289	HADOOP-14190. Add more on S3 regions to the s3a documentation. ... Contributed by Steve Loughran	2017-06-28 10:22:13 +01:00
Mingliang Liu	7e031c2c18	HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas	2017-06-27 17:32:07 -07:00
Mingliang Liu	686a634f01	HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran	2017-06-27 16:48:47 -07:00
Mingliang Liu	bc4dfe9c9c	HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu	2017-06-27 13:19:14 -07:00
Steve Loughran	48f4a229a9	HADOOP-14461 Azure: handle failure gracefully in case of missing account access key. ... Contributed by Mingliang Liu.	2017-06-26 13:39:47 +01:00
Akira Ajisaka	a594f96fb1	HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he.	2017-06-26 16:41:00 +09:00
Steve Loughran	c6bd73c6c5	HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations. ... Contributed by Thomas.	2017-06-23 21:38:41 +01:00
Jason Lowe	dd65eea74b	HADOOP-8143. Change distcp to have -pb on by default. Contributed by Mithun Radhakrishnan	2017-06-20 09:53:47 -05:00
Akira Ajisaka	3369540653	HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools.	2017-06-20 13:18:26 +09:00
Mingliang Liu	f214a9961f	HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran	2017-06-15 11:16:16 -07:00
Mingliang Liu	325163f23f	HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu	2017-06-09 13:07:11 -07:00
Mingliang Liu	df35e4cc3b	HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan	2017-06-07 13:54:52 -07:00
Carlo Curino	b65100c14b	YARN-6547. Enhance SLS-based tests leveraging invariant checker.	2017-06-06 17:19:15 -07:00
Mingliang Liu	536f057158	HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu	2017-06-06 11:09:28 -07:00
Mingliang Liu	6b5285bbcb	HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu	2017-06-06 11:06:49 -07:00
Mingliang Liu	5fd9742c83	HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan	2017-06-05 15:56:43 -07:00
Mingliang Liu	6aeda55bb8	HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu	2017-06-05 11:26:56 -07:00
Mingliang Liu	ece33208b8	HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu	2017-06-01 11:52:11 -07:00
Akira Ajisaka	6e6e96583f	HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.	2017-06-01 13:08:01 +09:00
Akira Ajisaka	07e60f85d8	HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.	2017-05-30 15:11:10 +09:00
John Zhuge	6c6a7a5962	HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge.	2017-05-28 22:25:00 -07:00
Mingliang Liu	89bb8bfe58	HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari	2017-05-26 17:54:00 -07:00
Andrew Wang	16ad896d5c	Update maven version for 3.0.0-alpha4 development	2017-05-26 14:09:44 -07:00
Akira Ajisaka	6a52b5e144	HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu.	2017-05-25 15:36:44 +09:00
Sunil G	b6f66b0da1	YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.	2017-05-22 14:10:06 +05:30
Yufei Gu	fcbdecc54d	YARN-6111. Rumen input does't work in SLS. Contributed by Yufei Gu.	2017-05-21 21:04:06 -07:00
Mingliang Liu	6672810eea	HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran	2017-05-19 11:51:43 -07:00
Yongjun Zhang	b4adc8392c	HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang)	2017-05-18 15:35:22 -07:00
Sunil G	40e6a85d25	YARN-6560. SLS doesn't honor node total resource specified in sls-runner.xml. Contributed by Wangda Tan.	2017-05-18 22:49:42 +05:30
Steve Loughran	ba70225cf6	HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries. ... Contributed by Steve Loughran. (cherry picked from commit 2ac5aab8d725f761a9f9723471a4426f6b5d78c4)	2017-05-18 15:44:39 +01:00
Akira Ajisaka	d4aa9e3c69	HADOOP-14415. Use java.lang.AssertionError instead of junit.framework.AssertionFailedError. Contributed by Chen Liang.	2017-05-17 15:38:28 -04:00
Robert Kanter	101852ca11	YARN-6535. Program needs to exit when SLS finishes. (yufeigu via rkanter)	2017-05-16 17:52:17 -07:00
Mingliang Liu	b415c6fe74	HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger. Contributed by Sivaguru Sankaridurg	2017-05-16 11:22:32 -07:00
Akira Ajisaka	0d5c8ed8e0	HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor.	2017-05-11 16:37:32 -05:00
Mingliang Liu	eed731496f	HADOOP-14361. Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently. Contributed by Santhosh G Nayak	2017-05-10 15:05:11 -07:00
Carlo Curino	5cb6e3e082	YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations.	2017-05-10 14:49:25 -07:00
Akira Ajisaka	2ba9903932	HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang.	2017-05-10 05:57:12 -05:00
Akira Ajisaka	c60164fb60	HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong.	2017-05-09 05:37:43 -05:00
Andrew Wang	54fd0e44b7	HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy.	2017-05-08 21:59:49 -07:00
Andrew Wang	8065129d87	HADOOP-14298. TestHadoopArchiveLogsRunner fails. Contribute dby Akira Ajisaka.	2017-05-07 13:59:15 -07:00
Steve Loughran	4e6bbd049d	HADOOP-14382 Remove usages of MoreObjects.toStringHelper. ... Contributed by Andrew Wang	2017-05-05 13:03:09 +01:00
Robert Kanter	3082552b3b	YARN-6522. Make SLS JSON input file format simple and scalable (yufeigu via rkanter)	2017-05-04 17:21:46 -07:00
Mingliang Liu	83dded556d	HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran	2017-05-03 11:22:44 -07:00
Andrew Wang	b062b323b7	HADOOP-14281. Fix TestKafkaMetrics#testPutMetrics. Contributed by Alison Yu.	2017-05-02 11:49:19 -07:00
Akira Ajisaka	62579b69a0	HADOOP-11614. Remove httpclient dependency from hadoop-openstack. Contributed by Akira Ajisaka, Brahma Reddy Battula, and Steve Loughran.	2017-04-27 16:57:08 +09:00
Mingliang Liu	8b5f2c372e	HADOOP-14351. Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache. Contributed by Santhosh G Nayak	2017-04-26 13:47:18 -07:00
Mingliang Liu	e4c32bccde	HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu	2017-04-25 20:54:56 -07:00
Robert Kanter	475f933b41	YARN-6423. Queue metrics doesn't work for Fair Scheduler in SLS (yufeigu via rkanter)	2017-04-25 16:26:09 -07:00
Steve Loughran	5f934f8386	HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus. ... Contributed by Steve Moist.	2017-04-24 20:33:19 +01:00
Wangda Tan	de69d6e811	YARN-6363. Extending SLS: Synthetic Load Generator. (Carlo Curino via wangda)	2017-04-20 21:54:30 -07:00
Mingliang Liu	667966c13c	HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran	2017-04-20 17:13:36 -07:00
Andrew Wang	7e075a50e3	YARN-6365. Get static SLS html resources from classpath. Contributed by Yufei Gu.	2017-04-19 11:08:59 -07:00
Steve Loughran	6b015d00c9	HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. ... Contributed by Steve Loughran	2017-04-19 10:21:44 +01:00
Mingliang Liu	0cab57223e	HADOOP-14274. Azure: Simplify Ranger-WASB policy model. Contributed by Sivaguru Sankaridurg	2017-04-12 16:07:10 -07:00
Mingliang Liu	b053fdc547	HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu	2017-04-12 14:30:34 -07:00
Chris Nauroth	b8305e6d06	HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. Contributed by Mingliang Liu.	2017-04-12 10:02:13 -07:00
John Zhuge	56e81f2a20	HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge. ... Signed-off-by: John Zhuge <jzhuge@apache.org>	2017-04-07 12:04:44 -07:00
Andrew Wang	7eea8d83fa	HADOOP-14280. Fix compilation of TestKafkaMetrics.	2017-04-05 15:12:03 -07:00
Mingliang Liu	56ab02eed9	HADOOP-14273. Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag. Contributed by Santhosh G Nayak	2017-04-04 11:10:28 -07:00
Mingliang Liu	27a44b60c1	HADOOP-14272. Azure: WasbRemoteCallHelper should use String equals for comparison. Contributed by Santhosh G Nayak	2017-04-04 11:03:59 -07:00
Chris Douglas	6eba79232f	HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang	2017-04-03 20:13:14 -07:00
Mingliang Liu	5faa949b78	HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka	2017-04-03 11:07:14 -07:00
Mingliang Liu	26172a94d6	HADOOP-14267. Make DistCpOptions immutable. Contributed by Mingliang Liu	2017-03-31 20:04:26 -07:00
Yongjun Zhang	bf3fb585aa	HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li.	2017-03-30 17:38:56 -07:00
Yongjun Zhang	144f1cf765	Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen." ... This reverts commit 064c8b25ec.	2017-03-30 17:38:18 -07:00
Yongjun Zhang	064c8b25ec	HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen.	2017-03-30 17:01:15 -07:00
Wei-Chiu Chuang	8c591b8d19	HDFS-10974. Document replication factor for EC files. Contributed by Yiqun Lin.	2017-03-30 11:16:05 -07:00
Akira Ajisaka	0d053eeb30	HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula.	2017-03-30 18:11:50 +09:00
Robert Kanter	6a5516c238	YARN-5654. Not be able to run SLS with FairScheduler (yufeigu via rkanter)	2017-03-29 16:18:13 -07:00
Andrew Wang	4966a6e26e	HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy.	2017-03-29 14:37:21 -07:00
Andrew Wang	0e6f8e4bc6	HDFS-10971. Distcp should not copy replication factor if source file is erasure coded. Contributed by Manoj Govindassamy.	2017-03-28 22:14:03 -07:00
John Zhuge	253e3e78ab	HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge.	2017-03-27 23:05:57 -07:00
John Zhuge	d1b7439b48	HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge.	2017-03-24 13:37:04 -07:00
Andrew Wang	52b00600df	HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy.	2017-03-24 11:44:46 -07:00
Chris Douglas	c55195588f	HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge	2017-03-23 12:17:27 -07:00
Chris Douglas	03955612b7	HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria	2017-03-23 12:13:23 -07:00
Mingliang Liu	2e30aa72e0	HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu	2017-03-23 11:33:29 -07:00
Steve Loughran	a5a4867f3b	HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject(). ... Contributed by Yuanbo Liu (cherry picked from commit 20878d052cebc715c2494a97362fdff08885a77f)	2017-03-23 12:54:54 +00:00
Robert Kanter	cc938e99ec	YARN-6326. Shouldn't use AppAttemptIds to fetch applications while AM Simulator tracks app in SLS (yufeigu via rkanter)	2017-03-21 15:21:11 -07:00
Steve Loughran	2841666f1f	HADOOP-14204 S3A multipart commit failing, ... "UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort". Contributed by Steve Loughran (cherry picked from commit 595501c9ddacd5fdb740ac1bf9bfeb0ffbbd6c0e)	2017-03-21 13:15:15 +00:00
Mingliang Liu	8e15e24059	HADOOP-13945. Azure: Add Kerberos and Delegation token support to WASB client. Contributed by Santhosh G Nayak ... Change-Id: I39fb0a3b0491bf2160571366939a0502a0045429	2017-03-20 16:31:24 -07:00
Arpit Agarwal	6c399a88e9	HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran.	2017-03-20 11:29:23 -07:00
Mingliang Liu	86035c1644	HADOOP-14192. Aliyun OSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu	2017-03-17 11:01:00 -07:00
Mingliang Liu	fa67a96d7b	HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu	2017-03-14 14:38:21 -07:00
Mingliang Liu	e6cda5819b	Revert "HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu" ... This reverts commit b8c69557b7.	2017-03-14 13:03:42 -07:00
Mingliang Liu	b8c69557b7	HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu	2017-03-13 14:15:02 -07:00
John Zhuge	5a40bafdae	HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge. ... Change-Id: I5dc6f885816b8834f718874542dfa373458b0333	2017-03-13 08:11:25 -07:00
Steve Loughran	092ec39fb9	HADOOP-14111 cut some obsolete, ignored s3 tests in TestS3Credentials. ... Contributed by Yuanbo Liu	2017-03-10 17:43:22 +00:00
Mingliang Liu	881ec4d97b	HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu	2017-03-10 00:16:09 -08:00
John Zhuge	c5ee7fded4	HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge. ... Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e	2017-03-09 18:30:17 -08:00
John Zhuge	98142d2f72	Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" ... This reverts commit 53a12fa721.	2017-03-08 09:54:22 -08:00
Mingliang Liu	28daaf0eb2	HADOOP-14150. Implement getHomeDirectory() method in NativeAzureFileSystem. Contributed by Santhosh G Nayak	2017-03-07 14:55:52 -08:00
Robert Kanter	1598fd3b79	YARN-6275. Fail to show real-time tracking charts in SLS (yufeigu via rkanter)	2017-03-07 13:47:52 -08:00
Akira Ajisaka	f597f4c43e	HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri.	2017-03-07 15:14:55 +09:00
Mingliang Liu	686823529b	HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth	2017-03-06 17:16:36 -08:00
Mingliang Liu	52d7d5aa1a	Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth" ... This reverts commit 6b7cd62b8c.	2017-03-06 17:10:11 -08:00
Mingliang Liu	c571cda5c7	HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA	2017-03-06 17:00:13 -08:00
Mingliang Liu	6b7cd62b8c	HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth	2017-03-03 17:09:12 -08:00
Akira Ajisaka	490abfb10f	HADOOP-14057. Fix package.html to compile with Java 9.	2017-03-04 00:25:22 +09:00
Steve Loughran	a97833e0ed	HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. ... Contributed by Steve Loughran (cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)	2017-03-02 21:44:30 +00:00
Allen Wittenauer	989bd56b9f	HADOOP-12556. KafkaSink jar files are created but not copied to target dist (Babak Behzad via aw)	2017-02-28 16:15:40 -08:00
Wangda Tan	eac6b4c35c	YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda)	2017-02-28 10:35:50 -08:00
Akira Ajisaka	258342e76c	HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR.	2017-02-28 13:47:44 +09:00
Yongjun Zhang	d235dcdf0b	HADOOP-14127. Add log4j configuration to enable logging in hadoop-distcp's tests. (Xiao Chen via Yongjun Zhang)	2017-02-27 20:42:13 -08:00
Steve Loughran	dab00da19f	HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures. ... Contributed by Steve Loughran. (cherry picked from commit 29fe5af017b945d8750c074ca39031b5b777eddd)	2017-02-25 15:35:19 +00:00
Sunil G	b32ffa2753	YARN-4779. Fix AM container allocation logic in SLS. Contributed by Wangda Tan.	2017-02-24 21:39:25 +05:30
Steve Loughran	e60c6543d5	HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran	2017-02-24 13:24:59 +00:00
Steve Loughran	9c22a91662	HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory. ... (cherry picked from commit ff87ca84418a710c6dc884fe8c70947fcc6489d5)	2017-02-24 10:41:36 +00:00
Mingliang Liu	82607fce39	HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari.	2017-02-23 13:48:44 -08:00
Mingliang Liu	a4d4a23785	HADOOP-13321. Deprecate FileSystem APIs that promote inefficient call patterns. Contributed by Chris Nauroth and Mingliang Liu	2017-02-23 12:55:40 -08:00
Mingliang Liu	0013090fb4	HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu	2017-02-22 13:34:20 -08:00
Steve Loughran	4f4250fbcc	HADOOP-14099 Split S3 testing documentation out into its own file. Contributed by Steve Loughran.	2017-02-22 11:43:48 +00:00
Mingliang Liu	924def7854	HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane	2017-02-21 13:44:42 -08:00
Steve Loughran	2158496f6b	HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory. ... (cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)	2017-02-21 18:28:49 +00:00
Steve Loughran	8035749c26	HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan	2017-02-20 16:21:46 +00:00
Steve Loughran	3a2e30fa9f	HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge ... (cherry picked from commit b1c1f05b1dc997906390d653dfafb4f0d7e193c4)	2017-02-18 18:17:11 +00:00
Mingliang Liu	658702efff	HADOOP-14040. Use shaded aws-sdk uber-JAR 1.11.86. Contributed by Steve Loughran and Sean Mackrory	2017-02-16 16:51:03 -08:00
Mingliang Liu	bdad8b7b97	HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran	2017-02-16 16:41:31 -08:00
Mingliang Liu	f432999025	HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane	2017-02-16 15:14:25 -08:00
Kai Zheng	cd3e59a3dc	HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu	2017-02-15 16:34:30 +08:00
Akira Ajisaka	b9f8491252	HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin.	2017-02-15 01:45:56 +09:00
Lei Xu	71c23c9fc9	HADOOP-13929. ADLS connector should not check in contract-test-options.xml. (John Zhuge via lei)	2017-02-13 13:33:13 -08:00
Kai Zheng	01be4503c3	HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui	2017-02-13 13:18:20 +08:00
Lei Xu	839b690ed5	HADOOP-13075. Add support for SSE-KMS and SSE-C in s3a filesystem. (Steve Moist via lei)	2017-02-11 13:59:03 -08:00
Arpit Agarwal	2b7a7bbe0f	HADOOP-14055. SwiftRestClient includes pass length in exception if auth fails. Contributed by Marcell Hegedus.	2017-02-10 10:09:31 -08:00
Kai Zheng	5b151290ae	HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation deleteDirs. Contributed by Genmao Yu	2017-02-10 14:53:18 +08:00
Kai Zheng	a8a594b4c8	HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui	2017-02-09 18:00:22 +08:00
Akira Ajisaka	98c35bb1a4	HADOOP-13858. TestGridmixMemoryEmulation and TestResourceUsageEmulators fail on the environment other than Linux or Windows.	2017-01-21 17:31:20 +09:00
Lei Xu	ccf2d66244	HADOOP-13962. Update ADLS SDK to 2.1.4. (John Zhuge via lei)	2017-01-21 10:52:40 +08:00
Lei Xu	e015b56319	HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei)	2017-01-20 14:48:04 +08:00
Andrew Wang	5d8b80ea9b	Preparing for 3.0.0-alpha3 development	2017-01-19 15:50:07 -08:00
Steve Loughran	ed33ce11dd	HADOOP-13496. Include file lengths in Mismatch in length error for distcp. Contributed by Ted Yu ... (cherry picked from commit 77401bd5fcca5127c9908156971eeec468371f47)	2017-01-19 11:25:40 +00:00
Steve Loughran	e648b6e138	HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran	2017-01-11 17:25:15 +00:00
Wangda Tan	287d3d6804	YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)	2017-01-09 08:40:39 -08:00
Chris Nauroth	4c8f9e1302	HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore.	2017-01-05 15:04:47 -08:00
Wangda Tan	0a55bd841e	YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)	2017-01-05 10:31:05 -08:00
Wangda Tan	5ed63e3e9d	YARN-4899. Queue metrics of SLS capacity scheduler only activated after app submit to the queue. (Jonathan Hung via wangda)	2017-01-04 19:17:11 -08:00
Akira Ajisaka	01d31fe938	HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang.	2017-01-04 14:10:36 +09:00
Mingliang Liu	c6a5b689db	HADOOP-13942. Build failure due to errors of javadoc build in hadoop-azure. Contributed by Kai Sasaki	2016-12-29 16:24:06 -08:00
Mingliang Liu	e92a77099b	HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by Dushyanth	2016-12-22 20:15:18 -08:00
Akira Ajisaka	ae401539ea	YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor.	2016-12-22 14:32:24 +09:00
Wangda Tan	3bcfe3a9de	YARN-5706. Fail to launch SLSRunner due to NPE. (Kai Sasaki via wangda)	2016-12-20 11:03:29 -08:00
Mingliang Liu	f92913c35b	HADOOP-13831. Correct check for error code to detect Azure Storage Throttling and provide retries. Contributed by Gaurav Kanade	2016-12-15 12:35:08 -08:00
Mingliang Liu	ef34bf2bb9	HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane	2016-12-13 10:34:32 -08:00
Mingliang Liu	c6a3923245	HADOOP-13871. ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful. Contributed by Steve Loughran	2016-12-12 14:55:34 -08:00
Mingliang Liu	15dd1f3381	HADOOP-13675. Bug in return value for delete() calls in WASB. Contributed by Dushyanth	2016-12-05 12:04:07 -08:00
Mingliang Liu	4113ec5fa5	HADOOP-13257. Improve Azure Data Lake contract tests. Contributed by Vishwajeet Dusane	2016-12-02 15:54:57 -08:00
Mingliang Liu	51211a7d7a	HADOOP-13855. Fix a couple of the s3a statistic names to be consistent with the rest. Contributed by Steve Loughran	2016-12-02 14:01:42 -08:00
Mingliang Liu	2ff84a0040	HADOOP-13857. S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs. Contributed by Steve Loughran	2016-12-02 13:36:04 -08:00
Mingliang Liu	3fd844b99f	HADOOP-13830. Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string". Contributed by Steve Loughran	2016-11-30 13:01:19 -08:00
Mingliang Liu	d60a60be8a	HADOOP-13823. s3a rename: fail if dest file exists. Contributed by Steve Loughran	2016-11-28 16:30:29 -08:00
Akira Ajisaka	209e805430	HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.	2016-11-28 14:34:57 +09:00
Akira Ajisaka	9e173d2bac	HADOOP-13816. Ambiguous plugin version warning from maven build. Contributed by Kai Sasaki.	2016-11-28 13:10:31 +09:00
Steve Loughran	0de0c32ddd	HADOOP-13801 regression: ITestS3AMiniYarnCluster failing. Contributed by Steve Loughran	2016-11-23 21:37:39 +00:00
Mingliang Liu	af791b774b	HADOOP-13050. Upgrade to AWS SDK 10.11+. Contributed by Chris Nauroth and Steve Loughran	2016-11-22 14:00:35 -08:00
Mingliang Liu	beb70fed4f	HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran ... This closes #131	2016-11-22 13:12:23 -08:00
Wei-Chiu Chuang	c0b1a44f6c	HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory.	2016-11-17 22:48:35 -06:00
Mingliang Liu	5af572b644	HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu	2016-11-15 10:57:00 -08:00
Chris Douglas	5c61ad2488	HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane	2016-11-11 11:15:07 -08:00
Mingliang Liu	89354f0475	HADOOP-13687. Provide a unified dependency artifact that transitively includes the cloud storage modules shipped with Hadoop. Contributed by Chris Nauroth	2016-11-10 08:58:37 -08:00
Jason Lowe	3f93ac0733	YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri	2016-11-08 22:01:26 +00:00
Xiao Chen	acd509dc57	HADOOP-13795. Skip testGlobStatusThrowsExceptionForUnreadableDir in TestFSMainOperationsSwift. Contributed by John Zhuge.	2016-11-07 09:21:01 -08:00
Akira Ajisaka	b970446b2c	HADOOP-13798. TestHadoopArchives times out.	2016-11-07 19:53:43 +09:00
Akira Ajisaka	de0132748d	MAPREDUCE-6796. Remove unused properties from JTConfig.java. Contributed by Haibo Chen.	2016-11-05 03:49:19 +09:00
Arun Suresh	0aafc122d4	YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh)	2016-11-04 07:31:54 -07:00
Xiao Chen	20c4d8efab	HADOOP-13787. Azure testGlobStatusThrowsExceptionForUnreadableDir fails. Contributed by John Zhuge.	2016-11-03 11:16:07 -07:00
Steve Loughran	a1761a841e	HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi.	2016-10-31 20:54:46 +00:00
Robert Kanter	5877f20f9c	HADOOP-10075. Update jetty dependency to version 9 (rkanter)	2016-10-27 16:09:00 -07:00
Masatake Iwasaki	0bdd263d82	HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran.	2016-10-27 15:46:59 +09:00
Chris Nauroth	1f8490a5ba	HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth.	2016-10-26 08:55:42 -07:00
Chris Nauroth	9cad3e2350	HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran.	2016-10-26 08:27:26 -07:00
Akira Ajisaka	d3bb69a667	YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton.	2016-10-26 15:32:07 +09:00
Yongjun Zhang	0f0c15f7a5	HDFS-11040. Add documentation for HDFS-9820 distcp improvement. Contributed by Yongjun Zhang.	2016-10-25 12:25:40 -07:00
Yongjun Zhang	3a60573039	Revert "Fix HDFS-11040" ... This reverts commit 54c1815790.	2016-10-25 12:25:02 -07:00
Yongjun Zhang	54c1815790	Fix HDFS-11040	2016-10-25 12:19:34 -07:00
Chris Nauroth	309a43925c	HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth.	2016-10-25 09:03:03 -07:00
Chris Nauroth	3372e94030	HADOOP-12774. s3a should use UGI.getCurrentUser.getShortname() for username. Contributed by Steve Loughran.	2016-10-24 21:54:06 -07:00
Chris Nauroth	d8fa1cfa67	HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth.	2016-10-24 21:22:34 -07:00
Chris Douglas	a1a0281e12	HADOOP-13626. Remove distcp dependency on FileStatus serialization	2016-10-24 12:46:54 -07:00
Anu Engineer	3fbf4cd5da	HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran.	2016-10-20 12:33:58 -07:00
Steve Loughran	9ae270af02	HADOOP-13735 ITestS3AFileContextStatistics.testStatistics() failing. Contributed by Pieter Reuse	2016-10-20 14:50:59 +01:00
Yongjun Zhang	8650cc84f2	HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.	2016-10-19 17:37:54 -07:00
Xiao Chen	efdf810cf9	HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge.	2016-10-18 18:18:43 -07:00
Steve Loughran	6c348c5691	HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran	2016-10-18 21:16:02 +01:00
Yongjun Zhang	0bc6d37f3c	Revert "HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang." ... This reverts commit 412c4c9a34.	2016-10-17 22:47:37 -07:00
Andrew Wang	987ee51141	HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei.	2016-10-17 13:25:58 -07:00
Yongjun Zhang	412c4c9a34	HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.	2016-10-17 11:04:42 -07:00
Mingliang Liu	d9f73f1b7c	HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu	2016-10-13 22:33:55 -07:00
Jing Zhao	0a85d07983	HADOOP-13024. Distcp with -delete feature on raw data not implemented. Contributed by Mavin Martin.	2016-10-13 13:24:54 -07:00
Kai Zheng	c31b5e61b1	HADOOP-13701. AbstractContractRootDirectoryTest can fail when handling delete "/". Contributed by Genmao Yu	2016-10-08 17:35:59 +06:00
Kai Zheng	a57bba470b	Merge branch 'trunk' into HADOOP-12756	2016-10-09 10:29:40 +08:00
Chris Nauroth	69620f9559	HADOOP-13692. hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts. Contributed by Chris Nauroth.	2016-10-07 11:41:19 -07:00
Steve Loughran	ebd4f39a39	HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran.	2016-10-07 12:51:40 +01:00
Steve Loughran	2cc841f16e	HADOOP-13678 Update jackson from 1.9.13 to 2.x in hadoop-tools. Contributed by Akira Ajisaka.	2016-10-06 16:31:00 +01:00
Steve Loughran	d6be1e75d8	HADOOP-12667 s3a to support createNonRecursive API. Contributed by Sean Mackrory	2016-10-05 15:02:11 +01:00
Akira Ajisaka	6cfe649375	HADOOP-13681. Reduce Kafka dependencies in hadoop-kafka module. Contributed by Grant Henke. ... This closes #133	2016-10-05 20:44:39 +09:00
Chris Nauroth	88b9444a81	HADOOP-13674. S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint. Contributed by Chris Nauroth.	2016-10-04 10:36:58 -07:00
Steve Loughran	ee0c722dc8	HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan.	2016-09-29 17:01:00 +01:00
Chris Nauroth	47f80922dc	HADOOP-13599. s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown. Contributed by Steve Loughran.	2016-09-28 16:19:32 -07:00
Kai Zheng	d1443988f8	Revert "HADOOP-13584. hdoop-aliyun: merge HADOOP-12756 branch back" ... This reverts commit 5707f88d85	2016-09-29 09:18:27 +08:00
Brahma Reddy Battula	e17a4970be	HDFS-9885. Correct the distcp counters name while displaying counters. Contributed by Surendra Singh Lilhore	2016-09-27 10:45:12 +05:30
Kai Zheng	5707f88d85	HADOOP-13584. hdoop-aliyun: merge HADOOP-12756 branch back. ... HADOOP-12756 branch: Incorporate Aliyun OSS file system implementation. Contributors: Mingfei Shi (mingfei.shi@intel.com) Genmao Yu (genmao.ygm@alibaba-inc.com)	2016-09-26 20:42:22 +08:00
Steve Loughran	e5ef51e717	HADOOP-13643. Math error in AbstractContractDistCpTest. Contributed by Aaron Fabbri.	2016-09-23 10:01:30 +01:00
Kai Zheng	26d5df390c	HADOOP-13634. Some configuration in doc has been outdated. Contributed by Genmao Yu	2016-09-23 08:44:28 +08:00
Kai Zheng	a49b3be38e	Merge branch 'trunk' into HADOOP-12756	2016-09-21 14:03:21 +08:00
Kai Zheng	22af6f8db3	HADOOP-13624. Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu	2016-09-21 14:02:44 +08:00
Kai Zheng	08b37603d9	HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu	2016-09-20 15:12:02 +08:00
Kai Zheng	9cd4760257	HADOOP-13609. Refine credential provider related codes for AliyunOss integration. Contributed by Genmao Yu	2016-09-20 11:39:02 +08:00
Kai Zheng	005f4528c7	Merge branch 'trunk' into HADOOP-12756	2016-09-20 08:09:29 +08:00
Chris Nauroth	98bdb51397	HADOOP-13169. Randomize file list in SimpleCopyListing. Contributed by Rajesh Balamohan.	2016-09-19 15:16:47 -07:00
Kai Zheng	a1940464a4	HADOOP-13610. Clean up AliyunOss integration tests. Contributed by Genmao Yu	2016-09-18 19:10:48 +08:00
Kai Zheng	dbb28eb1f2	Merge branch 'trunk' into HADOOP-12756	2016-09-18 19:03:39 +08:00
Mingliang Liu	96142efa2d	HADOOP-13621. s3:// should have been fully cut off from trunk. Contributed by Mingliang Liu.	2016-09-17 22:07:46 -07:00
Steve Loughran	53a12fa721	HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran	2016-09-14 15:45:04 +01:00
Kai Zheng	60f66a9306	Merge branch 'trunk' into HADOOP-12756	2016-09-14 16:12:40 +08:00
Allen Wittenauer	58ed4fa544	HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS ... This commit includes the following changes: HADOOP-13356. Add a function to handle command_subcommand_OPTS HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts HADOOP-13357. Modify common to use hadoop_subcommand_opts HADOOP-13359. Modify YARN to use hadoop_subcommand_opts HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity) HADOOP-13564. modify mapred to use hadoop_subcommand_opts HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug HADOOP-13360. Documentation for HADOOP_subcommand_OPTS This closes apache/hadoop#126	2016-09-12 11:10:00 -07:00
Ravi Prakash	9faccd1046	HADOOP-13587. distcp.map.bandwidth.mb is overwritten even when -bandwidth flag isn't set. Contributed by Zoran Dimitrijevic	2016-09-12 08:26:08 -07:00
Steve Loughran	4b6d795f28	HADOOP-13540 improve section on troubleshooting s3a auth problems. Contributed by Steve Loughran	2016-09-09 18:55:32 +01:00
Steve Loughran	7fdfcd8a6c	HADOOP-13541 explicitly declare the Joda time version S3A depends on. Contributed by Stevel Loughran	2016-09-07 12:25:23 +01:00
Mingfei	e671a0f52b	HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.	2016-09-07 11:17:43 +08:00
Mingfei	d33e928fbe	HADOOP-13529. Do some code refactoring. Contributed by Genmao Yu.	2016-09-07 11:15:48 +08:00
Mingfei	cdb77110e7	HADOOP-13498. The number of multi-part upload part should not bigger than 10000. Contributed by Genmao Yu.	2016-09-07 11:15:48 +08:00
Mingfei	6bb741b9f8	HADOOP-13499. Support session credentials for authenticating with Aliyun. Contributed by Genmao Yu.	2016-09-07 11:15:48 +08:00
Mingfei	bd2d97adee	HADOOP-13483. File create should throw error rather than overwrite directories. Contributed by Genmao Yu.	2016-09-07 11:15:47 +08:00
Mingfei	4d84c814fc	HADOOP-13491. Fix several warnings from findbugs. Contributed by Genmao Yu.	2016-09-07 11:15:47 +08:00
Kai Zheng	a5d5342228	HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou	2016-09-07 11:15:47 +08:00
Chris Nauroth	d152557cf7	HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth.	2016-09-06 09:36:21 -07:00
Arun Suresh	d6d9cff21b	YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh)	2016-08-30 15:52:29 -07:00
Xiaoyu Yao	8b57be108f	HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth. ... (cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41) (cherry picked from commit a65e159c59de41d26e12212dd1b7c28c28265cc3)	2016-08-29 15:30:49 -07:00
Chris Douglas	c37346d0e3	HADOOP-12726. Unsupported FS operations should throw UnsupportedOperationException. Contributed by Daniel Templeton.	2016-08-23 14:12:52 -07:00
Chris Nauroth	6f9c346e57	HADOOP-13446. Support running isolated unit tests separate from AWS integration tests. Contributed by Chris Nauroth.	2016-08-23 07:18:49 -07:00
Chris Nauroth	763f0497bb	HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran.	2016-08-19 10:48:10 -07:00
Chris Nauroth	ae4db25443	HADOOP-13513. Java 1.7 support for org.apache.hadoop.fs.azure testcases. Contributed by Tibor Kiss.	2016-08-18 10:44:36 -07:00
Junping Du	0da69c324d	YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. ... (cherry picked from commit d464483bf7f0b3e3be3ba32cd6c3eee546747ab5) Conflicts: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java	2016-08-18 07:27:23 -07:00
Steve Loughran	040c185d62	HADOOP-13405 doc for fs.s3a.acl.default indicates incorrect values. Contributed by Shen Yinjie	2016-08-18 14:36:55 +01:00
Chris Nauroth	822d661b8f	HADOOP-13208. S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories. Contributed by Steve Loughran.	2016-08-17 14:54:54 -07:00
Chris Nauroth	3808876c73	HADOOP-13324. s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints). Contributed by Steve Loughran.	2016-08-16 17:05:52 -07:00
Chris Nauroth	2ed58c40e5	HADOOP-13403. AzureNativeFileSystem rename/delete performance improvements. Contributed by Subramanyam Pattipaka.	2016-08-08 12:28:40 -07:00
Chris Nauroth	4ef1bc0bdb	HADOOP-13459. hadoop-azure runs several test cases repeatedly, causing unnecessarily long running time. Contributed by Chris Nauroth.	2016-08-02 22:32:21 -07:00
Chris Nauroth	b43de80031	HADOOP-13354. Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients. Contributed by Sivaguru Sankaridurg.	2016-07-27 15:50:38 -07:00
Akira Ajisaka	54fe17a607	HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.	2016-07-27 19:41:09 +09:00
Steve Loughran	86ae218893	HADOOP-13188 S3A file-create should throw error rather than overwrite directories. Contributed by Steve Loughran	2016-07-25 15:50:23 +01:00
Steve Loughran	be9e46b42d	HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and mkdirs(). Contributed by Rajesh Balamohan	2016-07-25 14:45:03 +01:00
Steve Loughran	7052ca8804	HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials. Contributed by Steven K. Wong	2016-07-23 18:00:08 +01:00
Chris Nauroth	c60a68ce1d	HADOOP-13392. [Azure Data Lake] OAuth2 configuration should be default set to true when AdlFileSystem is configured. Contributed by Vishwajeet Dusane and Chris Douglas.	2016-07-22 09:16:38 -07:00
Chris Douglas	dca6335dbc	HADOOP-13393. Omit unsupported fs.defaultFS setting in ADLS documentation	2016-07-21 23:21:47 -07:00
mattf-horton	12aa184479	HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley.	2016-07-21 14:43:57 -07:00
Steve Loughran	37362c2f92	HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan)	2016-07-20 13:42:51 +01:00
Andrew Wang	da456ffd62	Preparing for 3.0.0-alpha2 development	2016-07-15 19:04:17 -07:00
Jitendra Pandey	c2f9cd584c	HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu.	2016-07-15 14:28:53 -07:00
Jian He	819224dcf9	YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan	2016-07-11 22:36:20 -07:00
Andrew Wang	f292624bd8	HDFS-10300. TestDistCpSystem should share MiniDFSCluster. Contributed by John Zhuge.	2016-07-11 18:06:28 -07:00
Jitendra Pandey	6e597600f7	HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu.	2016-07-07 14:58:19 -07:00
Jitendra Pandey	aa42c7a6dd	HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu.	2016-07-01 15:34:03 -07:00
Steve Loughran	96fa0f848b	HADOOP-12709 Cut s3:// from trunk. Contributed by Mingliang Liu.	2016-06-29 16:04:50 +01:00
Akira Ajisaka	8d202f1258	HADOOP-13235. Use Date and Time API in KafkaSink.	2016-06-29 07:40:12 -07:00
Yongjun Zhang	8113855b3a	HDFS-10396. Using -diff option with DistCp may get "Comparison method violates its general contract" exception. Contributed by Yongjun Zhang.	2016-06-28 23:15:13 -07:00
Allen Wittenauer	422c73a865	HADOOP-13034. Log message about input options in distcp lacks some items (Takashi Ohnishi via aw)	2016-06-28 07:21:04 -07:00
Steve Loughran	4ee3543625	HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel.	2016-06-22 15:45:25 +01:00
Chris Nauroth	b2c596cdda	HADOOP-13287. TestS3ACredentials#testInstantiateFromURL fails if AWS secret key contains +. Contributed by Chris Nauroth.	2016-06-21 11:28:52 -07:00
Jitendra Pandey	b7c4cf7129	HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu.	2016-06-20 16:25:30 -07:00
Chris Nauroth	51d16e7b38	HADOOP-13242. Authenticate to Azure Data Lake using client ID and keys. Contributed by Atul Sikaria.	2016-06-16 23:35:20 -07:00
Chris Nauroth	127d2c7281	HADOOP-13241. document s3a better. Contributed by Steve Loughran.	2016-06-16 11:18:02 -07:00
Ravi Prakash	4aefe119a0	HADOOP-3733. "s3x:" URLs break when Secret Key contains a slash, even if encoded. Contributed by Steve Loughran.	2016-06-16 11:13:35 -07:00
Chris Nauroth	c9e71382a5	HADOOP-12875. [Azure Data Lake] Support for contract test and unit test cases. Contributed by Vishwajeet Dusane.	2016-06-16 09:46:05 -07:00
Wangda Tan	c77a1095dc	YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)	2016-06-14 15:06:38 -07:00
Chris Nauroth	9581fb715c	HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane.	2016-06-09 14:33:31 -07:00
Andrew Wang	58c3486850	HADOOP-13175. Remove hadoop-ant from hadoop-tools. Contributed by Chris Douglas.	2016-06-09 13:49:52 -07:00
Steve Loughran	31ffaf76f2	HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory.	2016-06-09 21:00:47 +01:00
Steve Loughran	656c460c0e	HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth	2016-06-09 17:28:49 +01:00
Steve Loughran	a3f78d8fa8	HADOOP-12807 S3AFileSystem should read AWS credentials from environment variables. Contributed by Tobin Baker.	2016-06-06 23:42:36 +02:00
Chris Nauroth	c58a59f708	HADOOP-13171. Add StorageStatistics to S3A; instrument some more operations. Contributed by Steve Loughran.	2016-06-03 08:55:33 -07:00
Chris Nauroth	16b1cc7af9	HADOOP-13131. Add tests to verify that S3A supports SSE-S3 encryption. Contributed by Steve Loughran.	2016-06-01 14:49:22 -07:00
Steve Loughran	5870611030	HADOOP-13162. Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs. (Rajesh Balamohan via stevel)	2016-06-01 14:18:20 +01:00
Varun Vasudev	42f90ab885	YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.	2016-05-29 21:24:16 +05:30
Yongjun Zhang	cfb860dee7	HADOOP-13199. Add doc for distcp -filters. (John Zhuge via Yongjun Zhang)	2016-05-26 23:30:31 -07:00
Steve Loughran	39ec1515a2	HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran)	2016-05-21 16:39:31 +01:00
Steve Loughran	757050ff35	HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong.	2016-05-20 13:52:15 +01:00
Steve Loughran	c918286b17	HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth.	2016-05-20 12:21:59 +01:00
Chris Nauroth	93258459fa	HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. Contributed by Steve Loughran.	2016-05-19 22:00:21 -07:00
Jing Zhao	03788d3015	HDFS-10397. Distcp should ignore -delete option if -diff option is provided instead of exiting. Contributed by Mingliang Liu.	2016-05-17 15:46:30 -07:00
Steve Loughran	08ea07f1b8	HADOOP-13158 S3AFileSystem#toString might throw NullPointerException due to null cannedACL. (Chris Nauroth via stevel)	2016-05-17 13:19:29 +01:00
Steve Loughran	c69a649257	HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel)	2016-05-17 13:00:18 +01:00
Allen Wittenauer	730bc746f9	HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw) ... This commit contains the following JIRA issues: HADOOP-12931. bin/hadoop work for dynamic subcommands HADOOP-12932. bin/yarn work for dynamic subcommands HADOOP-12933. bin/hdfs work for dynamic subcommands HADOOP-12934. bin/mapred work for dynamic subcommands HADOOP-12935. API documentation for dynamic subcommands HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands HADOOP-13086. enable daemonization of dynamic commands HADOOP-13087. env var doc update for dynamic commands HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement HADOOP-13089. hadoop distcp adds client opts twice when dynamic HADOOP-13094. hadoop-common unit tests for dynamic commands HADOOP-13095. hadoop-hdfs unit tests for dynamic commands HADOOP-13107. clean up how rumen is executed HADOOP-13108. dynamic subcommands need a way to manipulate arguments HADOOP-13110. add a streaming subcommand to mapred HADOOP-13111. convert hadoop gridmix to be dynamic HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow HADOOP-13117. clarify daemonization and security vars for dynamic commands HADOOP-13120. add a --debug message when dynamic commands have been used HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop HADOOP-13129. fix typo in dynamic subcommand docs HADOOP-13151. Underscores should be escaped in dynamic subcommands document HADOOP-13153. fix typo in debug statement for dynamic subcommands	2016-05-16 17:54:45 -07:00
Chris Nauroth	b9685e85d5	HADOOP-13148. TestDistCpViewFs to include IOExceptions in test error reports. Contributed by Steve Loughran.	2016-05-16 11:53:17 -07:00
Steve Loughran	dd3a8bed0a	HADOOP-13113 Enable parallel test execution for hadoop-aws. Chris Nauroth via stevel	2016-05-13 10:47:12 +01:00
Andrew Wang	fa440a39b7	HADOOP-12868. Fix hadoop-openstack undeclared and unused dependencies. Contributed by Masatake Iwasaki.	2016-05-12 23:58:18 -07:00
Andrew Wang	3c5c57af28	HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.	2016-05-12 18:27:28 -07:00
Andrew Wang	ca5613af91	Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." ... This reverts commit 6b53802cba.	2016-05-12 15:32:45 -07:00
Steve Loughran	27c4e90efc	HADOOP-13028 add low level counter metrics for S3A; use in read performance tests. contributed by: stevel ... patch includes HADOOP-12844 Recover when S3A fails on IOException in read() HADOOP-13058 S3A FS fails during init against a read-only FS if multipart purge HADOOP-13047 S3a Forward seek in stream length to be configurable	2016-05-12 19:24:20 +01:00
Andrew Wang	6b53802cba	Update project version to 3.0.0-alpha1-SNAPSHOT.	2016-05-12 11:05:05 -07:00
Steve Loughran	3f2816ab5a	HADOOP-13116 Jets3tNativeS3FileSystemContractTest does not run.	2016-05-12 16:35:57 +01:00
Steve Loughran	def2a6d385	HADOOP-13122 Customize User-Agent header sent in HTTP requests by S3A. Chris Nauroth via stevel.	2016-05-12 13:57:35 +01:00
Chris Nauroth	27242f211e	HADOOP-12801. Suppress obsolete S3FileSystem tests. Contributed by Chris Nauroth.	2016-05-10 15:21:06 -07:00
Steve Loughran	025219b12f	HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel)	2016-05-10 21:37:22 +01:00
Jian He	996a210ab0	YARN-4778. Support specifying resources for task containers in SLS. Contributed by Wangda Tan	2016-05-09 14:49:41 -07:00
Allen Wittenauer	1c5bbf6499	HADOOP-12866. add a subcommand for gridmix (Kai Sasaki via aw)	2016-05-06 14:07:02 -07:00
Jason Lowe	d0da13229c	YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla	2016-05-05 14:07:54 +00:00
Jing Zhao	af942585a1	HADOOP-12469. distcp should not ignore the ignoreFailures option. Contributed by Mingliang Liu.	2016-05-04 10:23:04 -07:00
Masatake Iwasaki	ef0870ad03	HADOOP-12378. Fix findbugs warnings in hadoop-tools module. Contributed by Akira AJISAKA.	2016-04-29 20:25:52 +09:00
Arun Suresh	341888a0aa	YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh)	2016-04-26 20:12:12 -07:00
Yongjun Zhang	959a28dd12	HDFS-10313. Distcp need to enforce the order of snapshot names passed to -diff. (Lin Yiqun via Yongjun Zhang)	2016-04-26 16:08:03 -07:00
Akira Ajisaka	02c51c27d9	HDFS-10298. Document the usage of distcp -diff option. Contributed by Takashi Ohnishi.	2016-04-25 22:33:09 +09:00
Jing Zhao	63e5412f1a	HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.	2016-04-22 15:14:40 -07:00
Steve Loughran	19f0f9608e	HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel)	2016-04-22 11:25:03 +01:00
Yongjun Zhang	a749ba0cea	HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via Yongjun Zhang)	2016-04-21 12:17:17 -07:00
Jing Zhao	404f57f328	HDFS-10216. Distcp -diff throws exception when handling relative path. Contributed by Takashi Ohnishi.	2016-04-14 10:35:22 -07:00
Steve Loughran	df18b6e984	HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel)	2016-04-14 12:44:55 +01:00
Jason Lowe	2a5da97f81	Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" ... This reverts commit 1cbcd4a491.	2016-04-11 15:51:01 +00:00
Steve Loughran	b9e3eff62a	HADOOP-12444 Support lazy seek in S3AInputStream. Rajesh Balamohan via stevel	2016-04-09 11:25:31 +01:00
Chris Nauroth	843ee8d59d	HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran.	2016-04-08 13:36:58 -07:00
Steve Loughran	ce5b4812f0	HADOOP-12753 S3A JUnit tests failing if using HTTP proxy. (Zoran Rajic via stevel)	2016-04-08 17:10:00 +01:00
Akira Ajisaka	9c32f8785e	HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.	2016-04-08 15:32:13 +09:00
Jason Lowe	1cbcd4a491	YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla	2016-04-05 13:40:19 +00:00
Steve Loughran	89c9347511	HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse. - omitted new S3A subclass	2016-04-04 17:02:04 +01:00
Steve Loughran	0ecdd4cffa	HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse.	2016-04-03 16:40:19 +01:00
Harsh J	256c82fe29	HADOOP-11687. Ignore x-* and response headers when copying an Amazon S3 object. Contributed by Aaron Peterson and harsh.	2016-04-01 14:18:10 +05:30
Allen Wittenauer	0a74610d1c	HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)	2016-03-31 07:51:05 -07:00
Akira Ajisaka	0050fa5f1c	MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.	2016-03-29 18:17:52 +09:00
Akira Ajisaka	115be193df	Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote." ... This reverts commit 2c268cc936.	2016-03-28 16:05:35 +09:00
Akira Ajisaka	18c7e58283	HDFS-9640. Remove hsftp from DistCp in trunk. Contributed by Wei-Chiu Chuang.	2016-03-28 15:32:38 +09:00
Akira Ajisaka	2c268cc936	MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.	2016-03-25 15:17:27 +09:00
Allen Wittenauer	738155063e	HADOOP-12857. rework hadoop-tools (aw)	2016-03-23 13:46:38 -07:00
Chris Nauroth	c50aad0f85	HADOOP-12717. NPE when trying to rename a directory in Windows Azure Storage FileSystem. Contributed by Robert Yokota and Gaurav Kanade.	2016-03-04 22:57:43 -08:00
Masatake Iwasaki	cbd31328a6	HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)	2016-03-04 14:11:36 +09:00
Masatake Iwasaki	33a412e8a4	HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)	2016-03-03 18:57:23 +09:00
Chris Nauroth	f98dff329b	HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty.	2016-03-01 13:18:42 -08:00
Chris Nauroth	d251e55415	HADOOP-12851. S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders. Contributed by Larry McCay.	2016-02-29 21:59:52 -08:00
Masatake Iwasaki	809ebc0b14	HADOOP-12552. Fix undeclared/unused dependency to httpclient (iwasakims)	2016-02-29 18:20:14 +09:00
Chris Nauroth	7634d404b7	HADOOP-12846. Credential Provider Recursive Dependencies. Contributed by Larry McCay.	2016-02-28 11:22:55 -08:00
cnauroth	3369a4f691	HADOOP-12535. Run FileSystem contract tests with hadoop-azure. Contributed by Madhumita Chakraborty.	2016-02-23 21:37:50 -08:00
cnauroth	27b77751c1	HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay.	2016-02-22 17:30:13 -08:00
cnauroth	d4f5fc23b2	HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki.	2016-02-22 13:39:38 -08:00
Yongjun Zhang	ba1c9d484a	HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang)	2016-02-19 10:17:37 -08:00
cnauroth	8ab7658025	HADOOP-11031. Design Document for Credential Provider API. Contributed by Larry McCay.	2016-02-18 14:06:38 -08:00
cnauroth	76fab26c5c	HADOOP-12548. Read s3a creds from a Credential Provider. Contributed by Larry McCay.	2016-02-17 12:19:59 -08:00
cnauroth	91a96eaa53	HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty.	2016-02-12 15:50:10 -08:00
Steve Loughran	29ae258013	HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel)	2016-02-06 15:05:16 +00:00
Akira Ajisaka	700a176fa7	HADOOP-12757. Findbug compilation fails for 'Kafka Library support'. (aajisaka)	2016-02-02 23:15:39 +09:00
Yongjun Zhang	eddd823cd6	HDFS-9638. Improve DistCp Help and documentation. (Wei-Chiu Chuang via Yongjun Zhang)	2016-01-29 12:11:55 -08:00
cnauroth	41da9a0773	MAPREDUCE-6563. Streaming documentation contains a stray % character. Contributed by Chris Nauroth.	2016-01-28 14:46:01 -08:00
Xuan	4efdf3a979	YARN-4612. Fix rumen and scheduler load simulator handle killed tasks ... properly. Contributed by Ming Ma.	2016-01-26 18:17:12 -08:00
Akira Ajisaka	736eb17a79	HADOOP-12731. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.	2016-01-25 13:47:29 +09:00
Akira Ajisaka	0bae506c22	HADOOP-12730. Hadoop streaming -mapper and -reducer options are wrongly documented as required. Contributed by Kengo Seki.	2016-01-22 21:08:31 +09:00
Xuan	1708a4cd23	YARN-4611. Fix scheduler load simulator to support multi-layer network ... location. Contributed by Ming Ma.	2016-01-20 15:41:50 -08:00
Ravi Prakash	1acc509b45	HADOOP-12696. Add tests for S3FileSystem Contract. Contributed by Matt Paduano	2016-01-19 13:57:08 -08:00
Wangda Tan	89d1fd5dac	HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda)	2016-01-19 21:27:38 +08:00
Wangda Tan	a44ce3f14f	YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)	2016-01-19 09:30:04 +08:00
Wangda Tan	150f5ae034	Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" ... This reverts commit 3fe5728563. Conflicts: hadoop-yarn-project/CHANGES.txt	2016-01-19 09:27:36 +08:00
cnauroth	8bc93db2e7	HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth.	2016-01-18 09:08:58 -08:00
Wangda Tan	3fe5728563	YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) ... (cherry picked from commit 805a9ed85e)	2016-01-18 17:06:05 +08:00
Wangda Tan	adf260a728	Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" ... This reverts commit 805a9ed85e.	2016-01-18 16:50:45 +08:00
Wangda Tan	805a9ed85e	YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)	2016-01-18 11:04:25 +08:00
Yongjun Zhang	a9c69ebeb7	HDFS-9612. DistCp worker threads are not terminated after jobs are done. (Wei-Chiu Chuang via Yongjun Zhang)	2016-01-15 10:03:09 -08:00
Colin Patrick Mccabe	b1ed28fa77	HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe)	2016-01-14 11:02:34 -08:00
Colin Patrick Mccabe	8315582c4f	HDFS-9517. Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes (Wei-Chiu Chuang via cmccabe)	2016-01-13 16:28:06 -08:00
Lei Xu	126705f67e	HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei)	2016-01-12 12:19:53 -08:00
Xiaoyu Yao	c2e2e13455	HDFS-8584. NPE in distcp when ssl configuration file does not exist in class path. Contributed by Surendra Singh Lilhore.	2016-01-11 17:08:26 -08:00
Zhe Zhang	95f32015ad	HDFS-9630. DistCp minor refactoring and clean up. Contributed by Kai Zheng. ... Change-Id: I363c4ffcac32116ddcdc0a22fac3db92f14a0db0	2016-01-11 09:46:56 -08:00