hadoop

Author	SHA1	Message	Date
Steve Loughran	e02eb24e0a	HADOOP-15183. S3Guard store becomes inconsistent after partial failure of rename. Contributed by Steve Loughran. Change-Id: I825b0bc36be960475d2d259b1cdab45ae1bb78eb	2019-06-20 09:56:40 +01:00
Sahil Takiar	28291a9e8a	HADOOP-16379: S3AInputStream.unbuffer should merge input stream stats into fs-wide stats Contributed by Sahil Takiar Change-Id: I2bcfaaea00d12c633757069402dcd0b91a5f5c05	2019-06-20 09:42:27 +01:00
Robert Levas	450c070a8f	HADOOP-16340. ABFS driver continues to retry on IOException responses from REST operations. Contributed by Robert Levas. This makes the HttpException constructor protected rather than public, so it is possible to implement custom subclasses of this exception -exceptions which will not be retried. Change-Id: Ie8aaa23a707233c2db35948784908b6778ff3a8f	2019-06-19 17:43:14 +01:00
Da Zhou	1da09bd9d5	HADOOP-16376. ABFS: Override access() to no-op. Contributed by Da Zhou. Change-Id: Ia0024bba32250189a87eb6247808b2473c331ed0	2019-06-16 19:20:46 +01:00
Gabor Bota	f9cc9e1621	HADOOP-16279. S3Guard: Implement time-based (TTL) expiry for entries (and tombstones). Contributed by Gabor Bota. Change-Id: I73a2d2861901dedfe7a0e783b310fbb95e7c1af9	2019-06-16 17:05:01 +01:00
Wanqiang Ji	b417a4c854	MAPREDUCE-7214. Remove unused pieces related to `mapreduce.job.userlog.retain.hours` Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-06-11 18:40:35 +09:00
Steve Loughran	4e38dafde4	HADOOP-15563. S3Guard to support creating on-demand DDB tables. Contributed by Steve Loughran Change-Id: I2262b5b9f52e42ded8ed6f50fd39756f96e77087	2019-06-07 18:26:10 +01:00
Akira Ajisaka	3ea4f41d9f	MAPREDUCE-6794. Remove unused properties from TTConfig.java	2019-06-07 10:27:41 +09:00
Steve Loughran	ec26c431f9	HADOOP-16117. Update AWS SDK to 1.11.563. Contributed by Steve Loughran. Change-Id: I7c46ed2a6378e1370f567acf4cdcfeb93e43fa13	2019-06-06 10:08:18 +01:00
Steve Loughran	309501c6fa	Revert "HADOOP-16050: s3a SSL connections should use OpenSSL" This reverts commit `b067f8acaa`. Change-Id: I584b050a56c0e6f70b11fa3f7db00d5ac46e7dd8	2019-06-05 13:54:55 +01:00
Steve Loughran	7724d8031b	Revert "HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors" This reverts commit `5906268f0d`.	2019-06-05 13:54:42 +01:00
Akira Ajisaka	afd844059c	HADOOP-16331. Fix ASF License check in pom.xml Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2019-05-29 17:25:13 +09:00
Steve Loughran	0c73dba3a6	HADOOP-16332. Remove S3A dependency on http core. Contributed by Steve Loughran. Change-Id: I53209c993a405fefdb5e1b692d5a56d027d3b845	2019-05-28 22:50:37 +01:00
Akira Ajisaka	9f933e6446	HADOOP-16323. https everywhere in Maven settings.	2019-05-27 15:24:59 +09:00
Sahil Takiar	5906268f0d	HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors	2019-05-21 11:30:45 -06:00
Ben Roling	a36274d699	HADOOP-16085. S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite. Contributed by Ben Roling. S3Guard will now track the etag of uploaded files and, if an S3 bucket is versioned, the object version. You can then control how to react to a mismatch between the data in the DynamoDB table and that in the store: warn, fail, or, when using versions, return the original value. This adds two new columns to the table: etag and version. This is transparent to older S3A clients -but when such clients add/update data to the S3Guard table, they will not add these values. As a result, the etag/version checks will not work with files uploaded by older clients. For a consistent experience, upgrade all clients to use the latest hadoop version.	2019-05-19 22:29:54 +01:00
Sahil Takiar	b067f8acaa	HADOOP-16050: s3a SSL connections should use OpenSSL (cherry picked from commit aebf229c175dfa19fff3b31e9e67596f6c6124fa)	2019-05-16 08:57:54 -06:00
Andrew Olson	c15b3bca86	HADOOP-16294: Enable access to input options by DistCp subclasses. Adding a protected-scope getter for the DistCpOptions, so that a subclass does not need to save its own copy of the inputOptions supplied to its constructor, if it wishes to override the createInputFileListing method with logic similar to the original implementation, i.e. calling CopyListing#buildListing with a path and input options. Author: Andrew Olson	2019-05-16 16:11:12 +02:00
Weiwei Yang	2d8282bb82	HADOOP-16306. AliyunOSS: Remove temporary files when upload small files to OSS. Contributed by wujinhu.	2019-05-14 14:05:39 -07:00
DadanielZ	ff27e8eabd	HADOOP-16251. ABFS: add FSMainOperationsBaseTest. Re-commit to fix git metadata. Author: Da Zhou	2019-05-10 13:57:30 -07:00
Aaron Fabbri	5f1a34649f	Revert "HADOOP-16251. ABFS: add FSMainOperationsBaseTest" This reverts commit `7c2d7c05a9`. Git Commit name and email were incorrect. Will re-commit.	2019-05-10 13:55:56 -07:00
DadanielZ	7c2d7c05a9	HADOOP-16251. ABFS: add FSMainOperationsBaseTest Author: Da Zhou	2019-05-10 12:46:03 -07:00
Rajat Khandelwal	591ca69823	HADOOP-16278. With S3A Filesystem, Long Running services End up Doing lot of GC and eventually die. Contributed by Rajat Khandelwal	2019-05-09 21:13:29 +01:00
Da Zhou	3418bbbb59	HADOOP-16269. ABFS: add listFileStatus with StartFrom. Author: Da Zhou	2019-05-08 17:20:46 +01:00
Giovanni Matteo Fumarola	7a3188d054	HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena.	2019-05-02 12:58:42 -07:00
Ben Roling	0af4011580	HADOOP-16221. S3Guard: add option to fail operation on metadata write failure.	2019-04-30 11:53:26 +01:00
Da Zhou	1cef194a28	HADOOP-16242. ABFS: add bufferpool to AbfsOutputStream. Contributed by Da Zhou.	2019-04-29 13:27:28 +01:00
Ben Roling	e1c5ddf2aa	HADOOP-16252. Add prefix to dynamo tables in tests. Contributed by Ben Roling.	2019-04-24 14:55:58 +01:00
Takanobu Asanuma	a5ceed26f3	HADOOP-16253. Update AssertJ to 3.12.2.	2019-04-16 12:53:52 +09:00
Akira Ajisaka	4b4fef2f0e	HADOOP-16227. Upgrade checkstyle to 8.19	2019-04-15 10:46:10 +09:00
Sahil Takiar	2382f63fc0	HADOOP-14747. S3AInputStream to implement CanUnbuffer. Author: Sahil Takiar <stakiar@cloudera.com>	2019-04-12 18:12:02 -07:00
Masatake Iwasaki	bbdbc7a9a1	HADOOP-14544. DistCp documentation for command line options is misaligned. Contributed by Masatake Iwasaki.	2019-04-12 11:52:18 +09:00
Steve Loughran	cf4efcab3b	HADOOP-16118. S3Guard to support on-demand DDB tables. This is the first step for on-demand operations: things recognize when they are using on-demand tables, as do the tests. Contributed by Steve Loughran.	2019-04-11 17:12:12 -07:00
Giovanni Matteo Fumarola	358e928622	YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Addendum. Contributed by Inigo Goiri.	2019-04-09 14:18:05 -07:00
Steve Loughran	215ffc792e	HADOOP-16197 S3AUtils.translateException to map CredentialInitializationException to AccessDeniedException Contributed by Steve Loughran. Change-Id: Ie98ca5210bf0009f297edbcacf1fc6dfe5ea70cd.	2019-04-04 21:14:18 +01:00
Steve Loughran	366186d999	HADOOP-16233. S3AFileStatus to declare that isEncrypted() is always true (#685 ) This is needed to fix up some confusion about caching of job.addCache() handling of S3A paths; all parent dirs -the files are downloaded by the NM without using the DTs of the user submitting the job. This means that when you submit jobs to an EC2 cluster with lower IAM permissions than the user, cached resources don't get downloaded and the job doesn't start. Production code changes: * S3AFileStatus Adds "true" to the superclass's encrypted flag during construction. Tests * Base AbstractContractOpenTest can control whether zero byte files created in tests are encrypted. Not done via an XML attribute, just a subclass point. Thoughts? * Verify that the filecache considers paths to not have the permissions which trigger reduce-privilege downloads * And extend ITestDelegatedMRJob to test a completely different bucket (open street map), to verify that cached resources do get their tokens picked up Docs: * Advise FS developers to say all files are encrypted. It's otherwise harmless and it'll stop other people seeing impossible to debug error messages on app launch. Contributed by Steve Loughran. Change-Id: Ifaae4c9d735ccc5eafeebd2584b65daf2d4e5da3	2019-04-03 21:23:40 +01:00
Akira Ajisaka	8b6deebb1d	HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji.	2019-04-03 19:35:02 +09:00
Steve Loughran	df578c07ec	HADOOP-16195 MarshalledCredentials toString Change-Id: I4f1bdd2be0d5760c5501dce6edb6122499108b53	2019-03-28 17:01:57 +00:00
Gabor Bota	b5db238383	HADOOP-15999. S3Guard: Better support for out-of-band operations. Author: Gabor Bota	2019-03-28 15:59:25 +00:00
Gabor Bota	cfb0186903	HADOOP-16186. S3Guard: NPE in DynamoDBMetadataStore.lambda$listChildren. Author: Gabor Bota	2019-03-28 15:49:56 +00:00
Siyao Meng	ce4bafdf44	HADOOP-16037. DistCp: Document usage of Sync (-diff option) in detail. Contributed by Siyao Meng	2019-03-26 18:42:54 +00:00
Lokesh Jain	ae2eb2dd42	HADOOP-16201: S3AFileSystem#innerMkdirs builds needless lists (#636 )	2019-03-22 11:42:00 +00:00
Andrew Olson	faba3591d3	HADOOP-16147. Allow CopyListing sequence file keys and values to be more easily customized. Author: Andrew Olson	2019-03-22 10:35:30 +00:00
Steve Loughran	9f1c017f44	HADOOP-16058. S3A tests to include Terasort. Contributed by Steve Loughran. This includes - HADOOP-15890. Some S3A committer tests don't match ITest* pattern; don't run in maven - MAPREDUCE-7090. BigMapOutput example doesn't work with paths off cluster fs - MAPREDUCE-7091. Terasort on S3A to switch to new committers - MAPREDUCE-7092. MR examples to work better against cloud stores	2019-03-21 11:15:37 +00:00
Ajay Kumar	399563fec6	HDFS-14176. Replace incorrect use of system property user.name. Contributed by Dinesh Chitlangia.	2019-03-20 11:20:45 -07:00
Weiwei Yang	568d3ab8b6	HADOOP-16191. AliyunOSS: improvements for copyFile/copyDirectory and logging. Contributed by wujinhu.	2019-03-19 09:55:32 +08:00
Adam Antal	c0427c84dd	HADOOP-16124. Extend documentation in testing.md about S3 endpoint constants. Contributed by Adam Antal.	2019-03-18 19:13:13 +00:00
Vishwajeet Dusane	1edf1914ac	HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file. Contributed by Vishwajeet Dusane.	2019-03-18 13:18:08 +00:00
Ben Roling	6fa229891e	HADOOP-15625. S3A input stream to use etags/version number to detect changed source files. Author: Ben Roling <ben.roling@gmail.com> Initial patch from Brahma Reddy Battula.	2019-03-13 20:37:11 +00:00
Denes Gerencser	ddede7ae6f	HADOOP-16174. Disable wildfly logs to the console. Follow-on to HADOOP-15851. Author: Denes Gerencser <dgerencser@cloudera.com>	2019-03-09 18:45:17 +00:00
Shweta Yakkali	bb8ad096e7	HADOOP-16157. [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore (Contributed by Shweta Yakkali via Daniel Templeton) Change-Id: I6269ae66378e46eed440a76f847ae1af1fa95450	2019-03-09 09:55:56 -08:00
Steve Loughran	0cbe9ad8c2	HADOOP-16109. Parquet reading S3AFileSystem causes EOF Nobody gets seek right. No matter how many times they think they have. Reproducible test from: Dave Christianson Fixed seek() logic: Steve Loughran	2019-03-09 16:00:34 +00:00
Da Zhou	e0260417ad	HADOOP-16169. ABFS: Bug fix for getPathProperties. Author: Da Zhou <da.zhou@microsoft.com>	2019-03-08 13:53:08 +00:00
Da Zhou	05df151d09	HADOOP-16163. NPE in setup/teardown of ITestAbfsDelegationTokens. Contributed by Da Zhou. Signed-off-by: Steve Loughran <stevel@apache.org>	2019-03-05 14:02:34 +00:00
Steve Loughran	65f60e56b0	HADOOP-16068. ABFS Authentication and Delegation Token plugins to optionally be bound to specific URI of the store. Contributed by Steve Loughran.	2019-02-28 14:22:32 +00:00
Abhishek Modi	52b2eab575	HADOOP-16093. Move DurationInfo from hadoop-aws to hadoop-common org.apache.hadoop.util. Contributed by Abhishek Modi	2019-02-26 17:10:41 +00:00
Da Zhou	3988e75ca3	HADOOP-16136. ABFS: Should only transform username to short name Contributed by Da Zhou.	2019-02-26 15:37:24 +00:00
Steve Loughran	a868f59d52	HADOOP-16105. WASB in secure mode does not set connectingUsingSAS. Contributed by Steve Loughran.	2019-02-21 14:08:46 +00:00
Masatake Iwasaki	aa3ad36605	HADOOP-16104. Wasb tests to downgrade to skip when test a/c is namespace enabled. Contributed by Masatake Iwasaki.	2019-02-20 22:00:57 +09:00
bibinchundatt	779dae4de7	YARN-9309. Improve graph text in SLS to avoid overlapping. Contributed by Bilwa S T.	2019-02-20 00:36:57 +05:30
Adam Antal	1e0ae6ed15	HADOOP-15843. s3guard bucket-info command to not print a stack trace on bucket-not-found. Contributed by Adam Antal. (Revised patch applied after stevel committed the wrong one; that has been reverted)	2019-02-19 11:33:02 +00:00
Steve Loughran	920a89627d	Revert "HADOOP-15843. s3guard bucket-info command to not print a stack trace on bucket-not-found." This reverts commit `c4a00d1ad3`.	2019-02-18 14:57:22 +00:00
Akira Ajisaka	0f2b65c3da	HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.	2019-02-18 15:21:46 +09:00
bibinchundatt	134ae8fc80	YARN-9293. Optimize MockAMLauncher event handling. Contributed by Bibin A Chundatt.	2019-02-14 22:56:52 +05:30
Masatake Iwasaki	6c999fe4b0	HADOOP-16098. Fix javadoc warnings in hadoop-aws. Contributed by Masatake Iwasaki.	2019-02-12 06:07:47 +09:00
Da Zhou	1f1655028e	HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. Contributed by Da Zhou and Junhua Gu.	2019-02-07 21:58:21 +00:00
Steve Loughran	668817a6ce	Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth." (accidentally mixed in two patches) This reverts commit `fa8cd1bf28`.	2019-02-07 21:57:22 +00:00
Ranith Sardar	546c5d70ef	HADOOP-16032. Distcp It should clear sub directory ACL before applying new ACL on.	2019-02-07 21:48:07 +00:00
Andrew Olson	de804e53b9	HADOOP-15281. Distcp to add no-rename copy option. Contributed by Andrew Olson.	2019-02-07 10:07:22 +00:00
Da Zhou	fa8cd1bf28	HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. Contributed by Da Zhou and Junhua Gu.	2019-02-05 19:23:15 +00:00
Da Zhou	ba9efe06fa	HADOOP-16074. WASB: Update container not found error code. Contributed by Da Zhou.	2019-02-05 14:39:45 +00:00
Steve Loughran	f365957c63	HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile(); S3A to implement S3 Select through this API. The new openFile() API is asynchronous, and implemented across FileSystem and FileContext. The MapReduce V2 inputs are moved to this API, and you can actually set must/may options to pass in. This is more useful for setting things like s3a seek policy than for S3 select, as the existing input format/record readers can't handle S3 select output where the stream is shorter than the file length, and splitting plain text is suboptimal. Future work is needed there. In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific configuration parameters which can be set in jobs and used to set filesystem input stream options (seek policy, retry, encryption secrets, etc). Contributed by Steve Loughran	2019-02-05 11:51:02 +00:00
Akira Ajisaka	1129288cf5	HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.	2019-01-29 18:29:56 -08:00
Shweta Yakkali	02eb91856e	HADOOP-16041. Include Hadoop version in User-Agent string for ABFS. Contributed by Shweta Yakkali. Signed-off-by: Sean Mackrory <mackrorysd@apache.org>	2019-01-29 16:50:14 -08:00
Akira Ajisaka	3c60303ac5	HADOOP-16065. -Ddynamodb should be -Ddynamo in AWS SDK testing document.	2019-01-25 10:27:59 +09:00
Da Zhou	00ad9e23e8	HADOOP-16048. ABFS: Fix Date format parser. Contributed by Da Zhou.	2019-01-22 16:40:42 +00:00
Da Zhou	30863c5ae3	HADOOP-16044. ABFS: Better exception handling of DNS errors followup Contributed by Da Zhou.	2019-01-14 19:32:56 +00:00
Steve Loughran	6d0bffe17e	HADOOP-14556. S3A to support Delegation Tokens. Contributed by Steve Loughran and Daryn Sharp.	2019-01-14 17:59:27 +00:00
Adam Antal	c4a00d1ad3	HADOOP-15843. s3guard bucket-info command to not print a stack trace on bucket-not-found. Contributed by Adam Antal.	2019-01-14 17:27:00 +00:00
Gabor Bota	04fcbef9c9	HADOOP-16043. NPE in ITestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is not set. Contributed by Gabor Bota.	2019-01-14 13:12:05 +00:00
Giovanni Matteo Fumarola	fb8932a727	HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena.	2019-01-11 10:54:49 -08:00
Da Zhou	dddad985d7	HADOOP-15975. ABFS: remove timeout check for DELETE and RENAME. Contributed by Da Zhou. (cherry picked from commit 8b5fbe7a125f9d08cbb9f5e5ae28dc984e0d73d8)	2019-01-11 11:13:41 +00:00
Da Zhou	852701f793	HADOOP-16036. WASB: Disable jetty logging configuration announcement. Contributed by Da Zhou.	2019-01-10 12:07:09 +00:00
Da Zhou	7211269142	HADOOP-15662. Better exception handling of DNS errors. Contributed by Da Zhou.	2019-01-10 12:02:58 +00:00
Da Zhou	e8d1900369	HADOOP-16040. ABFS: Bug fix for tolerateOobAppends configuration. Contributed by Da Zhou.	2019-01-10 11:58:39 +00:00
Sean Mackrory	3420e26ae5	HADOOP-16027. [DOC] Effective use of FS instances during S3A integration tests. Contributed by Gabor Bota.	2019-01-09 10:57:58 -07:00
Kai Xie	188bebbe7e	HADOOP-16018. DistCp won't reassemble chunks when blocks per chunk > 0. Contributed by Kai Xie.	2019-01-08 11:57:57 +00:00
Akira Ajisaka	7f78397036	Revert "HADOOP-14556. S3A to support Delegation Tokens." This reverts commit `d7152332b3`.	2019-01-08 14:51:30 +09:00
Steve Loughran	d7152332b3	HADOOP-14556. S3A to support Delegation Tokens. Contributed by Steve Loughran.	2019-01-07 13:18:03 +00:00
Weiwei Yang	f87b3b11c4	HADOOP-16030. AliyunOSS: bring fixes back from HADOOP-15671. Contributed by wujinhu.	2019-01-07 15:56:49 +08:00
Sunil G	1f425271a7	Revert "HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu." This reverts commit `e4fca6aae4`. Revert "HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu." This reverts commit `2b635125fb`.	2019-01-05 17:35:29 +09:00
Weiwei Yang	040a202b20	HADOOP-15323. AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore. Contributed wujinhu.	2019-01-03 21:37:49 +08:00
Sean Mackrory	13f0ee21f2	HADOOP-15860. ABFS: Throw exception when directory / file name ends with a period (.). Contributed by Shweta Yakkali.	2019-01-02 09:33:09 -07:00
Sean Mackrory	d8f670ff28	HADOOP-15819. FileSystem cache misused in S3A integration tests. Contributed by Adam Antal.	2018-12-27 08:19:25 -07:00
Eric Yang	f82922dcfa	YARN-5168. Added exposed port information for Docker container. Contributed by Xun Liu	2018-12-21 19:44:07 -05:00
Steve Loughran	a668f8e6c6	HADOOP-16015. Add bouncycastle jars to hadoop-aws as test dependencies. Contributed by Steve Loughran.	2018-12-20 18:09:01 +00:00
Vishwajeet Dusane	784ea3aff6	HADOOP-15965. Upgrade to ADLS SDK which has major performance improvement for ingress/egress. Contributed by Vishwajeet Dusane.	2018-12-20 18:00:05 +00:00
Da Zhou	346c0c8aff	HADOOP-16004. ABFS: Convert 404 error response in AbfsInputStream and AbfsOutPutStream to FileNotFoundException. Contributed by Da Zhou.	2018-12-17 11:15:20 +00:00
Da Zhou	62df60737c	HADOOP-15972 ABFS: reduce list page size to to 500. Contributed by Da Zhou. (cherry picked from commit d09dbcc8fb6889e51d4d4c914eb6159e40845e9f)	2018-12-17 11:10:22 +00:00
Da Zhou	b2523d8100	HADOOP-15969. ABFS: getNamespaceEnabled can fail blocking user access thru ACLs. Contributed by Da Zhou.	2018-12-17 11:04:40 +00:00
Sean Mackrory	82b798581d	HADOOP-15988. DynamoDBMetadataStore#innerGet should support empty directory flag when using authoritative listings. Contributed by Gabor Bota.	2018-12-12 09:30:13 -07:00
Sean Mackrory	1a25bbe9ec	HADOOP-15845. Require explicit URI on CLI for s3guard init and destroy. Contributed by Gabor Bota.	2018-12-11 08:33:13 -07:00
Sean Mackrory	c35de95a22	HADOOP-15987. ITestDynamoDBMetadataStore should check if table configured properly. Contributed by Gabor Bota.	2018-12-11 08:29:39 -07:00
Sean Mackrory	3ff8580f22	HADOOP-15428. s3guard bucket-info will create s3guard table if FS is set to do this automatically. (Contributed by Gabor Bota)	2018-12-10 14:03:08 -07:00
Inigo Goiri	8fc0d04517	HDFS-14001. [PROVIDED Storage] bootstrapStandby should manage the InMemoryAliasMap. Contributed by Virajith Jalaparti.	2018-12-07 18:30:36 -08:00
Da Zhou	a8bbd818d5	HADOOP-15968. ABFS: add try catch for UGI failure when initializing ABFS. Contributed by Da Zhou.	2018-12-04 13:39:10 +00:00
Da Zhou	7ccb640a66	HADOOP-15957. WASB: Add asterisk wildcard support for PageBlobDirSet. Contributed by Da Zhou.	2018-11-30 10:12:41 +00:00
Sean Mackrory	7eb0d3a324	HADOOP-14927. ITestS3GuardTool failures in testDestroyNoBucket(). Contributed by Gabor Bota.	2018-11-29 09:36:39 -07:00
Ewan Higgs	c1d24f8483	HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests. Contributed by Ewan Higgs and Steve Loughran.	2018-11-29 15:12:17 +00:00
Sean Mackrory	085f10e75d	HADOOP-15947. Fix ITestDynamoDBMetadataStore test error issues. Contributed by Gabor Bota.	2018-11-28 10:45:09 -07:00
Sean Mackrory	e148c3ff09	HADOOP-15798. LocalMetadataStore put() does not retain isDeleted in parent listing. Contributed by Gabor Bota.	2018-11-28 10:45:09 -07:00
Sean Mackrory	5d96b74f33	HADOOP-15370. S3A log message on rm s3a://bucket/ not intuitive. Contributed by Gabor Bota.	2018-11-28 10:45:09 -07:00
Steve Loughran	4c106fca0c	HADOOP-15932. Oozie unable to create sharelib in s3a filesystem. Contributed by Steve Loughran.	2018-11-27 20:39:54 +00:00
Da Zhou	300f772c36	HADOOP-15940. ABFS: For HNS account, avoid unnecessary get call when doing Rename. Contributed by Da Zhou <da.zhou@microsoft.com> (cherry picked from commit 1a3a4960d91aa594283e447963fb2e407d4c6af5)	2018-11-27 18:13:07 +00:00
Da Zhou	a8302e398c	HADOOP-15872. ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2. Contributed by Junhua Gu and Da Zhou.	2018-11-23 14:17:36 +00:00
Weiwei Yang	5ff0cf86a9	HADOOP-15943. AliyunOSS: add missing owner & group attributes for oss FileStatus. Contributed by wujinhu.	2018-11-23 13:56:54 +08:00
Weiwei Yang	be0708c6eb	HADOOP-15919. AliyunOSS: Enable Yarn to use OSS. Contributed by wujinhu.	2018-11-19 14:00:54 +08:00
Arpit Agarwal	914b0cf15f	HADOOP-12558. distcp documentation is woefully out of date. Contributed by Dinesh Chitlangia.	2018-11-15 13:58:13 -08:00
Akira Ajisaka	66b1335bb3	HADOOP-15926. Document upgrading the section in NOTICE.txt when upgrading the version of AWS SDK. Contributed by Dinesh Chitlangia.	2018-11-15 16:30:24 +09:00
Sammi Chen	3fade865ce	HADOOP-15917. AliyunOSS: fix incorrect ReadOps and WriteOps in statistics. Contributed by Jinhu Wu.	2018-11-14 12:58:57 +08:00
Da Zhou	a13be203b7	HADOOP-15876. Use keySet().removeAll() to remove multiple keys from Map in AzureBlobFileSystemStore Contributed by Da Zhou.	2018-11-13 21:46:18 +00:00
Da Zhou	9dbb2b67c6	HADOOP-15812. ABFS: Improve AbfsRestOperationException format to ensure full msg can be displayed on console. Author: Da Zhou <da.zhou@microsoft.com>	2018-11-09 11:06:23 +00:00
Junhua Gu	724c15007b	HADOOP-15846. ABFS: fix mask related bugs in setAcl, modifyAclEntries and removeAclEntries. Contributed by Junhua Gu. (cherry picked from commit 66715005f9e8f4f25faa352a06d142b75a029f0e)	2018-11-08 17:21:40 +00:00
Sammi Chen	7574d18538	HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu.	2018-10-26 15:19:56 +08:00
Daniel Templeton	e374584479	HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI (Contributed by Da Zhou via Daniel Templeton) Change-Id: I546ab3a1df1efec635c08c388148e718dc4a9843	2018-10-24 14:18:35 -07:00
Robert Kanter	5e294e950b	YARN-8919. Some tests fail due to NoClassDefFoundError for OperatorCreationException (tasanuma0829 via rkanter)	2018-10-23 13:37:17 -07:00
Ted Yu	e2cecb681e	HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2018-10-19 13:21:06 -07:00
Giovanni Matteo Fumarola	cc09b2b0c7	HDFS-14007. Incompatible layout when generating FSImage. Contributed by Inigo Goiri.	2018-10-18 17:24:23 -07:00
Steve Loughran	d59ca43bff	HADOOP-15826. @Retries annotation of putObject() call & uses wrong. Contributed by Steve Loughran and Ewan Higgs.	2018-10-16 20:02:54 +01:00
Steve Loughran	ef9dc6c44c	HADOOP-15851. Disable wildfly logs to the console. Contributed by Vishwajeet Dusane.	2018-10-15 22:53:55 +01:00
bibinchundatt	b4a38e7b3e	YARN-8830. SLS tool fix node addition. Contributed by Bibin A Chundatt.	2018-10-15 16:10:25 +05:30
Steve Loughran	e36ae9639f	HADOOP-15831. Include modificationTime in the toString method of CopyListingFileStatus. Contributed by Ted Yu.	2018-10-12 09:59:19 +01:00
Steve Loughran	ee816f1fd7	HADOOP-15837. DynamoDB table Update can fail S3A FS init. Contributed by Steve Loughran.	2018-10-11 14:57:38 +01:00
Steve Loughran	bd50fa956b	HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. Contributed by Da Zhou.	2018-10-09 20:02:12 +01:00
Jason Lowe	4b5b1ac3d1	MAPREDUCE-7130. Rumen crashes trying to handle MRAppMaster recovery events. Contributed by Peter Bacsko	2018-10-09 13:27:03 -05:00
Steve Loughran	7ba1cfdea7	HADOOP-15827. NPE in DynamoDBMetadataStore.lambda$listChildren for root + auth S3Guard. Contributed by Gabor Bota	2018-10-09 10:46:41 +01:00
Steve Loughran	273cc2d4e9	HADOOP-15809. ABFS: better exception handling when making getAccessToken call. Contributed by Da Zhou	2018-10-05 11:29:03 +01:00
Steve Loughran	7051bd78b1	HADOOP-15795. Make HTTPS the default protocol for ABFS. Contributed by Da Zhou.	2018-10-03 12:52:53 +01:00
Aaron Fabbri	4f752d442b	HADOOP-15621 2/2 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota	2018-10-03 00:24:29 -07:00
Aaron Fabbri	046b8768af	HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota	2018-10-02 21:22:49 -07:00
Steve Loughran	e8b8604314	HADOOP-15792. typo in AzureBlobFileSystem.getIsNamespaceEnabeld. Contributed by Abhishek Modi.	2018-10-02 18:57:20 +01:00
Sunil G	58fa96b697	Changed version in trunk to 3.3.0-SNAPSHOT.	2018-10-02 22:41:41 +05:30
Steve Loughran	a383ac47ca	HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore. Contributed by Da Zhou	2018-10-02 11:42:52 +01:00
Steve Loughran	43bc984891	HADOOP-15793. ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization Contributed by Yuan Gao.	2018-10-02 11:37:28 +01:00
Steve Loughran	a4abf02028	HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies. Contributed by Da Zhou.	2018-09-25 20:58:32 +01:00
Steve Loughran	d5da9928c9	HADOOP-15723. ABFS: Ranger Support. Contributed by Yuan Gao.	2018-09-25 19:13:10 +01:00
Sammi Chen	2b635125fb	HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu.	2018-09-25 19:48:30 +08:00
Mingliang Liu	c07715e378	HADOOP-15781 S3A assumed role tests failing due to changed error text in AWS exceptions. Contributed by Steve Loughran	2018-09-24 12:53:21 -07:00
Sunil G	d060cbea48	HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran.	2018-09-24 21:19:47 +05:30
Sean Mackrory	0def61482b	Merge branch 'HADOOP-15407' into trunk	2018-09-22 21:19:12 -06:00
Steve Loughran	d0b4624c88	HADOOP-15778. ABFS: Fix client side throttling for read. Contributed by Sneha Varma.	2018-09-21 11:06:24 +01:00
Steve Loughran	a5692c2da5	HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable. Contributed by Steve Loughran.	2018-09-20 17:36:18 +01:00
Sean Mackrory	8e831ba458	HADOOP-15773. Fixing checkstyle and other issues raised by Yetus.	2018-09-19 16:56:33 -06:00
Steve Loughran	a55d26b23e	HADOOP-15769. ABFS: distcp tests are always skipped. Contributed by Steve Loughran	2018-09-19 13:57:39 +01:00
Steve Loughran	df2166a643	HADOOP-15719. Fail-fast when using OAuth over http. Contributed by Da Zhou.	2018-09-18 12:20:52 +01:00
Steve Loughran	51d368982b	HADOOP-15714. Tune abfs/wasb parallel and sequential test execution. Contributed by Da Zhou.	2018-09-18 12:09:25 +01:00
Steve Loughran	524776625d	HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min. Contributed by Da Zhou	2018-09-18 11:48:46 +01:00
Steve Loughran	1cf38a38da	HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. Contributed by Steve Loughran.	2018-09-18 10:56:56 +01:00
Steve Loughran	26d0c63a1e	HADOOP-15754. s3guard: testDynamoTableTagging should clear existing config. Contributed by Gabor Bota.	2018-09-17 22:40:08 +01:00
Thomas Marquardt	b4c23043d3	HADOOP-15757. ABFS: remove dependency on common-codec Base64. Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	26211019c8	HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	e5593cbd83	HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. Contributed by Sean Mackrory.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	13c70e9ba3	HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. Contributed by Sneha Varma.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	6801b30733	HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation for storage account with no namespace feature. Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	347a52a866	Fixing findbugs and license issues related to: HADOOP-15703. ABFS - Implement client-side throttling. Contributed by Sneha Varma and Thomas Marquardt.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	97f06b3fc7	HADOOP-15703. ABFS - Implement client-side throttling. Contributed by Sneha Varma and Thomas Marquardt.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	4410eacba7	HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	81dc4a995c	HADOOP-15663. ABFS: Simplify configuration. Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	df57c6c3b1	HADOOP-15692. ABFS: extensible support for custom oauth. Contributed by Junhua Gu and Rajeev Bansal.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	dd2b22fa31	HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. Contributed by Thomas Marquardt.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	6b6f8cc2be	HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. Contributed by Sean Mackrory.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	9c1e4e8139	HADOOP-15661. ABFS: Add support for ACL. Contributed by Junhua Gu and Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	9149b9703e	HADOOP-15660. ABFS: Add support for OAuth Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	d6a4f39bd5	HADOOP-15669. ABFS: Improve HTTPS Performance. Contributed by Vishwajeet Dusane.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	cc5cc60c41	Fixing issue due to commit `2b2399d6` after rebase onto trunk.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	b54b0c1b67	HADOOP-15659. Code changes for bug fix and new tests. Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	ce03a93f78	HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. Contributed by Steve Loughran and Da Zhou.	2018-09-17 19:54:01 +00:00
Steve Loughran	a271fd0eca	HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Steve Loughran	f044deedbb	HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran	2018-09-17 19:54:01 +00:00
Steve Loughran	d7c0a08a1c	HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran)	2018-09-12 21:04:49 -07:00
Aaron Fabbri	d32a8d5d58	HADOOP-14734 add option to tag DDB table(s) created. (Contributed by Gabor Bota and Abe Fine)	2018-09-12 16:36:01 -07:00
Mingliang Liu	1f6c4545cf	HADOOP-15750. Remove obsolete S3A test ITestS3ACredentialsInURL. Contributed by Steve Loughran	2018-09-12 10:58:39 -07:00
Sean Mackrory	47b72c87eb	HADOOP-15635. s3guard set-capacity command to fail fast if bucket is unguarded. Contributed by Gabor Bota.	2018-09-12 09:12:38 -06:00
bibinchundatt	c44088ac19	YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan.	2018-09-12 16:01:01 +05:30
Naganarasimha	eb08543c7a	YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt.	2018-09-12 16:01:01 +05:30
Sunil G	89b3ebd11e	YARN-8092. Expose Node Attributes info via RM nodes REST API. Contributed by Weiwei Yang.	2018-09-12 16:01:00 +05:30
Mingliang Liu	87f63b6479	HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran	2018-09-11 17:18:42 -07:00
Steve Loughran	1eef0acfe5	HADOOP-15677. WASB: Add support for StreamCapabilities. Contributed by Thomas Marquardt.	2018-09-10 14:24:41 +01:00
Gabor Bota	36c7c78260	HADOOP-15709 Move S3Guard LocalMetadataStore constants to org.apache.hadoop.fs.s3a.Constants (Contributed by Gabor Bota)	2018-09-07 10:25:20 -07:00
Steve Loughran	5a0babf765	HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs. Contributed by Steve Loughran.	2018-08-30 14:49:53 +01:00
Steve Loughran	e8d138ca7c	HADOOP-15680. ITestNativeAzureFileSystemConcurrencyLive times out. Contributed by Andras Bokor.	2018-08-30 14:36:00 +01:00
Steve Loughran	2e6c1109dc	HADOOP-15667. FileSystemMultipartUploader should verify that UploadHandle has non-0 length. Contributed by Ewan Higgs	2018-08-30 14:33:16 +01:00
Yufei Gu	8563fd67be	YARN-8632. Threads in SLS quit without logging exception. Contributed by Xianghao Lu.	2018-08-24 11:23:17 -07:00
Surendra Singh Lilhore	96c4575d73	HDFS-13805. Journal Nodes should allow to format non-empty directories with -force option. Contributed by Surendra Singh Lilhore.	2018-08-24 08:14:57 +05:30
Aaron Fabbri	d7232857d8	HADOOP-14154 Persist isAuthoritative bit in DynamoDBMetaStore (Contributed by Gabor Bota)	2018-08-17 10:15:39 -07:00
Steve Loughran	0e832e7a74	HADOOP-15642. Update aws-sdk version to 1.11.375. Contributed by Steve Loughran.	2018-08-16 09:58:46 -07:00
Akira Ajisaka	3e3963b035	HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering.	2018-08-16 00:31:59 +09:00
Ewan Higgs	a13929ddcb	HADOOP-15645. ITestS3GuardToolLocal.testDiffCommand fails if bucket has per-bucket binding to DDB. Contributed by Steve Loughran.	2018-08-13 12:57:45 +02:00
Steve Loughran	da9a39eed1	HADOOP-15583. Stabilize S3A Assumed Role support. Contributed by Steve Loughran.	2018-08-08 22:57:24 -07:00
Ewan Higgs	2ec97abb2e	HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran.	2018-08-08 13:50:23 +02:00
Sean Mackrory	7862f1523f	HADOOP-15400. Improve S3Guard documentation on Authoritative Mode implementation. (Contributed by Gabor Bota)	2018-08-07 20:13:09 -06:00
Steve Loughran	48673bc2a8	HADOOP-15626. FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append. Contributed by Steve Loughran.	2018-08-03 16:06:00 -07:00
Inigo Goiri	9fea5c9ee7	YARN-8175. Add support for Node Labels in SLS. Contributed by Abhishek Modi.	2018-07-31 09:36:34 -07:00
Sammi Chen	0857f116b7	HADOOP-15607. AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream. Contributed by Jinhu Wu.	2018-07-30 10:53:44 +08:00
Sean Mackrory	59adeb8d7f	HADOOP-15636. Follow-up from HADOOP-14918; restoring test under new name. Contributed by Gabor Bota.	2018-07-27 18:23:29 -06:00
bibinchundatt	3d3158cea4	YARN-8577. Fix the broken anchor in SLS site-doc. Contributed by Weiwei Yang.	2018-07-25 18:59:05 +05:30
Steve Loughran	45d9568aaa	HADOOP-15547/ WASB: improve listStatus performance. Contributed by Thomas Marquardt. (cherry picked from commit 749fff577ed9afb4ef8a54b8948f74be083cc620)	2018-07-19 12:31:19 -07:00
Wangda Tan	752dcce5f4	YARN-8511. When AM releases a container, RM removes allocation tags before it is released by NM. (Weiwei Yang via wangda) Change-Id: I6f9f409f2ef685b405cbff547dea9623bf3322d9	2018-07-16 10:54:41 -07:00
Sean Mackrory	a08812a1b1	HADOOP-15349. S3Guard DDB retryBackoff to be more informative on limits exceeded. Contributed by Gabor Bota.	2018-07-12 17:24:01 +02:00
Sean Mackrory	d503f65b66	HADOOP-15541. [s3a] Shouldn't try to drain stream before aborting connection in case of timeout.	2018-07-10 17:52:57 +02:00
Steve Loughran	ca8b80bf59	HADOOP-15384. distcp numListstatusThreads option doesn't get to -delete scan. Contributed by Steve Loughran.	2018-07-10 10:43:59 +01:00
Aaron Fabbri	93ac01cb59	HADOOP-15215 s3guard set-capacity command to fail on read/write of 0 (Gabor Bota)	2018-07-03 13:50:11 -07:00
Inigo Goiri	1804a31515	HDFS-13536. [PROVIDED Storage] HA for InMemoryAliasMap. Contributed by Virajith Jalaparti.	2018-07-02 10:48:20 -07:00
Inigo Goiri	73746c5da7	HDFS-13707. [PROVIDED Storage] Fix failing integration tests in ITestProvidedImplementation. Contributed by Virajith Jalaparti.	2018-06-29 09:56:13 -07:00
Akira Ajisaka	2b2399d623	HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma.	2018-06-28 14:37:22 +09:00
Todd Lipcon	7a3c6e9c3c	HADOOP-15550. Avoid static initialization of ObjectMappers	2018-06-25 15:36:45 -07:00
Sean Mackrory	c687a6617d	HADOOP-15423. Merge fileCache and dirCache into ine single cache in LocalMetadataStore. Contributed by Gabor Bota.	2018-06-25 14:59:41 -06:00
Anu Engineer	e16e5b307d	HDDS-177. Create a releasable ozonefs artifact Contributed by Marton, Elek.	2018-06-24 01:05:04 -07:00
Anu Engineer	ca14fec02f	HDDS-184. Upgrade common-langs version to 3.7 in hadoop-tools/hadoop-ozone. Contributed by Takanobu Asanuma.	2018-06-23 15:49:44 -07:00
Sean Mackrory	55fad6a3de	HADOOP-15416. Clear error message in S3Guard diff if source not found. Contributed by Gabor Bota.	2018-06-22 11:36:56 -06:00
Sean Mackrory	b089a06793	HADOOP-14918. Remove the Local Dynamo DB test option. Contributed by Gabor Bota.	2018-06-20 16:45:08 -06:00
Chris Douglas	f34744603e	HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii	2018-06-17 23:12:18 -07:00
Chris Douglas	980031bb04	HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs	2018-06-17 11:54:26 -07:00
Jitendra Pandey	f4c7c91123	HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan.	2018-06-13 00:36:02 -07:00
Inigo Goiri	c35481594f	YARN-8422. TestAMSimulator failing with NPE. Contributed by Giovanni Matteo Fumarola.	2018-06-12 10:59:50 -07:00
Steve Loughran	d901be6795	HADOOP-15506. Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks. Contributed by Esfandiar Manii.	2018-06-06 18:28:14 +01:00
Anu Engineer	3b34148c4f	HDDS-88. Create separate message structure to represent ports in DatanodeDetails. Contributed by Nanda Kumar.	2018-05-30 08:52:07 -07:00
Aaron Fabbri	5f6769f796	HADOOP-15480 AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo (Gabor Bota)	2018-05-29 19:20:22 -07:00
Aaron Fabbri	30284d020d	HADOOP-14946 S3Guard testPruneCommandCLI can fail. Contributed by Gabor Bota.	2018-05-29 13:38:15 -07:00
Inigo Goiri	8fdc993a99	HADOOP-15498. TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows. Contributed by Anbang Hu.	2018-05-28 16:45:42 -07:00
Johan Gustavsson	d14e26b31f	HADOOP-15477. Make unjar in RunJar overrideable Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-05-28 17:29:59 +09:00
Steve Loughran	ba842847c9	HADOOP-15478. WASB: hflush() and hsync() regression. Contributed by Thomas Marquardt.	2018-05-21 11:12:34 +01:00
Steve Loughran	cc3600aabd	HADOOP-15469. S3A directory committer commit job fails if _temporary directory created under dest. Contributed by Steve Loughran. (cherry picked from commit 170f1040d46f9b1a084e6637def91e9864446acc)	2018-05-17 19:08:55 +01:00
Steve Loughran	b670837469	HADOOP-15442. ITestS3AMetrics.testMetricsRegister can't know metrics source's name. Contributed by Sean Mackrory.	2018-05-15 16:13:56 +01:00
Steve Loughran	ba051b0686	HADOOP-15354. hadoop-aliyun & hadoop-azure modules to mark hadoop-common as provided Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-05-10 18:38:08 +09:00
Anu Engineer	46e0f27862	HDDS-18. Ozone Shell should use RestClient and RpcClient. Contributed by Lokesh Jain.	2018-05-09 23:00:13 -07:00
Sean Mackrory	1cfe7506f7	HADOOP-15356. Make HTTP timeout configurable in ADLS connector. Contributed by Atul Sikaria and Sean Mackrory.	2018-05-09 15:15:51 -06:00
Mukul Kumar Singh	343b51dcf4	HDDS-28. Ozone:Duplicate declaration in hadoop-tools/hadoop-ozone/pom.xml. Contributed by Tsz Wo Nicholas Sze.	2018-05-09 20:20:21 +05:30
Mukul Kumar Singh	208b97e969	HDDS-20. Ozone: Add support for rename key within a bucket for rpc client. Contributed by Lokesh Jain.	2018-05-09 19:06:07 +05:30
Aaron Fabbri	8981674bbc	HADOOP-15420 s3guard ITestS3GuardToolLocal failures in diff tests (Gabor Bota)	2018-05-08 18:58:42 -07:00
Aaron Fabbri	69aac696d9	HADOOP-13649 s3guard: implement time-based (TTL) expiry for LocalMetadataStore (Gabor Bota)	2018-05-08 15:29:54 -07:00
Steve Loughran	5b11b9fd41	HADOOP-15446. WASB: PageBlobInputStream.skip breaks HBASE replication. Contributed by Thomas Marquardt	2018-05-07 11:54:08 +01:00
Aaron Fabbri	96c843f64b	HADOOP-15444 ITestS3GuardToolDynamo should only run with -Ddynamo (Aaron Fabbri)	2018-05-04 11:34:45 -07:00
Sean Mackrory	85381c7b60	HADOOP-15434. Upgrade to ADLS SDK that exposes current timeout.	2018-05-02 21:30:31 -06:00
Aaron Fabbri	919865a34b	HADOOP-15239 S3ABlockOutputStream.flush() be no-op when stream closed. Contributed by Gabor Bota.	2018-04-30 16:02:57 -07:00
Weiwei Yang	ba2db39c65	YARN-8152. Add chart in SLS to illustrate the throughput of the scheduler. Contributed by Tao Yang.	2018-04-27 15:36:21 +08:00
Aaron Fabbri	7d8bcf534a	HADOOP-13756 LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash. (Gabor Bota)	2018-04-26 20:41:29 -07:00
Owen O'Malley	2adda92de1	HDFS-7240 git commit fixup Signed-off-by: Owen O'Malley <omalley@apache.org>	2018-04-26 13:52:17 -07:00
Mukul Kumar Singh	ae8ac7f082	HDFS-13424. Ozone: Refactor MiniOzoneClassicCluster. Contributed by Nanda Kumar.	2018-04-26 05:36:04 -07:00
Nanda kumar	025058f251	HDFS-13446. Ozone: Fix OzoneFileSystem contract test failures. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	8b832f3c35	HDFS-13405. Ozone: Rename HDSL to HDDS. Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.	2018-04-26 05:36:04 -07:00
Nanda kumar	7ae9543a57	HDFS-13391. Ozone: Make dependency of internal sub-module scope as provided in maven. Contributed by Nanda kumar.	2018-04-26 05:36:04 -07:00
Anu Engineer	04e1287771	HDFS-13298. Ozone: Make ozone/hdsl/cblock modules turned off by default. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Anu Engineer	4e61bc431e	HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.	2018-04-26 05:36:04 -07:00
Anu Engineer	5e7164c614	HDFS-13137. Ozone: Ozonefs read fails because ChunkGroupInputStream#read does not iterate through all the blocks in the key. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	61651dcf5c	HDFS-13108. Ozone: OzoneFileSystem: Simplified url schema for Ozone File System. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Mukul Kumar Singh	cd807c9890	HDFS-13133. Ozone: OzoneFileSystem: Calling delete with non-existing path shouldn't be logged on ERROR level. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Mukul Kumar Singh	9272e1021d	HDFS-12636. Ozone: OzoneFileSystem: Implement seek functionality for rpc client. Contributed by Lokesh Jain.	2018-04-26 05:36:04 -07:00
Anu Engineer	4460ac9c57	HDFS-12756. Ozone: Add datanodeID to heartbeat responses and container protocol. Contributed by Anu Engineer.	2018-04-26 05:36:04 -07:00
Anu Engineer	740a06cdd7	HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Chen Liang	e395244e95	HDFS-12758. Ozone: Correcting assertEquals argument order in test cases. Contributed by Bharat Viswanadham.	2018-04-26 05:36:04 -07:00
Xiaoyu Yao	2ec24a7542	HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar.	2018-04-26 05:36:04 -07:00
Chen Liang	895a4cc60a	HDFS-12632. Ozone: OzoneFileSystem: Add contract tests to OzoneFileSystem. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	d90ab3365d	HDFS-12608. Ozone: Remove Warnings when building. Contributed by Bharat Viswanadham.	2018-04-26 05:36:04 -07:00
Xiaoyu Yao	9445a9267f	HDFS-12572. Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	0b3270a8de	HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar.	2018-04-26 05:36:04 -07:00
Xiaoyu Yao	5e4a6b686c	HDFS-12425. Ozone: OzoneFileSystem: read/write/create/open/getFileInfo APIs. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Xiaoyu Yao	f2d9360b9d	HDFS-12199. Ozone: OzoneFileSystem: OzoneFileystem initialization code. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Akira Ajisaka	3e2ca23a53	HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang.	2018-04-26 05:36:04 -07:00
Aaron Fabbri	989a3929a9	HADOOP-14756 S3Guard: expose capability query in MetadataStore and add tests of authoritative mode (Gabor Bota)	2018-04-23 15:26:53 -07:00
Inigo Goiri	fd24fd0ff7	YARN-8137. Parallelize node addition in SLS. Contributed by Abhishek Modi.	2018-04-20 09:09:12 -07:00
Yiqun Lin	1312f9ae4c	HADOOP-15391. Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules.	2018-04-18 16:04:00 +08:00
Inigo Goiri	78860372bd	YARN-8134. Support specifying node resources in SLS. Contributed by Abhishek Modi.	2018-04-17 13:11:34 -07:00
Aaron Fabbri	b0aff8a962	HADOOP-15346 S3ARetryPolicy for 400/BadArgument to be "fail". Contributed by Steve Loughran.	2018-04-10 23:55:38 -07:00
Xiao Chen	7c9cdad6d0	HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo.	2018-04-10 21:31:48 -07:00
Aaron Fabbri	ea3849f0cc	HADOOP-14759 S3GuardTool prune to prune specific bucket entries. Contributed by Gabor Bota.	2018-04-05 20:23:17 -07:00
Steve Loughran	5a174f8ac6	HADOOP-14758. S3GuardTool.prune to handle UnsupportedOperationException. Contributed by Gabor Bota.	2018-04-03 16:31:34 +01:00
Sammi Chen	6542d17ea4	HADOOP-14999. AliyunOSS: provide one asynchronous multi-part based uploading mechanism. Contributed by Genmao Yu.	2018-03-30 20:23:05 +08:00
Sean Mackrory	9d7a9031a5	HADOOP-15342. Updating ADLS connector to use the current SDK version (2.2.7). Contributed by Atul Sikaria.	2018-03-29 10:13:40 -06:00
Chris Douglas	081c350188	HADOOP-15320. Remove customized getFileBlockLocations for hadoop-azure and hadoop-azure-datalake. Contributed by Shanyu Zhao	2018-03-28 11:58:59 -07:00
Haibo Chen	fcea5a4d79	YARN-7794. SLSRunner is not loading timeline service jars, causing failure. (Yufei Gu via Haibo Chen)	2018-03-26 14:30:11 -07:00
Sean Mackrory	82665a7887	HADOOP-15299. Bump Jackson 2 version to Jackson 2.9.x.	2018-03-26 14:05:15 -06:00
Sean Mackrory	7ce6b41509	HADOOP-15332. Fix typos in hadoop-aws markdown docs. Contributed by Gabor Bota.	2018-03-20 21:12:20 -07:00
Sammi Chen	d67a5e2dec	HADOOP-15262. AliyunOSS: move files under a directory in parallel when rename a directory. Contributed by Jinhu Wu.	2018-03-19 15:02:37 +08:00
Wangda Tan	f47659fb97	MAPREDUCE-7047. Make HAR tool support IndexedLogAggregtionController. (Xuan Gong via wangda) Change-Id: Ice5ae1c706f2476361997dcbb29f2c33c77d4f0c	2018-03-15 13:26:01 -07:00
Steve Loughran	1976e0066e	HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories. Contributed by Steve Loughran.	2018-03-15 18:05:14 +00:00
fang zhenyi	5e013d50d1	HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-03-15 16:12:07 +09:00
Chris Douglas	45cccadd2e	HDFS-12780. Fix spelling mistake in DistCpUtils.java. Contributed by Jianfei Jiang	2018-03-13 11:08:11 -07:00
Chris Douglas	45d1b0fdcc	HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer	2018-03-12 20:05:39 -07:00
Steve Loughran	dd05871b8b	HADOOP-15297. Make S3A etag => checksum feature optional. Contributed by Steve Loughran.	2018-03-12 14:01:42 +00:00
Steve Loughran	7ef4d942dd	HADOOP-15273.distcp can't handle remote stores with different checksum algorithms. Contributed by Steve Loughran.	2018-03-08 11:24:06 +00:00
Steve Loughran	3bd6b1fd85	HADOOP-15292. Distcp's use of pread is slowing it down. Contributed by Virajith Jalaparti.	2018-03-08 11:15:46 +00:00
Steve Loughran	e0307e53e2	HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled. Contributed by Anis Elleuch. (cherry picked from commit 1dedc68f9d8d8544d715e67ee77cd3f017c21699)	2018-03-07 18:51:10 +00:00
Steve Loughran	2e1e049bd0	HADOOP-15288. TestSwiftFileSystemBlockLocation doesn't compile. Contributed by Steve Loughran.	2018-03-05 18:12:38 +00:00
Steve Loughran	8110d6a0d5	HADOOP-13761. S3Guard: implement retries for DDB failures and throttling; translate exceptions. Contributed by Aaron Fabbri.	2018-03-05 14:06:20 +00:00
Gergely Novák	e99a1e7767	MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-03-02 16:15:04 -08:00
Ray Chiang	923e1773e1	HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang)	2018-03-01 15:36:10 -08:00
fang zhenyi	29233c3c4a	HADOOP-15274. Move hadoop-openstack to slf4j Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-03-01 15:08:30 -08:00
Steve Loughran	edc9f1451b	HADOOP-15263. hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake. Contributed by Steve Loughran	2018-02-28 14:42:25 +00:00
Weiwei Yang	7af4f34de5	YARN-7929. Support to set container execution type in SLS. (Jiandan Yang via Weiwei Yang)	2018-02-28 17:58:09 +08:00
Arpit Agarwal	2fa7963c3d	HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi.	2018-02-24 14:41:55 -08:00
Xuan Gong	68ce193efc	MAPREDUCE-7027: HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails. Contributed by Gergely Novák	2018-02-23 14:37:26 -08:00
Carlo Curino	84cea0011f	YARN-7732. Support Generic AM Simulator from SynthGenerator. (Contributed by Young Chen via curino)	2018-02-20 17:00:34 -08:00
fang zhenyi	4d4dde5112	HADOOP-15223. Replace Collections.EMPTY* with empty* when available Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-02-18 22:19:39 +09:00
Steve Loughran	7ac88244c5	HADOOP-14507. Extend per-bucket secret key config with explicit getPassword() on fs.s3a.$bucket.secret.key. Contributed by Steve Loughran.	2018-02-16 16:37:06 +00:00
Steve Loughran	481d79fedc	HADOOP-13972. ADLS to support per-store configuration. Contributed by Sharad Sonker. (cherry picked from commit 050f5287b79324b7f6231b879c0bfc608203b980)	2018-02-15 16:27:31 +00:00
Steve Loughran	9a013b255f	HADOOP-15176. Enhance IAM Assumed Role support in S3A client. Contributed by Steve Loughran (cherry picked from commit 96c047fbb98c2378eed9693a724d4cbbd03c00fd)	2018-02-15 15:57:10 +00:00
Steve Loughran	b27ab7dd81	HADOOP-15076. Enhance S3A troubleshooting documents and add a performance document. Contributed by Steve Loughran. (cherry picked from commit c761e658f6594c4e519ed39ef36669de2c5cee15)	2018-02-15 14:57:56 +00:00
Steve Loughran	c9a373fb14	HADOOP-15090. Add ADL troubleshooting doc. Contributed by Steve Loughran. (cherry picked from commit 58a2120e8a31307f19551f87be4e81d4fb626de1)	2018-02-15 14:26:00 +00:00
Steve Loughran	f20dc0d577	HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. Contributed by Andras Bokor.	2018-02-14 16:20:14 +00:00
Steve Loughran	8cf88fcd1f	HADOOP-15187. Remove ADL mock test dependency on REST call invoked from Java SDK. Contributed by Vishwajeet Dusane. (cherry picked from commit bd5ab5912564d2d687651b01f552b8e4ca8c145a)	2018-02-12 15:13:00 +00:00
Wangda Tan	60f9e60b3b	Preparing for 3.2.0 development Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938	2018-02-11 11:17:38 +08:00
Anu Engineer	4304fcd5bd	HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen.	2018-02-06 13:43:45 -08:00
Arpit Agarwal	d4e13a4647	HADOOP-15198. Correct the spelling in CopyFilter.java. Contributed by Mukul Kumar Singh.	2018-02-02 11:37:51 -08:00
Sunil G	9b81cb0537	YARN-7779. Display allocation tags in RM web UI and expose same through REST API. Contributed by Weiwei Yang.	2018-01-31 01:30:17 -08:00
Arun Suresh	38af237969	YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh)	2018-01-31 01:30:17 -08:00
Steve Loughran	7fd287b4af	HADOOP-15186. Allow Azure Data Lake SDK dependency version to be set on the command line. Contributed by Vishwajeet Dusane. (cherry picked from commit 95a96b13e2a54e01ea6c6933045d912998477da3)	2018-01-29 09:48:14 -08:00
Jason Lowe	ff8378eb1b	YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang	2018-01-25 17:47:19 -06:00
Yufei Gu	16be42d309	YARN-7798. Refactor SLS Reservation Creation. Contributed by Young Chen.	2018-01-25 13:06:01 -08:00
Steve Loughran	f63d13f10d	HADOOP-15185. Update adls connector to use the current version of ADLS SDK. Contributed by Atul Sikaria.	2018-01-23 09:09:11 -08:00
Steve Loughran	1093a73689	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. Contributed by Aaron Fabbri	2018-01-18 13:13:58 +00:00
Steve Loughran	f274fe33ea	Revert "HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits." This reverts commit `35ad9b1dd2`.	2018-01-18 12:35:57 +00:00
Aaron Fabbri	268ab4e027	HADOOP-15141 Support IAM Assumed roles in S3A. Contributed by Steve Loughran.	2018-01-17 00:05:24 -08:00
Sammi Chen	9195a6e302	HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)	2018-01-17 15:55:59 +08:00
Steve Loughran	a0c71dcc33	HADOOP-15079. ITestS3AFileOperationCost#testFakeDirectoryDeletion failing after OutputCommitter patch. Contributed by Steve Loughran	2018-01-15 11:33:32 +00:00
Mukul Kumar Singh	2e1e9017aa	HADOOP-15172. Fix the javadoc warning in WriteOperationHelper.java Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-01-15 15:45:51 +09:00
Steve Loughran	1a09da7400	HADOOP-15163. Fix S3ACommitter documentation Contributed by Alessandro Andrioni. (cherry picked from commit 100e8a1ae1d930dde084af7d1281e491c7f124ec)	2018-01-10 15:37:07 +00:00
Steve Loughran	b62a5ece95	HADOOP-15161. s3a: Stream and common statistics missing from metrics Contributed by Sean Mackrory	2018-01-09 18:46:52 +00:00
Steve Loughran	52babbb4a0	HADOOP-15086. NativeAzureFileSystem file rename is not atomic. Contributed by Thomas Marquardt	2017-12-22 11:39:55 +00:00
Steve Loughran	c8ff0cc304	HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls. Contributed by Steve Loughran	2017-12-21 14:58:58 +00:00
Steve Loughran	ef450df443	HADOOP-15113. NPE in S3A getFileStatus: null instrumentation on using closed instance. Contributed by Steve Loughran.	2017-12-21 14:15:53 +00:00
Akira Ajisaka	5ab632baf5	HDFS-12949. Fix findbugs warning in ImageWriter.java.	2017-12-21 10:04:34 +09:00
Steve Loughran	1ba491ff90	HADOOP-14965. S3a input stream "normal" fadvise mode to be adaptive	2017-12-20 18:25:33 +00:00
Steve Loughran	35ad9b1dd2	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. Contributed by Aaron Fabbri	2017-12-18 21:19:06 +00:00
Chris Douglas	4b3a785914	HDFS-12903. [READ] Fix closing streams in ImageWriter. Contributed by Virajith Jalaparti	2017-12-15 17:51:42 -08:00
Chris Douglas	e515103a83	Revert "HDFS-12903. [READ] Fix closing streams in ImageWriter" This reverts commit c1bf2654b0e9118985b8518b0254eac4dd302a2f.	2017-12-15 17:51:42 -08:00
Virajith Jalaparti	8239e3afb3	HDFS-12712. [9806] Code style cleanup	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	80c3fec3a1	HDFS-12912. [READ] Fix configuration and implementation of LevelDB-based alias maps	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	c89b29bd42	HDFS-12893. [READ] Support replication of Provided blocks with non-default topologies.	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	0f6aa9564c	HDFS-12905. [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	962b5e722b	HDFS-12903. [READ] Fix closing streams in ImageWriter	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	4531588a94	HDFS-11640. [READ] Datanodes should use a unique identifier when reading from external stores	2017-12-15 17:51:41 -08:00
Virajith Jalaparti	fb996a32a9	HDFS-12894. [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	9c35be86e1	HDFS-12713. [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata. Contributed by Ewan Higgs	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	a027055dd2	HDFS-12885. Add visibility/stability annotations. Contributed by Chris Douglas	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	352f994b64	HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs.	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	4d59dabb7f	HDFS-12809. [READ] Fix the randomized selection of locations in {{ProvidedBlocksBuilder}}.	2017-12-15 17:51:40 -08:00
Virajith Jalaparti	3d3be87e30	HDFS-12778. [READ] Report multiple locations for PROVIDED blocks	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	3b1d30301b	HDFS-12775. [READ] Fix reporting of Provided volumes	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	6cd80b2521	HDFS-12779. [READ] Allow cluster id to be specified to the Image generation tool	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	90d1b47a2a	HDFS-12776. [READ] Increasing replication for PROVIDED files should create local replicas	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	87dc026bee	HDFS-12789. [READ] Image generation tool does not close an opened stream	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	c293cc8e9b	HDFS-12671. [READ] Test NameNode restarts when PROVIDED is configured	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	71d0a82571	HDFS-12607. [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED	2017-12-15 17:51:39 -08:00
Virajith Jalaparti	98f5ed5aa3	HDFS-11902. [READ] Merge BlockFormatProvider and FileRegionProvider.	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	d6a9a89973	HDFS-12605. [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	17052c4aff	HDFS-12584. [READ] Fix errors in image generation tool from latest rebase	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	aca023b72c	HDFS-12289. [READ] HDFS-12091 breaks the tests for provided block reads	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	546b95f484	HDFS-11673. [READ] Handle failures of Datanode with PROVIDED storage	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	4851f06bc2	HDFS-11791. [READ] Test for increasing replication of provided files.	2017-12-15 17:51:38 -08:00
Virajith Jalaparti	aa5ec85f7f	HDFS-11663. [READ] Fix NullPointerException in ProvidedBlocksBuilder	2017-12-15 17:51:37 -08:00
Virajith Jalaparti	d65df0f273	HDFS-11190. [READ] Namenode support for data stored in external stores.	2017-12-15 17:51:37 -08:00
Virajith Jalaparti	8da3a6e314	HDFS-10706. [READ] Add tool generating FSImage from external store	2017-12-15 17:51:37 -08:00
Kai Zheng	f86c81d923	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-14 11:14:52 +08:00
Kai Zheng	28792b6b7f	Revert "HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu" This reverts commit 479d6a5792262c977025c26fd4960574b0db6847	2017-12-14 11:05:11 +08:00
Surendra Singh Lilhore	00129c5314	HDFS-12833. Distcp : Update the usage of delete option for dependency with update and overwrite option. Contributed by usharani.	2017-12-12 00:28:02 +05:30
Kai Zheng	ce04340ec7	HADOOP-15104. AliyunOSS: change the default value of max error retry. Contributed by Jinhu Wu	2017-12-08 23:03:54 +08:00
Kai Zheng	73b86979d6	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-06 12:01:36 +08:00
Aaron Fabbri	6555af81a2	HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed by Younger and Sean Mackrory.	2017-12-05 11:06:32 -08:00
Steve Loughran	3150c019ae	HADOOP-15071 S3a troubleshooting docs to add a couple more failure modes. Contributed by Steve Loughran	2017-12-05 15:05:41 +00:00
Steve Loughran	0ea182d0fa	HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. Contributed by Rajesh Balamohan	2017-11-28 11:52:59 +00:00
Rohith Sharma K S	3cd75845da	MAPREDUCE-7014. Fix java doc errors in jdk1.8. Contributed by Steve Loughran.	2017-11-27 22:01:00 +05:30
Steve Loughran	de8b6ca5ef	HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints. Contributed by Steve Loughran and Ryan Blue.	2017-11-22 15:28:12 +00:00
Steve Loughran	c326fc89b0	HADOOP-15024 Support user agent configuration and include that & Hadoop version information to oss server. Contributed by Sammi Chen.	2017-11-20 18:56:42 +00:00
Akira Ajisaka	6903cf096e	HADOOP-13514. Upgrade maven surefire plugin to 2.20.1 Signed-off-by: Allen Wittenauer <aw@apache.org>	2017-11-19 12:39:37 -08:00
Kai Zheng	18621af7ae	HADOOP-14993. AliyunOSS: Override listFiles and listLocatedStatus. Contributed Genmao Yu	2017-11-14 17:58:37 +08:00
Daniel Templeton	ba8136615a	YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton) Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b	2017-11-09 12:09:48 -08:00
John Zhuge	bf6a660232	HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge.	2017-11-09 10:16:12 -08:00
Subru Krishnan	f2df6b8983	HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru).	2017-11-08 18:07:12 -08:00
Arun Suresh	53c0fb7efe	HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh)	2017-11-02 17:15:33 -07:00
Akira Ajisaka	cc3f3eca40	MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi.	2017-11-02 18:32:24 +09:00
Chris Douglas	139cc752d2	HADOOP-14991. Add missing figures to Resource Estimator tool. Contributed by Rui Li	2017-10-27 21:59:27 -07:00
Yufei Gu	99880d0a16	YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak)	2017-10-27 14:41:37 -07:00
Subru Krishnan	625039ef20	HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru).	2017-10-25 15:51:27 -07:00
Wangda Tan	281d83604d	YARN-7318. Fix shell check warnings of SLS. (Gergely Novák via wangda)	2017-10-20 15:54:15 -07:00
Steve Loughran	f36cbc8475	HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. Contributed by Andras Bokor.	2017-10-20 22:27:04 +01:00
Haibo Chen	46eb1033a8	YARN-6546. SLS is slow while loading 10k queues. (Yufei Gu via Haibo Chen)	2017-10-17 16:04:19 -07:00
Steve Loughran	9fcc3a1fc8	HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled. Contributed by Santhosh G Nayak	2017-10-16 16:01:47 +01:00
Steve Loughran	3de574413c	HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. Contributed by Varada Hemeswari.	2017-10-11 18:06:43 +01:00
Steve Loughran	572cdb5463	HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB Contributed by Kannapiran Srinivasan.	2017-10-06 17:43:38 +01:00
Haibo Chen	c071aad5da	YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen)	2017-10-05 15:50:13 -07:00
Steve Loughran	9288206cb3	HADOOP-14845. Azure wasb: getFileStatus not making any auth check. Contributed by Sivaguru Sankaridurg	2017-10-05 15:05:23 +01:00
Yufei Gu	79e37dc6db	YARN-7211. AMSimulator in SLS does't work due to responseId refactoring.(Contributed by Botong Huang via Yufei Gu)	2017-10-03 15:10:50 -07:00
Akira Ajisaka	c9b525bd94	YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari.	2017-10-04 07:01:54 +09:00
John Zhuge	8aca46e53f	HADOOP-14651. Update okhttp version to 2.7.5. Contributed by Ray Chiang and John Zhuge.	2017-09-29 12:15:06 -07:00
Steve Loughran	a530e7ab3b	HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB Contributed by Varada Hemeswari	2017-09-28 19:52:56 +01:00
Aaron Fabbri	47011d7dd3	HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran	2017-09-25 15:59:38 -07:00
Jason Lowe	b133dc5700	MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari	2017-09-22 15:21:42 -05:00
Steve Loughran	2d2d97fa7d	HADOOP-14553. Add (parallelized) integration tests to hadoop-azure Contributed by Steve Loughran	2017-09-15 17:03:01 +01:00
Aaron Fabbri	49467165a5	HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran.	2017-09-14 14:10:48 -07:00
ChenSammi	e0b3c644e1	HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen	2017-09-14 09:15:29 +08:00
John Zhuge	a4661850c1	HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria.	2017-09-08 11:51:03 -07:00
Steve Loughran	5bbca80428	HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A. Contributed by Aaron Fabbri	2017-09-08 12:07:02 +01:00
Steve Loughran	83449abb40	HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. Contributed by Yonger.	2017-09-07 19:55:56 +01:00
Steve Loughran	13eda50003	HADOOP-14520. WASB: Block compaction for Azure Block Blobs. Contributed by Georgi Chalakov	2017-09-07 18:35:03 +01:00
Xiaoyu Yao	63720ef574	HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin.	2017-09-05 23:34:55 -07:00
John Zhuge	50506e90a8	HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge.	2017-09-05 23:26:57 -07:00
Steve Loughran	792eff9ea7	HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. Contributed by Sivaguru Sankaridurg	2017-09-05 14:16:57 +01:00
Andrew Wang	0d419c984f	Preparing for 3.1.0 development	2017-09-01 11:53:48 -07:00
Allen Wittenauer	a3fee475f7	HADOOP-14414. Calling maven-site-plugin directly for docs profile is unnecessary (Andras Bokor via aw)	2017-09-01 08:38:30 -07:00
Steve Loughran	621b43e254	HADOOP-13345 HS3Guard: Improved Consistency for S3A. Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu, Sean Mackrory, Steve Loughran and others.	2017-09-01 14:13:41 +01:00
John Zhuge	7ecc6dbed6	HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria.	2017-08-31 21:13:22 -07:00
Allen Wittenauer	71bbb86d69	HADOOP-14670. Increase minimum cmake version for all platforms Signed-off-by: Chris Douglas <cdouglas@apache.org>	2017-08-30 21:47:24 -07:00
Haibo Chen	d04f85f387	MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen)	2017-08-30 10:07:48 -07:00
Yufei Gu	26fafc3597	YARN-7077. TestAMSimulator and TestNMSimulator fail. (Contributed by Akira Ajisaka via Yufei Gu)	2017-08-29 16:33:51 -07:00
Steve Loughran	9374f38820	HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory Contributed by Esfandiar Manii and Thomas Marquardt.	2017-08-29 19:03:44 +01:00
Steve Loughran	021974f4cb	HADOOP-14802. Add support for using container saskeys for all accesses. Contributed by Sivaguru Sankaridurg	2017-08-29 19:02:43 +01:00
Andrew Wang	f29a0fc288	HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou.	2017-08-25 14:14:23 -07:00
Arpit Agarwal	8b7cbe3840	HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar.	2017-08-25 09:55:46 -07:00
Steve Loughran	27ab5f7385	HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator. Contributed by Genmao Yu	2017-08-22 11:55:48 +01:00
Kai Zheng	267e19a09f	HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu	2017-08-21 13:36:28 +08:00
Arun Suresh	8410d862d3	YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh)	2017-08-20 07:54:09 -07:00
Steve Loughran	c6b4e656b7	HADOOP-14769. WASB: delete recursive should not fail if a file is deleted. Contributed by Thomas Marquardt	2017-08-18 14:13:40 +01:00
Andrew Wang	dd7916d3cd	HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas.	2017-08-17 09:35:36 -07:00
Chris Douglas	645a8f2a4d	HADOOP-14726. Mark FileStatus::isDir as final	2017-08-14 21:57:20 -07:00
John Zhuge	7769e96149	HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria.	2017-08-13 00:22:34 -07:00
Allen Wittenauer	4222c97108	HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw)	2017-08-11 09:25:56 -07:00
Steve Loughran	54356b1e83	HADOOP-14183. Remove service loader config file for wasb fs. Contributed by Esfandiar Manii.	2017-08-10 16:46:33 +01:00
Steve Loughran	f4e1aa0508	HADOOP-14715. TestWasbRemoteCallHelper failing. Contributed by Esfandiar Manii.	2017-08-08 23:37:47 +01:00
Chris Douglas	55a181f845	HADOOP-14730. Support protobuf FileStatus in AdlFileSystem.	2017-08-07 21:31:28 -07:00
Steve Loughran	d91b7a8451	HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek. Contributed by Thomas Marquardt	2017-08-06 20:19:23 +01:00
Steve Loughran	7fc324aabd	HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws. Contributed by Steve Loughran (cherry picked from commit ced547d5f0dbea571cbc472c5f55fe89d5900a6f)	2017-08-04 11:09:08 +01:00
Sean Mackrory	1a1bf6b7d0	HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer.	2017-08-02 12:25:05 -06:00
Steve Loughran	778d4edd9a	HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded. Contributed by Thomas Marquardt.	2017-08-01 21:36:28 +01:00
John Zhuge	f735ad1b67	HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge.	2017-07-28 09:46:54 -07:00
Jitendra Pandey	a92bf39e23	HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt.	2017-07-25 16:26:48 -07:00
Jitendra Pandey	f2921e51f0	HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov.	2017-07-24 23:01:01 -07:00
Ravi Prakash	94ca52ae9e	HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash.	2017-07-24 16:01:45 -07:00
Wei-Chiu Chuang	44350fdf49	HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default). Contributed by Bharat Viswanadham.	2017-07-20 18:23:13 -07:00
Haibo Chen	0ba8cda135	YARN-6685. Add job count in to SLS JSON input format. (Yufei Gu via Haibo Chen)	2017-07-20 08:15:46 -07:00
Jitendra Pandey	2843c688bc	HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg.	2017-07-19 00:13:06 -07:00
Jitendra Pandey	b0e78ae085	HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak.	2017-07-17 02:27:55 -07:00
Steve Loughran	7d272ea124	HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb. Contributed by Varada Hemeswari (cherry picked from commit 1e69e5260351effc8077d1bdc397cec57cf1ff1b)	2017-07-12 10:37:39 +01:00
Steve Loughran	d670c3a4da	HADOOP-14535 wasb: implement high-performance random access and seek of block blobs. Contributed by Thomas Marquardt	2017-07-11 21:34:27 +01:00
Haibo Chen	147df300bf	YARN-5067 Support specifying resources for AM containers in SLS. (Yufei Gu via Haibo Chen)	2017-06-30 17:03:44 -07:00
Mingliang Liu	38996fdcf0	HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak	2017-06-30 16:53:48 -07:00
Andrew Wang	af2773f609	Updating version for 3.0.0-beta1 development	2017-06-29 17:57:40 -07:00
Mingliang Liu	72993b33b7	HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc	2017-06-29 17:07:52 -07:00
Mingliang Liu	b08cc97396	HADOOP-14601. Azure: Reuse ObjectMapper. Contributed by Mingliang Liu	2017-06-29 09:22:00 -07:00
Allen Wittenauer	20ba86d66a	MAPREDUCE-6536. hadoop-pipes doesn't use maven properties for openssl Signed-off-by: Ravi Prakash <raviprak@apache.org>	2017-06-28 17:33:44 -07:00
Mingliang Liu	990aa34de2	HADOOP-14609. NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code. Contributed by Steve Loughran	2017-06-28 14:18:59 -07:00
Steve Loughran	ee243e5289	HADOOP-14190. Add more on S3 regions to the s3a documentation. Contributed by Steve Loughran	2017-06-28 10:22:13 +01:00
Mingliang Liu	7e031c2c18	HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas	2017-06-27 17:32:07 -07:00
Mingliang Liu	686a634f01	HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran	2017-06-27 16:48:47 -07:00
Mingliang Liu	bc4dfe9c9c	HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu	2017-06-27 13:19:14 -07:00
Steve Loughran	48f4a229a9	HADOOP-14461 Azure: handle failure gracefully in case of missing account access key. Contributed by Mingliang Liu.	2017-06-26 13:39:47 +01:00
Akira Ajisaka	a594f96fb1	HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he.	2017-06-26 16:41:00 +09:00
Steve Loughran	c6bd73c6c5	HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations. Contributed by Thomas.	2017-06-23 21:38:41 +01:00
Jason Lowe	dd65eea74b	HADOOP-8143. Change distcp to have -pb on by default. Contributed by Mithun Radhakrishnan	2017-06-20 09:53:47 -05:00
Akira Ajisaka	3369540653	HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools.	2017-06-20 13:18:26 +09:00
Mingliang Liu	f214a9961f	HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran	2017-06-15 11:16:16 -07:00
Mingliang Liu	325163f23f	HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu	2017-06-09 13:07:11 -07:00
Mingliang Liu	df35e4cc3b	HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan	2017-06-07 13:54:52 -07:00
Carlo Curino	b65100c14b	YARN-6547. Enhance SLS-based tests leveraging invariant checker.	2017-06-06 17:19:15 -07:00
Mingliang Liu	536f057158	HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu	2017-06-06 11:09:28 -07:00
Mingliang Liu	6b5285bbcb	HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu	2017-06-06 11:06:49 -07:00
Mingliang Liu	5fd9742c83	HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan	2017-06-05 15:56:43 -07:00
Mingliang Liu	6aeda55bb8	HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu	2017-06-05 11:26:56 -07:00
Mingliang Liu	ece33208b8	HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu	2017-06-01 11:52:11 -07:00
Akira Ajisaka	6e6e96583f	HADOOP-14466. Remove useless document from TestAliyunOSSFileSystemContract.java. Contributed by Chen Liang.	2017-06-01 13:08:01 +09:00
Akira Ajisaka	07e60f85d8	HADOOP-14458. Add missing imports to TestAliyunOSSFileSystemContract.java. Contributed by Mingliang Liu.	2017-05-30 15:11:10 +09:00
John Zhuge	6c6a7a5962	HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge.	2017-05-28 22:25:00 -07:00
Mingliang Liu	89bb8bfe58	HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari	2017-05-26 17:54:00 -07:00
Andrew Wang	16ad896d5c	Update maven version for 3.0.0-alpha4 development	2017-05-26 14:09:44 -07:00
Akira Ajisaka	6a52b5e144	HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu.	2017-05-25 15:36:44 +09:00
Sunil G	b6f66b0da1	YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.	2017-05-22 14:10:06 +05:30
Yufei Gu	fcbdecc54d	YARN-6111. Rumen input does't work in SLS. Contributed by Yufei Gu.	2017-05-21 21:04:06 -07:00
Mingliang Liu	6672810eea	HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran	2017-05-19 11:51:43 -07:00
Yongjun Zhang	b4adc8392c	HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang)	2017-05-18 15:35:22 -07:00
Sunil G	40e6a85d25	YARN-6560. SLS doesn't honor node total resource specified in sls-runner.xml. Contributed by Wangda Tan.	2017-05-18 22:49:42 +05:30
Steve Loughran	ba70225cf6	HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries. Contributed by Steve Loughran. (cherry picked from commit 2ac5aab8d725f761a9f9723471a4426f6b5d78c4)	2017-05-18 15:44:39 +01:00
Akira Ajisaka	d4aa9e3c69	HADOOP-14415. Use java.lang.AssertionError instead of junit.framework.AssertionFailedError. Contributed by Chen Liang.	2017-05-17 15:38:28 -04:00
Robert Kanter	101852ca11	YARN-6535. Program needs to exit when SLS finishes. (yufeigu via rkanter)	2017-05-16 17:52:17 -07:00
Mingliang Liu	b415c6fe74	HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger. Contributed by Sivaguru Sankaridurg	2017-05-16 11:22:32 -07:00
Akira Ajisaka	0d5c8ed8e0	HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor.	2017-05-11 16:37:32 -05:00
Mingliang Liu	eed731496f	HADOOP-14361. Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently. Contributed by Santhosh G Nayak	2017-05-10 15:05:11 -07:00
Carlo Curino	5cb6e3e082	YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations.	2017-05-10 14:49:25 -07:00
Akira Ajisaka	2ba9903932	HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang.	2017-05-10 05:57:12 -05:00
Akira Ajisaka	c60164fb60	HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong.	2017-05-09 05:37:43 -05:00
Andrew Wang	54fd0e44b7	HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy.	2017-05-08 21:59:49 -07:00
Andrew Wang	8065129d87	HADOOP-14298. TestHadoopArchiveLogsRunner fails. Contribute dby Akira Ajisaka.	2017-05-07 13:59:15 -07:00
Steve Loughran	4e6bbd049d	HADOOP-14382 Remove usages of MoreObjects.toStringHelper. Contributed by Andrew Wang	2017-05-05 13:03:09 +01:00
Robert Kanter	3082552b3b	YARN-6522. Make SLS JSON input file format simple and scalable (yufeigu via rkanter)	2017-05-04 17:21:46 -07:00
Mingliang Liu	83dded556d	HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran	2017-05-03 11:22:44 -07:00
Andrew Wang	b062b323b7	HADOOP-14281. Fix TestKafkaMetrics#testPutMetrics. Contributed by Alison Yu.	2017-05-02 11:49:19 -07:00
Akira Ajisaka	62579b69a0	HADOOP-11614. Remove httpclient dependency from hadoop-openstack. Contributed by Akira Ajisaka, Brahma Reddy Battula, and Steve Loughran.	2017-04-27 16:57:08 +09:00
Mingliang Liu	8b5f2c372e	HADOOP-14351. Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache. Contributed by Santhosh G Nayak	2017-04-26 13:47:18 -07:00
Mingliang Liu	e4c32bccde	HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu	2017-04-25 20:54:56 -07:00
Robert Kanter	475f933b41	YARN-6423. Queue metrics doesn't work for Fair Scheduler in SLS (yufeigu via rkanter)	2017-04-25 16:26:09 -07:00
Steve Loughran	5f934f8386	HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus. Contributed by Steve Moist.	2017-04-24 20:33:19 +01:00
Wangda Tan	de69d6e811	YARN-6363. Extending SLS: Synthetic Load Generator. (Carlo Curino via wangda)	2017-04-20 21:54:30 -07:00
Mingliang Liu	667966c13c	HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran	2017-04-20 17:13:36 -07:00
Andrew Wang	7e075a50e3	YARN-6365. Get static SLS html resources from classpath. Contributed by Yufei Gu.	2017-04-19 11:08:59 -07:00
Steve Loughran	6b015d00c9	HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. Contributed by Steve Loughran	2017-04-19 10:21:44 +01:00
Mingliang Liu	0cab57223e	HADOOP-14274. Azure: Simplify Ranger-WASB policy model. Contributed by Sivaguru Sankaridurg	2017-04-12 16:07:10 -07:00
Mingliang Liu	b053fdc547	HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu	2017-04-12 14:30:34 -07:00
Chris Nauroth	b8305e6d06	HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. Contributed by Mingliang Liu.	2017-04-12 10:02:13 -07:00
John Zhuge	56e81f2a20	HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge. Signed-off-by: John Zhuge <jzhuge@apache.org>	2017-04-07 12:04:44 -07:00
Andrew Wang	7eea8d83fa	HADOOP-14280. Fix compilation of TestKafkaMetrics.	2017-04-05 15:12:03 -07:00
Mingliang Liu	56ab02eed9	HADOOP-14273. Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag. Contributed by Santhosh G Nayak	2017-04-04 11:10:28 -07:00
Mingliang Liu	27a44b60c1	HADOOP-14272. Azure: WasbRemoteCallHelper should use String equals for comparison. Contributed by Santhosh G Nayak	2017-04-04 11:03:59 -07:00
Chris Douglas	6eba79232f	HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang	2017-04-03 20:13:14 -07:00
Mingliang Liu	5faa949b78	HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka	2017-04-03 11:07:14 -07:00
Mingliang Liu	26172a94d6	HADOOP-14267. Make DistCpOptions immutable. Contributed by Mingliang Liu	2017-03-31 20:04:26 -07:00
Yongjun Zhang	bf3fb585aa	HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li.	2017-03-30 17:38:56 -07:00
Yongjun Zhang	144f1cf765	Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen." This reverts commit `064c8b25ec`.	2017-03-30 17:38:18 -07:00
Yongjun Zhang	064c8b25ec	HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen.	2017-03-30 17:01:15 -07:00
Wei-Chiu Chuang	8c591b8d19	HDFS-10974. Document replication factor for EC files. Contributed by Yiqun Lin.	2017-03-30 11:16:05 -07:00
Akira Ajisaka	0d053eeb30	HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula.	2017-03-30 18:11:50 +09:00
Robert Kanter	6a5516c238	YARN-5654. Not be able to run SLS with FairScheduler (yufeigu via rkanter)	2017-03-29 16:18:13 -07:00
Andrew Wang	4966a6e26e	HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy.	2017-03-29 14:37:21 -07:00
Andrew Wang	0e6f8e4bc6	HDFS-10971. Distcp should not copy replication factor if source file is erasure coded. Contributed by Manoj Govindassamy.	2017-03-28 22:14:03 -07:00
John Zhuge	253e3e78ab	HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge.	2017-03-27 23:05:57 -07:00
John Zhuge	d1b7439b48	HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge.	2017-03-24 13:37:04 -07:00
Andrew Wang	52b00600df	HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy.	2017-03-24 11:44:46 -07:00
Chris Douglas	c55195588f	HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge	2017-03-23 12:17:27 -07:00
Chris Douglas	03955612b7	HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria	2017-03-23 12:13:23 -07:00
Mingliang Liu	2e30aa72e0	HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu	2017-03-23 11:33:29 -07:00
Steve Loughran	a5a4867f3b	HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject(). Contributed by Yuanbo Liu (cherry picked from commit 20878d052cebc715c2494a97362fdff08885a77f)	2017-03-23 12:54:54 +00:00
Robert Kanter	cc938e99ec	YARN-6326. Shouldn't use AppAttemptIds to fetch applications while AM Simulator tracks app in SLS (yufeigu via rkanter)	2017-03-21 15:21:11 -07:00
Steve Loughran	2841666f1f	HADOOP-14204 S3A multipart commit failing, "UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort". Contributed by Steve Loughran (cherry picked from commit 595501c9ddacd5fdb740ac1bf9bfeb0ffbbd6c0e)	2017-03-21 13:15:15 +00:00
Mingliang Liu	8e15e24059	HADOOP-13945. Azure: Add Kerberos and Delegation token support to WASB client. Contributed by Santhosh G Nayak Change-Id: I39fb0a3b0491bf2160571366939a0502a0045429	2017-03-20 16:31:24 -07:00
Arpit Agarwal	6c399a88e9	HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran.	2017-03-20 11:29:23 -07:00
Mingliang Liu	86035c1644	HADOOP-14192. Aliyun OSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu	2017-03-17 11:01:00 -07:00
Mingliang Liu	fa67a96d7b	HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu	2017-03-14 14:38:21 -07:00
Mingliang Liu	e6cda5819b	Revert "HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu" This reverts commit `b8c69557b7`.	2017-03-14 13:03:42 -07:00
Mingliang Liu	b8c69557b7	HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu	2017-03-13 14:15:02 -07:00
John Zhuge	5a40bafdae	HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge. Change-Id: I5dc6f885816b8834f718874542dfa373458b0333	2017-03-13 08:11:25 -07:00
Steve Loughran	092ec39fb9	HADOOP-14111 cut some obsolete, ignored s3 tests in TestS3Credentials. Contributed by Yuanbo Liu	2017-03-10 17:43:22 +00:00
Mingliang Liu	881ec4d97b	HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu	2017-03-10 00:16:09 -08:00
John Zhuge	c5ee7fded4	HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge. Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e	2017-03-09 18:30:17 -08:00
John Zhuge	98142d2f72	Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" This reverts commit `53a12fa721`.	2017-03-08 09:54:22 -08:00
Mingliang Liu	28daaf0eb2	HADOOP-14150. Implement getHomeDirectory() method in NativeAzureFileSystem. Contributed by Santhosh G Nayak	2017-03-07 14:55:52 -08:00
Robert Kanter	1598fd3b79	YARN-6275. Fail to show real-time tracking charts in SLS (yufeigu via rkanter)	2017-03-07 13:47:52 -08:00
Akira Ajisaka	f597f4c43e	HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri.	2017-03-07 15:14:55 +09:00
Mingliang Liu	686823529b	HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth	2017-03-06 17:16:36 -08:00
Mingliang Liu	52d7d5aa1a	Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth" This reverts commit `6b7cd62b8c`.	2017-03-06 17:10:11 -08:00
Mingliang Liu	c571cda5c7	HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA	2017-03-06 17:00:13 -08:00
Mingliang Liu	6b7cd62b8c	HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth	2017-03-03 17:09:12 -08:00
Akira Ajisaka	490abfb10f	HADOOP-14057. Fix package.html to compile with Java 9.	2017-03-04 00:25:22 +09:00
Steve Loughran	a97833e0ed	HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. Contributed by Steve Loughran (cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)	2017-03-02 21:44:30 +00:00
Allen Wittenauer	989bd56b9f	HADOOP-12556. KafkaSink jar files are created but not copied to target dist (Babak Behzad via aw)	2017-02-28 16:15:40 -08:00
Wangda Tan	eac6b4c35c	YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda)	2017-02-28 10:35:50 -08:00
Akira Ajisaka	258342e76c	HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR.	2017-02-28 13:47:44 +09:00
Yongjun Zhang	d235dcdf0b	HADOOP-14127. Add log4j configuration to enable logging in hadoop-distcp's tests. (Xiao Chen via Yongjun Zhang)	2017-02-27 20:42:13 -08:00
Steve Loughran	dab00da19f	HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures. Contributed by Steve Loughran. (cherry picked from commit 29fe5af017b945d8750c074ca39031b5b777eddd)	2017-02-25 15:35:19 +00:00
Sunil G	b32ffa2753	YARN-4779. Fix AM container allocation logic in SLS. Contributed by Wangda Tan.	2017-02-24 21:39:25 +05:30
Steve Loughran	e60c6543d5	HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran	2017-02-24 13:24:59 +00:00
Steve Loughran	9c22a91662	HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory. (cherry picked from commit ff87ca84418a710c6dc884fe8c70947fcc6489d5)	2017-02-24 10:41:36 +00:00
Mingliang Liu	82607fce39	HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari.	2017-02-23 13:48:44 -08:00
Mingliang Liu	a4d4a23785	HADOOP-13321. Deprecate FileSystem APIs that promote inefficient call patterns. Contributed by Chris Nauroth and Mingliang Liu	2017-02-23 12:55:40 -08:00
Mingliang Liu	0013090fb4	HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu	2017-02-22 13:34:20 -08:00
Steve Loughran	4f4250fbcc	HADOOP-14099 Split S3 testing documentation out into its own file. Contributed by Steve Loughran.	2017-02-22 11:43:48 +00:00
Mingliang Liu	924def7854	HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane	2017-02-21 13:44:42 -08:00
Steve Loughran	2158496f6b	HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory. (cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)	2017-02-21 18:28:49 +00:00
Steve Loughran	8035749c26	HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan	2017-02-20 16:21:46 +00:00
Steve Loughran	3a2e30fa9f	HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge (cherry picked from commit b1c1f05b1dc997906390d653dfafb4f0d7e193c4)	2017-02-18 18:17:11 +00:00
Mingliang Liu	658702efff	HADOOP-14040. Use shaded aws-sdk uber-JAR 1.11.86. Contributed by Steve Loughran and Sean Mackrory	2017-02-16 16:51:03 -08:00
Mingliang Liu	bdad8b7b97	HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran	2017-02-16 16:41:31 -08:00
Mingliang Liu	f432999025	HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane	2017-02-16 15:14:25 -08:00
Kai Zheng	cd3e59a3dc	HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu	2017-02-15 16:34:30 +08:00
Akira Ajisaka	b9f8491252	HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin.	2017-02-15 01:45:56 +09:00
Lei Xu	71c23c9fc9	HADOOP-13929. ADLS connector should not check in contract-test-options.xml. (John Zhuge via lei)	2017-02-13 13:33:13 -08:00
Kai Zheng	01be4503c3	HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui	2017-02-13 13:18:20 +08:00
Lei Xu	839b690ed5	HADOOP-13075. Add support for SSE-KMS and SSE-C in s3a filesystem. (Steve Moist via lei)	2017-02-11 13:59:03 -08:00
Arpit Agarwal	2b7a7bbe0f	HADOOP-14055. SwiftRestClient includes pass length in exception if auth fails. Contributed by Marcell Hegedus.	2017-02-10 10:09:31 -08:00
Kai Zheng	5b151290ae	HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation `deleteDirs`. Contributed by Genmao Yu	2017-02-10 14:53:18 +08:00
Kai Zheng	a8a594b4c8	HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui	2017-02-09 18:00:22 +08:00
Akira Ajisaka	98c35bb1a4	HADOOP-13858. TestGridmixMemoryEmulation and TestResourceUsageEmulators fail on the environment other than Linux or Windows.	2017-01-21 17:31:20 +09:00
Lei Xu	ccf2d66244	HADOOP-13962. Update ADLS SDK to 2.1.4. (John Zhuge via lei)	2017-01-21 10:52:40 +08:00
Lei Xu	e015b56319	HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei)	2017-01-20 14:48:04 +08:00
Andrew Wang	5d8b80ea9b	Preparing for 3.0.0-alpha3 development	2017-01-19 15:50:07 -08:00
Steve Loughran	ed33ce11dd	HADOOP-13496. Include file lengths in Mismatch in length error for distcp. Contributed by Ted Yu (cherry picked from commit 77401bd5fcca5127c9908156971eeec468371f47)	2017-01-19 11:25:40 +00:00
Steve Loughran	e648b6e138	HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran	2017-01-11 17:25:15 +00:00
Wangda Tan	287d3d6804	YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)	2017-01-09 08:40:39 -08:00
Chris Nauroth	4c8f9e1302	HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore.	2017-01-05 15:04:47 -08:00
Wangda Tan	0a55bd841e	YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)	2017-01-05 10:31:05 -08:00
Wangda Tan	5ed63e3e9d	YARN-4899. Queue metrics of SLS capacity scheduler only activated after app submit to the queue. (Jonathan Hung via wangda)	2017-01-04 19:17:11 -08:00
Akira Ajisaka	01d31fe938	HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang.	2017-01-04 14:10:36 +09:00
Mingliang Liu	c6a5b689db	HADOOP-13942. Build failure due to errors of javadoc build in hadoop-azure. Contributed by Kai Sasaki	2016-12-29 16:24:06 -08:00
Mingliang Liu	e92a77099b	HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by Dushyanth	2016-12-22 20:15:18 -08:00
Akira Ajisaka	ae401539ea	YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor.	2016-12-22 14:32:24 +09:00
Wangda Tan	3bcfe3a9de	YARN-5706. Fail to launch SLSRunner due to NPE. (Kai Sasaki via wangda)	2016-12-20 11:03:29 -08:00
Mingliang Liu	f92913c35b	HADOOP-13831. Correct check for error code to detect Azure Storage Throttling and provide retries. Contributed by Gaurav Kanade	2016-12-15 12:35:08 -08:00
Mingliang Liu	ef34bf2bb9	HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane	2016-12-13 10:34:32 -08:00
Mingliang Liu	c6a3923245	HADOOP-13871. ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful. Contributed by Steve Loughran	2016-12-12 14:55:34 -08:00
Mingliang Liu	15dd1f3381	HADOOP-13675. Bug in return value for delete() calls in WASB. Contributed by Dushyanth	2016-12-05 12:04:07 -08:00
Mingliang Liu	4113ec5fa5	HADOOP-13257. Improve Azure Data Lake contract tests. Contributed by Vishwajeet Dusane	2016-12-02 15:54:57 -08:00
Mingliang Liu	51211a7d7a	HADOOP-13855. Fix a couple of the s3a statistic names to be consistent with the rest. Contributed by Steve Loughran	2016-12-02 14:01:42 -08:00
Mingliang Liu	2ff84a0040	HADOOP-13857. S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs. Contributed by Steve Loughran	2016-12-02 13:36:04 -08:00

... 10 11 12 13 14 ...

1717 Commits