big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,889 Commits 2 Branches 2 Tags 264 MiB
45382bf047
Commit Graph

3323 Commits

Author SHA1 Message Date
Xiao Chen
5d182949ba HADOOP-13597. Switch KMS from Tomcat to Jetty. Contributed by John Zhuge. 2017-01-05 17:21:57 -08:00
Jason Lowe
a0a2761621 HADOOP-13578. Add Codec for ZStandard Compression. Contributed by churro morales 2017-01-04 14:46:25 +00:00
Akira Ajisaka
01d31fe938 HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 2017-01-04 14:10:36 +09:00
Mingliang Liu
88731c731a Revert "HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran" 
This reverts commit 451efb08fe.
 2017-01-03 14:44:26 -08:00
Mingliang Liu
451efb08fe HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran 2017-01-03 13:08:38 -08:00
Robert Kanter
591fb15944 YARN-5529. Create new DiskValidator class with metrics (yufeigu via rkanter) 2017-01-03 12:13:32 -08:00
Brahma Reddy Battula
6938b67711 HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 2016-12-31 16:16:02 +05:30
Akira Ajisaka
0665c5f09a HADOOP-13943. TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass fails after HADOOP-13863. Contributed by Brahma Reddy Battula. 2016-12-28 01:39:30 +09:00
Daniel Templeton
ded2d08f33 HADOOP-13940. Document the missing envvars commands (Contributed by Yiqun Lin via Daniel Templeton) 2016-12-27 07:18:08 -08:00
Xiaoyu Yao
4e9029653d HDFS-10917. Collect peer performance statistics on DataNode. Contributed by Xiaobing Zhou. 2016-12-22 23:46:58 -08:00
Mingliang Liu
e92a77099b HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by Dushyanth 2016-12-22 20:15:18 -08:00
Akira Ajisaka
22befbd585 MAPREDUCE-6704. Update the documents to run MapReduce application. Contributed by Bibin A Chundatt. 2016-12-23 04:48:04 +09:00
Kai Zheng
56a13a6a59 HDFS-11216. Add remoteBytesRead counter metrics for erasure coding reconstruction task. Contributed by Sammi Chen 2016-12-22 14:18:54 +08:00
Akira Ajisaka
f6e2521eb2 HDFS-11261. Document missing NameNode metrics. Contributed by Yiqun Lin. 2016-12-21 14:17:47 +09:00
Jason Lowe
631f1daee3 HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger 2016-12-15 20:52:40 +00:00
Jason Lowe
169bfc0903 Revert "HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger" 
This reverts commit 9947aeb60c.
 2016-12-15 20:51:24 +00:00
Xiaoyu Yao
f5e0bd30fd HADOOP-13890. Maintain HTTP/host as SPNEGO SPN support and fix KerberosName parsing. Contributed by Xiaoyu Yao. 2016-12-14 13:45:21 -08:00
Arpit Agarwal
6ba9587d37 HDFS-10958. Add instrumentation hooks around Datanode disk IO. 2016-12-14 11:18:58 -08:00
Akira Ajisaka
72bff192cd HDFS-11204. Document the missing options of hdfs zkfc command. Contributed by Yiqun Lin. 2016-12-14 18:50:43 +09:00
Jason Lowe
9947aeb60c HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger 2016-12-13 22:55:09 +00:00
Steve Loughran
f66f61892a HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-12 13:58:27 +00:00
Wei-Chiu Chuang
b606e025f1 HADOOP-13824. FsShell can suppress the real error if no error message is present. Contributed by John Zhuge. 2016-12-09 15:22:21 -08:00
Brahma Reddy Battula
80b8023276 HADOOP-13883. Add description of -fs option in generic command usage. Contributed By Yiqun Lin. 2016-12-09 18:21:42 +05:30
Akira Ajisaka
7d8e440eee Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran" 
This reverts commit c265515725.
 2016-12-09 12:49:27 +09:00
Steve Loughran
c265515725 HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-08 17:57:15 +00:00
Brahma Reddy Battula
0ef796174e HADOOP-13867. FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations. Contributed By Vinayakumar B. 2016-12-08 18:57:43 +05:30
Akira Ajisaka
deb4743b2c Revert "HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin." 
This reverts commit 0857641f62.
 2016-12-08 13:37:06 +09:00
Akira Ajisaka
0857641f62 HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin. 2016-12-08 13:34:45 +09:00
Ming Ma
c73e08a6da HDFS-10206. Datanodes not sorted properly by distance when the reader isn't a datanode. (Nandakumar via mingma) 2016-12-07 08:26:09 -08:00
Karthik Kambatla
c8d0a049b0 HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha) 2016-12-06 14:10:58 -08:00
Andrew Wang
f53b481ffe HDFS-11172. Support an erasure coding policy using RS 10 + 4. Contributed by Wei Zhou. 2016-12-06 12:49:38 -08:00
Xiao Chen
79d90b810c HADOOP-13827. Add reencryptEncryptedKey interface to KMS. 2016-12-06 12:04:04 -08:00
Andrew Wang
7b988e8899 HADOOP-13861. Spelling errors in logging and exceptions for code. Contributed by Grant Sohn. 2016-12-05 23:18:18 -08:00
Akira Ajisaka
b2a3d6c519 HADOOP-13835. Move Google Test Framework code from mapreduce to hadoop-common. Contributed by Varun Vasudev. 2016-12-06 14:01:47 +09:00
Xiao Chen
a2b5d60220 HADOOP-13864. KMS should not require truststore password. Contributed by Mike Yoder. 2016-12-05 17:36:00 -08:00
Xiao Chen
291df5c7fb HADOOP-13847. KMSWebApp should close KeyProviderCryptoExtension. Contributed by John Zhuge. 2016-12-05 09:35:17 -08:00
Mingliang Liu
19f373a46b HDFS-11132. Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files. Contributed by Vishwajeet Dusane 2016-12-01 12:54:28 -08:00
Andrew Wang
1f7613be95 HADOOP-13840. Implement getUsed() for ViewFileSystem. Contributed by Manoj Govindassamy. 2016-11-30 17:55:12 -08:00
Kai Zheng
51e6c1cc3f HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen 2016-11-30 15:52:56 +08:00
Kai Zheng
cfd8076f81 Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen" 
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
 2016-11-30 15:44:52 +08:00
Akira Ajisaka
67d9f2808e HADOOP-13706. Update jackson from 1.9.13 to 2.x in hadoop-common-project. 2016-11-29 14:07:19 +09:00
Robert Kanter
47ca9e26fb HADOOP-13838. KMSTokenRenewer should close providers (xiaochen via rkanter) 2016-11-28 18:08:09 -08:00
Andrew Wang
a2b1ff0257 HADOOP-13828. Implement getFileChecksum(path, length) for ViewFileSystem. Contributed by Manoj Govindassamy. 2016-11-28 11:54:43 -08:00
Kai Zheng
5d5614f847 HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen 2016-11-28 14:34:44 +08:00
Harsh J
07825f2b49 HADOOP-1381. The distance between sync blocks in SequenceFiles should be configurable rather than hard coded to 2000 bytes. Contributed by Harsh J. 2016-11-25 22:22:23 +05:30
Brahma Reddy Battula
ee3d437a33 HADOOP-13820. Replace ugi.getUsername() with ugi.getShortUserName() in viewFS. Contributed By Brahma Reddy Battula. 2016-11-25 19:54:50 +05:30
Steve Loughran
01665e456d HADOOP-10776 Open up already widely-used APIs for delegation-token fetching & renewal to ecosystem projects. Contributed by Vinod Kumar Vavilapalli 
Cherry picked from 24715cefe80fba531dd16a44ec1cead67c389eb1
 2016-11-24 17:54:10 +00:00
Steve Loughran
3df3fa396c HADOOP-13833 TestSymlinkHdfsFileSystem#testCreateLinkUsingPartQualPath2 fails after HADOOP-13605. Contributed by Brahma Reddy Battula. 2016-11-24 17:44:36 +00:00
Brahma Reddy Battula
13501053dd HADOOP-11603. Metric Snapshot log can be changed #MetricsSystemImpl.java since all the services will be initialized. Contributed By Brahma Reddy Battula. 2016-11-24 22:23:22 +05:30
Steve Loughran
abb9fa7fc6 HADOOP-13018. Make Kdiag check whether hadoop.token.files points to existent and valid files. Contributed by Ravi Prakash 2016-11-24 16:41:59 +00:00
Andrew Wang
c7a5f2906f HADOOP-13766. Fix a typo in the comments of RPC.getProtocolVersion. Contributed by Ethan Li. 2016-11-23 17:08:38 -08:00
Mingliang Liu
860d49aa6a HADOOP-13605. Clean up FileSystem javadocs, logging; improve diagnostics on FS load. Contributed by Steve Loughran 2016-11-23 16:42:27 -08:00
Andrew Wang
de4894936a HDFS-11368. Erasure Coding: Deprecate replication-related config keys. Contributed by Rakesh R. 2016-11-23 16:42:06 -08:00
Andrew Wang
dd98a8005a HDFS-11058. Implement 'hadoop fs -df' command for ViewFileSystem. Contributed by Manoj Govindassamy. 2016-11-23 16:40:39 -08:00
Jian He
3d94da1e00 HADOOP-11552. Allow handoff on the server side for RPC requests. Contributed by Siddharth Seth 2016-11-23 16:01:55 -08:00
Mingliang Liu
2bf9a15e8a HADOOP-13166. Add getFileStatus("/") test to AbstractContractGetFileStatusTest. Contributed by Steve Loughran 2016-11-22 14:27:23 -08:00
Lei Xu
3a09e5970d HDFS-11145. Implement getTrashRoot() for ViewFileSystem. (Manoj Govindassamy via lei) 2016-11-22 13:52:09 -08:00
Mingliang Liu
beb70fed4f HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran 
This closes #131
 2016-11-22 13:12:23 -08:00
Brahma Reddy Battula
afcf8d38e7 HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula. 2016-11-22 19:45:45 +05:30
Wei-Chiu Chuang
c0b1a44f6c HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory. 2016-11-17 22:48:35 -06:00
Mingliang Liu
ae8849fe37 HADOOP-11601. Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty files. Contributed by Steve Loughran 2016-11-17 14:11:38 -08:00
Kihwal Lee
bd3735554f HADOOP-13742. Expose NumOpenConnectionsPerUser as a metric. Brahma Reddy Battula. 2016-11-17 12:16:38 -06:00
Mingliang Liu
aab9737a05 HADOOP-13822. Use GlobalStorageStatistics.INSTANCE.reset() at FileSystem#clearStatistics(). Contribute by Brahma Reddy Battula 2016-11-16 20:08:48 -08:00
Mingliang Liu
5af572b644 HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 2016-11-15 10:57:00 -08:00
Mingliang Liu
04014c4c73 HADOOP-13810. Add a test to verify that Configuration handles &-encoded characters. Contributed by Steve Loughran 
This closes #158
 2016-11-14 19:38:10 -08:00
Zhe Zhang
ff0b99eafe HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 2016-11-14 11:05:29 -08:00
Chris Douglas
5c61ad2488 HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane 2016-11-11 11:15:07 -08:00
Yongjun Zhang
fd2f22adec HADOOP-13720. Add more info to the msgs printed in AbstractDelegationTokenSecretManager. Contributed by Yongjun Zhang. 2016-11-10 22:42:59 -08:00
Yongjun Zhang
470bdaa27a HADOOP-12718. Incorrect error message by fs -put local dir without permission. (John Zhuge via Yongjun Zhang) 2016-11-10 22:38:38 -08:00
Akira Ajisaka
8848a8a76c HDFS-11116. Fix javac warnings caused by deprecation of APIs in TestViewFsDefaultValue. Contributed by Yiqun Lin. 2016-11-11 13:45:30 +09:00
Xiao Chen
822ae88f7d HADOOP-13346. DelegationTokenAuthenticationHandler writes via closed writer. Contributed by Gregory Chanan and Hrishikesh Gadre. 2016-11-09 09:33:00 -08:00
Xiao Chen
367c3d4121 HADOOP-13590. Retry until TGT expires even if the UGI renewal thread encountered exception. 2016-11-09 09:07:12 -08:00
Akira Ajisaka
c074880096 HADOOP-13800. Remove unused HADOOP_AUDIT_LOGGER from hadoop-env.sh. Contributed by Yiqun Lin. 2016-11-09 22:02:40 +09:00
Zhe Zhang
77c13c3857 HADOOP-13782. Make MutableRates metrics thread-local write, aggregate-on-read. Contributed by Erik Krogen. 2016-11-08 16:07:36 -08:00
Mingliang Liu
2a65eb121e HADOOP-13802. Make generic options help more consistent, and aligned. Contributed by Grant Sohn 2016-11-08 15:40:22 -08:00
Subru Krishnan
29e3b3417c YARN-5833. Add validation to ensure default ports are unique in Configuration. (Konstantinos Karanasos via Subru). 2016-11-08 14:38:18 -08:00
Brahma Reddy Battula
dbb133ccfc HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed. Contributed by Yuanbo Liu. 2016-11-08 20:55:10 +05:30
Zhe Zhang
3dbad5d823 HADOOP-13804. MutableStat mean loses accuracy if add(long, long) is used. Contributed by Erik Krogen. 2016-11-07 16:08:10 -08:00
Steve Loughran
f76895573d HADOOP-13797 Remove hardcoded absolute path for ls. Contributed by Christine Koppelt 2016-11-07 12:36:10 +00:00
Xiao Chen
d8bab3dcb6 HDFS-10756. Expose getTrashRoot to HTTPFS and WebHDFS. Contributed by Yuanbo Liu. 2016-11-04 18:06:55 -07:00
Akira Ajisaka
de0132748d MAPREDUCE-6796. Remove unused properties from JTConfig.java. Contributed by Haibo Chen. 2016-11-05 03:49:19 +09:00
Mingliang Liu
abfc15d5ef HADOOP-13792. Stackoverflow for schemeless defaultFS with trailing slash. Contributed by John Zhuge 2016-11-04 10:46:27 -07:00
Xiao Chen
7154a20bcb HADOOP-12453. Support decoding KMS Delegation Token with its own Identifier. Contributed by Xiaoyu Yao. 2016-11-03 13:09:03 -07:00
Andrew Wang
7e521c5a49 HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory. 2016-11-02 19:11:05 -07:00
Daniel Templeton
0dc2a6a3a5 HADOOP-13667. Fix typing mistake of inline document in hadoop-metrics2.properties (Contributed by Rui Gao via Daniel Templeton) 2016-11-02 06:53:33 -07:00
Arpit Agarwal
1b6ecaf016 HADOOP-13738. DiskChecker should perform some disk IO. 2016-11-01 18:02:23 -07:00
Steve Loughran
a1761a841e HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi. 2016-10-31 20:54:46 +00:00
Wei-Chiu Chuang
112f04eb1b HDFS-11061. Update dfs -count -t command line help and documentation. Contributed by Yiqun Lin. 2016-10-28 15:33:27 -07:00
Jason Lowe
c017171da0 HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 2016-10-28 14:57:50 +00:00
Robert Kanter
5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Brahma Reddy Battula
0c837db8a8 HADOOP-13201. Print the directory paths when ViewFs denies the rename operation on internal dirs. Contributed by Tianyin Xiu 2016-10-27 19:46:41 +05:30
Masatake Iwasaki
0bdd263d82 HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran. 2016-10-27 15:46:59 +09:00
Andrew Wang
22ff0eff4d HADOOP-8299. ViewFileSystem link slash mount point crashes with IndexOutOfBoundsException. Contributed by Manoj Govindassamy. 2016-10-26 14:25:03 -07:00
Chris Nauroth
1f8490a5ba HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth. 2016-10-26 08:55:42 -07:00
Chris Nauroth
9cad3e2350 HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran. 2016-10-26 08:27:26 -07:00
Wei-Chiu Chuang
d88dca844a HADOOP-11798. Native raw erasure coder in XOR codes. Contributed by SammiChen. 2016-10-25 16:53:53 -07:00
Andrew Wang
084bdab156 HDFS-5684. Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as VisibleForTesting. Contributed by Manoj Govindassamy. 2016-10-25 15:35:39 -07:00
Chris Nauroth
309a43925c HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth. 2016-10-25 09:03:03 -07:00
Chris Nauroth
d8fa1cfa67 HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth. 2016-10-24 21:22:34 -07:00
Akira Ajisaka
dc3272bfe8 HDFS-11046. Duplicate '-' in the daemon log name. 2016-10-25 10:07:43 +09:00
Xiaoyu Yao
d0a347984d HADOOP-13749. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 2016-10-23 10:58:36 -07:00
Xiaoyu Yao
da901b6c14 Revert "HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao." 
This reverts commit be72372248.
 2016-10-23 08:25:37 -07:00
Wei-Chiu Chuang
25438526ed HDFS-11033. Add documents for native raw erasure coder in XOR codes. Contributed by SammiChen. 2016-10-21 16:20:07 -07:00
Xiaoyu Yao
be72372248 HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 2016-10-21 14:23:02 -07:00
Chris Douglas
ae8bccd509 HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du 2016-10-21 12:59:54 -07:00
Anu Engineer
262827cf75 HADOOP-13737. Cleanup DiskChecker interface. Contributed by Arpit Agarwal. 2016-10-20 13:35:26 -07:00
Anu Engineer
3fbf4cd5da HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-20 12:33:58 -07:00
Brahma Reddy Battula
f872c6bc03 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 2016-10-20 21:47:17 +05:30
Benoy Antony
4bca385241 HADOOP-12082 Support multiple authentication schemes via AuthenticationFilter 2016-10-18 18:32:01 -07:00
Xiao Chen
efdf810cf9 HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge. 2016-10-18 18:18:43 -07:00
Steve Loughran
6c348c5691 HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran 2016-10-18 21:16:02 +01:00
Kai Zheng
c023c74886 HADOOP-13061. Refactor erasure coders. Contributed by Kai Sasaki 2016-10-18 12:02:53 +06:00
Akira Ajisaka
bedfec0c10 HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini. 2016-10-18 15:00:44 +09:00
Andrew Wang
b671ee6846 HDFS-11013. Correct typos in native erasure coding dump code. Contributed by László Bence Nagy. 2016-10-17 14:14:50 -07:00
Andrew Wang
987ee51141 HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei. 2016-10-17 13:25:58 -07:00
Andrew Wang
0f4afc8100 HADOOP-13722. Code cleanup -- ViewFileSystem and InodeTree. Contributed by Manoj Govindassamy. 2016-10-17 13:15:11 -07:00
Wei-Chiu Chuang
ed9fcbec54 HADOOP-13661. Upgrade HTrace version. Contributed by Sean Mackrory. 2016-10-17 05:04:49 -07:00
Andrew Wang
aee538be6c HADOOP-13721. Remove stale method ViewFileSystem#getTrashCanLocation. Contributed by Manoj Govindassamy. 2016-10-14 14:08:31 -07:00
Xiaoyu Yao
dbe663d524 HADOOP-13686. Adding additional unit test for Trash (I). Contributed by Weiwei Yang. 2016-10-13 23:05:16 -07:00
Arpit Agarwal
008122b3c9 HADOOP-13710. Supress CachingGetSpaceUsed from logging interrupted exception stacktrace. Contributed by Hanisha Koneru. 2016-10-13 11:37:03 -07:00
Andrew Wang
12d739a34b HADOOP-13700. Remove unthrown IOException from TrashPolicy#initialize and #getInstance signatures. 2016-10-12 15:19:52 -07:00
Kihwal Lee
85cd06f663 HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 2016-10-12 15:11:42 -05:00
Xiao Chen
b84c4891f9 HADOOP-13698. Document caveat for KeyShell when underlying KeyProvider does not delete a key. 2016-10-11 17:05:00 -07:00
Andrew Wang
8a09bf7c19 HADOOP-13705. Revert HADOOP-13534 Remove unused TrashPolicy#getInstance and initialize code. 2016-10-11 13:46:07 -07:00
Wei-Chiu Chuang
4b32b1420d HADOOP-13684. Snappy may complain Hadoop is built without snappy if libhadoop is not found. Contributed by Wei-Chiu Chuang. 2016-10-11 13:21:33 -07:00
Mingliang Liu
2fb392a587 HADOOP-13697. LogLevel#main should not throw exception if no arguments. Contributed by Mingliang Liu 2016-10-11 10:57:08 -07:00
Mingliang Liu
c874fa914d HDFS-10985. o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertions. Contributed by Mingliang Liu 2016-10-10 13:33:07 -07:00
Andrew Wang
03060075c5 HADOOP-13699. Configuration does not substitute multiple references to the same var. 2016-10-10 12:19:26 -07:00
Kai Zheng
3d59b18d49 HADOOP-13641. Update UGI#spawnAutoRenewalThreadForUserCreds to reduce indentation. Contributed by Huafeng Wang 2016-10-09 15:53:36 +06:00
Kai Zheng
ec0b70716c HADOOP-12579. Deprecate WriteableRPCEngine. Contributed by Wei Zhou 2016-10-09 15:07:03 +06:00
Xiao Chen
2e853be657 HADOOP-13627. Have an explicit KerberosAuthException for UGI to throw, text from public constants. Contributed by Xiao Chen. 2016-10-07 13:46:27 -07:00
Steve Loughran
ebd4f39a39 HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran. 2016-10-07 12:51:40 +01:00
Chris Nauroth
1d330fbaf6 HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth. 2016-10-06 13:19:16 -07:00
Chris Nauroth
2d46c3f6b7 HADOOP-13323. Downgrade stack trace on FS load from Warn to debug. Contributed by Steve Loughran. 2016-10-06 10:57:01 -07:00
Brahma Reddy Battula
35b9d7de9f HADOOP-13690. Fix typos in core-default.xml. Contributed by Yiqun Lin 2016-10-06 18:06:08 +05:30
Brahma Reddy Battula
b90fc70d67 HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen 2016-10-06 17:47:31 +05:30
Wei-Chiu Chuang
c5ca216915 HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge. 2016-10-05 17:36:50 -07:00
Mingliang Liu
00160f71b6 HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang 2016-10-05 16:50:01 -04:00
Akira Ajisaka
9a61b52fbe HADOOP-13685. Document -safely option of rm command. Contributed by Xiaoyu Yao. 2016-10-05 20:41:08 +09:00
Xiao Chen
853d65a157 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 2016-10-03 16:01:54 -07:00
Wei-Chiu Chuang
57aec2b46b HADOOP-12974. Create a CachingGetSpaceUsed implementation that uses df. Contributed by Elliott Clark. 2016-09-30 12:58:37 -07:00
Chris Nauroth
1e0ea27e96 Revert "HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin." 
This reverts commit 0458a2af6e.
 2016-09-29 13:59:09 -07:00
Kihwal Lee
236ac773c9 HADOOP-13537. Support external calls in the RPC call queue. Contributed by Daryn Sharp. 2016-09-29 13:27:30 -05:00
Steve Loughran
ee0c722dc8 HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan. 2016-09-29 17:01:00 +01:00
Steve Loughran
1518cb9532 HADOOP-13663 Index out of range in SysInfoWindows. Contributed by Inigo Goiri 2016-09-29 11:35:00 +01:00
Kihwal Lee
e19b37ead2 HADOOP-11780. Prevent IPC reader thread death. Contributed by Daryn Sharp. 2016-09-28 08:24:21 -05:00
Mingliang Liu
9a44a832a9 HADOOP-13658. Replace config key literal strings with names I: hadoop common. Contributed by Chen Liang 2016-09-27 17:36:53 -07:00
Brahma Reddy Battula
edf0d0f8b2 HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki 2016-09-27 07:04:37 +05:30
Konstantin V Shvachko
6eb700ee74 HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko. 2016-09-23 14:10:11 -07:00
Brahma Reddy Battula
bbdf350ff9 HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document. 2016-09-23 19:21:34 +05:30
Mingliang Liu
e80386d69d HADOOP-13601. Fix a log message typo in AbstractDelegationTokenSecretManager. Contributed by Mehran Hassani. 2016-09-20 13:20:01 -07:00
Kai Zheng
734d54c1a8 HDFS-10861. Refactor StripeReaders and use ECChunk version decode API. Contributed by Sammi Chen 2016-09-21 21:34:48 +08:00
Xiao Chen
ea839bd48e HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen. 2016-09-17 22:25:39 -07:00
Kai Zheng
58bae35447 YARN-5163. Migrate TestClientToAMTokens and TestClientRMTokens tests from the old RPC engine. Contributed by Wei Zhou and Kai Zheng 2016-09-18 08:43:36 +08:00
Andrew Wang
8a40953058 HDFS-10823. Implement HttpFSFileSystem#listStatusIterator. 2016-09-16 15:37:36 -07:00
Sangjin Lee
fcbac00212 Revert "HADOOP-13410. RunJar adds the content of the jar twice to the classpath (Yuanbo Liu via sjlee)" 
This reverts commit 4d3ea92f4f.
 2016-09-15 10:18:56 -07:00
Akira Ajisaka
7cad7b7046 HADOOP-13616. Broken code snippet area in Hadoop Benchmarking. Contributed by Kai Sasaki. 2016-09-15 23:42:34 +09:00
Steve Loughran
53a12fa721 HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran 2016-09-14 15:45:04 +01:00
Kai Zheng
ea0c2b8b05 HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng 2016-09-15 10:46:00 +08:00
Jing Zhao
08d8e0ba25 HADOOP-13546. Override equals and hashCode of the default retry policy to avoid connection leakage. Contributed by Xiaobing Zhou. 2016-09-13 11:12:52 -07:00
Yongjun Zhang
d2466ec3e0 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 2016-09-12 21:25:06 -07:00
Mingliang Liu
59d59667a8 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang 2016-09-12 16:17:08 -07:00
Allen Wittenauer
58ed4fa544 HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS 
This commit includes the following changes:

	HADOOP-13356. Add a function to handle command_subcommand_OPTS
	HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
	HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
	HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
	HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
	HADOOP-13357. Modify common to use hadoop_subcommand_opts
	HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
	HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
	HADOOP-13564. modify mapred to use hadoop_subcommand_opts
	HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
	HADOOP-13360. Documentation for HADOOP_subcommand_OPTS

This closes apache/hadoop#126
 2016-09-12 11:10:00 -07:00
Arpit Agarwal
a99bf26a08 HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy) 2016-09-10 18:22:25 -07:00
Chris Douglas
9f192cc5ac MAPREDUCE-6628. Potential memory leak in CryptoOutputStream. Contributed by Mariappan Asokan 2016-09-09 11:12:44 -07:00
Kihwal Lee
d4d076876a HADOOP-10940. RPC client does no bounds checking of responses. Contributed by Daryn Sharp. 2016-09-09 10:39:35 -05:00
Chris Douglas
011f3b24d4 HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang 2016-09-08 17:53:40 -07:00
Chris Douglas
1192781a78 HADOOP-13519. Make Path Serializable. Contributed by Steve Loughran 2016-09-08 17:46:12 -07:00
Allen Wittenauer
cae331186d Revert "YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang)" 
This reverts commit 05ede00386.
 2016-09-08 12:40:30 -07:00
Zhe Zhang
401db4fc65 HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng. 2016-09-08 11:54:33 -07:00
Kai Zheng
d355573f56 Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng" 
This reverts commit 62a9667136
 2016-09-08 05:50:17 +08:00
Anu Engineer
f414d5e118 HADOOP-13388. Clean up TestLocalFileSystemPermission. Contributed by Andras Bokor. 2016-09-07 11:09:41 -07:00
Xiao Chen
680be58aac HADOOP-13558. UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticket. Contributed by Xiao Chen. 2016-09-06 20:54:17 -07:00
Chris Nauroth
f0d5382ff3 HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth. 2016-09-06 11:02:39 -07:00
Chris Douglas
d37dc5d1b8 HDFS-9847. HDFS configuration should accept time units. Contributed by Yiqun Lin 2016-09-06 10:38:04 -07:00
Andrew Wang
f388a20943 Add CHANGES and release notes for 3.0.0-alpha1 to site 2016-09-06 09:51:55 -07:00
Kihwal Lee
39d1b1d747 HADOOP-13549. Eliminate intermediate buffer for server-side PB encoding. Contributed by Daryn Sharp. 2016-09-06 10:02:42 -05:00
Kai Zheng
62a9667136 HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng 2016-09-07 17:05:33 +08:00
Kihwal Lee
23abb09c1f HADOOP-13547. Optimize IPC client protobuf decoding. Contributed by Daryn Sharp. 2016-09-02 11:03:18 -05:00
Kihwal Lee
76cd81f4b6 HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp. 2016-09-01 15:44:44 -05:00
Chris Douglas
3069df7551 FsPermission string constructor does not recognize sticky bit. Contributed by Atul Sikaria 2016-09-01 11:32:26 -07:00
Mingliang Liu
dcd21d083a HADOOP-13375. o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flaky. (Contributed by Weiwei Yang) 2016-09-01 11:03:06 -07:00
Ray Chiang
05ede00386 YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang) 2016-08-29 15:55:33 -07:00
Mingliang Liu
6fcb04c178 HADOOP-13559. Remove close() within try-with-resources in ChecksumFileSystem/ChecksumFs classes. (Contributed by Aaron Fabbri) 2016-08-29 13:04:28 -07:00
Anu Engineer
e1ad598cef HADOOP-7363. TestRawLocalFileSystemContract is needed. Contributed by Andras Bokor. 2016-08-29 09:26:46 -07:00
Jason Lowe
92d8f37155 HADOOP-13552. RetryInvocationHandler logs all remote exceptions. Contributed by Jason Lowe 2016-08-29 15:55:38 +00:00
Kihwal Lee
81485dbfc1 Revert "HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp." 
This reverts commit d288a0ba83.
 2016-08-25 16:04:54 -05:00
Kihwal Lee
d288a0ba83 HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp. 2016-08-25 11:44:13 -05:00
Albert Chu
52b25fe6ae HADOOP-13533. Do not require user to set HADOOP_SSH_OPTS to a non-null string, allow setting of an empty string. (Albert Chu via aw) closes #121 
Signed-off-by: Allen Wittenauer <aw@apache.org>
 2016-08-25 07:11:06 -07:00
Allen Wittenauer
601599b036 HADOOP-13532. Fix typo in hadoop_connect_to_hosts error message (Albert Chu via aw) closes #120 2016-08-25 06:58:42 -07:00
Akira Ajisaka
ab3b727b5f HADOOP-13534. Remove unused TrashPolicy#getInstance and initialize code. Contributed by Yiqun Lin. 2016-08-25 13:54:57 +09:00
Akira Ajisaka
092b4d5bfd HADOOP-13538. Deprecate getInstance and initialize methods with Path in TrashPolicy. Contributed by Yiqun Lin. 2016-08-24 16:47:52 +09:00
Chris Douglas
c37346d0e3 HADOOP-12726. Unsupported FS operations should throw UnsupportedOperationException. Contributed by Daniel Templeton. 2016-08-23 14:12:52 -07:00
Masatake Iwasaki
8aae8d6bf0 HADOOP-13497. fix wrong command in CredentialProviderAPI.md. Contributed by Yuanbo Liu. 2016-08-24 00:03:27 +09:00
Wei-Chiu Chuang
f0efea490e HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-23 04:14:24 -07:00
Eric Yang
c49333becf HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang) 2016-08-22 18:29:56 -07:00
Xiao Chen
4070caad70 HADOOP-13526. Add detailed logging in KMS for the authentication failure of proxy user. Contributed by Suraj Acharya. 2016-08-22 18:09:35 -07:00
Xiao Chen
f4d4d3474c HADOOP-13487. Hadoop KMS should load old delegation tokens from Zookeeper on startup. Contributed by Xiao Chen. 2016-08-22 14:42:13 -07:00
Mingliang Liu
115ecb52a8 HADOOP-13527. Add Spark to CallerContext LimitedPrivate scope. (Contributed by Weiqing Yang) 2016-08-21 09:40:29 -07:00
Chris Nauroth
763f0497bb HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran. 2016-08-19 10:48:10 -07:00