big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,875 Commits 2 Branches 2 Tags 264 MiB
e4f34ecb04
Commit Graph

1581 Commits

Author SHA1 Message Date
Kihwal Lee
a0bfb41504 HADOOP-14032. Reduce fair call queue priority inversion. Contributed by Daryn Sharp. 2017-02-09 10:04:28 -06:00
Xiao Chen
2007e0cf2a HDFS-11210. Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key is rolled. 2017-02-07 20:36:11 -08:00
Junping Du
cce35c3815 HADOOP-12097. Allow port range to be specified while starting webapp. Contributed by Varun Saxena. 2017-02-05 19:42:11 -08:00
Chris Douglas
59c5f18784 HADOOP-13895. Make FileStatus Serializable 2017-02-01 10:19:36 -08:00
Jason Lowe
9e19f758c1 YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger 2017-01-25 21:41:43 +00:00
Steve Loughran
7fc3e68a87 HADOOP-13433 Race in UGI.reloginFromKeytab. Contributed by Duo Zhang. 2017-01-25 21:29:27 +00:00
Arpit Agarwal
b57368b6f8 HDFS-11194. Maintain aggregated peer performance metrics on NameNode. 2017-01-24 16:58:20 -08:00
Eric Yang
a847903b6e HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu. 2017-01-21 15:43:18 -08:00
Allen Wittenauer
0eb4b513b7 HADOOP-13673. Update scripts to be smarter when running with privilege 
Signed-off-by: Andrew Wang <wang@apache.org>
Signed-off-by: Ravi Prakash <raviprak@apache.org>
 2017-01-18 14:39:05 -08:00
Akira Ajisaka
9130af3f74 HADOOP-13955. Replace deprecated HttpServer2 and SSLFactory constants. Contributed by John Zhuge. 2017-01-18 18:11:33 +09:00
Kihwal Lee
706d630eb9 HADOOP-13976. Path globbing does not match newlines. Contributed by Eric Badger. 2017-01-17 15:10:24 -06:00
Lei Xu
ed09c1418d HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error. (John Zhuge via lei) 2017-01-15 14:02:11 +08:00
Andrew Wang
511d39e074 HADOOP-13885. Implement getLinkTarget for ViewFileSystem. Contributed by Manoj Govindassamy. 2017-01-09 18:01:37 -08:00
Wei-Chiu Chuang
0a212a40fc HADOOP-13953. Make FTPFileSystem's data connection mode and transfer mode configurable. Contributed by Xiao Chen. 2017-01-09 15:18:26 -08:00
Xiao Chen
5d182949ba HADOOP-13597. Switch KMS from Tomcat to Jetty. Contributed by John Zhuge. 2017-01-05 17:21:57 -08:00
Jason Lowe
a0a2761621 HADOOP-13578. Add Codec for ZStandard Compression. Contributed by churro morales 2017-01-04 14:46:25 +00:00
Akira Ajisaka
01d31fe938 HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 2017-01-04 14:10:36 +09:00
Robert Kanter
591fb15944 YARN-5529. Create new DiskValidator class with metrics (yufeigu via rkanter) 2017-01-03 12:13:32 -08:00
Akira Ajisaka
0665c5f09a HADOOP-13943. TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass fails after HADOOP-13863. Contributed by Brahma Reddy Battula. 2016-12-28 01:39:30 +09:00
Xiaoyu Yao
4e9029653d HDFS-10917. Collect peer performance statistics on DataNode. Contributed by Xiaobing Zhou. 2016-12-22 23:46:58 -08:00
Kai Zheng
56a13a6a59 HDFS-11216. Add remoteBytesRead counter metrics for erasure coding reconstruction task. Contributed by Sammi Chen 2016-12-22 14:18:54 +08:00
Jason Lowe
631f1daee3 HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger 2016-12-15 20:52:40 +00:00
Jason Lowe
169bfc0903 Revert "HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger" 
This reverts commit 9947aeb60c.
 2016-12-15 20:51:24 +00:00
Xiaoyu Yao
f5e0bd30fd HADOOP-13890. Maintain HTTP/host as SPNEGO SPN support and fix KerberosName parsing. Contributed by Xiaoyu Yao. 2016-12-14 13:45:21 -08:00
Jason Lowe
9947aeb60c HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger 2016-12-13 22:55:09 +00:00
Wei-Chiu Chuang
b606e025f1 HADOOP-13824. FsShell can suppress the real error if no error message is present. Contributed by John Zhuge. 2016-12-09 15:22:21 -08:00
Brahma Reddy Battula
0ef796174e HADOOP-13867. FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations. Contributed By Vinayakumar B. 2016-12-08 18:57:43 +05:30
Karthik Kambatla
c8d0a049b0 HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha) 2016-12-06 14:10:58 -08:00
Xiao Chen
79d90b810c HADOOP-13827. Add reencryptEncryptedKey interface to KMS. 2016-12-06 12:04:04 -08:00
Xiao Chen
a2b5d60220 HADOOP-13864. KMS should not require truststore password. Contributed by Mike Yoder. 2016-12-05 17:36:00 -08:00
Mingliang Liu
19f373a46b HDFS-11132. Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files. Contributed by Vishwajeet Dusane 2016-12-01 12:54:28 -08:00
Andrew Wang
1f7613be95 HADOOP-13840. Implement getUsed() for ViewFileSystem. Contributed by Manoj Govindassamy. 2016-11-30 17:55:12 -08:00
Akira Ajisaka
67d9f2808e HADOOP-13706. Update jackson from 1.9.13 to 2.x in hadoop-common-project. 2016-11-29 14:07:19 +09:00
Harsh J
07825f2b49 HADOOP-1381. The distance between sync blocks in SequenceFiles should be configurable rather than hard coded to 2000 bytes. Contributed by Harsh J. 2016-11-25 22:22:23 +05:30
Brahma Reddy Battula
ee3d437a33 HADOOP-13820. Replace ugi.getUsername() with ugi.getShortUserName() in viewFS. Contributed By Brahma Reddy Battula. 2016-11-25 19:54:50 +05:30
Steve Loughran
3df3fa396c HADOOP-13833 TestSymlinkHdfsFileSystem#testCreateLinkUsingPartQualPath2 fails after HADOOP-13605. Contributed by Brahma Reddy Battula. 2016-11-24 17:44:36 +00:00
Steve Loughran
abb9fa7fc6 HADOOP-13018. Make Kdiag check whether hadoop.token.files points to existent and valid files. Contributed by Ravi Prakash 2016-11-24 16:41:59 +00:00
Mingliang Liu
860d49aa6a HADOOP-13605. Clean up FileSystem javadocs, logging; improve diagnostics on FS load. Contributed by Steve Loughran 2016-11-23 16:42:27 -08:00
Andrew Wang
dd98a8005a HDFS-11058. Implement 'hadoop fs -df' command for ViewFileSystem. Contributed by Manoj Govindassamy. 2016-11-23 16:40:39 -08:00
Jian He
3d94da1e00 HADOOP-11552. Allow handoff on the server side for RPC requests. Contributed by Siddharth Seth 2016-11-23 16:01:55 -08:00
Mingliang Liu
2bf9a15e8a HADOOP-13166. Add getFileStatus("/") test to AbstractContractGetFileStatusTest. Contributed by Steve Loughran 2016-11-22 14:27:23 -08:00
Lei Xu
3a09e5970d HDFS-11145. Implement getTrashRoot() for ViewFileSystem. (Manoj Govindassamy via lei) 2016-11-22 13:52:09 -08:00
Wei-Chiu Chuang
c0b1a44f6c HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory. 2016-11-17 22:48:35 -06:00
Mingliang Liu
ae8849fe37 HADOOP-11601. Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty files. Contributed by Steve Loughran 2016-11-17 14:11:38 -08:00
Kihwal Lee
bd3735554f HADOOP-13742. Expose NumOpenConnectionsPerUser as a metric. Brahma Reddy Battula. 2016-11-17 12:16:38 -06:00
Mingliang Liu
04014c4c73 HADOOP-13810. Add a test to verify that Configuration handles &-encoded characters. Contributed by Steve Loughran 
This closes #158
 2016-11-14 19:38:10 -08:00
Yongjun Zhang
fd2f22adec HADOOP-13720. Add more info to the msgs printed in AbstractDelegationTokenSecretManager. Contributed by Yongjun Zhang. 2016-11-10 22:42:59 -08:00
Yongjun Zhang
470bdaa27a HADOOP-12718. Incorrect error message by fs -put local dir without permission. (John Zhuge via Yongjun Zhang) 2016-11-10 22:38:38 -08:00
Xiao Chen
822ae88f7d HADOOP-13346. DelegationTokenAuthenticationHandler writes via closed writer. Contributed by Gregory Chanan and Hrishikesh Gadre. 2016-11-09 09:33:00 -08:00
Xiao Chen
367c3d4121 HADOOP-13590. Retry until TGT expires even if the UGI renewal thread encountered exception. 2016-11-09 09:07:12 -08:00
Zhe Zhang
77c13c3857 HADOOP-13782. Make MutableRates metrics thread-local write, aggregate-on-read. Contributed by Erik Krogen. 2016-11-08 16:07:36 -08:00
Subru Krishnan
29e3b3417c YARN-5833. Add validation to ensure default ports are unique in Configuration. (Konstantinos Karanasos via Subru). 2016-11-08 14:38:18 -08:00
Brahma Reddy Battula
dbb133ccfc HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed. Contributed by Yuanbo Liu. 2016-11-08 20:55:10 +05:30
Zhe Zhang
3dbad5d823 HADOOP-13804. MutableStat mean loses accuracy if add(long, long) is used. Contributed by Erik Krogen. 2016-11-07 16:08:10 -08:00
Mingliang Liu
abfc15d5ef HADOOP-13792. Stackoverflow for schemeless defaultFS with trailing slash. Contributed by John Zhuge 2016-11-04 10:46:27 -07:00
Andrew Wang
7e521c5a49 HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory. 2016-11-02 19:11:05 -07:00
Arpit Agarwal
1b6ecaf016 HADOOP-13738. DiskChecker should perform some disk IO. 2016-11-01 18:02:23 -07:00
Wei-Chiu Chuang
112f04eb1b HDFS-11061. Update dfs -count -t command line help and documentation. Contributed by Yiqun Lin. 2016-10-28 15:33:27 -07:00
Robert Kanter
5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Andrew Wang
22ff0eff4d HADOOP-8299. ViewFileSystem link slash mount point crashes with IndexOutOfBoundsException. Contributed by Manoj Govindassamy. 2016-10-26 14:25:03 -07:00
Chris Nauroth
1f8490a5ba HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth. 2016-10-26 08:55:42 -07:00
Chris Nauroth
9cad3e2350 HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran. 2016-10-26 08:27:26 -07:00
Wei-Chiu Chuang
d88dca844a HADOOP-11798. Native raw erasure coder in XOR codes. Contributed by SammiChen. 2016-10-25 16:53:53 -07:00
Andrew Wang
084bdab156 HDFS-5684. Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as VisibleForTesting. Contributed by Manoj Govindassamy. 2016-10-25 15:35:39 -07:00
Chris Douglas
ae8bccd509 HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du 2016-10-21 12:59:54 -07:00
Anu Engineer
262827cf75 HADOOP-13737. Cleanup DiskChecker interface. Contributed by Arpit Agarwal. 2016-10-20 13:35:26 -07:00
Anu Engineer
3fbf4cd5da HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-20 12:33:58 -07:00
Xiao Chen
efdf810cf9 HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge. 2016-10-18 18:18:43 -07:00
Steve Loughran
6c348c5691 HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran 2016-10-18 21:16:02 +01:00
Kai Zheng
c023c74886 HADOOP-13061. Refactor erasure coders. Contributed by Kai Sasaki 2016-10-18 12:02:53 +06:00
Akira Ajisaka
bedfec0c10 HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini. 2016-10-18 15:00:44 +09:00
Andrew Wang
0f4afc8100 HADOOP-13722. Code cleanup -- ViewFileSystem and InodeTree. Contributed by Manoj Govindassamy. 2016-10-17 13:15:11 -07:00
Xiaoyu Yao
dbe663d524 HADOOP-13686. Adding additional unit test for Trash (I). Contributed by Weiwei Yang. 2016-10-13 23:05:16 -07:00
Kihwal Lee
85cd06f663 HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 2016-10-12 15:11:42 -05:00
Andrew Wang
8a09bf7c19 HADOOP-13705. Revert HADOOP-13534 Remove unused TrashPolicy#getInstance and initialize code. 2016-10-11 13:46:07 -07:00
Mingliang Liu
c874fa914d HDFS-10985. o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertions. Contributed by Mingliang Liu 2016-10-10 13:33:07 -07:00
Andrew Wang
03060075c5 HADOOP-13699. Configuration does not substitute multiple references to the same var. 2016-10-10 12:19:26 -07:00
Steve Loughran
ebd4f39a39 HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran. 2016-10-07 12:51:40 +01:00
Chris Nauroth
2d46c3f6b7 HADOOP-13323. Downgrade stack trace on FS load from Warn to debug. Contributed by Steve Loughran. 2016-10-06 10:57:01 -07:00
Wei-Chiu Chuang
c5ca216915 HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge. 2016-10-05 17:36:50 -07:00
Mingliang Liu
00160f71b6 HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang 2016-10-05 16:50:01 -04:00
Wei-Chiu Chuang
57aec2b46b HADOOP-12974. Create a CachingGetSpaceUsed implementation that uses df. Contributed by Elliott Clark. 2016-09-30 12:58:37 -07:00
Chris Nauroth
1e0ea27e96 Revert "HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin." 
This reverts commit 0458a2af6e.
 2016-09-29 13:59:09 -07:00
Kihwal Lee
236ac773c9 HADOOP-13537. Support external calls in the RPC call queue. Contributed by Daryn Sharp. 2016-09-29 13:27:30 -05:00
Steve Loughran
ee0c722dc8 HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan. 2016-09-29 17:01:00 +01:00
Steve Loughran
1518cb9532 HADOOP-13663 Index out of range in SysInfoWindows. Contributed by Inigo Goiri 2016-09-29 11:35:00 +01:00
Mingliang Liu
9a44a832a9 HADOOP-13658. Replace config key literal strings with names I: hadoop common. Contributed by Chen Liang 2016-09-27 17:36:53 -07:00
Brahma Reddy Battula
edf0d0f8b2 HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki 2016-09-27 07:04:37 +05:30
Konstantin V Shvachko
6eb700ee74 HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko. 2016-09-23 14:10:11 -07:00
Kai Zheng
58bae35447 YARN-5163. Migrate TestClientToAMTokens and TestClientRMTokens tests from the old RPC engine. Contributed by Wei Zhou and Kai Zheng 2016-09-18 08:43:36 +08:00
Andrew Wang
8a40953058 HDFS-10823. Implement HttpFSFileSystem#listStatusIterator. 2016-09-16 15:37:36 -07:00
Kai Zheng
ea0c2b8b05 HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng 2016-09-15 10:46:00 +08:00
Jing Zhao
08d8e0ba25 HADOOP-13546. Override equals and hashCode of the default retry policy to avoid connection leakage. Contributed by Xiaobing Zhou. 2016-09-13 11:12:52 -07:00
Yongjun Zhang
d2466ec3e0 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 2016-09-12 21:25:06 -07:00
Mingliang Liu
59d59667a8 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang 2016-09-12 16:17:08 -07:00
Allen Wittenauer
58ed4fa544 HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS 
This commit includes the following changes:

	HADOOP-13356. Add a function to handle command_subcommand_OPTS
	HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
	HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
	HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
	HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
	HADOOP-13357. Modify common to use hadoop_subcommand_opts
	HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
	HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
	HADOOP-13564. modify mapred to use hadoop_subcommand_opts
	HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
	HADOOP-13360. Documentation for HADOOP_subcommand_OPTS

This closes apache/hadoop#126
 2016-09-12 11:10:00 -07:00
Chris Douglas
9f192cc5ac MAPREDUCE-6628. Potential memory leak in CryptoOutputStream. Contributed by Mariappan Asokan 2016-09-09 11:12:44 -07:00
Kihwal Lee
d4d076876a HADOOP-10940. RPC client does no bounds checking of responses. Contributed by Daryn Sharp. 2016-09-09 10:39:35 -05:00
Chris Douglas
1192781a78 HADOOP-13519. Make Path Serializable. Contributed by Steve Loughran 2016-09-08 17:46:12 -07:00
Allen Wittenauer
cae331186d Revert "YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang)" 
This reverts commit 05ede00386.
 2016-09-08 12:40:30 -07:00