big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,750 Commits 2 Branches 2 Tags 264 MiB
b8305e6d06
Commit Graph

3283 Commits

Author SHA1 Message Date
Arpit Agarwal
959940b0ab HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru. 2017-03-07 10:12:35 -08:00
Mingliang Liu
686823529b HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth 2017-03-06 17:16:36 -08:00
Mingliang Liu
52d7d5aa1a Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth" 
This reverts commit 6b7cd62b8c.
 2017-03-06 17:10:11 -08:00
Ray Chiang
d9dc444dc7 YARN-5665. Enhance documentation for yarn.resourcemanager.scheduler.class property. (Yufei Gu via rchiang) 2017-03-06 14:02:49 -08:00
Akira Ajisaka
209ecd1a5c
HADOOP-14108. CLI MiniCluster: add an option to specify NameNode HTTP port. Contributed by Takanobu Asanuma. 2017-03-07 01:34:51 +09:00
Mingliang Liu
6b7cd62b8c HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth 2017-03-03 17:09:12 -08:00
Jason Lowe
05237636d3 YARN-6274. Documentation refers to incorrect nodemanager health checker interval property. Contributed by WeiWei Yang 2017-03-03 07:57:37 -06:00
Harsh J
eb5a17954a HADOOP-6801. io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java. 
This closes #146

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-03-03 18:38:04 +09:00
Rakesh Radhakrishnan
3749152b66 HDFS-11416. Refactor out system default erasure coding policy. Contributed by Andrew Wang. 2017-03-03 09:28:03 +05:30
Andrew Wang
8f4817f2c5 Revert "HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru." 
This reverts commit e61491d476.
 2017-03-02 16:20:48 -08:00
Arpit Agarwal
e61491d476 HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru. 2017-03-02 15:38:25 -08:00
Arpit Agarwal
eeca8b0c4e HDFS-11419. HDFS specific network topology classes with storage type info included. Contributed by Chen Liang. 2017-03-02 09:21:27 -08:00
Brahma Reddy Battula
dcd03df9f9 HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula. 2017-03-01 10:45:56 +05:30
Yiqun Lin
9db2e0c8d4 HADOOP-14119. Remove unused imports from GzipCodec.java. Contributed by Yiqun Lin. 2017-02-27 18:39:14 +08:00
Xuan
289bc50e66 HADOOP-14116:FailoverOnNetworkExceptionRetry does not wait when failover 
on certain exception. Contributed by Jian He
 2017-02-24 11:42:23 -08:00
Andrew Wang
c1a52b04d0 HDFS-11427. Rename rs-default to rs. 2017-02-24 10:58:45 -08:00
Harsh J
e8694deb6a HADOOP-13817. Add a finite shell command timeout to ShellBasedUnixGroupsMapping. (harsh) 2017-02-24 21:34:00 +05:30
Akira Ajisaka
50decd3613 HADOOP-14097. Remove Java6 specific code from GzipCodec.java. Contributed by Elek, Marton. 2017-02-25 00:28:31 +09:00
Andrew Wang
132f758e3d HDFS-11426. Refactor EC CLI to be similar to storage policies CLI. 2017-02-23 16:00:11 -08:00
Mingliang Liu
a4d4a23785 HADOOP-13321. Deprecate FileSystem APIs that promote inefficient call patterns. Contributed by Chris Nauroth and Mingliang Liu 2017-02-23 12:55:40 -08:00
Mingliang Liu
003ae00693 HDFS-11430. Separate class InnerNode from class NetworkTopology and make it extendable. Contributed by Tsz Wo Nicholas Sze 2017-02-21 15:32:46 -08:00
Eric Yang
172b23af33 HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu. 2017-02-18 18:34:13 -08:00
Yongjun Zhang
4c26c241ad HADOOP-13805. UGI.getCurrentUser() fails if user does not have a keytab associated. Contributed by Xiao Chen, Wei-Chiu Chuang, Yongjun Zhang. 2017-02-17 09:18:50 -08:00
Wei-Chiu Chuang
5690b51ef7 HDFS-11100. Recursively deleting file protected by sticky bit should fail. Contributed by John Zhuge. 2017-02-16 05:39:37 -08:00
Wei-Chiu Chuang
cc45da79fd HADOOP-13233. help of stat is confusing. Contributed by Attila Bukor. 2017-02-13 10:14:45 -08:00
Arpit Agarwal
0aacd8fd25 HDFS-11403. Zookeper ACLs on NN HA enabled clusters should be handled consistently. Contributed by Hanisha Koneru. 2017-02-11 01:17:56 -08:00
Kihwal Lee
0c01cf5798 HADOOP-14033. Reduce fair call queue lock contention. Contributed by Daryn Sharp. 2017-02-09 16:17:24 -06:00
Kihwal Lee
b6bb99c18a HADOOP-14034. Allow ipc layer exceptions to selectively close connections. Contributed by Daryn Sharp. 2017-02-09 10:47:29 -06:00
Kihwal Lee
a0bfb41504 HADOOP-14032. Reduce fair call queue priority inversion. Contributed by Daryn Sharp. 2017-02-09 10:04:28 -06:00
Junping Du
eec52e158b YARN-6145. Improve log message on fail over. Contributed by Jian He. 2017-02-08 02:11:19 -08:00
Xiao Chen
2007e0cf2a HDFS-11210. Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key is rolled. 2017-02-07 20:36:11 -08:00
Xiao Chen
69b23632c4 HDFS-10860. Switch HttpFS from Tomcat to Jetty. Contributed by John Zhuge. 2017-02-07 09:36:55 -08:00
Junping Du
cce35c3815 HADOOP-12097. Allow port range to be specified while starting webapp. Contributed by Varun Saxena. 2017-02-05 19:42:11 -08:00
Yiqun Lin
3ea6d3517e HDFS-11371. Document missing metrics of erasure coding. Contributed by Yiqun Lin. 2017-02-04 11:01:27 +08:00
Akira Ajisaka
cb8f3f3adf HADOOP-14053. Update the link to HTrace SpanReceivers. Contributed by Yiqun Lin. 2017-02-04 10:58:08 +09:00
Xiao Chen
ba75bc7593 HADOOP-14044. Synchronization issue in delegation token cancel functionality. Contributed by Hrishikesh Gadre. 2017-02-03 17:13:53 -08:00
Chris Douglas
59c5f18784 HADOOP-13895. Make FileStatus Serializable 2017-02-01 10:19:36 -08:00
Xiao Chen
ebd40056a0 HADOOP-13992. KMS should load SSL configuration the same way as SSLFactory. Contributed by John Zhuge. 2017-01-27 10:49:26 -08:00
Xiaoyu Yao
2034315763 HADOOP-14029. Fix KMSClientProvider for non-secure proxyuser use case. Contributed by Xiaoyu Yao. 2017-01-26 20:34:32 -08:00
Andrew Wang
ff02bdfe65 Add CHANGES, RELEASENOTES, and jdiff for 3.0.0-alpha2 release. 2017-01-25 15:39:40 -08:00
Jason Lowe
9e19f758c1 YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger 2017-01-25 21:41:43 +00:00
Xiaoyu Yao
a46933e8ce HADOOP-13988. KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser. Contributed by Greg Senia and Xiaoyu Yao. 2017-01-25 13:33:06 -08:00
Steve Loughran
7fc3e68a87 HADOOP-13433 Race in UGI.reloginFromKeytab. Contributed by Duo Zhang. 2017-01-25 21:29:27 +00:00
Xiao Chen
9c0a4d3e71 HADOOP-13990. Document KMS usage of CredentialProvider API. Contributed by John Zhuge. 2017-01-24 21:30:10 -08:00
Arpit Agarwal
b57368b6f8 HDFS-11194. Maintain aggregated peer performance metrics on NameNode. 2017-01-24 16:58:20 -08:00
Eric Yang
a847903b6e HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu. 2017-01-21 15:43:18 -08:00
Lei Xu
e015b56319 HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei) 2017-01-20 14:48:04 +08:00
Daniel Templeton
f5839fd6df YARN-6028. Add document for container metrics (Contributed by Weiwei Yang via Daniel Templeton) 2017-01-19 12:45:17 -08:00
Akira Ajisaka
1763467210 HADOOP-14001. Improve delegation token validity checking. 2017-01-19 17:56:39 +09:00
Yiqun Lin
383aa9c9f5 HADOOP-13965. Groups should be consistent in using default group mapping class. Contributed by Yiqun Lin. 2017-01-19 09:42:56 +08:00
Allen Wittenauer
0eb4b513b7 HADOOP-13673. Update scripts to be smarter when running with privilege 
Signed-off-by: Andrew Wang <wang@apache.org>
Signed-off-by: Ravi Prakash <raviprak@apache.org>
 2017-01-18 14:39:05 -08:00
Akira Ajisaka
9130af3f74 HADOOP-13955. Replace deprecated HttpServer2 and SSLFactory constants. Contributed by John Zhuge. 2017-01-18 18:11:33 +09:00
Kihwal Lee
706d630eb9 HADOOP-13976. Path globbing does not match newlines. Contributed by Eric Badger. 2017-01-17 15:10:24 -06:00
Akira Ajisaka
e407449ddb HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 2017-01-17 10:10:23 +09:00
Lei Xu
ed09c1418d HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error. (John Zhuge via lei) 2017-01-15 14:02:11 +08:00
Steve Loughran
e648b6e138 HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran 2017-01-11 17:25:15 +00:00
Arpit Agarwal
4046794a53 HDFS-11299. Support multiple Datanode File IO hooks. Contributed by Hanisha Koneru. 2017-01-10 10:43:11 -08:00
Andrew Wang
511d39e074 HADOOP-13885. Implement getLinkTarget for ViewFileSystem. Contributed by Manoj Govindassamy. 2017-01-09 18:01:37 -08:00
Wei-Chiu Chuang
0a212a40fc HADOOP-13953. Make FTPFileSystem's data connection mode and transfer mode configurable. Contributed by Xiao Chen. 2017-01-09 15:18:26 -08:00
Arpit Agarwal
4a659ff40f HDFS-11282. Document the missing metrics of DataNode Volume IO operations. Contributed by Yiqun Lin. 2017-01-05 19:34:32 -08:00
Xiao Chen
5d182949ba HADOOP-13597. Switch KMS from Tomcat to Jetty. Contributed by John Zhuge. 2017-01-05 17:21:57 -08:00
Jason Lowe
a0a2761621 HADOOP-13578. Add Codec for ZStandard Compression. Contributed by churro morales 2017-01-04 14:46:25 +00:00
Akira Ajisaka
01d31fe938 HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 2017-01-04 14:10:36 +09:00
Mingliang Liu
88731c731a Revert "HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran" 
This reverts commit 451efb08fe.
 2017-01-03 14:44:26 -08:00
Mingliang Liu
451efb08fe HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran 2017-01-03 13:08:38 -08:00
Robert Kanter
591fb15944 YARN-5529. Create new DiskValidator class with metrics (yufeigu via rkanter) 2017-01-03 12:13:32 -08:00
Brahma Reddy Battula
6938b67711 HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 2016-12-31 16:16:02 +05:30
Akira Ajisaka
0665c5f09a HADOOP-13943. TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass fails after HADOOP-13863. Contributed by Brahma Reddy Battula. 2016-12-28 01:39:30 +09:00
Daniel Templeton
ded2d08f33 HADOOP-13940. Document the missing envvars commands (Contributed by Yiqun Lin via Daniel Templeton) 2016-12-27 07:18:08 -08:00
Xiaoyu Yao
4e9029653d HDFS-10917. Collect peer performance statistics on DataNode. Contributed by Xiaobing Zhou. 2016-12-22 23:46:58 -08:00
Mingliang Liu
e92a77099b HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by Dushyanth 2016-12-22 20:15:18 -08:00
Akira Ajisaka
22befbd585 MAPREDUCE-6704. Update the documents to run MapReduce application. Contributed by Bibin A Chundatt. 2016-12-23 04:48:04 +09:00
Kai Zheng
56a13a6a59 HDFS-11216. Add remoteBytesRead counter metrics for erasure coding reconstruction task. Contributed by Sammi Chen 2016-12-22 14:18:54 +08:00
Akira Ajisaka
f6e2521eb2 HDFS-11261. Document missing NameNode metrics. Contributed by Yiqun Lin. 2016-12-21 14:17:47 +09:00
Jason Lowe
631f1daee3 HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger 2016-12-15 20:52:40 +00:00
Jason Lowe
169bfc0903 Revert "HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger" 
This reverts commit 9947aeb60c.
 2016-12-15 20:51:24 +00:00
Xiaoyu Yao
f5e0bd30fd HADOOP-13890. Maintain HTTP/host as SPNEGO SPN support and fix KerberosName parsing. Contributed by Xiaoyu Yao. 2016-12-14 13:45:21 -08:00
Arpit Agarwal
6ba9587d37 HDFS-10958. Add instrumentation hooks around Datanode disk IO. 2016-12-14 11:18:58 -08:00
Akira Ajisaka
72bff192cd HDFS-11204. Document the missing options of hdfs zkfc command. Contributed by Yiqun Lin. 2016-12-14 18:50:43 +09:00
Jason Lowe
9947aeb60c HADOOP-13709. Ability to clean up subprocesses spawned by Shell when the process exits. Contributed by Eric Badger 2016-12-13 22:55:09 +00:00
Steve Loughran
f66f61892a HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-12 13:58:27 +00:00
Wei-Chiu Chuang
b606e025f1 HADOOP-13824. FsShell can suppress the real error if no error message is present. Contributed by John Zhuge. 2016-12-09 15:22:21 -08:00
Brahma Reddy Battula
80b8023276 HADOOP-13883. Add description of -fs option in generic command usage. Contributed By Yiqun Lin. 2016-12-09 18:21:42 +05:30
Akira Ajisaka
7d8e440eee Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran" 
This reverts commit c265515725.
 2016-12-09 12:49:27 +09:00
Steve Loughran
c265515725 HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-08 17:57:15 +00:00
Brahma Reddy Battula
0ef796174e HADOOP-13867. FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations. Contributed By Vinayakumar B. 2016-12-08 18:57:43 +05:30
Akira Ajisaka
deb4743b2c Revert "HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin." 
This reverts commit 0857641f62.
 2016-12-08 13:37:06 +09:00
Akira Ajisaka
0857641f62 HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin. 2016-12-08 13:34:45 +09:00
Ming Ma
c73e08a6da HDFS-10206. Datanodes not sorted properly by distance when the reader isn't a datanode. (Nandakumar via mingma) 2016-12-07 08:26:09 -08:00
Karthik Kambatla
c8d0a049b0 HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha) 2016-12-06 14:10:58 -08:00
Andrew Wang
f53b481ffe HDFS-11172. Support an erasure coding policy using RS 10 + 4. Contributed by Wei Zhou. 2016-12-06 12:49:38 -08:00
Xiao Chen
79d90b810c HADOOP-13827. Add reencryptEncryptedKey interface to KMS. 2016-12-06 12:04:04 -08:00
Andrew Wang
7b988e8899 HADOOP-13861. Spelling errors in logging and exceptions for code. Contributed by Grant Sohn. 2016-12-05 23:18:18 -08:00
Akira Ajisaka
b2a3d6c519 HADOOP-13835. Move Google Test Framework code from mapreduce to hadoop-common. Contributed by Varun Vasudev. 2016-12-06 14:01:47 +09:00
Xiao Chen
a2b5d60220 HADOOP-13864. KMS should not require truststore password. Contributed by Mike Yoder. 2016-12-05 17:36:00 -08:00
Xiao Chen
291df5c7fb HADOOP-13847. KMSWebApp should close KeyProviderCryptoExtension. Contributed by John Zhuge. 2016-12-05 09:35:17 -08:00
Mingliang Liu
19f373a46b HDFS-11132. Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files. Contributed by Vishwajeet Dusane 2016-12-01 12:54:28 -08:00
Andrew Wang
1f7613be95 HADOOP-13840. Implement getUsed() for ViewFileSystem. Contributed by Manoj Govindassamy. 2016-11-30 17:55:12 -08:00
Kai Zheng
51e6c1cc3f HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen 2016-11-30 15:52:56 +08:00
Kai Zheng
cfd8076f81 Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen" 
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
 2016-11-30 15:44:52 +08:00