big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,991 Commits 2 Branches 2 Tags 264 MiB
bd6a2172e0
Commit Graph

203 Commits

Author SHA1 Message Date
Andrew Wang
713cb71820 HADOOP-13155. Implement TokenRenewer to renew and cancel delegation tokens in KMS. Contributed by Xiao Chen. 2016-06-03 16:48:54 -07:00
Chris Nauroth
d82bc85018 HADOOP-13105. Support timeouts in LDAP queries in LdapGroupsMapping. Contributed by Mingliang Liu. 2016-06-03 16:38:30 -07:00
Steve Loughran
757050ff35 HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong. 2016-05-20 13:52:15 +01:00
Kai Zheng
182fc1986a HADOOP-12782. Faster LDAP group name resolution with ActiveDirectory. Contributed by Wei-Chiu Chuang 2016-05-19 07:15:52 -07:00
Sangjin Lee
61f46be071 HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee) 2016-05-16 18:49:47 -07:00
Steve Loughran
27c4e90efc HADOOP-13028 add low level counter metrics for S3A; use in read performance tests. contributed by: stevel 
patch includes
HADOOP-12844 Recover when S3A fails on IOException in read()
HADOOP-13058 S3A FS fails during init against a read-only FS if multipart purge
HADOOP-13047 S3a Forward seek in stream length to be configurable
 2016-05-12 19:24:20 +01:00
Steve Loughran
def2a6d385 HADOOP-13122 Customize User-Agent header sent in HTTP requests by S3A. Chris Nauroth via stevel. 2016-05-12 13:57:35 +01:00
Steve Loughran
025219b12f HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel) 2016-05-10 21:37:22 +01:00
Arpit Agarwal
ea5475d1c1 HADOOP-13039. Add documentation for configuration property ipc.maximum.data.length (Contributed by Mingliang Liu) 2016-04-26 20:46:38 -07:00
Steve Loughran
19f0f9608e HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel) 2016-04-22 11:25:03 +01:00
Kai Zheng
b5d4c7dc76 HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li. 2016-04-20 15:47:01 +08:00
Steve Loughran
df18b6e984 HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel) 2016-04-14 12:44:55 +01:00
Masatake Iwasaki
85ec5573eb HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims) 2016-04-06 03:22:48 +09:00
Junping Du
90fcb16c0f MAPREDUCE-6655. Fix a typo (STRICT_IE6) in Encrypted Shuffle. Contributed by Wei-Chiu Chuang. 2016-03-28 05:30:42 -07:00
Steve Loughran
754299695b Revert "HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims)" 
This reverts commit 682adc6ba9.
 2016-03-11 17:00:17 +00:00
Masatake Iwasaki
682adc6ba9 HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims) 2016-03-11 15:03:40 +09:00
Masatake Iwasaki
f70e317ccc HADOOP-12793. Write a new group mapping service guide (Wei-Chiu Chuang via iwasakims) 2016-03-03 17:14:32 +09:00
Steve Loughran
03cfb454fe HADOOP-12825. Log slow name resolutions. (Sidharta Seethana via stevel) 2016-02-27 20:23:51 +00:00
Lei Xu
2093acf6b6 HADOOP-12841. Update s3-related properties in core-default.xml. (Wei-Chiu Chuang via lei) 2016-02-26 10:47:49 -08:00
Robert Kanter
a365a3941c HADOOP-12817. Enable TLS v1.1 and 1.2 (rkanter) 2016-02-18 11:09:50 -08:00
Sangjin Lee
58acbf940a HADOOP-12773. HBase classes fail to load with client/job classloader enabled (sjlee) 2016-02-08 13:55:30 -08:00
Steve Loughran
29ae258013 HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel) 2016-02-06 15:05:16 +00:00
Colin Patrick Mccabe
9086dd58c3 HADOOP-12766. The default value of "hadoop.workaround.non.threadsafe.getpwuid" is different between core-default.xml and NativeIO.java (Akira AJISAKA via cmccabe) 2016-02-05 10:56:54 -08:00
Colin Patrick Mccabe
2e8ab3d465 HADOOP-12735. core-default.xml misspells hadoop.workaround.non.threadsafe.getpwuid (Ray Chiang via cmccabe) 2016-01-27 11:39:55 -08:00
Allen Wittenauer
d22c4239a4 HDFS-9525. hadoop utilities need to support provided delegation tokens (HeeSoo Kim via aw) 2016-01-22 12:15:22 -08:00
Lei Xu
126705f67e HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei) 2016-01-12 12:19:53 -08:00
Akira Ajisaka
64ae85fd2e HADOOP-11252. RPC client does not time out by default. Contributed by Wilfred Spiegelenburg and Masatake Iwasaki. 2016-01-04 14:31:22 +09:00
Allen Wittenauer
576b569b6c Revert "HDFS-9525. hadoop utilities need to support provided delegation tokens (HeeSoo Kim via aw)" 
This reverts commit 832b3cbde1.
 2015-12-11 10:59:42 -08:00
Robert Kanter
e27fffdb90 HADOOP-12625. Add a config to disable the /logs endpoints (rkanter) 2015-12-09 14:32:20 -08:00
Allen Wittenauer
832b3cbde1 HDFS-9525. hadoop utilities need to support provided delegation tokens (HeeSoo Kim via aw) 2015-12-08 12:56:38 -08:00
Akira Ajisaka
4a3b8f1ed2 HADOOP-12568. Update core-default.xml to describe posixGroups support. Contributed by Wei-Chiu Chuang. 2015-11-18 17:23:00 +09:00
Lei Xu
bff7c90a56 HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei) 2015-11-05 18:35:15 -08:00
Jonathan Eagles
f8adeb712d YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 2015-10-23 10:34:08 -05:00
Colin Patrick Mccabe
892ade689f HDFS-9080. Update htrace version to 4.0.1 (cmccabe) 2015-09-28 07:42:12 -07:00
Arpit Agarwal
df31c446bf HADOOP-12437. Allow SecurityUtil to lookup alternate hostnames. (Contributed by Arpit Agarwal) 2015-09-24 11:41:48 -07:00
Andrew Wang
e1feaf6db0 HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao. 2015-09-04 13:42:55 -07:00
Lei Xu
6ab2d19f5c HADOOP-12346. Increase some default timeouts / retries for S3a connector. (Sean Mackrory via Lei (Eddy) Xu) 2015-08-29 09:59:30 -07:00
Arpit Agarwal
bdbe53c676 HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) 2015-08-29 09:52:37 -07:00
Xiaoyu Yao
48774d0a45 HADOOP-12325. RPC Metrics : Add the ability track and log slow RPCs. Contributed by Anu Engineer 2015-08-24 14:31:24 -07:00
Lei Xu
d5403747b5 HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu) 2015-08-04 18:51:52 -07:00
Jakob Homan
0712a8103f HDFS-8180. AbstractFileSystem Implementation for WebHdfs. Contributed by Sathosh G Nayak. 2015-07-28 21:03:31 -07:00
Vinod Kumar Vavilapalli
722aa1db1f HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. 2015-06-15 14:22:34 -07:00
Andrew Wang
374ddd9f9e HADOOP-12043. Display warning if defaultFs is not set when running fs commands. Contributed by Eddy Xu. 2015-05-29 17:15:58 -07:00
cnauroth
860b8373c3 HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop. Contributed by Larry McCay. 2015-05-28 15:01:42 -07:00
Akira Ajisaka
0e4f1081c7 HADOOP-10366. Add whitespaces between classes for values in core-default.xml to fit better in browser. Contributed by kanaka kumar avvaru. 2015-05-21 17:52:03 +09:00
Colin Patrick Mccabe
8f7c2364d7 HDFS-8284. Update documentation about how to use HTrace with HDFS (Masatake Iwasaki via Colin P. McCabe) 2015-05-08 12:30:03 -07:00
Akira Ajisaka
ac281e3fc8 HADOOP-11627. Remove io.native.lib.available. Contributed by Brahma Reddy Battula. 2015-04-24 08:08:55 +09:00
Gera Shegalov
64eb068ee8 DelegateToFileSystem erroneously uses default FS's port in constructor. (Brahma Reddy Battula via gera) 2015-03-10 13:52:06 -07:00
Tsuyoshi Ozawa
54639c7d7a HADOOP-11226. Add a configuration to set ipc.Client's traffic class with IPTOS_LOWDELAY|IPTOS_RELIABILITY. Contributed by Gopal V. 2015-03-10 13:08:29 +09:00
Steve Loughran
15b7076ad5 HADOOP-11183. Memory-based S3AOutputstream. (Thomas Demoor via stevel) 2015-03-03 16:18:51 -08:00
Akira Ajisaka
ab5976161f HADOOP-8642. Document that io.native.lib.available only controls native bz2 and zlib compression codecs. (aajisaka) 2015-02-23 17:34:43 -08:00
Steve Loughran
00b80958d8 HADOOP-11521. Make connection timeout configurable in s3a. (Thomas Demoor via stevel) 2015-02-17 20:06:27 +00:00
Steve Loughran
78a7e8d3a6 HADOOP-11522. Update S3A Documentation. (Thomas Demoor via stevel) 2015-02-17 18:15:02 +00:00
Steve Loughran
c3da2db48f YARN-2683. [YARN-913] registry config options: document and move to core-default. (stevel) 2015-02-11 14:26:37 +00:00
Akira Ajisaka
42548f4dc2 HADOOP-11544. Remove unused configuration keys for tracing. Contributed by Masatake Iwasaki. 2015-02-04 04:18:17 -08:00
Harsh J
ffc75d6ebe HADOOP-11488. Difference in default connection timeout for S3A FS. Contributed by Daisuke Kobayashi. 2015-02-01 00:17:04 +05:30
Kihwal Lee
8dc59cb9e0 HADOOP-9137. Support connection limiting in IPC server. Contributed by Kihwal Lee. 2015-01-30 17:21:36 -06:00
yliu
fc20129c56 HDFS-7623. Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace. (yliu) 2015-01-21 00:59:47 +08:00
Karthik Kambatla
3c5f5af118 HADOOP-11243. SSLFactory shouldn't allow SSLv3. (Wei Yan via kasha) 2014-10-28 18:03:00 -07:00
yliu
e7859015bc HADOOP-11233. hadoop.security.kms.client.encrypted.key.cache.expiry property spelled wrong in core-default. (Stephen Chu via yliu) 2014-10-28 14:21:26 +08:00
Andrew Wang
2d8e6e2c4a HADOOP-11151. Automatically refresh auth token and retry on auth failure. Contributed by Arun Suresh. 2014-10-02 19:54:57 -07:00
Jason Lowe
cf4631d7c9 HADOOP-11049. javax package system class default is too broad. Contributed by Sangjin Lee 2014-09-29 14:36:29 +00:00
Aaron T. Myers
24d920b80e HADOOP-10400. Incorporate new S3A FileSystem implementation. Contributed by Jordan Mendelson and Dave Wang. 2014-09-15 08:27:07 -07:00
Colin Patrick Mccabe
5ec7fcd9dd HADOOP-11074. Move s3-related FS connector code to hadoop-aws. (David S. Wang via Colin Patrick McCabe) 2014-09-10 16:14:53 -07:00
Andrew Wang
c3fa0b5884 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614553 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 06:09:33 +00:00
Arpit Agarwal
260fd25568 HADOOP-8069. Enable TCP_NODELAY by default for IPC. (Contributed by Todd Lipcon) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614242 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 02:55:27 +00:00
Andrew Wang
22ba00de60 Merge from trunk to branch pt 2 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612941 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 20:45:10 +00:00
Yi Liu
b52b80d7bd HADOOP-10853. Refactor get instance of CryptoCodec and support create via algorithm/mode/padding. (yliu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612513 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 08:38:38 +00:00
Andrew Wang
d3bf8186ae HADOOP-10755. Support negative caching of user-group mapping. Contributed by Lei Xu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612408 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 21:52:17 +00:00
Andrew Wang
cb20bb6117 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612403 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 21:44:50 +00:00
Alejandro Abdelnur
0c1469ece3 HADOOP-10720. KMS: Implement generateEncryptedKey and decryptEncryptedKey in the REST API. (asuresh via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612399 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 20:55:42 +00:00
Yi Liu
beb0c19cde HADOOP-10735. Fall back AesCtrCryptoCodec implementation from OpenSSL to JCE if non native support. (yliu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1610887 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 00:36:30 +00:00
Colin McCabe
905c90b066 HADOOP-10734. Implement high-performance secure random number sources. (Yi Liu via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1609874 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-12 01:29:26 +00:00
Andrew Wang
dda85637df Merge trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1608603 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:43:56 +00:00
Brandon Li
f194aaa0d1 HADOOP-8943. Support multiple group mapping providers. Contributed by Kai Zheng 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605857 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-26 17:14:11 +00:00
Andrew Wang
97583dbb0a Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1603993 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 18:32:13 +00:00
Chris Nauroth
db437b7cbb HADOOP-10716. Cannot use more than 1 har filesystem. Contributed by Rushabh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603668 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 23:27:45 +00:00
Andrew Wang
5e5e1a6202 Merge trunk r1603663 to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1603664 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 23:15:04 +00:00
Owen O'Malley
c79728478c HADOOP-10607. Create API to separate credential/password storage from 
applications. (Larry McCay via omalley)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603491 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 15:45:20 +00:00
Charles Lamb
871616b993 merge from trunk r1598783 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1598785 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 00:01:20 +00:00
Brandon Li
42391d260d HDFS-6056. Clean up NFS config settings. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598782 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 23:53:00 +00:00
Yi Liu
f7921030cd HADOOP-10635. Add a method to CryptoCodec to generate SRNs for IV. Contributed by Yi Liu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1598493 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 08:22:41 +00:00
Charles Lamb
333562b767 merge from trunk r1598430 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1598435 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 22:27:25 +00:00
Charles Lamb
2e5ae1aad7 HADOOP-10628. Javadoc and few code style improvement for Crypto input and output streams. (yliu via clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1598429 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 22:09:51 +00:00
Arpit Agarwal
4a4868e523 HADOOP-10448. Support pluggable mechanism to specify proxy user settings (Contributed by Benoy Antony) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598396 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 20:52:01 +00:00
Yi Liu
b20180ffa6 HADOOP-10603. Crypto input and output streams implementing Hadoop stream interfaces. Contributed by Yi Liu and Charles Lamb. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1597230 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-24 01:19:06 +00:00
Alejandro Abdelnur
17d4fbbf0a HADOOP-10433. Key Management Server based on KeyProvider API. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592637 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-05 21:43:14 +00:00
Aaron Myers
1ac2ce067c HADOOP-10568. Add s3 server-side encryption. Contributed by David S. Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592133 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-03 00:25:09 +00:00
Jonathan Turner Eagles
7a655d5633 HADOOP-10454. Provide FileContext version of har file system. (Kihwal Lee via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584431 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-03 22:20:47 +00:00
Arpit Agarwal
f799618828 HADOOP-10221. Add a plugin to specify SaslProperties for RPC protocol based on connection properties. (Contributed by Benoy Antony and Daryn Sharp) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579382 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-19 20:16:55 +00:00
Arpit Agarwal
097e8b205e HADOOP-10211. Enable RPC protocol to negotiate SASL-QOP values between clients and servers. (Contributed by Benoy Antony) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574697 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 22:30:56 +00:00
Aaron Myers
4dfdee0c56 HADOOP-9454. Support multipart uploads for s3native. Contributed by Jordan Mendelson and Akira AJISAKA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572235 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:28:41 +00:00
Andrew Wang
a3151a9f3d HADOOP-10305. Add rpc.metrics.quantile.enable and rpc.metrics.percentiles.intervals to core-default.xml. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562659 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 01:03:28 +00:00
Owen O'Malley
7730629164 HADOOP-10141. Create KeyProvider API to separate encryption key storage 
from the applications. (omalley)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552462 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-20 00:25:42 +00:00
Colin McCabe
7f059104d2 HADOOP-10142. Avoid groups lookup for unprivileged users such as dr.who (vinay via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548763 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-07 00:11:15 +00:00
Jing Zhao
d02baff9a0 HDFS-5536. Implement HTTP policy for Namenode and DataNode. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547925 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-04 21:40:57 +00:00
Sanford Ryza
f7fe50d55f HADOOP-10127. Add ipc.client.connect.retry.interval to control the frequency of connection retries (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547626 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-03 22:44:46 +00:00
Colin McCabe
b7aca6bd90 HADOOP-10079. log a warning message if group resolution takes too long (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538108 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-02 00:42:59 +00:00
Steve Loughran
3caca924bc HADOOP-8545. Filesystem Implementation for OpenStack Swift 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526854 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-27 11:32:53 +00:00
Brandon Li
3582026204 HDFS-5246. Make Hadoop nfs server port and mount daemon port configurable. Contributed by Jinghui Wang 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526316 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-25 21:59:35 +00:00