big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,448 Commits 2 Branches 2 Tags 264 MiB
63392376c9
Commit Graph

254 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
4ffe5eb1dd
HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 (#2959) 
* HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
Change-Id: I0f31409923ece24a82dfba4c4610d8a38c52d9fb

* HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661). Contributed by Vinayakumar B.

(cherry picked from commit c4c0683dff)

Co-authored-by: Xiaoyu Yao <xyao@apache.org>
Co-authored-by: Vinayakumar B <vinayakumarb@apache.org>
 2021-05-17 18:57:46 -07:00
Takanobu Asanuma
14fddba715 HDFS-15252. HttpFS: setWorkingDirectory should not accept invalid paths. Contributed by hemanthboyina. 
(cherry picked from commit 736659e0e1)
 2021-04-02 17:13:59 +08:00
Inigo Goiri
41fad5056c HDFS-15265. HttpFS: validate content-type in HttpFSUtils. Contributed by hemanthboyina. 
(cherry picked from commit 31b2f687ef)
 2021-04-02 14:06:52 +08:00
Takanobu Asanuma
9a0c5d0ca2 HDFS-15275. HttpFS: Response of Create was not correct with noredirect and data are true. Contributed by hemanthboyina. 
(cherry picked from commit 1fdfaebd98)
 2021-03-24 17:44:29 +08:00
Takanobu Asanuma
9e1e89b1b9 HDFS-15902. Improve the log for HTTPFS server operation. Contributed by Bhavik Patel. 
(cherry picked from commit b4d97a8dc7)
 2021-03-24 17:53:19 +09:00
Akira Ajisaka
de2904f123
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 23b343aed1)

 Conflicts:
	dev-support/docker/Dockerfile
	hadoop-project/pom.xml
 2021-03-11 14:57:03 +09:00
Wei-Chiu Chuang
94c126cc9e HDFS-15719. [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout (#2533) 
(cherry picked from commit 2b4febcf57)
 2021-01-04 20:56:18 -08:00
Jim Brennan
27e455a1a0 HDFS-15711. Add Metrics to HttpFS Server. (#2521) Contributed by Ahmed Hussein and Kihwal Lee 2020-12-10 20:57:43 +00:00
Ayush Saxena
8378ab9f92 HADOOP-17288. Use shaded guava from thirdparty. Contributed by Ayush Saxena. #2505 2020-12-10 05:50:55 +05:30
Ahmed Hussein
a7dbd3b6f9 HDFS-15706. HttpFS: Log more information on request failures. (#2515) 
(cherry picked from commit 07655a7040)
 2020-12-03 15:15:18 -08:00
Akira Ajisaka
db04195afd
HADOOP-17394. [JDK 11] Fix error in mvn package -Pdocs (#2488) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 2ce2198287)
 2020-11-26 11:34:39 +09:00
Kihwal Lee
41a3c9bc95 HDFS-15628. HttpFS server throws NPE if a file is a symlink. Contributed by Ahmed Hussein. 
(cherry picked from commit e45407128d)
 2020-10-14 17:28:02 -05:00
Kihwal Lee
7eb91ac1b2 HDFS-15581. Access Controlled HttpFS Server. Contributed by Richard Ross. 
(cherry picked from commit dfc2682213)
 2020-09-22 10:55:26 -05:00
Mingliang Liu
1cd1f978da
HDFS-15499. Clean up httpfs/pom.xml to remove aws-java-sdk-s3 exclusion (#2188) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-08-04 20:49:50 -07:00
Akira Ajisaka
dfa7f160a5
Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
Takanobu Asanuma
9a3a28018a HDFS-15190. HttpFS: Add Support for Storage Policy Satisfier. Contributed by hemanthboyina. 2020-02-29 10:30:26 +09:00
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Inigo Goiri
061421fc6d HDFS-15145. HttpFS: getAclStatus() returns permission as null. Contributed by hemanthboyina. 2020-01-28 10:04:38 -08:00
Akira Ajisaka
f6d20daf40
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li. 2020-01-21 18:03:24 +09:00
Wei-Chiu Chuang
6b86a5110e HDFS-15097. Purge log in KMS and HttpFS. Contributed by Doris Gu. 2020-01-13 10:48:53 -08:00
Takanobu Asanuma
9da294a140 HDFS-15110. HttpFS: post requests are not supported for path "/". Contributed by hemanthboyina. 2020-01-10 17:53:19 +09:00
Takanobu Asanuma
782c0556fb HDFS-15102. HttpFS: put requests are not supported for path "/". Contributed by hemanthboyina. 2020-01-10 09:52:13 +09:00
Takanobu Asanuma
59aac00283 HDFS-15066. HttpFS: Implement setErasureCodingPolicy , unsetErasureCodingPolicy , getErasureCodingPolicy. Contributed by hemanthboyina. 2020-01-07 11:10:32 +09:00
Takanobu Asanuma
074050ca59 HDFS-15063. HttpFS: getFileStatus doesn't return ecPolicy. Contributed by hemanthboyina. 2020-01-01 11:26:38 +09:00
Masatake Iwasaki
4dffd81bb7
HDFS-14522. Allow compact property description in xml in httpfs. (#1737) 2019-12-10 10:30:57 +09:00
Takanobu Asanuma
4ede8bce28 HDFS-9695. HTTPFS - CHECKACCESS operation missing. Contributed by hemanthboyina 2019-12-03 09:38:36 +09:00
Inigo Goiri
3037762b2c HDFS-14949. Add getServerDefaults() support to HttpFS. Contributed by hemanthboyina. 2019-11-20 10:51:58 -08:00
Yuming Wang
3d41f33018 HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng. 
Co-authored-By: Siyao Meng <smeng@cloudera.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-10-18 08:25:02 -07:00
Ayush Saxena
eeb58a07e2 HDFS-14898. Use Relative URLS in Hadoop HDFS HTTP FS. Contributed by David Mollitor. 2019-10-10 01:03:33 +05:30
Inigo Goiri
d8313b2274 HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun. 2019-09-28 17:20:44 -07:00
Steve Loughran
e346e3638c HADOOP-15691 Add PathCapabilities to FileSystem and FileContext. 
Contributed by Steve Loughran.

This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.

This is intended to allow applications to determine

* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.

As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
 fs. + schema + .

The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.

Note

* The remote store is not expected to be checked for the feature;
  It is more a check of client API and the client's configuration/knowledge
  of the state of the remote system.
* Permissions are not checked.

Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
 2019-09-25 12:16:41 +01:00
Prabhu Joseph
3f89084ac7
HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-09-25 10:19:30 +09:00
Malcolm Taylor
56248f9d87
HADOOP-16556. Fix some alerts raised by LGTM. 
Contributed by Malcolm Taylor.

Change-Id: Ic60c3f4681dd9d48b3afcba7520bd1e4d3cc4231
 2019-09-19 16:00:05 +01:00
Siyao Meng
6ae8bc3a4a HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields (#1267) Contributed by Siyao Meng. 2019-08-13 16:27:57 -07:00
Siyao Meng
17e8cf501b HDFS-14686. HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null (#1192) Contributed by Siyao Meng. 2019-08-01 17:15:22 -07:00
Chao Sun
3ae775d740 HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-07-30 16:01:17 -07:00
Takanobu Asanuma
98d2065643 HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-20 20:17:24 -07:00
Takanobu Asanuma
35f1014b3e HDFS-13654. Use a random secret when a secret file doesn't exist in HttpFS. This should be default. 2019-05-31 10:29:24 +09:00
Akira Ajisaka
afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Giovanni Matteo Fumarola
7a3188d054 HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena. 2019-05-02 12:58:42 -07:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Weiwei Yang
ad5256e44d HDFS-14063. Support noredirect param for CREATE/APPEND/OPEN/GETFILECHECKSUM in HttpFS. Contributed by Íñigo Goiri. 2018-11-17 01:12:22 +08:00
Yiqun Lin
4e3df75eb7 HDFS-14049. TestHttpFSServerWebServer fails on Windows because of missing winutils.exe. Contributed by Inigo Goiri. 2018-11-04 09:15:53 +08:00
Siyao Meng
8fe85af63b HDFS-13996. Make HttpFS' ACLs RegEx configurable. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-11-01 17:36:50 -07:00
Siyao Meng
6dcfef79af HDFS-13878. HttpFS: Implement GETSNAPSHOTTABLEDIRECTORYLIST. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-11 15:01:50 -07:00
Akira Ajisaka
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Siyao Meng
396ce0d9f4 HDFS-13877. HttpFS: Implement GETSNAPSHOTDIFF. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-04 15:17:47 -07:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Akira Ajisaka
284aca8b63
HDFS-13938. Add a missing break in BaseTestHttpFSWith. Contributed by Siyao Meng. 2018-09-28 15:44:46 +09:00