big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,797 Commits 2 Branches 2 Tags 264 MiB
0debe55d6c
Commit Graph

193 Commits

Author SHA1 Message Date
Mike
18d7dfbf35
HDFS-1820. FTPFileSystem attempts to close the outputstream even when it is not initialised. (#1952) 
Contributed by Mikhail Pryakhin.
 2020-04-27 14:43:51 +01:00
Brahma Reddy Battula
8914cf9167 Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Sahil Takiar
f206b736f0
HADOOP-16346. Stabilize S3A OpenSSL support. 
Introduces `openssl` as an option for `fs.s3a.ssl.channel.mode`.
The new option is documented and marked as experimental.

For details on how to use this, consult the peformance document
in the s3a documentation.

This patch is the successor to HADOOP-16050 "S3A SSL connections
should use OpenSSL" -which was reverted because of
incompatibilities between the wildfly OpenSSL client and the AWS
HTTPS servers (HADOOP-16347). With the Wildfly release moved up
to 1.0.7.Final (HADOOP-16405) everything should now work.

Related issues:

* HADOOP-15669. ABFS: Improve HTTPS Performance
* HADOOP-16050: S3A SSL connections should use OpenSSL
* HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8
* HADOOP-16405: Upgrade Wildfly Openssl version to 1.0.7.Final

Contributed by Sahil Takiar

Change-Id: I80a4bc5051519f186b7383b2c1cea140be42444e
 2020-01-21 16:37:51 +00:00
Akira Ajisaka
bc366d4ea7
HADOOP-16773. Fix duplicate assertj-core dependency in hadoop-common module. Contributed by Xieming Li. 2020-01-07 20:49:24 +09:00
Vinayakumar B
e2a5448d2b
HADOOP-16774. TestDiskChecker and TestReadWriteDiskValidator fails when run with -Pparallel-tests (#1776). Contributed by Vinayakumar B. 2019-12-21 00:08:15 +05:30
Vinayakumar B
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494). Contributed by Vinayakumar B. 2019-09-23 12:37:47 +05:30
Sahil Takiar
55ce454ce4
HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8. 
Contributed by Sahil Takiar.

This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common
as the DelegatingSSLSocketFactory and binds the S3A connector to it so that
it can avoid using those HTTPS algorithms which are underperformant on Java 8.

Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd
 2019-09-17 11:32:03 +01:00
Takanobu Asanuma
54f9f75a44 HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles. 2019-06-14 10:17:25 +09:00
Sammi Chen
d1aad44490 HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00
Steve Loughran
309501c6fa
Revert "HADOOP-16050: s3a SSL connections should use OpenSSL" 
This reverts commit b067f8acaa.

Change-Id: I584b050a56c0e6f70b11fa3f7db00d5ac46e7dd8
 2019-06-05 13:54:55 +01:00
Akira Ajisaka
afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Eric Yang
ea0b1d8fba HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol impersonation. 
Contributed by Prabhu Joseph
 2019-05-23 11:36:32 -04:00
Akira Ajisaka
a771e2a638
HADOOP-12948. Remove the defunct startKdc profile from hadoop-common. Contributed by Wei-Chiu Chuang. 2019-05-23 13:59:42 +09:00
Sahil Takiar
b067f8acaa HADOOP-16050: s3a SSL connections should use OpenSSL 
(cherry picked from commit aebf229c175dfa19fff3b31e9e67596f6c6124fa)
 2019-05-16 08:57:54 -06:00
Todd Lipcon
65deb1ac42 HADOOP-16179. hadoop-common pom should not depend on kerb-simplekdc 
The hadoop-common pom currently has a dependency on kerb-simplekdc. In
fact, the only classes used from Kerby are in kerb-core and kerb-util
(which is a transitive dependency frmo kerb-core). Depending on
kerb-simplekdc pulls a bunch of other unnecessary classes into the
hadoop-common classpath.

This changes the hadoop-common pom to depend only on kerb-core.

hadoop-minikdc already had the appropriate dependency on kerb-simplekdc
so it continues to pull in what it needs.

Signed-off-by: Todd Lipcon <todd@apache.org>
 2019-04-10 08:49:46 -07:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Akira Ajisaka
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Takanobu Asanuma
9bbeb52486 HADOOP-15775. [JDK9] Add missing javax.activation-api dependency. Contributed by Akira Ajisaka. 2018-10-09 13:06:14 +09:00
Akira Ajisaka
31fa9bb799
HADOOP-15767. [JDK10] Building native package on JDK10 fails due to missing javah. Contributed by Takanobu Asanuma. 2018-10-03 11:29:18 +09:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Ewan Higgs
429a07e08c HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka. 2018-09-20 15:13:55 +02:00
Allen Wittenauer
4b8c2b1c1c
HADOOP-14671. Upgrade Apache Yetus to 0.8.0 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-09-11 08:55:34 -07:00
Akira Ajisaka
88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Akira Ajisaka
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Chris Douglas
3fc3fa9711 HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-03-19 16:05:55 -07:00
Chris Douglas
45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Andrew Wang
0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Steve Loughran
621b43e254
HADOOP-13345 HS3Guard: Improved Consistency for S3A. 
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
 2017-09-01 14:13:41 +01:00
Allen Wittenauer
190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build 
Signed-off-by: Andrew Wang <wang@apache.org>
 2017-08-31 19:06:49 -07:00
wenxinhe
691bf5ec5d
HADOOP-14706. Adding a helper method to determine whether a log is Log4j implement. 
This closes #257

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-08-04 14:35:57 +09:00
Chris Douglas
12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Steve Loughran
09653ea098
HADOOP-14634. Remove jline from main Hadoop pom.xml. 
Contributed by Ray Chiang.
 2017-07-10 11:53:13 +01:00
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Jonathan Eagles
a81916ea89 HADOOP-14501. Switch from aalto-xml to woodstox to handle odd XML features (jeagles) 2017-06-12 17:07:53 -05:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Haohui Mai
ff5ec3b841 HADOOP-14383. Implement FileSystem that reads from HTTP / HTTPS endpoints. 2017-05-08 15:28:45 -07:00
Chris Douglas
a5e57df3c5 HADOOP-14225. Remove xmlenc dependency 2017-04-10 11:48:40 -07:00
Jonathan Eagles
523f467d93 HADOOP-14216. Improve Configuration XML Parsing Performance (jeagles) 2017-03-29 10:12:02 -05:00
Mingliang Liu
c04fb35c45 HADOOP-14191. Duplicate hadoop-minikdc dependency in hadoop-common module. Contributed by Xiaobing Zhou 2017-03-16 16:08:32 -07:00
Akira Ajisaka
258342e76c HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR. 2017-02-28 13:47:44 +09:00
Steve Loughran
7fc3e68a87 HADOOP-13433 Race in UGI.reloginFromKeytab. Contributed by Duo Zhang. 2017-01-25 21:29:27 +00:00
Andrew Wang
5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
Jason Lowe
a0a2761621 HADOOP-13578. Add Codec for ZStandard Compression. Contributed by churro morales 2017-01-04 14:46:25 +00:00
Akira Ajisaka
7d8e440eee Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran" 
This reverts commit c265515725.
 2016-12-09 12:49:27 +09:00
Steve Loughran
c265515725 HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-08 17:57:15 +00:00
Akira Ajisaka
b2a3d6c519 HADOOP-13835. Move Google Test Framework code from mapreduce to hadoop-common. Contributed by Varun Vasudev. 2016-12-06 14:01:47 +09:00
Akira Ajisaka
67d9f2808e HADOOP-13706. Update jackson from 1.9.13 to 2.x in hadoop-common-project. 2016-11-29 14:07:19 +09:00