big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,306 Commits 2 Branches 2 Tags 264 MiB
d00b3acd5e
Commit Graph

242 Commits

Author SHA1 Message Date
Brahma Reddy Battula
8914cf9167 Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Sahil Takiar
f206b736f0
HADOOP-16346. Stabilize S3A OpenSSL support. 
Introduces `openssl` as an option for `fs.s3a.ssl.channel.mode`.
The new option is documented and marked as experimental.

For details on how to use this, consult the peformance document
in the s3a documentation.

This patch is the successor to HADOOP-16050 "S3A SSL connections
should use OpenSSL" -which was reverted because of
incompatibilities between the wildfly OpenSSL client and the AWS
HTTPS servers (HADOOP-16347). With the Wildfly release moved up
to 1.0.7.Final (HADOOP-16405) everything should now work.

Related issues:

* HADOOP-15669. ABFS: Improve HTTPS Performance
* HADOOP-16050: S3A SSL connections should use OpenSSL
* HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8
* HADOOP-16405: Upgrade Wildfly Openssl version to 1.0.7.Final

Contributed by Sahil Takiar

Change-Id: I80a4bc5051519f186b7383b2c1cea140be42444e
 2020-01-21 16:37:51 +00:00
Akira Ajisaka
bc366d4ea7
HADOOP-16773. Fix duplicate assertj-core dependency in hadoop-common module. Contributed by Xieming Li. 2020-01-07 20:49:24 +09:00
Vinayakumar B
e2a5448d2b
HADOOP-16774. TestDiskChecker and TestReadWriteDiskValidator fails when run with -Pparallel-tests (#1776). Contributed by Vinayakumar B. 2019-12-21 00:08:15 +05:30
Vinayakumar B
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494). Contributed by Vinayakumar B. 2019-09-23 12:37:47 +05:30
Sahil Takiar
55ce454ce4
HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8. 
Contributed by Sahil Takiar.

This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common
as the DelegatingSSLSocketFactory and binds the S3A connector to it so that
it can avoid using those HTTPS algorithms which are underperformant on Java 8.

Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd
 2019-09-17 11:32:03 +01:00
Takanobu Asanuma
54f9f75a44 HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles. 2019-06-14 10:17:25 +09:00
Sammi Chen
d1aad44490 HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00
Steve Loughran
309501c6fa
Revert "HADOOP-16050: s3a SSL connections should use OpenSSL" 
This reverts commit b067f8acaa.

Change-Id: I584b050a56c0e6f70b11fa3f7db00d5ac46e7dd8
 2019-06-05 13:54:55 +01:00
Akira Ajisaka
afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Eric Yang
ea0b1d8fba HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol impersonation. 
Contributed by Prabhu Joseph
 2019-05-23 11:36:32 -04:00
Akira Ajisaka
a771e2a638
HADOOP-12948. Remove the defunct startKdc profile from hadoop-common. Contributed by Wei-Chiu Chuang. 2019-05-23 13:59:42 +09:00
Sahil Takiar
b067f8acaa HADOOP-16050: s3a SSL connections should use OpenSSL 
(cherry picked from commit aebf229c175dfa19fff3b31e9e67596f6c6124fa)
 2019-05-16 08:57:54 -06:00
Todd Lipcon
65deb1ac42 HADOOP-16179. hadoop-common pom should not depend on kerb-simplekdc 
The hadoop-common pom currently has a dependency on kerb-simplekdc. In
fact, the only classes used from Kerby are in kerb-core and kerb-util
(which is a transitive dependency frmo kerb-core). Depending on
kerb-simplekdc pulls a bunch of other unnecessary classes into the
hadoop-common classpath.

This changes the hadoop-common pom to depend only on kerb-core.

hadoop-minikdc already had the appropriate dependency on kerb-simplekdc
so it continues to pull in what it needs.

Signed-off-by: Todd Lipcon <todd@apache.org>
 2019-04-10 08:49:46 -07:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Akira Ajisaka
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Takanobu Asanuma
9bbeb52486 HADOOP-15775. [JDK9] Add missing javax.activation-api dependency. Contributed by Akira Ajisaka. 2018-10-09 13:06:14 +09:00
Akira Ajisaka
31fa9bb799
HADOOP-15767. [JDK10] Building native package on JDK10 fails due to missing javah. Contributed by Takanobu Asanuma. 2018-10-03 11:29:18 +09:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Ewan Higgs
429a07e08c HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka. 2018-09-20 15:13:55 +02:00
Allen Wittenauer
4b8c2b1c1c
HADOOP-14671. Upgrade Apache Yetus to 0.8.0 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-09-11 08:55:34 -07:00
Akira Ajisaka
88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Akira Ajisaka
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Chris Douglas
3fc3fa9711 HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-03-19 16:05:55 -07:00
Chris Douglas
45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Andrew Wang
0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Steve Loughran
621b43e254
HADOOP-13345 HS3Guard: Improved Consistency for S3A. 
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
 2017-09-01 14:13:41 +01:00
Allen Wittenauer
190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build 
Signed-off-by: Andrew Wang <wang@apache.org>
 2017-08-31 19:06:49 -07:00
wenxinhe
691bf5ec5d
HADOOP-14706. Adding a helper method to determine whether a log is Log4j implement. 
This closes #257

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-08-04 14:35:57 +09:00
Chris Douglas
12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Steve Loughran
09653ea098
HADOOP-14634. Remove jline from main Hadoop pom.xml. 
Contributed by Ray Chiang.
 2017-07-10 11:53:13 +01:00
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Jonathan Eagles
a81916ea89 HADOOP-14501. Switch from aalto-xml to woodstox to handle odd XML features (jeagles) 2017-06-12 17:07:53 -05:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Haohui Mai
ff5ec3b841 HADOOP-14383. Implement FileSystem that reads from HTTP / HTTPS endpoints. 2017-05-08 15:28:45 -07:00
Chris Douglas
a5e57df3c5 HADOOP-14225. Remove xmlenc dependency 2017-04-10 11:48:40 -07:00
Jonathan Eagles
523f467d93 HADOOP-14216. Improve Configuration XML Parsing Performance (jeagles) 2017-03-29 10:12:02 -05:00
Mingliang Liu
c04fb35c45 HADOOP-14191. Duplicate hadoop-minikdc dependency in hadoop-common module. Contributed by Xiaobing Zhou 2017-03-16 16:08:32 -07:00
Akira Ajisaka
258342e76c HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR. 2017-02-28 13:47:44 +09:00
Steve Loughran
7fc3e68a87 HADOOP-13433 Race in UGI.reloginFromKeytab. Contributed by Duo Zhang. 2017-01-25 21:29:27 +00:00
Andrew Wang
5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
Jason Lowe
a0a2761621 HADOOP-13578. Add Codec for ZStandard Compression. Contributed by churro morales 2017-01-04 14:46:25 +00:00
Akira Ajisaka
7d8e440eee Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran" 
This reverts commit c265515725.
 2016-12-09 12:49:27 +09:00
Steve Loughran
c265515725 HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-08 17:57:15 +00:00
Akira Ajisaka
b2a3d6c519 HADOOP-13835. Move Google Test Framework code from mapreduce to hadoop-common. Contributed by Varun Vasudev. 2016-12-06 14:01:47 +09:00
Akira Ajisaka
67d9f2808e HADOOP-13706. Update jackson from 1.9.13 to 2.x in hadoop-common-project. 2016-11-29 14:07:19 +09:00
Akira Ajisaka
209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Wei-Chiu Chuang
c0b1a44f6c HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory. 2016-11-17 22:48:35 -06:00
Andrew Wang
e1c6ef2efa HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 2016-11-08 16:33:55 -08:00
Robert Kanter
5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Wei-Chiu Chuang
d88dca844a HADOOP-11798. Native raw erasure coder in XOR codes. Contributed by SammiChen. 2016-10-25 16:53:53 -07:00
Andrew Wang
86c735bcb1 Revert "HADOOP-13696. change hadoop-common dependency scope of jsch to provided. Contributed by Yuanbo Liu." 
This reverts commit cef61d505e.
 2016-10-25 13:45:52 -07:00
Steve Loughran
cef61d505e HADOOP-13696. change hadoop-common dependency scope of jsch to provided. Contributed by Yuanbo Liu. 2016-10-10 12:32:39 +01:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
99603e9022 HADOOP-13428. Fixed hadoop-common build files to generate jdiff by working around a jdiff-bug. Contributed by Wangda Tan. 2016-08-19 19:08:53 -07:00
Wei-Chiu Chuang
03a9343d57 HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen. 2016-08-19 09:22:49 -07:00
Kai Zheng
34ccaa8367 HADOOP-11540. Raw Reed-Solomon coder using Intel ISA-L library. Contributed by Kai Zheng 2016-08-01 06:34:26 +08:00
Andrew Wang
da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Tsuyoshi Ozawa
5d58858bb6 HADOOP-9613. [JDK8] Update jersey version to latest 1.x release. 2016-06-21 08:05:32 +09:00
Allen Wittenauer
e2f640942b HADOOP-13245. Fix up some misc create-release issues (aw) 2016-06-14 16:03:20 -07:00
Kai Zheng
723432b338 HADOOP-13220. Follow on fixups after upgraded mini-kdc using Kerby. Contributed by Jiajia Li 2016-06-09 15:56:12 +08:00
Kai Zheng
916140604f HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li 2016-05-28 14:23:39 +08:00
Andrew Wang
3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang
ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Chris Nauroth
89b16d27e2 HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth. 2016-03-08 21:22:06 -08:00
Masatake Iwasaki
809ebc0b14 HADOOP-12552. Fix undeclared/unused dependency to httpclient (iwasakims) 2016-02-29 18:20:14 +09:00
Allen Wittenauer
79d7949fbb HADOOP-12492. maven install triggers bats test (aw) 2016-01-27 11:27:27 -08:00
Allen Wittenauer
8cecad2d56 HADOOP-12651. Replace dev-support with wrappers to Yetus (aw) 2016-01-21 12:22:03 -08:00
Colin Patrick Mccabe
b1ed28fa77 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 2016-01-14 11:02:34 -08:00
Colin Patrick Mccabe
f349d0a76c HADOOP-12647. Add /usr/lib default for ISAL library and fix missing .c file in CMakeLists (Kai Zheng via cmccabe) 2015-12-18 09:27:26 -08:00
Allen Wittenauer
607473e1d0 HADOOP-12192. update releasedocmaker command line (aw) 2015-12-16 06:23:58 -08:00
Colin Patrick Mccabe
482e35c55a HADOOP-11887. Introduce Intel ISA-L erasure coding library for native erasure encoding support (Kai Zheng via Colin P. McCabe) 2015-11-05 10:44:47 -08:00
Allen Wittenauer
6e0d353235 HADOOP-12541. make re2j dependency consistent (Matthew Paduano via aw) 2015-11-02 20:39:46 -08:00
Allen Wittenauer
4c0bae240b HADOOP-12436. GlobPattern regex library has performance issues with wildcard characters (Matthew Paduano via aw) 2015-10-22 08:19:56 -07:00
Allen Wittenauer
dc45a7a7c4 Revert "HADOOP-12436. GlobPattern regex library has performance issues with wildcard characters (Matthew Paduano via aw)" 
This reverts commit 0d77e85f0a.
 2015-10-15 08:26:06 -07:00
Allen Wittenauer
0d77e85f0a HADOOP-12436. GlobPattern regex library has performance issues with wildcard characters (Matthew Paduano via aw) 2015-10-14 08:59:41 -07:00
Colin Patrick Mccabe
892ade689f HDFS-9080. Update htrace version to 4.0.1 (cmccabe) 2015-09-28 07:42:12 -07:00
Vinayakumar B
f0f984e4e6 HADOOP-11984. Enable parallel JUnit tests in pre-commit (Contributed by Chris Nauroth) 2015-09-27 14:12:07 +05:30
yliu
f4d96be6c6 HADOOP-12367. Move TestFileUtil's test resources to resources folder. (wang via yliu) 2015-09-01 16:20:56 +08:00
Allen Wittenauer
a890a31529 HADOOP-10854. unit tests for the shell scripts (aw) 2015-07-31 14:34:48 -07:00
Chris Douglas
559425dcb9 HADOOP-5732. Add SFTPFileSystem. Contributed by Ramtin Boustani and Inigo Goiri 2015-07-07 10:08:59 -07:00
Andrew Wang
7673d4f205 HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang) 2015-05-29 14:54:18 -07:00
Allen Wittenauer
ae14543420 HADOOP-12022. fix site -Pdocs -Pdist in hadoop-project-dist; cleanout remaining forrest bits (aw) 2015-05-28 13:24:01 -07:00
Vinayakumar B
55ed6558ba HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 2015-05-22 16:45:19 +05:30
Allen Wittenauer
f325522c14 HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) 2015-05-06 10:34:09 -07:00
Allen Wittenauer
f383fd9b6c HADOOP-11731. Rework the changelog and releasenotes (aw) 2015-04-01 17:52:22 -07:00
Allen Wittenauer
b30ca8ce0e HADOOP-11553. Foramlize the shell API (aw) 2015-03-26 15:09:51 -07:00
Colin Patrick Mccabe
09ad9a868a HADOOP-11498. Bump the version of HTrace to 3.1.0-incubating (Masatake Iwasaki via Colin P. McCabe) 2015-01-30 16:01:59 -08:00
Colin Patrick Mccabe
fb20797b62 HADOOP-11410. Make the rpath of libhadoop.so configurable (cmccabe) 2014-12-15 18:55:16 -08:00
Haohui Mai
1eed102023 HADOOP-10847. Remove the usage of sun.security.x509.* in testing code. Contributed by Pascal Oliva. 2014-11-04 10:52:05 -08:00
yliu
2a6be65c8c HADOOP-11216. Improve Openssl library finding. (cmccabe via yliu) 2014-10-30 10:35:14 +08:00
Jian He
3b12fd6cfb YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 2014-10-22 15:57:46 -07:00
Colin Patrick Mccabe
d8d628d1e0 HADOOP-11184. Update Hadoop's lz4 to r123 (cmccabe) 2014-10-09 17:05:35 -07:00
Colin Patrick Mccabe
55302ccfba HDFS-6956. Allow dynamically changing the tracing level in Hadoop servers (cmccabe) 2014-09-26 10:30:30 -07:00
Alejandro Abdelnur
db890eef32 HADOOP-11017. KMS delegation token secret manager should be able to use zookeeper as store. (asuresh via tucu) 2014-09-20 08:21:44 -07:00
Alejandro Abdelnur
c0c7e6fabd HADOOP-11062. CryptoCodec testcases requiring OpenSSL should be run only if -Pnative is used. (asuresh via tucu) 2014-09-16 23:36:36 -07:00
Colin Patrick Mccabe
6962510f72 HDFS-6879. Adding tracing to Hadoop RPC. Contributed by Masatake Iwasaki. 2014-08-27 14:12:05 -07:00