big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,478 Commits 2 Branches 2 Tags 264 MiB
5ca1ea89b3
Commit Graph

678 Commits

Author SHA1 Message Date
He Xiaoqiao
005b854f6b
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:44:12 +08:00
Liang-Chi Hsieh
c2cecfc9b9
HADOOP-17425. Bump up snappy-java to 1.1.8.2. (#2536) 2020-12-10 13:09:44 +09:00
Steve Loughran
fb79be932c
HADOOP-17343. Upgrade AWS SDK to 1.11.901 (#2468) 
Contributed by Steve Loughran.
 2020-11-23 14:08:12 +00:00
Liang-Chi Hsieh
34aa6137bd
HADOOP-17292. Using lz4-java in Lz4Codec (#2350) 
Contributed by Liang-Chi Hsieh.
 2020-11-18 12:03:25 -08:00
Akira Ajisaka
fc961b63d1
Revert "HADOOP-17269. [JDK 11] Upgrade SpotBugs to 4.1.3 to fix false-positive warnings. (#2374)" (#2460) 
This reverts commit 2de7240d4f.
 2020-11-12 15:52:36 +09:00
Masatake Iwasaki
0946f58cae
HADOOP-17369. Bump up snappy-java to 1.1.8.1. (#2451) 2020-11-10 15:01:24 +09:00
Dongjoon Hyun
2cb9e8f66a
HADOOP-17341. Upgrade commons-codec to 1.15 (#2428) 
Reviewed-by: Chao Sun <sunchao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-11-04 09:12:31 -08:00
Wei-Chiu Chuang
e580280a8b HADOOP-17030. Remove unused joda-time. Contributed by Wei-Chiu Chuang. 2020-11-02 16:07:04 -08:00
Brahma Reddy Battula
eb84793af1 HADOOP-17236. Bump up snakeyaml to 1.26 to mitigate CVE-2017-18640. Contributed by Brahma Reddy Battula. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-28 09:27:32 -07:00
Akira Ajisaka
3a0922fb42
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 2020-10-19 10:47:49 +09:00
Ayush Saxena
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. (#2342). Contributed by Ayush Saxena. 2020-10-17 12:01:18 +05:30
Pranav Bheda
be3edd0532
HADOOP-17223 update org.apache.httpcomponents:httpclient to 4.5.13 and httpcore to 4.4.13 (#2242) 
* update org.apache.httpcomponents:httpclient from 4.5.6 to 4.5.13
* update org.apache.httpcomponents:httpcore from 4.4.10 to 4.4.13
 2020-10-13 17:54:41 +09:00
Akira Ajisaka
2de7240d4f
HADOOP-17269. [JDK 11] Upgrade SpotBugs to 4.1.3 to fix false-positive warnings. (#2374) 2020-10-10 00:28:05 +09:00
Doroszlai, Attila
1cfe5916e2 HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-06 14:58:42 -07:00
Liang-Chi Hsieh
c9ea344f98
HADOOP-17125. Use snappy-java in SnappyCodec (#2297) 
This switches the SnappyCodec to use the java-snappy codec, rather than the native one. 

To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.

This comesin as an avro dependency,  so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7

Contributed by DB Tsai and Liang-Chi Hsieh
 2020-10-06 17:07:54 +01:00
Szilard Nemeth
6e3d7050a9 YARN-10386. Create new JSON schema for Placement Rules. Contributed by Peter Bacsko and Benjamin Teke 2020-08-28 17:09:31 +02:00
Brahma Reddy Battula
15a0fed637 HADOOP-17220. Upgrade slf4j to 1.7.30 ( To Address: CVE-2018-8088). Contributed by Brahma Reddy Battula. 2020-08-24 19:03:22 +05:30
Akira Ajisaka
40542024df
HADOOP-17182. Remove breadcrumbs from web site (#2190) 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2020-08-07 23:29:52 -07:00
Akira Ajisaka
ed3ab4b87d
HADOOP-17179. [JDK 11] Fix javadoc error while detecting Java API link (#2186) 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-08-04 12:09:14 -07:00
Akira Ajisaka
c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors (#2098) 2020-08-03 10:46:51 +09:00
Masatake Iwasaki
2ba44a73bf
HADOOP-16866. Upgrade spotbugs to 4.0.6. (#2146) 2020-07-17 19:58:44 +09:00
Yiqun Lin
ff8bb67200 HDFS-15374. Add documentation for fedbalance tool. Contributed by Jinglun. 2020-07-01 14:18:18 +08:00
Thomas Marquardt
4b5b54c73f
HADOOP-17089: WASB: Update azure-storage-java SDK 
Contributed by Thomas Marquardt

DETAILS: WASB depends on the Azure Storage Java SDK. There is a concurrency
bug in the Azure Storage Java SDK that can cause the results of a list blobs
operation to appear empty. This causes the Filesystem listStatus and similar
APIs to return empty results. This has been seen in Spark work loads when jobs
use more than one executor core.

See Azure/azure-storage-java#546 for details on the bug in the Azure Storage SDK.

TESTS: A new test was added to validate the fix. All tests are passing:

wasb:
mvn -T 1C -Dparallel-tests=wasb -Dscale -DtestsThreadCount=8 clean verify
Tests run: 248, Failures: 0, Errors: 0, Skipped: 11
Tests run: 651, Failures: 0, Errors: 0, Skipped: 65

abfs:
mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify
Tests run: 64, Failures: 0, Errors: 0, Skipped: 0
Tests run: 437, Failures: 0, Errors: 0, Skipped: 33
Tests run: 206, Failures: 0, Errors: 0, Skipped: 24
 2020-06-25 02:32:42 +00:00
Yiqun Lin
9cbd76cc77 HDFS-15346. FedBalance tool implementation. Contributed by Jinglun. 2020-06-18 13:33:25 +08:00
Uma Maheswara Rao G
76fa0222f0
HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G. 2020-06-05 10:58:21 -07:00
Akira Ajisaka
52b21de1d8
HADOOP-17049. javax.activation-api and jakarta.activation-api define overlapping classes (#2027) 
* Removed javax.activation-api from dependency
 2020-05-22 11:19:11 +09:00
Surendra Singh Lilhore
1958cb7c2b YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng 2020-05-14 15:09:34 +05:30
Ayush Saxena
0918433b4d YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng. 2020-05-14 00:36:20 +05:30
Wei-Chiu Chuang
bd342bef64
HADOOP-17033. Update commons-codec from 1.11 to 1.14. (#2000) 2020-05-11 08:41:14 -07:00
Sammi Chen
82ff7bc9ab HADOOP-16959. Resolve hadoop-cos dependency conflict. Contributed by Yang Yu. 2020-04-20 18:14:11 +08:00
sunlisheng
ae474e1ae5 HADOOP-16982. Update Netty to 4.1.48.Final. Contributed by Lisheng Sun. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2020-04-15 06:41:32 +09:00
Akira Ajisaka
e53d472bb0
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 (#1933) 2020-04-09 01:47:31 +09:00
Brahma Reddy Battula
3eeb2466e9 upate the hadoop.version property in the root pom.xml and hadoop.assemblies.version in hadoop-project/pom.xml (see HADOOP-15369) 2020-03-29 23:44:20 +05:30
Brahma Reddy Battula
8914cf9167 Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
Vinayakumar B
f02d5abacd
HADOOP-16927. Update hadoop-thirdparty dependency version to 1.0.0 (#1900). Contributed by Vinayakumar B. 2020-03-20 15:13:07 +05:30
Wei-Chiu Chuang
cd91ef649e
YARN-10195. Dependency divergence building Timeline Service on HBase 2.2.0 and above. (#1895) 2020-03-13 11:20:28 -07:00
Brahma Reddy Battula
c6b8a30386 HADOOP-16871. Upgrade Netty version to 4.1.45.Final to handle CVE-2019-20444,CVE-2019-16869 2020-03-09 19:21:58 +05:30
John Zhuge
999096d82e
HADOOP-14206. TestSFTPFileSystem#testFileExists failure: Invalid encoding for signature. Contributed by Jim Brennan. 2020-03-07 22:56:11 -08:00
Wei-Chiu Chuang
69faaa1d58
HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. (#1876) 2020-03-07 12:18:01 +09:00
Siyao Meng
e36b272608
HADOOP-16891. Upgrade jackson-databind to 2.9.10.3 (#1865) 2020-02-28 17:02:41 -08:00
Akira Ajisaka
7f35676f90
HADOOP-16869. Upgrade findbugs-maven-plugin to 3.0.5 to fix mvn findbugs:findbugs failure (#1855) 2020-02-21 12:22:16 +09:00
Akira Ajisaka
d6d7f8d8c5
YARN-8374. Upgrade objenesis to 2.6 (#1798) 2020-02-19 09:50:37 +09:00
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Zhankun Tang
d40d7cc4f9 HADOOP-16670. Stripping Submarine code from Hadoop codebase. Contributed by Zhankun Tang. 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-01-21 20:07:31 -08:00
Sahil Takiar
f206b736f0
HADOOP-16346. Stabilize S3A OpenSSL support. 
Introduces `openssl` as an option for `fs.s3a.ssl.channel.mode`.
The new option is documented and marked as experimental.

For details on how to use this, consult the peformance document
in the s3a documentation.

This patch is the successor to HADOOP-16050 "S3A SSL connections
should use OpenSSL" -which was reverted because of
incompatibilities between the wildfly OpenSSL client and the AWS
HTTPS servers (HADOOP-16347). With the Wildfly release moved up
to 1.0.7.Final (HADOOP-16405) everything should now work.

Related issues:

* HADOOP-15669. ABFS: Improve HTTPS Performance
* HADOOP-16050: S3A SSL connections should use OpenSSL
* HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8
* HADOOP-16405: Upgrade Wildfly Openssl version to 1.0.7.Final

Contributed by Sahil Takiar

Change-Id: I80a4bc5051519f186b7383b2c1cea140be42444e
 2020-01-21 16:37:51 +00:00
Masatake Iwasaki
7f3510048d HADOOP-16803. Upgrade jackson-databind to 2.9.10.2. (#1805) 2020-01-15 19:50:25 +09:00
Adam Antal
20a90c0b5b
YARN-10071. Sync Mockito version with other modules 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-01-10 17:41:04 +09:00
Akira Ajisaka
a40dc9ee31 HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. (#1796) 2020-01-09 16:24:58 +09:00
Tamás Pénzes
f1f3f23c3c HADOOP-16772. Extract version numbers to head of pom.xml (addendum) (#1773) 
Follow up task of HADOOP-16729, extract even more version numbers.

Change-Id: I2aba142657f3978b24be2560ed6161f1132a8f9e
 2020-01-08 12:25:01 +01:00
Ayush Saxena
1b04bcc0d9 HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena. 2020-01-02 21:42:36 +05:30