big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,489 Commits 2 Branches 2 Tags 264 MiB
0d6939ffc4
Commit Graph

547 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
5cebd02bbe
HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9a0a808338)
 2021-06-01 11:36:27 +08:00
Steve Loughran
bb04b60197
HADOOP-17735. Upgrade AWS SDK to 1.11.1026 (#3057) 
Change-Id: Ia9b32ab35454ada5f5e3d47b72cd9f2a2446a309
 2021-05-31 21:49:39 +01:00
Takanobu Asanuma
d8689f1a08 Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) 
This reverts commit 0774116756.

Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53ff2dfed3)
 2021-05-27 13:17:24 +09:00
Akira Ajisaka
86682a83aa
HADOOP-17663. Remove useless property hadoop.assemblies.version in pom file. (#3025) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3f5a66c158)

 Conflicts:
	hadoop-project/pom.xml
 2021-05-20 10:49:40 +09:00
Wei-Chiu Chuang
fa4915fdbb
Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
Wei-Chiu Chuang
611ed04aff
HADOOP-17426. Upgrade to hadoop-thirdparty-1.1.0. (#3024) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ibf27246b3eb1b074b938f44f4906db9184466303
(cherry picked from commit 2960d83c25)
 2021-05-19 14:57:45 +09:00
dependabot[bot]
b2897fdd66
HADOOP-17683. Update commons-io to 2.8.0 (#2974) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 29105ffb63)
 2021-05-12 10:58:39 +09:00
Wei-Chiu Chuang
99e4e9bad6
HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2956) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: I7de255cb9a167e3a5e25ae339b050e82cb4b7c30
(cherry picked from commit 64264d726d)
 2021-04-27 10:43:18 +09:00
Wei-Chiu Chuang
1425512032 Revert "HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952)" 
This reverts commit d525e1e33d.
 2021-04-26 16:42:44 +08:00
Wei-Chiu Chuang
d525e1e33d HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 66dda86fec)
 2021-04-26 14:07:35 +08:00
Akira Ajisaka
842714d99f
HADOOP-17655. Upgrade Jetty to 9.4.40. (#2947) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 0b04c9694d)
 2021-04-23 19:53:30 +09:00
Viraj Jasani
49f6326a9f HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs (#2895). Contributed by Viraj Jasani. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-04-16 12:41:06 +05:30
Wei-Chiu Chuang
7f7535534d HADOOP-17625. Update to Jetty 9.4.39. (#2870) 
Reviewed-by: cxorm <lianp964@gmail.com>
(cherry picked from commit 6040e86e99)
 2021-04-08 14:28:03 +08:00
Wei-Chiu Chuang
cc0cc7ab5f HADOOP-17614. Bump netty to the latest 4.1.61. (#2850) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 29a6769b68)
 2021-04-05 08:37:10 +08:00
Akira Ajisaka
80d4636067
HADOOP-17602. Upgrade JUnit to 4.13.1. Contributed by Ahmed Hussein. 
(cherry picked from commit c5929c00a3)

 Conflicts:
	hadoop-cloud-storage-project/hadoop-huaweicloud/pom.xml
 2021-03-25 19:54:51 +09:00
Mingliang Liu
23082ac6c7
YARN-10706. Upgrade com.github.eirslett:frontend-maven-plugin to 1.11.2 (#2791) 
Contributed by Mingliang Liu.

Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-21 21:14:12 -07:00
Ayush Saxena
28e89509a3 HADOOP-17586. Upgrade org.codehaus.woodstox:stax2-api to 4.2.1. (#2769). Contributed by Ayush Saxena. 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-03-13 15:10:27 +05:30
Viraj Jasani
36313b38cb
HADOOP-17571 : Bump up woodstox-core to 5.3.0 due to security concerns (#2757) 
Contributed by Viraj Jasani.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-11 11:55:09 -08:00
Akira Ajisaka
de2904f123
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 23b343aed1)

 Conflicts:
	dev-support/docker/Dockerfile
	hadoop-project/pom.xml
 2021-03-11 14:57:03 +09:00
Takanobu Asanuma
e607d03995 HADOOP-17563. Update Bouncy Castle to 1.68. (#2740) 
(cherry picked from commit 0774116756)
 2021-03-05 22:59:09 +09:00
Akira Ajisaka
852db790d3
HADOOP-17534. Update Jackson to 2.10.5 and Jackson databind to 2.10.5.1 (#2708) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e119063a07)
 2021-02-22 17:34:43 +09:00
Akira Ajisaka
98eec71218
HADOOP-17516. Upgrade ant to 1.10.9. (#2683) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 933fbeffee)
 2021-02-11 08:07:55 +09:00
Akira Ajisaka
7106a789f4
HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage (#2556) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 9cf408d6ba)
 2021-01-24 22:52:51 +09:00
Wei-Chiu Chuang
6340ac857b HADOOP-17371. Bump Jetty to the latest version 9.4.34. Contributed by Wei-Chiu Chuang. (#2453) 
(cherry picked from commit 66ee0a6df0)
 2021-01-04 11:28:26 -08:00
Liang-Chi Hsieh
87064df1f2 HADOOP-17292. Using lz4-java in Lz4Codec (#2350) 
Contributed by Liang-Chi Hsieh.
 2020-12-29 13:17:26 -08:00
He Xiaoqiao
3736f6eb57
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:47:16 +08:00
Liang-Chi Hsieh
485e4e3348 HADOOP-17425. Bump up snappy-java to 1.1.8.2. (#2536) 
(cherry picked from commit c2cecfc9b9)

 Conflicts:
	hadoop-project/pom.xml
 2020-12-10 13:58:49 +00:00
Masatake Iwasaki
3a50148809 HADOOP-17369. Bump up snappy-java to 1.1.8.1. (#2451) 
(cherry picked from commit 0946f58cae)
 2020-12-10 13:57:35 +00:00
Ayush Saxena
8378ab9f92 HADOOP-17288. Use shaded guava from thirdparty. Contributed by Ayush Saxena. #2505 2020-12-10 05:50:55 +05:30
Steve Loughran
e4bc64cce0 HADOOP-17343. Upgrade AWS SDK to 1.11.901 (#2468) 
Contributed by Steve Loughran.
 2020-11-23 14:09:14 +00:00
Brahma Reddy Battula
fe8b22ca8f HADOOP-17236. Bump up snakeyaml to 1.26 to mitigate CVE-2017-18640. Contributed by Brahma Reddy Battula. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit eb84793af1)
 2020-10-28 09:28:09 -07:00
Akira Ajisaka
37c2e34b02
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 
(cherry picked from commit 3a0922fb42)
 2020-10-19 10:49:21 +09:00
Pranav Bheda
054dba68c2
HADOOP-17223 update org.apache.httpcomponents:httpclient to 4.5.13 and httpcore to 4.4.13 (#2242) 
* update org.apache.httpcomponents:httpclient from 4.5.6 to 4.5.13
* update org.apache.httpcomponents:httpcore from 4.4.10 to 4.4.13

(cherry picked from commit be3edd0532)
 2020-10-13 17:56:38 +09:00
Doroszlai, Attila
13e0c5f6e0 HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-08 23:44:56 -07:00
Liang-Chi Hsieh
8f60a90688 HADOOP-17125. Use snappy-java in SnappyCodec (#2297) 
This switches the SnappyCodec to use the java-snappy codec, rather than the native one.

To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.

This comesin as an avro dependency,  so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7

Contributed by DB Tsai and Liang-Chi Hsieh

Change-Id: Id52a404a0005480e68917cd17f0a27b7744aea4e
 2020-10-06 17:15:17 +01:00
Brahma Reddy Battula
d05051c840 HADOOP-17220. Upgrade slf4j to 1.7.30 ( To Address: CVE-2018-8088). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 15a0fed637)
 2020-08-24 19:05:27 +05:30
Thomas Marquardt
ee192c4826
HADOOP-17089: WASB: Update azure-storage-java SDK 
Contributed by Thomas Marquardt

DETAILS: WASB depends on the Azure Storage Java SDK. There is a concurrency
bug in the Azure Storage Java SDK that can cause the results of a list blobs
operation to appear empty. This causes the Filesystem listStatus and similar
APIs to return empty results. This has been seen in Spark work loads when jobs
use more than one executor core.

See Azure/azure-storage-java#546 for details on the bug in the Azure Storage SDK.

TESTS: A new test was added to validate the fix. All tests are passing:

wasb:
mvn -T 1C -Dparallel-tests=wasb -Dscale -DtestsThreadCount=8 clean verify
Tests run: 248, Failures: 0, Errors: 0, Skipped: 11
Tests run: 651, Failures: 0, Errors: 0, Skipped: 65

abfs:
mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify
Tests run: 64, Failures: 0, Errors: 0, Skipped: 0
Tests run: 437, Failures: 0, Errors: 0, Skipped: 33
Tests run: 206, Failures: 0, Errors: 0, Skipped: 24
 2020-06-25 05:43:32 +00:00
Akira Ajisaka
6b54f259e7
HADOOP-17049. javax.activation-api and jakarta.activation-api define overlapping classes (#2027) 
* Removed javax.activation-api from dependency

(cherry picked from commit 52b21de1d8)
 2020-05-22 11:20:16 +09:00
Surendra Singh Lilhore
7cef7109de YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng 2020-05-14 15:17:13 +05:30
Ayush Saxena
74887ab2ad YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng. 2020-05-14 00:37:09 +05:30
Akira Ajisaka
dfa7f160a5
Preparing for 3.3.1 development 2020-04-30 13:33:42 +09:00
Sammi Chen
9c81b17153 HADOOP-16959. Resolve hadoop-cos dependency conflict. Contributed by Yang Yu. 
(cherry picked from commit 82ff7bc9ab)
 2020-04-20 18:57:49 +08:00
sunlisheng
7aad965198 HADOOP-16982. Update Netty to 4.1.48.Final. Contributed by Lisheng Sun. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit ae474e1ae5)
 2020-04-15 07:22:54 +09:00
Akira Ajisaka
e8520fd132
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 (#1933) 
(cherry picked from commit e53d472bb0)
 2020-04-09 01:49:01 +09:00
Vinayakumar B
f02d5abacd
HADOOP-16927. Update hadoop-thirdparty dependency version to 1.0.0 (#1900). Contributed by Vinayakumar B. 2020-03-20 15:13:07 +05:30
Wei-Chiu Chuang
cd91ef649e
YARN-10195. Dependency divergence building Timeline Service on HBase 2.2.0 and above. (#1895) 2020-03-13 11:20:28 -07:00
Brahma Reddy Battula
c6b8a30386 HADOOP-16871. Upgrade Netty version to 4.1.45.Final to handle CVE-2019-20444,CVE-2019-16869 2020-03-09 19:21:58 +05:30
John Zhuge
999096d82e
HADOOP-14206. TestSFTPFileSystem#testFileExists failure: Invalid encoding for signature. Contributed by Jim Brennan. 2020-03-07 22:56:11 -08:00
Wei-Chiu Chuang
69faaa1d58
HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. (#1876) 2020-03-07 12:18:01 +09:00
Siyao Meng
e36b272608
HADOOP-16891. Upgrade jackson-databind to 2.9.10.3 (#1865) 2020-02-28 17:02:41 -08:00