big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
25,146 Commits 2 Branches 2 Tags
Commit Graph

750 Commits

Author SHA1 Message Date
Takanobu Asanuma
038fa67db4 HADOOP-17955. Bump netty to the latest 4.1.68. (#3528) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit cf4d2d826cad941ae3648ff9b6057856bb199fc4)
 2021-10-08 10:37:14 +09:00
Warren Zhu
e33509b094
HADOOP-17941. Update xerces to 2.12.1 (#3496) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1db5eb43ad30e2e238dfee51e789ff8e6b9cd505)
 2021-09-29 18:49:52 +09:00
Takanobu Asanuma
f00ab40b4d HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 2068b0041c05a7b53f32606cc24922cfac299ca8)
 2021-09-28 13:31:53 +09:00
Dongjoon Hyun
ca7fb6a813
HADOOP-17939. Support building on Apple Silicon (#3486) 
Support building on Apple Silicon with ARM CPUs by using the x86_64 version of protoc.

Contributed by  Dongjoon Hyun

Change-Id: I4b8330098822f1fd28f0113650eb709d53bcc690
 2021-09-27 13:27:56 +01:00
Ahmed Hussein
fc5d67dfb4 HADOOP-17886. Upgrade ant to 1.10.11 (#3371) 
(cherry picked from commit 051207375b30ffa126cf9c961ee97535c1dfabb1)
 2021-09-02 16:17:32 -05:00
Siyao Meng
226f94b4fc HADOOP-17834. Bump aliyun-sdk-oss to 3.13.0 (#3261) 
Change-Id: I335d4a2cb08c75dc24ef36bdfab51111f87e0762
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3aaac8a1f682ecda4d41221e8a836bc5401e3b1d)
 2021-08-16 00:32:05 +09:00
Renukaprasad C
5b566c3914 HADOOP-17844. Upgrade JSON smart to 2.4.7 (#3299) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b90389ae9840ccfdcab411e286547895a647feed)

 Conflicts:
	LICENSE-binary
 2021-08-14 20:00:38 +09:00
Akira Ajisaka
025ecf42be
HADOOP-17370. Upgrade commons-compress to 1.21 (#3274) 
(cherry picked from commit 3565c9477dffd104a3aa3603845f7a1305815ac3)
 2021-08-08 11:25:26 +09:00
Renukaprasad C
d37192a613
HADOOP-17796. Upgrade jetty version to 9.4.43 (#3208) 
(cherry picked from commit dbd255f4a96474b31af2bd45c952ac7151a265b9)
 2021-07-22 16:37:39 +08:00
Ahmed Hussein
22e7567475
HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery (#3130). Contributed by Ahmed Hussein. (#3138) 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 581f43dce15d5753deb33f72ec275d0ea67b1403)
 2021-06-25 22:48:56 +05:30
Wei-Chiu Chuang
5cebd02bbe
HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9a0a808338558eb2e8074a7611619a7fc991f994)
 2021-06-01 11:36:27 +08:00
Steve Loughran
bb04b60197
HADOOP-17735. Upgrade AWS SDK to 1.11.1026 (#3057) 
Change-Id: Ia9b32ab35454ada5f5e3d47b72cd9f2a2446a309
 2021-05-31 21:49:39 +01:00
Takanobu Asanuma
d8689f1a08 Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) 
This reverts commit 077411675679900f94adba5329d0e33a4a528793.

Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53ff2dfed3b784ff2120fadc85ba70226ecfd5ca)
 2021-05-27 13:17:24 +09:00
Akira Ajisaka
86682a83aa
HADOOP-17663. Remove useless property hadoop.assemblies.version in pom file. (#3025) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3f5a66c1583f6b8f21f4e8db73ce0d65d019e33c)

 Conflicts:
	hadoop-project/pom.xml
 2021-05-20 10:49:40 +09:00
Wei-Chiu Chuang
fa4915fdbb
Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
Wei-Chiu Chuang
611ed04aff
HADOOP-17426. Upgrade to hadoop-thirdparty-1.1.0. (#3024) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ibf27246b3eb1b074b938f44f4906db9184466303
(cherry picked from commit 2960d83c255a00a549f8809882cd3b73a6266b6d)
 2021-05-19 14:57:45 +09:00
dependabot[bot]
b2897fdd66
HADOOP-17683. Update commons-io to 2.8.0 (#2974) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 29105ffb634ceb44a331ba65280d418544b902a6)
 2021-05-12 10:58:39 +09:00
Wei-Chiu Chuang
99e4e9bad6
HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2956) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: I7de255cb9a167e3a5e25ae339b050e82cb4b7c30
(cherry picked from commit 64264d726d81c919d3f682228ff4bd7fe349fcae)
 2021-04-27 10:43:18 +09:00
Wei-Chiu Chuang
1425512032 Revert "HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952)" 
This reverts commit d525e1e33ddfb58fdd76862db19df0ab91f3e6f5.
 2021-04-26 16:42:44 +08:00
Wei-Chiu Chuang
d525e1e33d HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 66dda86fecee966c7b0625b19f700a43db9787a3)
 2021-04-26 14:07:35 +08:00
Akira Ajisaka
842714d99f
HADOOP-17655. Upgrade Jetty to 9.4.40. (#2947) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 0b04c9694da86941e04a480d55913701f9fe514d)
 2021-04-23 19:53:30 +09:00
Viraj Jasani
49f6326a9f HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs (#2895). Contributed by Viraj Jasani. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-04-16 12:41:06 +05:30
Wei-Chiu Chuang
7f7535534d HADOOP-17625. Update to Jetty 9.4.39. (#2870) 
Reviewed-by: cxorm <lianp964@gmail.com>
(cherry picked from commit 6040e86e99aae5e29c17b03fddb0a805da8fcae8)
 2021-04-08 14:28:03 +08:00
Wei-Chiu Chuang
cc0cc7ab5f HADOOP-17614. Bump netty to the latest 4.1.61. (#2850) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 29a6769b68f32fc3eb0686a955040b9412a24a66)
 2021-04-05 08:37:10 +08:00
Akira Ajisaka
80d4636067
HADOOP-17602. Upgrade JUnit to 4.13.1. Contributed by Ahmed Hussein. 
(cherry picked from commit c5929c00a35a8e58bcf1c80d9d414c3cbdce4483)

 Conflicts:
	hadoop-cloud-storage-project/hadoop-huaweicloud/pom.xml
 2021-03-25 19:54:51 +09:00
Mingliang Liu
23082ac6c7
YARN-10706. Upgrade com.github.eirslett:frontend-maven-plugin to 1.11.2 (#2791) 
Contributed by Mingliang Liu.

Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-21 21:14:12 -07:00
Ayush Saxena
28e89509a3 HADOOP-17586. Upgrade org.codehaus.woodstox:stax2-api to 4.2.1. (#2769). Contributed by Ayush Saxena. 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-03-13 15:10:27 +05:30
Viraj Jasani
36313b38cb
HADOOP-17571 : Bump up woodstox-core to 5.3.0 due to security concerns (#2757) 
Contributed by Viraj Jasani.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-11 11:55:09 -08:00
Akira Ajisaka
de2904f123
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 23b343aed17119b905dfa61ab1ce32ab554bfb57)

 Conflicts:
	dev-support/docker/Dockerfile
	hadoop-project/pom.xml
 2021-03-11 14:57:03 +09:00
Takanobu Asanuma
e607d03995 HADOOP-17563. Update Bouncy Castle to 1.68. (#2740) 
(cherry picked from commit 077411675679900f94adba5329d0e33a4a528793)
 2021-03-05 22:59:09 +09:00
Masatake Iwasaki
cc42f5dafd HADOOP-17538. Add kms-default.xml and httpfs-default.xml to site index. (#2716) 
(cherry picked from commit e9a3c2950e869482a8f1a9d7027c1b0b0f1bbd0d)
 2021-02-23 22:55:10 +00:00
Akira Ajisaka
852db790d3
HADOOP-17534. Update Jackson to 2.10.5 and Jackson databind to 2.10.5.1 (#2708) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e119063a078f06523158dba3b9e16ca04768e920)
 2021-02-22 17:34:43 +09:00
Akira Ajisaka
98eec71218
HADOOP-17516. Upgrade ant to 1.10.9. (#2683) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 933fbeffee1e9e2b9d78b5be31466d4d640768c5)
 2021-02-11 08:07:55 +09:00
Akira Ajisaka
7106a789f4
HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage (#2556) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 9cf408d6baf2fa9071729cac57516d366e928a9f)
 2021-01-24 22:52:51 +09:00
Wei-Chiu Chuang
6340ac857b HADOOP-17371. Bump Jetty to the latest version 9.4.34. Contributed by Wei-Chiu Chuang. (#2453) 
(cherry picked from commit 66ee0a6df0dc0dd8242018153fd652a3206e73b5)
 2021-01-04 11:28:26 -08:00
Liang-Chi Hsieh
87064df1f2 HADOOP-17292. Using lz4-java in Lz4Codec (#2350) 
Contributed by Liang-Chi Hsieh.
 2020-12-29 13:17:26 -08:00
He Xiaoqiao
3736f6eb57
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:47:16 +08:00
Liang-Chi Hsieh
485e4e3348 HADOOP-17425. Bump up snappy-java to 1.1.8.2. (#2536) 
(cherry picked from commit c2cecfc9b951e21d38783227379bec63f34b7702)

 Conflicts:
	hadoop-project/pom.xml
 2020-12-10 13:58:49 +00:00
Masatake Iwasaki
3a50148809 HADOOP-17369. Bump up snappy-java to 1.1.8.1. (#2451) 
(cherry picked from commit 0946f58caecd47c035b1805e2a668dd5d2954dec)
 2020-12-10 13:57:35 +00:00
Ayush Saxena
8378ab9f92 HADOOP-17288. Use shaded guava from thirdparty. Contributed by Ayush Saxena. #2505 2020-12-10 05:50:55 +05:30
Steve Loughran
e4bc64cce0 HADOOP-17343. Upgrade AWS SDK to 1.11.901 (#2468) 
Contributed by Steve Loughran.
 2020-11-23 14:09:14 +00:00
Brahma Reddy Battula
fe8b22ca8f HADOOP-17236. Bump up snakeyaml to 1.26 to mitigate CVE-2017-18640. Contributed by Brahma Reddy Battula. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit eb84793af1e48db05ab827d0cf09963a430615ed)
 2020-10-28 09:28:09 -07:00
Akira Ajisaka
37c2e34b02
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 
(cherry picked from commit 3a0922fb4207af2f06f8ef57e83ff9759871d0cb)
 2020-10-19 10:49:21 +09:00
Pranav Bheda
054dba68c2
HADOOP-17223 update org.apache.httpcomponents:httpclient to 4.5.13 and httpcore to 4.4.13 (#2242) 
* update org.apache.httpcomponents:httpclient from 4.5.6 to 4.5.13
* update org.apache.httpcomponents:httpcore from 4.4.10 to 4.4.13

(cherry picked from commit be3edd05321a0cbe5e6d60536b4ab6ed34f2cef6)
 2020-10-13 17:56:38 +09:00
Doroszlai, Attila
13e0c5f6e0 HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-08 23:44:56 -07:00
Liang-Chi Hsieh
8f60a90688 HADOOP-17125. Use snappy-java in SnappyCodec (#2297) 
This switches the SnappyCodec to use the java-snappy codec, rather than the native one.

To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.

This comesin as an avro dependency,  so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7

Contributed by DB Tsai and Liang-Chi Hsieh

Change-Id: Id52a404a0005480e68917cd17f0a27b7744aea4e
 2020-10-06 17:15:17 +01:00
Brahma Reddy Battula
d05051c840 HADOOP-17220. Upgrade slf4j to 1.7.30 ( To Address: CVE-2018-8088). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 15a0fed637129be04bbbb9300eb363341f8f5365)
 2020-08-24 19:05:27 +05:30
Thomas Marquardt
ee192c4826
HADOOP-17089: WASB: Update azure-storage-java SDK 
Contributed by Thomas Marquardt

DETAILS: WASB depends on the Azure Storage Java SDK. There is a concurrency
bug in the Azure Storage Java SDK that can cause the results of a list blobs
operation to appear empty. This causes the Filesystem listStatus and similar
APIs to return empty results. This has been seen in Spark work loads when jobs
use more than one executor core.

See Azure/azure-storage-java#546 for details on the bug in the Azure Storage SDK.

TESTS: A new test was added to validate the fix. All tests are passing:

wasb:
mvn -T 1C -Dparallel-tests=wasb -Dscale -DtestsThreadCount=8 clean verify
Tests run: 248, Failures: 0, Errors: 0, Skipped: 11
Tests run: 651, Failures: 0, Errors: 0, Skipped: 65

abfs:
mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify
Tests run: 64, Failures: 0, Errors: 0, Skipped: 0
Tests run: 437, Failures: 0, Errors: 0, Skipped: 33
Tests run: 206, Failures: 0, Errors: 0, Skipped: 24
 2020-06-25 05:43:32 +00:00
Uma Maheswara Rao G
418580446b HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 76fa0222f0d2e2d92b4a1eedba8b3e38002e8c23)
 2020-06-16 16:54:01 -07:00
Akira Ajisaka
6b54f259e7
HADOOP-17049. javax.activation-api and jakarta.activation-api define overlapping classes (#2027) 
* Removed javax.activation-api from dependency

(cherry picked from commit 52b21de1d8857151461341fbcf2fdb38c64485b1)
 2020-05-22 11:20:16 +09:00