big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
25,020 Commits 2 Branches 2 Tags
Commit Graph

730 Commits

Author SHA1 Message Date
Ayush Saxena
9890a4aea4
Revert "HADOOP-18417. Upgrade to M7 of surefire plugin (#4795)" 
This reverts commit 1ff121041c647f0a7980b59e06335faac9dc2f45.
 2022-08-25 03:53:34 +05:30
Steve Vaughan
98dd2b534f
HADOOP-18417. Upgrade to M7 of surefire plugin (#4795) 
This addresses an issue where the plugin's default classpath
for executing tests fails to include
org.junit.platform.launcher.core.LauncherFactory.

Contributed by: Steve Vaughan Jr
 2022-08-24 11:07:34 +01:00
Wei-Chiu Chuang
c4d94f5623
HADOOP-18333. Upgrade jetty version to 9.4.48.v20220622 (#4600) 
* HADOOP-18001. Upgrade jetty version to 9.4.44 (#3700). Contributed by Yuan Luo.

Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit b85c66a035b3091dd5269e39cfb9b0b3e82a8637)

* HADOOP-18333.Upgrade jetty version to 9.4.48.v20220622 (#4553)

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
(cherry picked from commit e664f81ce720e1f2aecfd795656538cfd171a2a0)

 Conflicts:
	LICENSE-binary

Change-Id: I5a758df2551539c2780e170c3738c5b21eb0c79d

Co-authored-by: better3471 <46600375+better3471@users.noreply.github.com>
Co-authored-by: Ashutosh Gupta <ashutosh.gupta@st.niituniversity.in>
 2022-08-24 08:16:49 +08:00
Steve Loughran
7aebacef77 HADOOP-18344. Upgrade AWS SDK to 1.12.262 (#4637) 
Fixes CVE-2018-7489 in shaded jackson.

+Add more commands in testing.md
 to the CLI tests needed when qualifying
 a release

Contributed by Steve Loughran
 2022-07-28 11:39:40 +01:00
Wei-Chiu Chuang
0c12873487
HADOOP-18079. Upgrade Netty to 4.1.77. (#3977) (#4592) 
Upgrade netty to address

CVE-2019-20444,
CVE-2019-20445
CVE-2022-24823

Contributed by Wei-Chiu Chuang

(cherry picked from commit a55ace7bc0c173f609b51e46cb0d4d8bcda3d79d)
 2022-07-27 03:10:20 +08:00
PJ Fanning
36cb8a6a2b
HADOOP-18354. Upgrade reload4j to 1.22.2 due to XXE vulnerability (#4607). Contributed by PJ Fanning. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-07-24 16:01:47 +05:30
PJ Fanning
6733ba56b8
HADOOP-18332. Remove rs-api dependency by downgrading jackson to 2.12.7. (#4552) 
This downgrades jackson from the version switched to in 
HADOOP-18033 (2.13.0), to Jackson 2.12.7.
This removes the dependency on javax.ws.rs-api,
so avoiding runtime problems with applications using
jersey-core v1 and/or jsr311-api.

The 2.12.7 release still contains the fix for CVE-2020-36518.

Contributed by PJ Fanning
 2022-07-16 18:18:52 +01:00
Mukund Thakur
5c348c41ab HADOOP-11867. Add a high-performance vectored read API. (#3904) 
part of HADOOP-18103.
Add support for multiple ranged vectored read api in PositionedReadable.
The default iterates through the ranges to read each synchronously,
but the intent is that FSDataInputStream subclasses can make more
efficient readers especially in object stores implementation.

Also added implementation in S3A where smaller ranges are merged and
sliced byte buffers are returned to the readers. All the merged ranged are
fetched from S3 asynchronously.

Contributed By: Owen O'Malley and Mukund Thakur

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
	pom.xml
 2022-06-23 17:09:16 -05:00
Igor Dvorzhak
d41e0a9cc3 HADOOP-18300. Upgrade Gson dependency to version 2.9.0 (#4454) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit 77d1b194c77ec62dc31f89e15594bc1250d88de3)
 2022-06-22 23:42:59 +00:00
Steve Loughran
9ca4ac0af0
HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 
Updating the hadoop version of branch-3.3 to 3.3.9-SNAPSHOT
pending agreement on what number its future release should take.

Using 3.3.9-SNAPSHOT puts space in for other incremental releases,
while avoiding creating JIRA release ordering and autocompletion
confusion the way adding a 3.3.10 or higher version would do.

Contributed by Steve Loughran
 2022-06-22 13:09:50 +01:00
Steve Loughran
03c2941d4b
HADOOP-18275. Update os-maven-plugin to 1.7.0 (#4397) 
Contributed by Steve Loughran

Change-Id: Ic4d442a37299dc8098b0bca3cc51beca6f058283
 2022-06-06 13:20:00 +01:00
Ashutosh Gupta
57fe613299
HDFS-16453. Upgrade okhttp from 2.7.5 to 4.9.3 (#4229) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fb910bd9068f12f89136c2776ca6a45caac0408f)

 Conflicts:
	hadoop-project/pom.xml
 2022-05-21 03:17:15 +09:00
Ashutosh Gupta
b0012ee578 HADOOP-18237. Upgrade Apache Xerces Java to 2.12.2 (#4318) 
Upgrade Apache Xerces Java to 2.12.2 due to handle vulnerability CVE-2022-23437

Contributed by Ashutosh Gupta
 2022-05-17 20:37:30 +01:00
Steve Loughran
caecec45f5
HADOOP-17650. Bump solr to unblock build failure with Maven 3.8.1 (#2939) 
Reviewed-by: Siyao Meng <siyao@apache.org>

Contributed by Viraj Jasani
 2022-04-20 16:36:51 +01:00
Dongjoon Hyun
af3558d61a
HADOOP-17341. Upgrade commons-codec to 1.15 (#2428) 
Change-Id: Iab26db901570b507ab25ddbf316a9579a9e92620
Reviewed-by: Chao Sun <sunchao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-04-20 12:29:00 +01:00
Steve Loughran
44e662272f
HADOOP-18198. Preparing for 3.3.4 development 
Change-Id: I2bf19beb541739af22fced38c2545f09c4e1bd53
 2022-04-12 14:09:08 +01:00
Akira Ajisaka
603367c54f
HADOOP-18178. Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 (#4147) 
(cherry picked from commit 4b786c797a76d00ef33237fde76f2bd6edda7f3a)

 Conflicts:
	LICENSE-binary

Co-authored-by: PJ Fanning <pjfanning@users.noreply.github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-04-11 14:58:28 +09:00
Masatake Iwasaki
160b6d106d
HADOOP-18088. Replace log4j 1.x with reload4j. (#4052) 
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-04-07 08:33:13 +09:00
Steve Loughran
3238bdab89
HADOOP-18163. hadoop-azure support for the Manifest Committer of MAPREDUCE-7341 
Follow-on patch to MAPREDUCE-7341, adding ABFS support and tests

* resilient rename
* tests for job commit through the manifest committer.

contains
- HADOOP-17976. ABFS etag extraction inconsistent between LIST and HEAD calls
- HADOOP-16204. ABFS tests to include terasort

Contributed by Steve Loughran.

Change-Id: I0a7d4043bdf19bcb00c033fc389730109b93b77f
 2022-03-17 11:47:15 +00:00
PJ Fanning
a302a19b48 HADOOP-18126. update junit 5 version due to build issues (#3993) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 5f6a294fab46012950dd5582df69b50c2b020fd8)
 2022-02-17 14:07:57 +09:00
Akira Ajisaka
8032b680fb YARN-10561. Upgrade node.js to 12.22.1 and yarn to 1.22.5 in YARN application catalog webapp (#2591) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 9cb535caf234c315270903bde0d34d1bcd8bba52)
 2022-01-28 15:52:33 +09:00
Steve Loughran
8ccc586af6
HADOOP-17409. Remove s3guard from S3A module (#3534) 
Completely removes S3Guard support from the S3A codebase.

If the connector is configured to use any metastore other than
the null and local stores (i.e. DynamoDB is selected) the s3a client
will raise an exception and refuse to initialize.

This is to ensure that there is no mix of S3Guard enabled and disabled
deployments with the same configuration but different hadoop releases
-it must be turned off completely.

The "hadoop s3guard" command has been retained -but the supported
subcommands have been reduced to those which are not purely S3Guard
related: "bucket-info" and "uploads".

This is major change in terms of the number of files
changed; before cherry picking subsequent s3a patches into
older releases, this patch will probably need backporting
first.

Goodbye S3Guard, your work is done. Time to die.

Contributed by Steve Loughran.
 2022-01-18 18:04:48 +00:00
Steve Loughran
47ba977ca9
HADOOP-18068. upgrade AWS SDK to 1.12.132 (#3864) 
With this update, the versions of key shaded dependencies are

  jackson    2.12.3
  httpclient 4.5.13

This backport patch does not include the TestArn changes needed
for the test to work with this version of the SDK; it is only
to be applied to branches without HADOOP-17198. "Support S3 Access Points".
If that patch is backported later, that test suite MUST be
updated to the latest version.

Contributed by Steve Loughran

Change-Id: I8d2b71781ee8472b16469531f9cd0de32dd3356f
 2022-01-18 12:20:12 +00:00
Ayush Saxena
53249a40db
HADOOP-18061. Update the year to 2022. (#3845). Contributed by Ayush Saxena. 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2022-01-04 07:59:45 +05:30
Igor Dvorzhak
5d72fdfcb2 HADOOP-13464. Upgrade Gson dependency to version 2.8.9 (#2524) 
Change-Id: Ifd3fb9ec6ebfc8874bb799bc198219511fe55a2f

Update pom.xml

Update pom.xml

(cherry picked from commit 795054882af72a9d5d4f4e96f775e96473d75c23)
 2021-12-30 21:37:14 +00:00
Viraj Jasani
b0c1158829
HADOOP-18033. Upgrade fasterxml Jackson to 2.13.0 (#3764) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-13 13:52:44 +09:00
Akira Ajisaka
35c5c6bb83 HADOOP-18040. Use maven.test.failure.ignore instead of ignoreTestFailure (#3774) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 9b9e2ef87ffe7b33dc01699892acf62965fb3550)

 Conflicts:
	hadoop-tools/hadoop-federation-balance/pom.xml
 2021-12-10 01:38:26 +09:00
Chao Sun
e079fa6577 Preparing for 3.3.3 development 2021-11-16 16:02:34 -08:00
Renukaprasad C
3bb4a09295
HADOOP-17946. Upgrade commons-lang to 3.12.0 (#3575) 
(cherry picked from commit b923fa7a1c74ffd5859803e6e2571d367d066efb)
 2021-11-16 22:59:25 +08:00
Masatake Iwasaki
9e2936f8d1
HADOOP-17424. Replace HTrace with No-Op tracer (#3520) 
(cherry picked from commit 1a205cc3adffa568c814a5241e041b08e2fcd3eb)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>
 2021-10-12 00:07:09 +09:00
Takanobu Asanuma
038fa67db4 HADOOP-17955. Bump netty to the latest 4.1.68. (#3528) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit cf4d2d826cad941ae3648ff9b6057856bb199fc4)
 2021-10-08 10:37:14 +09:00
Warren Zhu
e33509b094
HADOOP-17941. Update xerces to 2.12.1 (#3496) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1db5eb43ad30e2e238dfee51e789ff8e6b9cd505)
 2021-09-29 18:49:52 +09:00
Takanobu Asanuma
f00ab40b4d HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 2068b0041c05a7b53f32606cc24922cfac299ca8)
 2021-09-28 13:31:53 +09:00
Dongjoon Hyun
ca7fb6a813
HADOOP-17939. Support building on Apple Silicon (#3486) 
Support building on Apple Silicon with ARM CPUs by using the x86_64 version of protoc.

Contributed by  Dongjoon Hyun

Change-Id: I4b8330098822f1fd28f0113650eb709d53bcc690
 2021-09-27 13:27:56 +01:00
Ahmed Hussein
fc5d67dfb4 HADOOP-17886. Upgrade ant to 1.10.11 (#3371) 
(cherry picked from commit 051207375b30ffa126cf9c961ee97535c1dfabb1)
 2021-09-02 16:17:32 -05:00
Siyao Meng
226f94b4fc HADOOP-17834. Bump aliyun-sdk-oss to 3.13.0 (#3261) 
Change-Id: I335d4a2cb08c75dc24ef36bdfab51111f87e0762
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3aaac8a1f682ecda4d41221e8a836bc5401e3b1d)
 2021-08-16 00:32:05 +09:00
Renukaprasad C
5b566c3914 HADOOP-17844. Upgrade JSON smart to 2.4.7 (#3299) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b90389ae9840ccfdcab411e286547895a647feed)

 Conflicts:
	LICENSE-binary
 2021-08-14 20:00:38 +09:00
Akira Ajisaka
025ecf42be
HADOOP-17370. Upgrade commons-compress to 1.21 (#3274) 
(cherry picked from commit 3565c9477dffd104a3aa3603845f7a1305815ac3)
 2021-08-08 11:25:26 +09:00
Renukaprasad C
d37192a613
HADOOP-17796. Upgrade jetty version to 9.4.43 (#3208) 
(cherry picked from commit dbd255f4a96474b31af2bd45c952ac7151a265b9)
 2021-07-22 16:37:39 +08:00
Ahmed Hussein
22e7567475
HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery (#3130). Contributed by Ahmed Hussein. (#3138) 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 581f43dce15d5753deb33f72ec275d0ea67b1403)
 2021-06-25 22:48:56 +05:30
Wei-Chiu Chuang
5cebd02bbe
HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9a0a808338558eb2e8074a7611619a7fc991f994)
 2021-06-01 11:36:27 +08:00
Steve Loughran
bb04b60197
HADOOP-17735. Upgrade AWS SDK to 1.11.1026 (#3057) 
Change-Id: Ia9b32ab35454ada5f5e3d47b72cd9f2a2446a309
 2021-05-31 21:49:39 +01:00
Takanobu Asanuma
d8689f1a08 Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) 
This reverts commit 077411675679900f94adba5329d0e33a4a528793.

Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53ff2dfed3b784ff2120fadc85ba70226ecfd5ca)
 2021-05-27 13:17:24 +09:00
Akira Ajisaka
86682a83aa
HADOOP-17663. Remove useless property hadoop.assemblies.version in pom file. (#3025) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3f5a66c1583f6b8f21f4e8db73ce0d65d019e33c)

 Conflicts:
	hadoop-project/pom.xml
 2021-05-20 10:49:40 +09:00
Wei-Chiu Chuang
fa4915fdbb
Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
Wei-Chiu Chuang
611ed04aff
HADOOP-17426. Upgrade to hadoop-thirdparty-1.1.0. (#3024) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ibf27246b3eb1b074b938f44f4906db9184466303
(cherry picked from commit 2960d83c255a00a549f8809882cd3b73a6266b6d)
 2021-05-19 14:57:45 +09:00
dependabot[bot]
b2897fdd66
HADOOP-17683. Update commons-io to 2.8.0 (#2974) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 29105ffb634ceb44a331ba65280d418544b902a6)
 2021-05-12 10:58:39 +09:00
Wei-Chiu Chuang
99e4e9bad6
HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2956) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: I7de255cb9a167e3a5e25ae339b050e82cb4b7c30
(cherry picked from commit 64264d726d81c919d3f682228ff4bd7fe349fcae)
 2021-04-27 10:43:18 +09:00
Wei-Chiu Chuang
1425512032 Revert "HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952)" 
This reverts commit d525e1e33ddfb58fdd76862db19df0ab91f3e6f5.
 2021-04-26 16:42:44 +08:00
Wei-Chiu Chuang
d525e1e33d HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 66dda86fecee966c7b0625b19f700a43db9787a3)
 2021-04-26 14:07:35 +08:00