Commit Graph

96 Commits

Author SHA1 Message Date
ahmarsuhail
d25cba7e85
S3A: Upgrade AWS SDK version to 2.21.33 for Amazon S3 Express One Zone support (#6306)
With this upgrade, it is possible to connect to an Amazon S3 Express One Zone bucket.

Some tests from the S3A test suite will currently fail against a one zone bucket, as one zone buckets
do not support some S3 standard features (eg: SSE-KMS), and certain operations behave slightly
differently (eg: listMPU will return a directory that has incomplete MPUs).

Contributed by Ahmar Suhail
2023-11-29 13:16:19 +00:00
Steve Loughran
d634deea4e
HADOOP-18487. Protobuf 2.5 removal part 2: stop exporting protobuf-2.5 (#6185)
Followup to the previous HADOOP-18487 patch: changes the scope of
protobuf-2.5 in hadoop-common and elsewhere from "compile" to "provided".

This means that protobuf-2.5 is
* No longer included in hadoop distributions
* No longer exported by hadoop common POM files
* No longer exported transitively by other hadoop modules.
* No longer listed in LICENSE-binary.

Contributed by Steve Loughran
2023-11-06 17:52:05 +00:00
PJ Fanning
b9c9c42b29
HADOOP-18936. Upgrade to jetty 9.4.53 (#6181). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-10-29 13:09:12 +05:30
PJ Fanning
bbf905dc99
HADOOP-18933. upgrade to netty 4.1.100 due to CVE (#6173)
Mitigates Netty security advisory GHSA-xpw8-rcwv-8f8p
"HTTP/2 Rapid Reset Attack - DDoS vector in the HTTP/2 protocol due RST frames"

Contributed by PJ Fanning
2023-10-25 14:06:13 +01:00
Masatake Iwasaki
24fe1ef4dd HADOOP-18942 addendum. update LICENSE-binary. 2023-10-22 22:22:56 +09:00
Steve Loughran
42e695d510
HADOOP-18932. S3A. upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565 (#6178)
v1 => 1.12.565
v2 => 2.20.160
Only the v2 one is distributed; v1 is needed in deployments only to support v1 credential providers

Contributed by Steve Loughran
2023-10-17 12:59:50 +01:00
PJ Fanning
2bf5a9ed11
HADOOP-18917. Upgrade to commons-io 2.14.0 (#6133). Contributed by PJ Fanning
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-10-06 01:58:21 +05:30
PJ Fanning
35c42e4039
HADOOP-18912. upgrade snappy-java to 1.1.10.4 (#6115). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-09-28 11:22:31 +05:30
PJ Fanning
c16484ffb2
HADOOP-18890. Remove use of okhttp in runtime code (#6057)
Contributed by PJ Fanning
2023-09-19 12:38:36 +01:00
PJ Fanning
dea446419f
HADOOP-18895. Upgrade to commons-compress 1.24.0 (#6062)
Contributed by PJ Fanning
2023-09-14 17:49:12 +01:00
Steve Loughran
81d90fd65b
HADOOP-18073. S3A: Upgrade AWS SDK to V2 (#5995)
This patch migrates the S3A connector to use the V2 AWS SDK.

This is a significant change at the source code level.
Any applications using the internal extension/override points in
the filesystem connector are likely to break.

This includes but is not limited to:
- Code invoking methods on the S3AFileSystem class
  which used classes from the V1 SDK.
- The ability to define the factory for the `AmazonS3` client, and
  to retrieve it from the S3AFileSystem. There is a new factory
  API and a special interface S3AInternals to access a limited
  set of internal classes and operations.
- Delegation token and auditing extensions.
- Classes trying to integrate with the AWS SDK.

All standard V1 credential providers listed in the option 
fs.s3a.aws.credentials.provider will be automatically remapped to their
V2 equivalent.

Other V1 Credential Providers are supported, but only if the V1 SDK is
added back to the classpath.  

The SDK Signing plugin has changed; all v1 signers are incompatible.
There is no support for the S3 "v2" signing algorithm.

Finally, the aws-sdk-bundle JAR has been replaced by the shaded V2
equivalent, "bundle.jar", which is now exported by the hadoop-aws module.

Consult the document aws_sdk_upgrade for the full details.

Contributed by Ahmar Suhail + some bits by Steve Loughran
2023-09-11 14:30:25 +01:00
Viraj Jasani
911e9e0c01
HADOOP-18832. Upgrade aws-java-sdk to 1.12.499 (#5908)
Contributed by Viraj Jasani
2023-08-16 14:34:36 +01:00
rohit-kb
b1ed23654c
HADOOP-18837. Upgrade okio to 3.4.0 due to CVE-2023-3635. (#5914)
Contributed by Rohit Kumar
2023-08-08 13:37:20 +01:00
PJ Fanning
5a35fb5a72
HADOOP-18783. Upgrade to netty 4.1.94 due to CVE (#5774). Contributed by PJ Fanning. 2023-07-02 14:08:13 +05:30
PJ Fanning
56ef05a9ca
HADOOP-18782. Upgrade to snappy-java 1.1.10.1 due to CVEs (#5773)
Addresses CVE-2023-34454

Contributed by PJ Fanning
2023-06-27 11:53:02 +01:00
liangxs
cebcb44d37
HADOOP-18713. Update solr from 8.8.2 to 8.11.2 (#5459). Contributed by Xuesen Liang.
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-22 13:15:57 +05:30
Viraj Jasani
2fe3b2a73f
HADOOP-18763. Upgrade aws-java-sdk to 1.12.367 (#5741)
Contributed By: Viraj Jasani
2023-06-15 01:09:41 +05:30
slfan1989
a2dda0ce03
HADOOP-18359. Update commons-cli from 1.2 to 1.5. (#5095). Contributed by Shilun Fan.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-10 01:42:12 +05:30
PJ Fanning
b683769fc9
HADOOP-18712. Upgrade to jetty 9.4.51 due to cve (#5574). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-24 01:01:51 +05:30
dependabot[bot]
3b7783c549
HADOOP-18689. Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project (#5502)
Co-authored-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-22 16:19:21 +05:30
PJ Fanning
ad49ddda0e
HADOOP-18711. upgrade nimbus jwt jar due to issues in its embedded shaded json-smart code. (#5573). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-22 14:01:09 +05:30
PJ Fanning
0918c87fa2
HADOOP-18687. Remove json-smart dependency. (#5549). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 18:28:09 +05:30
Ayush Saxena
9e3d5c754b
Revert "HADOOP-18687. Remove json-smart dependency. (#5549). Contributed by PJ Fanning."
This reverts commit b6c0ec796e.
2023-04-20 10:26:08 +05:30
PJ Fanning
b6c0ec796e
HADOOP-18687. Remove json-smart dependency. (#5549). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 00:47:22 +05:30
PJ Fanning
476340c699
HADOOP-18658. snakeyaml dependency: upgrade to v2.0 (#5467). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-03-13 10:08:04 +05:30
nao
734f7abfb8
HADOOP-18646. Upgrade Netty to 4.1.89.Final to fix CVE-2022-41881 (#5435)
This fixes CVE-2022-41881.

This also upgrades io.opencensus dependencies to 0.12.3
 
Contributed by Aleksandr Nikolaev
2023-03-10 15:27:22 +00:00
rohit-kb
487368c4b9
HADOOP-18655. Upgrade kerby to 2.0.3 due to CVE-2023-25613 (#5458)
Upgrade kerby to 2.0.3 due to the CVE https://nvd.nist.gov/vuln/detail/CVE-2023-25613


Contributed by Rohit Kumar Badeau
2023-03-08 15:31:03 +00:00
Steve Loughran
dcd9dc6983
HADOOP-18641. Cloud connector dependency and LICENSE fixup. (#5429)
POM and LICENSE fixup of transient dependencies
* Exclude hadoop-cloud-storage imports which come in with hadoop-common
* Add explicit import of hadoop's org.codehaus.jettison declaration
  to hadoop-aliyun
* Tune aliyun jars imports
* Update LICENSE-binary for the current set of libraries.

Contributed by Steve Loughran
2023-02-28 10:48:54 +00:00
Viraj Jasani
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future (#5315) 2023-02-14 03:24:06 +08:00
Szilard Nemeth
b677d40ab5 HADOOP-18602. Remove netty3 dependency 2023-01-27 16:32:50 +01:00
Steve Loughran
970ebaeded
HADOOP-17717. Update wildfly openssl to 1.1.3.Final. (#5310)
Contributed by Wei-Chiu Chuang
2023-01-27 11:50:17 +00:00
PJ Fanning
b9eb760ed2
HADOOP-18587: upgrade to jettison 1.5.3 due to cve (#5270)
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-06 15:35:50 -08:00
Steve Loughran
5f08e51b72
HADOOP-18561. Update commons-net to 3.9.0 (#5214)
Addresses CVE-2021-37533, which *only* relates to FTP.

Applications not using the ftp:// filesystem, which, as
anyone who has used it will know is very minimal and
so rarely used, is not a critical part of the project.

Furthermore, the FTP-related issue is at worst information leakage
if someone connects to a malicious server.

This is a due diligence PR rather than an emergency fix.

Contributed by Steve Loughran
2022-12-15 16:45:05 +00:00
Murali Krishna
2e88096266
HADOOP-18538. Upgrade kafka to 2.8.2 (#5164)
Signed-off-by: Brahma Reddy Battula <brahma@apache.org>
2022-12-06 22:27:46 +05:30
PJ Fanning
e09e81abe4
HADOOP-18496: remove unused okhttp.version (#5140). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-11-27 18:59:40 +05:30
PJ Fanning
d340c4a7a1
HADOOP-18496. Upgrade okhttp3 and dependencies due to kotlin CVEs (#5035)
Updates okhttp3 and okio so their transitive dependency on Kotlin
stdlib is free from recent CVEs.

okhttp3:okhttp => 4.10.0
okio:okio => 3.2.0
kotlin stdlib => 1.6.20

kotlin CVEs fixed:
 CVE-2022-24329
 CVE-2020-29582
 
Contributed by PJ Fanning.
2022-11-12 14:14:19 +00:00
Ashutosh Gupta
e62ba16a02
HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853 (#4991) 2022-11-02 08:41:27 +01:00
PJ Fanning
7ba304d1c6
HADOOP-18512: upgrade woodstox-core to 5.4.0 for security fix (#5087). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-11-02 00:11:41 +05:30
PJ Fanning
d6a65a4180
HADOOP-18472. Upgrade to snakeyaml 1.33 (#4958)
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 02:30:41 +09:00
Bence Kosztolnik
562b693374 YARN-11356. Upgrade DataTables to 1.11.5 to fix CVEs. Contributed by Bence Kosztolnik. 2022-10-26 22:29:01 +02:00
Hexiaoqiao
babb050fa3
HADOOP-18497. Upgrade commons-text version to fix CVE-2022-42889. (#5037). Contributed by PJ Fanning.
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-10-18 11:28:56 +08:00
PJ Fanning
4ff6c9b8de
HADOOP-18493: upgrade jackson-databind to 2.12.7.1 (#5011). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-17 10:03:10 +05:30
slfan1989
3ff8f58f8c
HADOOP-18360. Update commons-csv from 1.0 to 1.9.0. (#4928). Contributed by fanshilun.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-13 12:10:54 +05:30
Ashutosh Gupta
d6b1e1eeb6
HDFS-16777. datatables@1.10.17 sonatype-2020-0988 vulnerability (#5003) 2022-10-12 14:51:12 -07:00
Steve Loughran
540a660429
HADOOP-18480. Upgrade aws sdk to 1.12.316 (#4972)
Contributed by Steve Loughran
2022-10-10 10:23:50 +01:00
PJ Fanning
5eddec8c46
HADOOP-18468: Upgrade jettison to 1.5.1 to fix CVE-2022-40149 (#4937)
Contributed by PJ Fanning
2022-10-07 15:44:01 +01:00
Ashutosh Gupta
7923cac86b
HADOOP-18443. Upgrade snakeyaml to 1.32 (#4906)
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:49:48 +09:00
PJ Fanning
e6d2c336cb
HADOOP-18341: upgrade commons-configuration2 to 2.8.0 and commons-text to 1.9 (#4578)
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-09-22 09:45:20 +09:00
slfan1989
4d9bb81b16
HADOOP-18451. Update hsqldb.version from 2.3.4 to 2.5.2. (#4880) 2022-09-20 11:10:51 -07:00
Colm O hEigeartaigh
272844ee57
HADOOP-15072 - Update Apache Kerby to 2.0.2 (#4473) 2022-09-15 00:43:25 +08:00