big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,446 Commits 2 Branches 2 Tags 264 MiB
50e6b49e05
Commit Graph

898 Commits

Author SHA1 Message Date
Steve Loughran
d56977e909
HADOOP-18470. More in the 3.3.5 index.html about security (#5383) 
Expands on the comments in cluster config to tell people
they shouldn't be running a cluster without a private VLAN
in cloud, that Knox is good here, and unsecured clusters
without a VLAN are just computation-as-a-service to crypto miners

Contributed by Steve Loughran
 2023-02-14 17:22:59 +00:00
Viraj Jasani
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future (#5315) 2023-02-14 03:24:06 +08:00
Viraj Jasani
4fcceff535
HADOOP-18620 Avoid using grizzly-http-* APIs (#5356) 2023-02-09 10:45:07 +08:00
Szilard Nemeth
b677d40ab5 HADOOP-18602. Remove netty3 dependency 2023-01-27 16:32:50 +01:00
Steve Loughran
970ebaeded
HADOOP-17717. Update wildfly openssl to 1.1.3.Final. (#5310) 
Contributed by Wei-Chiu Chuang
 2023-01-27 11:50:17 +00:00
PJ Fanning
b9eb760ed2
HADOOP-18587: upgrade to jettison 1.5.3 due to cve (#5270) 
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
 2023-01-06 15:35:50 -08:00
Ayush Saxena
b93b1c69cc
HADOOP-18586. Update the year to 2023. (#5265). Contributed by Ayush Saxena. 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
 2023-01-01 22:36:33 +05:30
Steve Loughran
52c72fafe4
HADOOP-18470. Update index md with section on ABFS prefetching 2022-12-19 13:04:26 +00:00
Steve Loughran
5f08e51b72
HADOOP-18561. Update commons-net to 3.9.0 (#5214) 
Addresses CVE-2021-37533, which *only* relates to FTP.

Applications not using the ftp:// filesystem, which, as
anyone who has used it will know is very minimal and
so rarely used, is not a critical part of the project.

Furthermore, the FTP-related issue is at worst information leakage
if someone connects to a malicious server.

This is a due diligence PR rather than an emergency fix.

Contributed by Steve Loughran
 2022-12-15 16:45:05 +00:00
Murali Krishna
2e88096266
HADOOP-18538. Upgrade kafka to 2.8.2 (#5164) 
Signed-off-by: Brahma Reddy Battula <brahma@apache.org>
 2022-12-06 22:27:46 +05:30
Steve Loughran
84b33b897c
HADOOP-18470. index.md update for 3.3.5 release 2022-12-05 16:13:24 +00:00
PJ Fanning
e09e81abe4
HADOOP-18496: remove unused okhttp.version (#5140). Contributed by PJ Fanning. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-11-27 18:59:40 +05:30
PJ Fanning
d340c4a7a1
HADOOP-18496. Upgrade okhttp3 and dependencies due to kotlin CVEs (#5035) 
Updates okhttp3 and okio so their transitive dependency on Kotlin
stdlib is free from recent CVEs.

okhttp3:okhttp => 4.10.0
okio:okio => 3.2.0
kotlin stdlib => 1.6.20

kotlin CVEs fixed:
 CVE-2022-24329
 CVE-2020-29582
 
Contributed by PJ Fanning.
 2022-11-12 14:14:19 +00:00
Steve Vaughan
2ba982a061
MAPREDUCE-7386. Maven parallel builds (skipping tests) fail (#4415) 
Contributed by Steve Vaughan Jr
 2022-11-04 11:50:43 +00:00
Ashutosh Gupta
e62ba16a02
HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853 (#4991) 2022-11-02 08:41:27 +01:00
PJ Fanning
7ba304d1c6
HADOOP-18512: upgrade woodstox-core to 5.4.0 for security fix (#5087). Contributed by PJ Fanning. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-11-02 00:11:41 +05:30
PJ Fanning
d6a65a4180
HADOOP-18472. Upgrade to snakeyaml 1.33 (#4958) 
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-30 02:30:41 +09:00
Willi Raschkowski
c4aa41aa80
HADOOP-18500. Upgrade maven-shade-plugin to 3.3.0 (#5045) 
Contributed by Willi Raschkowski
 2022-10-20 18:47:33 +01:00
Hexiaoqiao
babb050fa3
HADOOP-18497. Upgrade commons-text version to fix CVE-2022-42889. (#5037). Contributed by PJ Fanning. 
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-10-18 11:28:56 +08:00
PJ Fanning
4ff6c9b8de
HADOOP-18493: upgrade jackson-databind to 2.12.7.1 (#5011). Contributed by PJ Fanning. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-10-17 10:03:10 +05:30
slfan1989
3ff8f58f8c
HADOOP-18360. Update commons-csv from 1.0 to 1.9.0. (#4928). Contributed by fanshilun. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-10-13 12:10:54 +05:30
Steve Loughran
540a660429
HADOOP-18480. Upgrade aws sdk to 1.12.316 (#4972) 
Contributed by Steve Loughran
 2022-10-10 10:23:50 +01:00
PJ Fanning
5eddec8c46
HADOOP-18468: Upgrade jettison to 1.5.1 to fix CVE-2022-40149 (#4937) 
Contributed by PJ Fanning
 2022-10-07 15:44:01 +01:00
Steve Loughran
38b2ed2151
HADOOP-18442. Remove openstack support (#4855) 
Contributed by Steve Loughran
 2022-10-06 11:49:38 +01:00
Ashutosh Gupta
7923cac86b
HADOOP-18443. Upgrade snakeyaml to 1.32 (#4906) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-09-25 23:49:48 +09:00
PJ Fanning
e6d2c336cb
HADOOP-18341: upgrade commons-configuration2 to 2.8.0 and commons-text to 1.9 (#4578) 
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2022-09-22 09:45:20 +09:00
slfan1989
4d9bb81b16
HADOOP-18451. Update hsqldb.version from 2.3.4 to 2.5.2. (#4880) 2022-09-20 11:10:51 -07:00
Colm O hEigeartaigh
272844ee57
HADOOP-15072 - Update Apache Kerby to 2.0.2 (#4473) 2022-09-15 00:43:25 +08:00
Ashutosh Gupta
832d0e0d76
HADOOP-18443. Upgrade snakeyaml to 1.31 to mitigate CVE-2022-25857 (#4856) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Brahma Reddy Battula <brahma@apache.org>
 2022-09-08 19:58:38 +05:30
Ayush Saxena
880686d1e3
Revert "HADOOP-18417. Upgrade to M7 of surefire plugin (#4795)" 
This reverts commit 1ff121041c.
 2022-08-25 03:44:49 +05:30
slfan1989
052d7f286e
HADOOP-18361. Update commons-net from 3.6 to 3.8.0. (#4683). Contributed by fanshilun. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-08-24 20:05:17 +05:30
Steve Vaughan
1ff121041c
HADOOP-18417. Upgrade to M7 of surefire plugin (#4795) 
This addresses an issue where the plugin's default classpath for executing tests fails to include org.junit.platform.launcher.core.LauncherFactory.

Contributed by: Steve Vaughan Jr
 2022-08-24 11:04:04 +01:00
Ashutosh Gupta
69f6fdb757
HADOOP-18301. Upgrade commons-io to 2.11.0 (#4455) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-08-03 10:44:39 +09:00
slfan1989
13fbfd5dea
HADOOP-18358. Update commons-math3 from 3.1.1 to 3.6.1. (#4619). Contributed by fanshilun. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-08-02 01:48:47 +05:30
Steve Loughran
58ed621304
HADOOP-18344. Upgrade AWS SDK to 1.12.262 (#4637) 
Fixes CVE-2018-7489 in shaded jackson.

+Add more commands in testing.md
 to the CLI tests needed when qualifying
 a release

Contributed by Steve Loughran
 2022-07-28 11:29:38 +01:00
Ashutosh Gupta
e664f81ce7
HADOOP-18333.Upgrade jetty version to 9.4.48.v20220622 (#4553) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-21 00:15:39 +08:00
Wei-Chiu Chuang
a55ace7bc0
HADOOP-18079. Upgrade Netty to 4.1.77. (#3977) 
Upgrade netty to address

CVE-2019-20444,
CVE-2019-20445
CVE-2022-24823

Contributed by Wei-Chiu Chuang
 2022-07-18 10:41:00 +01:00
PJ Fanning
34e548cb62
HADOOP-18332: remove rs-api dependency as it conflicts with jsr311-api (#4547) 
This downgrades jackson from the version switched to in
    HADOOP-18033 (2.13.0), to Jackson 2.12.7.
    This removes the dependency on javax.ws.rs-api,
    so avoiding runtime problems with applications using
    jersey-core v1 and/or jsr311-api.
    
    The 2.12.7 release still contains the fix for CVE-2020-36518.
    
    Contributed by PJ Fanning
 2022-07-17 21:37:54 +05:30
Colm O hEigeartaigh
25f8bdcd21
HADOOP-18308 - Update to Apache LDAP API 2.0.x (#4477) 
Update the dependencies of the LDAP libraries used for testing:

ldap-api.version = 2.0.0
apacheds.version = 2.0.0.AM26

Contributed by Colm O hEigeartaigh.
 2022-06-27 11:15:18 +01:00
Igor Dvorzhak
77d1b194c7
HADOOP-18300. Upgrade Gson dependency to version 2.9.0 (#4454) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
 2022-06-22 16:37:22 -07:00
Mukund Thakur
2daf0a814f HADOOP-11867. Add a high-performance vectored read API. (#3904) 
part of HADOOP-18103.
Add support for multiple ranged vectored read api in PositionedReadable.
The default iterates through the ranges to read each synchronously,
but the intent is that FSDataInputStream subclasses can make more
efficient readers especially in object stores implementation.

Also added implementation in S3A where smaller ranges are merged and
sliced byte buffers are returned to the readers. All the merged ranged are
fetched from S3 asynchronously.

Contributed By: Owen O'Malley and Mukund Thakur
 2022-06-22 17:29:32 +01:00
Steve Loughran
a234d00c1c
HADOOP-18275. Update os-maven-plugin to 1.7.0 (#4397) 
Contributed by Steve Loughran
 2022-06-06 13:17:32 +01:00
Viraj Jasani
0733e968ab
HADOOP-18224. Upgrade maven compiler plugin to 3.10.1 (#4267) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-21 03:21:13 +09:00
Ashutosh Gupta
fb910bd906
HDFS-16453. Upgrade okhttp from 2.7.5 to 4.9.3 (#4229) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-21 02:53:14 +09:00
Ashutosh Gupta
3ecdf39943
HADOOP-18237. Upgrade Apache Xerces Java to 2.12.2 (#4318) 
Upgrade Apache Xerces Java to 2.12.2 due to handle vulnerability CVE-2022-23437

Contributed by Ashutosh Gupta
 2022-05-17 20:34:12 +01:00
PJ Fanning
63187083cc
HADOOP-15983. Use jersey-json that is built to use jackson2 (#3988) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-04-28 14:18:19 +09:00
Viraj Jasani
8ea3358380
HADOOP-18196. Remove replace-guava from replacer plugin (#4152) 2022-04-15 23:24:02 +09:00
PJ Fanning
bfde9102be
HADOOP-18195. Make jackson 1 a runtime scope dependency (#4149) 
Contributed by PJ Fanning
 2022-04-08 10:24:09 +01:00
PJ Fanning
4b786c797a
HADOOP-18178. Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 (#4111) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-04-07 16:19:36 +09:00
Akira Ajisaka
4e32318ace
HDFS-16523. Fix dependency error in hadoop-hdfs on M1 Mac (#4112) 
The version of hawtjni-runtime has been fixed to 1.11 to avoid build failures

Contributed by Akira Ajisaka
 2022-03-29 16:44:55 +01:00
PJ Fanning
61e809b245
HADOOP-13386. Upgrade Avro to 1.9.2 (#3990) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-03-26 20:31:16 +09:00
Steve Loughran
8294bd5a37
HADOOP-18163. hadoop-azure support for the Manifest Committer of MAPREDUCE-7341 
Follow-on patch to MAPREDUCE-7341, adding ABFS support and tests

* resilient rename
* tests for job commit through the manifest committer.

contains
- HADOOP-17976. ABFS etag extraction inconsistent between LIST and HEAD calls
- HADOOP-16204. ABFS tests to include terasort

Contributed by Steve Loughran.

Change-Id: I0a7d4043bdf19bcb00c033fc389730109b93b77f
 2022-03-17 11:24:51 +00:00
Viraj Jasani
66b72406bd
HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies (#4000) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2022-03-08 17:27:04 +09:00
PJ Fanning
da5a774018
HADOOP-17563. Upgrade BouncyCastle to 1.68 (#3980) 
Contributed by PJ Fanning
 2022-03-07 22:21:10 +00:00
Wei-Chiu Chuang
007c2011ef
YARN-11068. Update transitive log4j2 dependency to 2.17.1 (#3963) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-02-21 13:33:38 +09:00
PJ Fanning
5f6a294fab
HADOOP-18126. update junit 5 version due to build issues (#3993) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-02-17 14:03:39 +09:00
Aswin Shakil Balasubramanian
41c86b6464
HADOOP-18101. Bump aliyun-sdk-oss to 3.13.2 and jdom2 to 2.0.6.1 (#3951) 2022-02-03 15:50:38 -08:00
Akira Ajisaka
9cb535caf2
YARN-10561. Upgrade node.js to 12.22.1 and yarn to 1.22.5 in YARN application catalog webapp (#2591) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2022-01-28 15:51:21 +09:00
Steve Loughran
d8ab84275e
HADOOP-18068. upgrade AWS SDK to 1.12.132 (#3864) 
With this update, the versions of key shaded dependencies are

  jackson    2.12.3
  httpclient 4.5.13

Contributed by Steve Loughran
 2022-01-18 10:31:28 +00:00
Steve Loughran
14ba19af06
HADOOP-17409. Remove s3guard from S3A module (#3534) 
Completely removes S3Guard support from the S3A codebase.

If the connector is configured to use any metastore other than
the null and local stores (i.e. DynamoDB is selected) the s3a client
will raise an exception and refuse to initialize.

This is to ensure that there is no mix of S3Guard enabled and disabled
deployments with the same configuration but different hadoop releases
-it must be turned off completely.

The "hadoop s3guard" command has been retained -but the supported
subcommands have been reduced to those which are not purely S3Guard
related: "bucket-info" and "uploads".

This is major change in terms of the number of files
changed; before cherry picking subsequent s3a patches into
older releases, this patch will probably need backporting
first.

Goodbye S3Guard, your work is done. Time to die.

Contributed by Steve Loughran.
 2022-01-17 18:08:57 +00:00
Viraj Jasani
f64fda0f00
HADOOP-18055. Async Profiler endpoint for Hadoop daemons (#3824) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2022-01-06 17:56:49 +08:00
Ayush Saxena
0c51dec124
HADOOP-18061. Update the year to 2022. (#3845). Contributed by Ayush Saxena. 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2022-01-04 07:48:03 +05:30
Igor Dvorzhak
795054882a
HADOOP-13464. Upgrade Gson dependency to version 2.8.9 (#2524) 
Change-Id: Ifd3fb9ec6ebfc8874bb799bc198219511fe55a2f

Update pom.xml

Update pom.xml
 2021-12-30 13:27:54 -08:00
Viraj Jasani
6185056f4d
YARN-11047. ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled (#3802) 2021-12-20 18:02:57 +08:00
Viraj Jasani
04b6b9a87b
HADOOP-16908. Prune Jackson 1 from the codebase and restrict it's usage for future (#3789) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-20 16:01:34 +09:00
Wei-Chiu Chuang
bdec546671
Revert "HDFS-16384. Upgrade Netty to 4.1.72.Final (#3798)" 
This reverts commit a4557f9ed9.
 2021-12-16 21:27:08 +08:00
Tamás Pénzes
a4557f9ed9
HDFS-16384. Upgrade Netty to 4.1.72.Final (#3798) 2021-12-16 12:38:42 +08:00
Akira Ajisaka
40464a5e6c
HADOOP-18043. Use mina-core 2.0.22 to fix LDAP unit test failures (#3792) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-12-13 17:49:22 +09:00
Viraj Jasani
4e326f0c97
HADOOP-18039. Upgrade hbase2 version and fix TestTimelineWriterHBaseDown (#3768) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-13 14:03:32 +09:00
Akira Ajisaka
9b9e2ef87f
HADOOP-18040. Use maven.test.failure.ignore instead of ignoreTestFailure (#3774) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-12-10 01:36:31 +09:00
better3471
b85c66a035
HADOOP-18001. Upgrade jetty version to 9.4.44 (#3700). Contributed by Yuan Luo. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-12-09 13:16:10 +05:30
dependabot[bot]
8d9dec7e80
HADOOP-18034. Bump mina-core from 2.0.16 to 2.1.5 in /hadoop-project. (#3706) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-12-09 00:08:54 +05:30
Viraj Jasani
53edd0de5a
HADOOP-18033. Upgrade fasterxml Jackson to 2.13.0 (#3749) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-08 16:52:22 +09:00
Viraj Jasani
b34dcb5b3a
HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile (#3722) 2021-12-02 10:40:30 +08:00
Renukaprasad C
b923fa7a1c
HADOOP-17946. Upgrade commons-lang to 3.12.0 (#3575) 2021-10-26 09:16:50 +08:00
Takanobu Asanuma
cf4d2d826c
HADOOP-17955. Bump netty to the latest 4.1.68. (#3528) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-10-08 10:34:51 +09:00
Warren Zhu
1db5eb43ad
HADOOP-17941. Update xerces to 2.12.1 (#3496) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-09-29 18:48:52 +09:00
Takanobu Asanuma
2068b0041c
HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-09-28 13:21:55 +09:00
Dongjoon Hyun
b79cacf97b
HADOOP-17939. Support building on Apple Silicon (#3486) 
Support building on Apple Silicon with ARM CPUs by using the x86_64 version of protoc.

Contributed by  Dongjoon Hyun
 2021-09-27 13:27:10 +01:00
Ahmed Hussein
051207375b
HADOOP-17886. Upgrade ant to 1.10.11 (#3371) 2021-09-02 16:11:07 -05:00
Siyao Meng
3aaac8a1f6
HADOOP-17834. Bump aliyun-sdk-oss to 3.13.0 (#3261) 
Change-Id: I335d4a2cb08c75dc24ef36bdfab51111f87e0762
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-08-14 21:19:10 +09:00
Renukaprasad C
b90389ae98
HADOOP-17844. Upgrade JSON smart to 2.4.7 (#3299) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-08-14 19:55:32 +09:00
Akira Ajisaka
3565c9477d
HADOOP-17370. Upgrade commons-compress to 1.21 (#3274) 2021-08-08 11:23:28 +09:00
Tamas Domok
a5811dda7b
YARN-10874. Refactor NM ContainerLaunch#getEnvDependencies's unit tests (#3248) 
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
 2021-08-03 16:19:04 +02:00
Viraj Jasani
ccfa072dc7
HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (#3241) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-08-03 14:44:00 +09:00
Akira Ajisaka
4c35466359
HADOOP-17317. [JDK 11] Upgrade dnsjava to remove illegal access warnings (#2442) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-07-24 14:48:25 +09:00
Renukaprasad C
dbd255f4a9
HADOOP-17796. Upgrade jetty version to 9.4.43 (#3208) 2021-07-22 01:15:00 -07:00
Viraj Jasani
d0ee065cc4
HADOOP-16272. Upgrade HikariCP to 4.0.3 (#3204) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-07-16 12:17:17 +09:00
Ahmed Hussein
581f43dce1
HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery (#3130). Contributed by Ahmed Hussein. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-06-24 17:57:52 +05:30
Wei-Chiu Chuang
9a0a808338
HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-31 20:34:48 -07:00
Steve Loughran
bd7b97ba13
HADOOP-17735. Upgrade AWS SDK to 1.11.1026 (#3057) 2021-05-31 21:43:26 +01:00
Takanobu Asanuma
53ff2dfed3
Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) 
This reverts commit 0774116756.

Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-27 13:14:17 +09:00
Akira Ajisaka
3f5a66c158
HADOOP-17663. Remove useless property hadoop.assemblies.version in pom file. (#3025) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-05-20 10:47:33 +09:00
Wei-Chiu Chuang
2960d83c25
HADOOP-17426. Upgrade to hadoop-thirdparty-1.1.0. (#3024) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ibf27246b3eb1b074b938f44f4906db9184466303
 2021-05-19 14:55:47 +09:00
dependabot[bot]
29105ffb63
HADOOP-17683. Update commons-io to 2.8.0 (#2974) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-05-12 10:57:45 +09:00
Wei-Chiu Chuang
64264d726d
HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2956) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: I7de255cb9a167e3a5e25ae339b050e82cb4b7c30
 2021-04-27 10:42:42 +09:00
Wei-Chiu Chuang
bbf62a0562 Revert "HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952)" 
This reverts commit 66dda86fec.
 2021-04-26 16:42:32 +08:00
Viraj Jasani
538ce9c354
HADOOP-17650. Bump solr to unblock build failure with Maven 3.8.1 (#2939) 
Reviewed-by: Siyao Meng <siyao@apache.org>
 2021-04-25 23:48:39 -07:00
Wei-Chiu Chuang
66dda86fec
HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
 2021-04-25 23:04:52 -07:00
Akira Ajisaka
0b04c9694d
HADOOP-17655. Upgrade Jetty to 9.4.40. (#2947) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-04-23 19:52:41 +09:00
Viraj Jasani
f182798695
HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs (#2895). Contributed by Viraj Jasani. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-04-16 12:36:01 +05:30
Wei-Chiu Chuang
6040e86e99
HADOOP-17625. Update to Jetty 9.4.39. (#2870) 
Reviewed-by: cxorm <lianp964@gmail.com>
 2021-04-07 23:25:11 -07:00
Wei-Chiu Chuang
29a6769b68
HADOOP-17614. Bump netty to the latest 4.1.61. (#2850) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-04-04 17:35:30 -07:00
Akira Ajisaka
c5929c00a3
HADOOP-17602. Upgrade JUnit to 4.13.1. Contributed by Ahmed Hussein. 2021-03-25 19:53:30 +09:00
Mingliang Liu
648bbbdad6
YARN-10706. Upgrade com.github.eirslett:frontend-maven-plugin to 1.11.2 (#2791) 
Contributed by Mingliang Liu.

Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-21 21:12:27 -07:00
Ayush Saxena
167603ba0d
HADOOP-17586. Upgrade org.codehaus.woodstox:stax2-api to 4.2.1. (#2769). Contributed by Ayush Saxena. 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-03-13 15:09:03 +05:30
Viraj Jasani
54ae6bcfc3
HADOOP-17571 : Bump up woodstox-core to 5.3.0 due to security concerns (#2757) 
Contributed by Viraj Jasani.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-11 11:51:24 -08:00
Akira Ajisaka
23b343aed1
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-03-11 10:56:07 +09:00
Takanobu Asanuma
0774116756
HADOOP-17563. Update Bouncy Castle to 1.68. (#2740) 2021-03-05 22:56:51 +09:00
Masatake Iwasaki
e9a3c2950e
HADOOP-17538. Add kms-default.xml and httpfs-default.xml to site index. (#2716) 2021-02-24 07:46:02 +09:00
Akira Ajisaka
e119063a07
HADOOP-17534. Update Jackson to 2.10.5 and Jackson databind to 2.10.5.1 (#2708) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-02-22 17:33:02 +09:00
Akira Ajisaka
9a298d180d
Revert "HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)" 
This reverts commit 4cf3531583.
 2021-02-19 11:09:10 +09:00
Akira Ajisaka
4cf3531583
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) 
Use spotbugs instead of findbugs. Removed findbugs from the hadoop build images,
and added spotbugs in the images instead.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
 2021-02-17 10:38:20 +09:00
Akira Ajisaka
933fbeffee
HADOOP-17516. Upgrade ant to 1.10.9. (#2683) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-02-11 08:07:08 +09:00
Siyao Meng
1a205cc3ad
HADOOP-17424. Replace HTrace with No-Op tracer (#2645) 2021-02-01 13:42:44 +09:00
Yuming Wang
2fa73a2387
HADOOP-17452. Upgrade Guice to 4.2.3 (#2582) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-01-25 18:00:48 +09:00
Akira Ajisaka
9cf408d6ba
HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage (#2556) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-01-24 22:50:05 +09:00
Szilard Nemeth
85b1c017ee YARN-10563. Fix dependency exclusion problem in poms. Contributed by Peter Bacsko 2021-01-12 15:29:20 +01:00
Brahma Reddy Battula
1ec5c67b5e HADOOP-16492. Support HuaweiCloud Object Storage as a Hadoop Backend File System. Contributed by zhongjun. 2021-01-11 18:12:12 +05:30
Wei-Chiu Chuang
66ee0a6df0
HADOOP-17371. Bump Jetty to the latest version 9.4.34. Contributed by Wei-Chiu Chuang. (#2453) 2021-01-04 09:43:58 -08:00
He Xiaoqiao
005b854f6b
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:44:12 +08:00
Liang-Chi Hsieh
c2cecfc9b9
HADOOP-17425. Bump up snappy-java to 1.1.8.2. (#2536) 2020-12-10 13:09:44 +09:00
Steve Loughran
fb79be932c
HADOOP-17343. Upgrade AWS SDK to 1.11.901 (#2468) 
Contributed by Steve Loughran.
 2020-11-23 14:08:12 +00:00
Liang-Chi Hsieh
34aa6137bd
HADOOP-17292. Using lz4-java in Lz4Codec (#2350) 
Contributed by Liang-Chi Hsieh.
 2020-11-18 12:03:25 -08:00
Akira Ajisaka
fc961b63d1
Revert "HADOOP-17269. [JDK 11] Upgrade SpotBugs to 4.1.3 to fix false-positive warnings. (#2374)" (#2460) 
This reverts commit 2de7240d4f.
 2020-11-12 15:52:36 +09:00
Masatake Iwasaki
0946f58cae
HADOOP-17369. Bump up snappy-java to 1.1.8.1. (#2451) 2020-11-10 15:01:24 +09:00
Dongjoon Hyun
2cb9e8f66a
HADOOP-17341. Upgrade commons-codec to 1.15 (#2428) 
Reviewed-by: Chao Sun <sunchao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-11-04 09:12:31 -08:00
Wei-Chiu Chuang
e580280a8b HADOOP-17030. Remove unused joda-time. Contributed by Wei-Chiu Chuang. 2020-11-02 16:07:04 -08:00
Brahma Reddy Battula
eb84793af1 HADOOP-17236. Bump up snakeyaml to 1.26 to mitigate CVE-2017-18640. Contributed by Brahma Reddy Battula. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-28 09:27:32 -07:00
Akira Ajisaka
3a0922fb42
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 2020-10-19 10:47:49 +09:00
Ayush Saxena
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. (#2342). Contributed by Ayush Saxena. 2020-10-17 12:01:18 +05:30
Pranav Bheda
be3edd0532
HADOOP-17223 update org.apache.httpcomponents:httpclient to 4.5.13 and httpcore to 4.4.13 (#2242) 
* update org.apache.httpcomponents:httpclient from 4.5.6 to 4.5.13
* update org.apache.httpcomponents:httpcore from 4.4.10 to 4.4.13
 2020-10-13 17:54:41 +09:00
Akira Ajisaka
2de7240d4f
HADOOP-17269. [JDK 11] Upgrade SpotBugs to 4.1.3 to fix false-positive warnings. (#2374) 2020-10-10 00:28:05 +09:00
Doroszlai, Attila
1cfe5916e2 HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-06 14:58:42 -07:00
Liang-Chi Hsieh
c9ea344f98
HADOOP-17125. Use snappy-java in SnappyCodec (#2297) 
This switches the SnappyCodec to use the java-snappy codec, rather than the native one. 

To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.

This comesin as an avro dependency,  so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7

Contributed by DB Tsai and Liang-Chi Hsieh
 2020-10-06 17:07:54 +01:00
Szilard Nemeth
6e3d7050a9 YARN-10386. Create new JSON schema for Placement Rules. Contributed by Peter Bacsko and Benjamin Teke 2020-08-28 17:09:31 +02:00
Brahma Reddy Battula
15a0fed637 HADOOP-17220. Upgrade slf4j to 1.7.30 ( To Address: CVE-2018-8088). Contributed by Brahma Reddy Battula. 2020-08-24 19:03:22 +05:30
Akira Ajisaka
40542024df
HADOOP-17182. Remove breadcrumbs from web site (#2190) 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2020-08-07 23:29:52 -07:00
Akira Ajisaka
ed3ab4b87d
HADOOP-17179. [JDK 11] Fix javadoc error while detecting Java API link (#2186) 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-08-04 12:09:14 -07:00
Akira Ajisaka
c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors (#2098) 2020-08-03 10:46:51 +09:00
Masatake Iwasaki
2ba44a73bf
HADOOP-16866. Upgrade spotbugs to 4.0.6. (#2146) 2020-07-17 19:58:44 +09:00
Yiqun Lin
ff8bb67200 HDFS-15374. Add documentation for fedbalance tool. Contributed by Jinglun. 2020-07-01 14:18:18 +08:00
Thomas Marquardt
4b5b54c73f
HADOOP-17089: WASB: Update azure-storage-java SDK 
Contributed by Thomas Marquardt

DETAILS: WASB depends on the Azure Storage Java SDK. There is a concurrency
bug in the Azure Storage Java SDK that can cause the results of a list blobs
operation to appear empty. This causes the Filesystem listStatus and similar
APIs to return empty results. This has been seen in Spark work loads when jobs
use more than one executor core.

See Azure/azure-storage-java#546 for details on the bug in the Azure Storage SDK.

TESTS: A new test was added to validate the fix. All tests are passing:

wasb:
mvn -T 1C -Dparallel-tests=wasb -Dscale -DtestsThreadCount=8 clean verify
Tests run: 248, Failures: 0, Errors: 0, Skipped: 11
Tests run: 651, Failures: 0, Errors: 0, Skipped: 65

abfs:
mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify
Tests run: 64, Failures: 0, Errors: 0, Skipped: 0
Tests run: 437, Failures: 0, Errors: 0, Skipped: 33
Tests run: 206, Failures: 0, Errors: 0, Skipped: 24
 2020-06-25 02:32:42 +00:00
Yiqun Lin
9cbd76cc77 HDFS-15346. FedBalance tool implementation. Contributed by Jinglun. 2020-06-18 13:33:25 +08:00
Uma Maheswara Rao G
76fa0222f0
HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G. 2020-06-05 10:58:21 -07:00
Akira Ajisaka
52b21de1d8
HADOOP-17049. javax.activation-api and jakarta.activation-api define overlapping classes (#2027) 
* Removed javax.activation-api from dependency
 2020-05-22 11:19:11 +09:00
Surendra Singh Lilhore
1958cb7c2b YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng 2020-05-14 15:09:34 +05:30
Ayush Saxena
0918433b4d YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng. 2020-05-14 00:36:20 +05:30
Wei-Chiu Chuang
bd342bef64
HADOOP-17033. Update commons-codec from 1.11 to 1.14. (#2000) 2020-05-11 08:41:14 -07:00
Sammi Chen
82ff7bc9ab HADOOP-16959. Resolve hadoop-cos dependency conflict. Contributed by Yang Yu. 2020-04-20 18:14:11 +08:00