big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,159 Commits 2 Branches 2 Tags 264 MiB
6dfebdbce4
Commit Graph

308 Commits

Author SHA1 Message Date
huangzhaobo
6dfebdbce4
HADOOP-19065. Update Protocol Buffers installation to 3.21.12 (#6526) 
Instructions and docker script for installing protobuf updated from 3.7.1 to 3.21.12

Contributed by huangzhaobo
 2024-02-22 17:09:46 +00:00
slfan1989
555faf28ce
HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. (#6537) Contributed by Shilun Fan 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2024-02-11 07:41:46 +08:00
slfan1989
4c3d4e6a57
HADOOP-19038. Improve create-release RUN script. (#6448) Contributed by Shilun Fan. 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2024-01-18 19:12:12 +08:00
slfan1989
0f8b74b03f
HADOOP-19034. Fix Download Maven Url Not Found. (#6438). Contributed by Shilun Fan. 
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2024-01-14 18:30:40 +08:00
Gautham B A
98656db736
HADOOP-19017. Setup pre-commit CI for Windows 10 (#5820) 
* This PR adds a Jenkinsfile for pre-commit CI
  to validate the Hadoop PRs on Windows 10.
 2023-12-29 16:41:10 +05:30
Gautham B A
cbb153b69e
HDFS-17246. Fix shaded client for building Hadoop on Windows (#5943) 2023-11-01 09:10:15 -07:00
Hexiaoqiao
59a7836d13
HADOOP-18789. Remove ozone from hadoop dev support. (#5800). Contributed by Xiaoqiao He. 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
 2023-07-02 15:23:32 +08:00
Gautham B A
afe850ca2c
HADOOP-18746. Install Python 3 for Windows 10 docker image (#5679) 
* This PR installs Python 3.10.11 for
  Windows 10 Docker image to fix
  the issue with building mvnsite.
* After installing Python 3.10.11, it
  creates the hardlink python -> python3
  as required by the script.
 2023-05-21 21:10:04 +05:30
Gautham B A
a80e3dba3b
HADOOP-18734. Create qbt.sh symlink on Windows (#5626) 2023-05-08 09:55:15 -07:00
Gautham B A
0d06fd77de
HADOOP-18134. Setup Jenkins nightly CI for Windows 10 (#5062) 
This PR gets Yetus to run on Windows 10
against the Hadoop codebase. It introduces
the following changes to allow us to setup
the nightly CI on Jenkins for Hadoop on
Windows 10.
* Hadoop personality changes for Yetus.
  Additional arguments have been passed,
  which are necessary to build and test
  Hadoop on Windows 10.
* Docker image for building Hadoop on
  Windows 10.
  Installs the necessary tools that are
  needed to run Yetus.
* dev-support/jenkins.sh file.
  Passing of some flags are handled here
  which are needed for the nightly CI.
 2023-05-03 22:44:54 +05:30
Gautham B A
5147106b59
HADOOP-18725. Avoid cross-platform build for irrelevant Dockerfile changes (#5610) 2023-05-01 09:35:50 -07:00
Chris Nauroth
14c5810d5e HADOOP-18680: Insufficient heap during full test runs in Docker container. 
Closes #5522

Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-04-03 22:53:29 +00:00
Gautham B A
5694d7e25f
Add Dockerfile_windows_10 (#4936) 2022-10-12 09:30:54 -07:00
Mukund Thakur
2daf0a814f HADOOP-11867. Add a high-performance vectored read API. (#3904) 
part of HADOOP-18103.
Add support for multiple ranged vectored read api in PositionedReadable.
The default iterates through the ranges to read each synchronously,
but the intent is that FSDataInputStream subclasses can make more
efficient readers especially in object stores implementation.

Also added implementation in S3A where smaller ranges are merged and
sliced byte buffers are returned to the readers. All the merged ranged are
fetched from S3 asynchronously.

Contributed By: Owen O'Malley and Mukund Thakur
 2022-06-22 17:29:32 +01:00
Gautham B A
7ac271307c
HADOOP-18274. Use CMake 3.19.0 in Debian 10 (#4395) 2022-06-02 11:16:05 -07:00
Gautham B A
712e009172
HADOOP-18268. Install maven from Apache archives (#4373) 
* The remote server hosting maven
  was temporarily unavailable causing
  the Hadoop Jenkins CI to fail.
* Thus, we're switching to use a CDN
  to mitigate such issues.
 2022-06-02 17:05:18 +05:30
Masatake Iwasaki
6b331dde31
HADOOP-18251. Fix failure of extracting JIRA id from commit message in git_jira_fix_version_check.py. (#4344) 2022-05-26 12:23:30 +09:00
Ashutosh Gupta
84b0455cf8
HADOOP-18240. Upgrade Yetus to 0.14.0 (#4328) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-25 17:31:18 +09:00
Viraj Jasani
c85b80018b
HADOOP-18228. Update hadoop-vote to use HADOOP_RC_VERSION dir (#4272) 
Signed-off-by: stack@apache.org
 2022-05-16 07:34:39 -07:00
Steve Loughran
aa917b1cd4
HADOOP-18198. add -mvnargs option to create-release command line (#4239) 
This allows for builds to be run with options like
--mvnargs="-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false"

Contributed by Ayush Saxena.

Change-Id: I396e82d0915d679657d063a948f865041bcdde29
 2022-05-03 14:08:39 +01:00
Steve Loughran
d7fd61d616
HADOOP-18202. create-release fails fatal: unsafe repository (#4188) 
Since April 2022/CVE-2022-24765, git refuses to work in directories
whose owner != the current user, unless explicitly told to trust it.

This patches the create-release script to trust the /build/source
dir mounted from the hosting OS, whose userid is inevitably different
from that of the account in the container running git.

Contributed by: Steve Loughran, Ayush Saxena and the new git error messages
 2022-04-18 19:24:35 +01:00
Gautham B A
19561846df
HADOOP-18151. Switch the baseurl for Centos 8 (#4047) 
* Centos 8 has reached its
  End-of-Life and thus its
  packages are no longer
  accessible from
  mirror.centos.org.
* This PR switches the baseurl
  to vault.centos.org where
  the packages are archived.
 2022-03-05 07:11:22 +05:30
Viraj Jasani
697e5d4636
HADOOP-18125. Utility to identify git commit / Jira fixVersion discrepancies for RC preparation (#3991) 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-02-22 11:00:38 +08:00
Viraj Jasani
b39b334873
HADOOP-18098. Basic verification for the release candidate vote (#3944) 2022-02-07 09:34:58 +08:00
Gautham B A
bccf2f3ef4
HADOOP-17985. Disable JIRA plugin for Yetus run (#3601) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-11-01 02:18:23 +09:00
Viraj Jasani
329b95b620
HADOOP-17892. Add Hadoop code formatter in dev-support (#3387) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Gautham B A <gautham.bangalore@gmail.com>
 2021-09-23 12:34:19 +08:00
Gautham B A
d6eb077247
HADOOP-17902. Fix Hadoop build on Debian 10 (#3408) 2021-09-18 11:51:33 -07:00
Liang-Chi Hsieh
a424878e0e
HADOOP-17891. Fix compilation error under skipShade (ADDENDUM) (#3441) 2021-09-16 09:58:28 -07:00
Gautham B A
16ca362564
HADOOP-17913. Filter deps with release labels (#3437) 2021-09-16 09:18:58 -07:00
Liang-Chi Hsieh
b8f7c7527a
HADOOP-17891. Exclude snappy-java and lz4-java from relocation in shaded hadoop client libraries (#3385) 2021-09-14 11:17:18 -07:00
Ayush Saxena
4d18a2eb99
HADOOP-17904. Test Result Not Working In Jenkins Result. (#3413). Contributed by Ayush Saxena. 2021-09-11 05:15:54 +05:30
Gautham B A
c8e5864838
HADOOP-17854. Run junit in Jenkins only if surefire reports exist (#3319) 2021-08-24 10:26:26 -07:00
Gautham B A
4fd97e01e5
HADOOP-17787. Refactor fetching of credentials in Jenkins (#3167) 2021-08-06 11:56:27 -07:00
Gautham B A
e80b5a08e2
HADOOP-17816. Run optional CI for changes in C (#3238) 
* HADOOP-17816. Run optional CI for changes in C

* We need to ensure that we run
  the CI for all the platforms
  when there are changes in C files.

* Trigger CI

* Revert "Trigger CI"

This reverts commit ecb88350b91c692cac128fc6f72dcd8971932041.
 2021-08-05 12:13:33 +00:00
Gautham B A
13467f4524
HADOOP-17815. Run CI for Centos 7 (#3231) 2021-07-29 10:50:57 -07:00
Gautham B A
97c88c97de
HADOOP-17807. Use separate src dir for platform builds (#3210) 2021-07-26 13:09:36 -07:00
Masatake Iwasaki
c42441a1ec
HADOOP-17775. Remove JavaScript package from Docker environment. (#3137) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-07-07 11:07:10 +09:00
Gautham B A
7c999e2d9a
HADOOP-17778. CI for Centos 8 (#3151) 2021-06-30 22:04:54 -07:00
Gautham B A
ef5dbc7283
HADOOP-17766. CI for Debian 10 (#3129) 2021-06-23 10:02:33 -07:00
Gautham B A
51991c4907
HADOOP-17724. Add Dockerfile for Debian 10 (#3038) 2021-06-17 10:14:44 -07:00
Takanobu Asanuma
ae9630f580
HADOOP-17756. Increase precommit job timeout from 20 hours to 24 hours. (#3090) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
 2021-06-14 10:17:39 +09:00
Gautham B A
1c0b2edde9
HADOOP-17727. Modularize docker images (#3043) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2021-06-07 20:11:08 -07:00
Wei-Chiu Chuang
1576f81dfe
HADOOP-17723. [build] fix the Dockerfile for ARM (#3037) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-24 03:19:58 -07:00
Wei-Chiu Chuang
ad923ad564
HADOOP-17718. Explicitly set locale in the Dockerfile. (#3034) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-21 09:26:50 -07:00
Wei-Chiu Chuang
110cda3de6
HADOOP-17703. checkcompatibility.py errors out when specifying annotations. (#3017) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-17 20:21:54 -07:00
Ayush Saxena
fcd4140e5f
HDFS-15988. Stabilise HDFS Pre-Commit. (#2860). Contributed by Ayush Saxena. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2021-05-16 00:45:11 +05:30
Gautham B A
2e58fb671b
HADOOP-17693. Dockerfile for building on Centos 8 (#3006) 2021-05-13 09:28:27 -07:00
Gautham B A
8f850b46a0
HADOOP-17678. Add Dockerfile for Centos 7 (#2967) 2021-05-10 10:05:14 -07:00
Gautham B A
02b9506c5a
HDFS-15948. Fix test4tests for libhdfspp (#2873) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-07 19:06:25 +09:00
Akira Ajisaka
2b62b12d0d
HADOOP-17582. Replace GitHub App Token with GitHub OAuth token (#2766) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-03-12 18:04:00 +09:00