big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-19065. Update Protocol Buffers installation to 3.21.12 (#6526)

Browse Source 
Instructions and docker script for installing protobuf updated from 3.7.1 to 3.21.12

Contributed by huangzhaobo
This commit is contained in:
huangzhaobo 2024-02-23 01:09:46 +08:00 committed by GitHub
parent d1daf26b85
commit 6dfebdbce4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 24 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
BUILDING.txt
View File

@ -7,7 +7,7 @@ Requirements:
* JDK 1.8 * JDK 1.8
* Maven 3.3 or later * Maven 3.3 or later
* Boost 1.72 (if compiling native code) * Boost 1.72 (if compiling native code)
* Protocol Buffers 3.7.1 (if compiling native code) * Protocol Buffers 3.21.12 (if compiling native code)
* CMake 3.19 or newer (if compiling native code) * CMake 3.19 or newer (if compiling native code)
* Zlib devel (if compiling native code) * Zlib devel (if compiling native code)
* Cyrus SASL devel (if compiling native code) * Cyrus SASL devel (if compiling native code)
@ -74,10 +74,10 @@ Refer to dev-support/docker/Dockerfile):
$ ./bootstrap $ ./bootstrap
$ make -j$(nproc) $ make -j$(nproc)
$ sudo make install $ sudo make install
* Protocol Buffers 3.7.1 (required to build native code) * Protocol Buffers 3.21.12 (required to build native code)
$ curl -L -s -S https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz -o protobuf-3.7.1.tar.gz $ curl -L https://github.com/protocolbuffers/protobuf/archive/refs/tags/v3.21.12.tar.gz > protobuf-3.21.12.tar.gz
$ mkdir protobuf-3.7-src $ tar -zxvf protobuf-3.21.12.tar.gz && cd protobuf-3.21.12
$ tar xzf protobuf-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7-src && cd protobuf-3.7-src $ ./autogen.sh
$ ./configure $ ./configure
$ make -j$(nproc) $ make -j$(nproc)
$ sudo make install $ sudo make install
@ -433,10 +433,10 @@ Installing required dependencies for clean install of macOS 10.14:
* Install native libraries, only openssl is required to compile native code, * Install native libraries, only openssl is required to compile native code,
you may optionally install zlib, lz4, etc. you may optionally install zlib, lz4, etc.
$ brew install openssl $ brew install openssl
* Protocol Buffers 3.7.1 (required to compile native code) * Protocol Buffers 3.21.12 (required to compile native code)
$ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz $ curl -L https://github.com/protocolbuffers/protobuf/archive/refs/tags/v3.21.12.tar.gz > protobuf-3.21.12.tar.gz
$ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 $ tar -zxvf protobuf-3.21.12.tar.gz && cd protobuf-3.21.12
$ cd protobuf-3.7 $ ./autogen.sh
$ ./configure $ ./configure
$ make $ make
$ make check $ make check
@ -472,11 +472,10 @@ Building on CentOS 8
* Install python2 for building documentation. * Install python2 for building documentation.
$ sudo dnf install python2 $ sudo dnf install python2
* Install Protocol Buffers v3.7.1. * Install Protocol Buffers v3.21.12.
$ git clone https://github.com/protocolbuffers/protobuf $ curl -L https://github.com/protocolbuffers/protobuf/archive/refs/tags/v3.21.12.tar.gz > protobuf-3.21.12.tar.gz
$ cd protobuf $ tar -zxvf protobuf-3.21.12.tar.gz && cd protobuf-3.21.12
$ git checkout v3.7.1 $ ./autogen.sh
$ autoreconf -i
$ ./configure --prefix=/usr/local $ ./configure --prefix=/usr/local
$ make $ make
$ sudo make install $ sudo make install
@ -531,7 +530,7 @@ Requirements:
* JDK 1.8 * JDK 1.8
* Maven 3.0 or later (maven.apache.org) * Maven 3.0 or later (maven.apache.org)
* Boost 1.72 (boost.org) * Boost 1.72 (boost.org)
* Protocol Buffers 3.7.1 (https://github.com/protocolbuffers/protobuf/releases) * Protocol Buffers 3.21.12 (https://github.com/protocolbuffers/protobuf/tags)
* CMake 3.19 or newer (cmake.org) * CMake 3.19 or newer (cmake.org)
* Visual Studio 2019 (visualstudio.com) * Visual Studio 2019 (visualstudio.com)
* Windows SDK 8.1 (optional, if building CPU rate control for the container executor. Get this from * Windows SDK 8.1 (optional, if building CPU rate control for the container executor. Get this from

2
dev-support/docker/Dockerfile
View File

@ -66,7 +66,7 @@ ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV SPOTBUGS_HOME /opt/spotbugs ENV SPOTBUGS_HOME /opt/spotbugs
####### #######
# Set env vars for Google Protobuf 3.7.1 # Set env vars for Google Protobuf 3.21.12
####### #######
ENV PROTOBUF_HOME /opt/protobuf ENV PROTOBUF_HOME /opt/protobuf
ENV PATH "${PATH}:/opt/protobuf/bin" ENV PATH "${PATH}:/opt/protobuf/bin"

2
dev-support/docker/Dockerfile_aarch64
View File

@ -66,7 +66,7 @@ ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-arm64
ENV SPOTBUGS_HOME /opt/spotbugs ENV SPOTBUGS_HOME /opt/spotbugs
####### #######
# Set env vars for Google Protobuf 3.7.1 # Set env vars for Google Protobuf 3.21.12
####### #######
ENV PROTOBUF_HOME /opt/protobuf ENV PROTOBUF_HOME /opt/protobuf
ENV PATH "${PATH}:/opt/protobuf/bin" ENV PATH "${PATH}:/opt/protobuf/bin"

2
dev-support/docker/Dockerfile_centos_7
View File

@ -76,7 +76,7 @@ ENV JAVA_HOME /usr/lib/jvm/java-1.8.0
ENV SPOTBUGS_HOME /opt/spotbugs ENV SPOTBUGS_HOME /opt/spotbugs
####### #######
# Set env vars for Google Protobuf # Set env vars for Google Protobuf 3.21.12
####### #######
ENV PROTOBUF_HOME /opt/protobuf ENV PROTOBUF_HOME /opt/protobuf
ENV PATH "${PATH}:/opt/protobuf/bin" ENV PATH "${PATH}:/opt/protobuf/bin"

2
dev-support/docker/Dockerfile_centos_8
View File

@ -101,7 +101,7 @@ ENV JAVA_HOME /usr/lib/jvm/java-1.8.0
ENV SPOTBUGS_HOME /opt/spotbugs ENV SPOTBUGS_HOME /opt/spotbugs
####### #######
# Set env vars for Google Protobuf # Set env vars for Google Protobuf 3.21.12
####### #######
ENV PROTOBUF_HOME /opt/protobuf ENV PROTOBUF_HOME /opt/protobuf
ENV PATH "${PATH}:/opt/protobuf/bin" ENV PATH "${PATH}:/opt/protobuf/bin"

2
dev-support/docker/Dockerfile_debian_10
View File

@ -66,7 +66,7 @@ ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
ENV SPOTBUGS_HOME /opt/spotbugs ENV SPOTBUGS_HOME /opt/spotbugs
####### #######
# Set env vars for Google Protobuf 3.7.1 # Set env vars for Google Protobuf 3.21.12
####### #######
ENV PROTOBUF_HOME /opt/protobuf ENV PROTOBUF_HOME /opt/protobuf
ENV PATH "${PATH}:/opt/protobuf/bin" ENV PATH "${PATH}:/opt/protobuf/bin"

9
dev-support/docker/pkg-resolver/install-protobuf.sh
View File

@ -27,25 +27,26 @@ if [ $? -eq 1 ]; then
exit 1 exit 1
fi fi
default_version="3.7.1" default_version="3.21.12"
version_to_install=$default_version version_to_install=$default_version
if [ -n "$2" ]; then if [ -n "$2" ]; then
version_to_install="$2" version_to_install="$2"
fi fi
if [ "$version_to_install" != "3.7.1" ]; then if [ "$version_to_install" != "3.21.12" ]; then
echo "WARN: Don't know how to install version $version_to_install, installing the default version $default_version instead" echo "WARN: Don't know how to install version $version_to_install, installing the default version $default_version instead"
version_to_install=$default_version version_to_install=$default_version
fi fi
if [ "$version_to_install" == "3.7.1" ]; then if [ "$version_to_install" == "3.21.12" ]; then
# hadolint ignore=DL3003 # hadolint ignore=DL3003
mkdir -p /opt/protobuf-src && mkdir -p /opt/protobuf-src &&
curl -L -s -S \ curl -L -s -S \
https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz \ https://github.com/protocolbuffers/protobuf/archive/refs/tags/v3.21.12.tar.gz \
-o /opt/protobuf.tar.gz && -o /opt/protobuf.tar.gz &&
tar xzf /opt/protobuf.tar.gz --strip-components 1 -C /opt/protobuf-src && tar xzf /opt/protobuf.tar.gz --strip-components 1 -C /opt/protobuf-src &&
cd /opt/protobuf-src && cd /opt/protobuf-src &&
./autogen.sh &&
./configure --prefix=/opt/protobuf && ./configure --prefix=/opt/protobuf &&
make "-j$(nproc)" && make "-j$(nproc)" &&
make install && make install &&