From d634deea4ed7b018ece6cef6185abf98bb443338 Mon Sep 17 00:00:00 2001 From: Steve Loughran Date: Mon, 6 Nov 2023 17:52:05 +0000 Subject: [PATCH] HADOOP-18487. Protobuf 2.5 removal part 2: stop exporting protobuf-2.5 (#6185) Followup to the previous HADOOP-18487 patch: changes the scope of protobuf-2.5 in hadoop-common and elsewhere from "compile" to "provided". This means that protobuf-2.5 is * No longer included in hadoop distributions * No longer exported by hadoop common POM files * No longer exported transitively by other hadoop modules. * No longer listed in LICENSE-binary. Contributed by Steve Loughran --- BUILDING.txt | 24 +++++++++++++++--------- LICENSE-binary | 1 - hadoop-project/pom.xml | 4 ++-- 3 files changed, 17 insertions(+), 12 deletions(-) diff --git a/BUILDING.txt b/BUILDING.txt index 91d317ebe2..77561c5546 100644 --- a/BUILDING.txt +++ b/BUILDING.txt @@ -319,15 +319,11 @@ Controlling the redistribution of the protobuf-2.5 dependency the Hadoop codebase; alongside the move to Protobuf 3.x a private successor class, org.apache.hadoop.ipc.internal.ShadedProtobufHelper is now used. - The hadoop-common JAR still declares a dependency on protobuf-2.5, but this - is likely to change in the future. The maven scope of the dependency can be - set with the common.protobuf2.scope option. - It can be set to "provided" in a build: - -Dcommon.protobuf2.scope=provided - If this is done then protobuf-2.5.0.jar will no longer be exported as a dependency, - and will then be omitted from the share/hadoop/common/lib/ directory of - any Hadoop distribution built. Any application declaring a dependency on hadoop-commmon - will no longer get the dependency; if they need it then they must explicitly declare it: + The hadoop-common module no longer exports its compile-time dependency on + protobuf-2.5. Hadoop distributions no longer include it. + Any application declaring a dependency on hadoop-commmon will no longer get + the artifact added to their classpath. + If is still required, then they must explicitly declare it: com.google.protobuf @@ -335,6 +331,16 @@ Controlling the redistribution of the protobuf-2.5 dependency 2.5.0 + In Hadoop builds the scope of the dependency can be set with the + option "common.protobuf2.scope". + This can be upgraded from "provided" to "compile" on the maven command line: + + -Dcommon.protobuf2.scope=compile + + If this is done then protobuf-2.5.0.jar will again be exported as a + hadoop-common dependency, and included in the share/hadoop/common/lib/ + directory of any Hadoop distribution built. + ---------------------------------------------------------------------------------- Building components separately diff --git a/LICENSE-binary b/LICENSE-binary index 3a0e19c582..0ec5ee30f5 100644 --- a/LICENSE-binary +++ b/LICENSE-binary @@ -392,7 +392,6 @@ hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/d3-3.5.17.min.js leveldb v1.13 -com.google.protobuf:protobuf-java:2.5.0 com.google.protobuf:protobuf-java:3.6.1 com.google.re2j:re2j:1.1 com.jcraft:jsch:0.1.54 diff --git a/hadoop-project/pom.xml b/hadoop-project/pom.xml index 5b1c569d21..33e34a7d80 100644 --- a/hadoop-project/pom.xml +++ b/hadoop-project/pom.xml @@ -88,8 +88,8 @@ 2.5.0 - - compile + + provided ${common.protobuf2.scope}