big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26,941 Commits 2 Branches 2 Tags 264 MiB
615a2a42cf
Commit Graph

104 Commits

Author SHA1 Message Date
Steve Loughran
9bc159f4ac
HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996) 
Protobuf 2.5 JAR is no longer needed at runtime. 

The option common.protobuf.scope defines whether the protobuf 2.5.0
dependency is marked as provided or not.

* New package org.apache.hadoop.ipc.internal for internal only protobuf classes
  ...with a ShadedProtobufHelper in there which has shaded protobuf refs
  only, so guaranteed not to need protobuf-2.5 on the CP
* All uses of org.apache.hadoop.ipc.ProtobufHelper have
  been replaced by uses of org.apache.hadoop.ipc.internal.ShadedProtobufHelper
* The scope of protobuf-2.5 is set by the option common.protobuf2.scope
  In this patch is it is still "compile"
* There is explicit reference to it in modules where it may be needed.
*  The maven scope of the dependency can be set with the common.protobuf2.scope
   option. It can be set to "provided" in a build:
       -Dcommon.protobuf2.scope=provided
* Add new ipc(callable) method to catch and convert shaded protobuf
  exceptions raised during invocation of the supplied lambda expression
* This is adopted in the code where the migration is not traumatically
  over-complex. RouterAdminProtocolTranslatorPB is left alone for this
  reason.

Contributed by Steve Loughran
 2023-10-13 13:48:38 +01:00
Gautham B A
833750f72a
HADOOP-18506. Update build instructions for Windows using VS2019 (#5066) 2022-10-24 09:28:29 -07:00
Giovambattista Vieri
97ed0293ee
Update BUILDING.txt (#3811) 
java-8-openjdk become openjdk-8-jdk (see both ubuntu and debian package's name)

Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-12-22 13:08:14 +08:00
Masatake Iwasaki
65d9523aa2
HADOOP-17925. BUILDING.txt should not encourage to activate docs profile on building binary artifacts. (#3509) 2021-10-11 17:55:40 +09:00
Rintaro Ikeda
962068d2d8
HADOOP-17926. Maven-eclipse-plugin is no longer needed since Eclipse can import Maven projects by itself. (#3465) 2021-09-22 15:26:38 +09:00
Wanqiang Ji
c005195b78
HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-29 18:32:16 -07:00
Masatake Iwasaki
e66d7d91be
HADOOP-17635. Update the ubuntu version in the build instruction. (#2931) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-19 16:44:16 +09:00
Akira Ajisaka
23b343aed1
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-03-11 10:56:07 +09:00
Akira Ajisaka
9a298d180d
Revert "HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)" 
This reverts commit 4cf3531583.
 2021-02-19 11:09:10 +09:00
Akira Ajisaka
4cf3531583
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) 
Use spotbugs instead of findbugs. Removed findbugs from the hadoop build images,
and added spotbugs in the images instead.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
 2021-02-17 10:38:20 +09:00
Gautham B A
a163aa7394
HADOOP-17508. Simplify deps install instructions (#2664) 2021-02-02 14:33:17 +09:00
Gautham B A
6fafdd7682
HADOOP-17507. Add install notes GCC9, CMake 3.19 (#2663) 2021-01-31 23:40:56 +09:00
Gautham B A
9801b78edf
HADOOP-17506. Fix typo in BUILDING.txt (#2662) 2021-01-31 23:18:24 +09:00
Gautham B A
ad483fd66e
HDFS-15740. Add x-platform utilities (#2567) 2021-01-29 10:11:38 -08:00
Liang-Chi Hsieh
c9ea344f98
HADOOP-17125. Use snappy-java in SnappyCodec (#2297) 
This switches the SnappyCodec to use the java-snappy codec, rather than the native one. 

To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.

This comesin as an avro dependency,  so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7

Contributed by DB Tsai and Liang-Chi Hsieh
 2020-10-06 17:07:54 +01:00
tkuramoto33
0207f5cf46
HADOOP-17240. Fix wrong command line for setting up CentOS 8. (#2268) 2020-09-02 15:12:03 +09:00
Masatake Iwasaki
aee3b97d09
HADOOP-17206. Add python2 to required package on CentOS 8 for building documentation. (#2227) 2020-08-14 20:44:18 +09:00
Masatake Iwasaki
4b1816c7d0
HADOOP-17153. Add boost installation steps to build instruction on CentOS 8. (#2169) 2020-07-26 06:38:54 +09:00
Gautham B A
cce5a6f609
HDFS-15385 Upgrade boost library to 1.72 (#2051) 
* Removed the asio-1.10.2 that was
  checked in under libhdfspp/third_party
  directory.
 2020-07-15 13:39:48 +09:00
Masatake Iwasaki
749e45dfdb
HADOOP-16856. cmake is missing in the CentOS 8 section of BUILDING.txt. (#1841) 2020-02-12 21:17:33 +09:00
Masatake Iwasaki
d5467d299d HADOOP-16739. Fix native build failure of hadoop-pipes on CentOS 8. 2020-02-10 13:13:11 +09:00
Masatake Iwasaki
18059acb6a
HADOOP-16744. Fix building instruction to enable zstd. (#1736) 2019-12-06 15:25:20 +09:00
Sandeep Nemuri
9f0610fb83 HADOOP-16654:Delete hadoop-ozone and hadoop-hdds subprojects from apache trunk 
Signed-off-by: Dinesh Chitlangia <dineshchitlangia@gmail.com>
 2019-11-15 14:53:28 -05:00
Ayush Saxena
104ccca916 HDFS-14900. Fix build failure of hadoop-hdfs-native-client. Contributed by Masatake Iwasaki. 2019-10-10 21:36:02 +05:30
Akira Ajisaka
0e026cb0ce
HADOOP-16620. [pb-upgrade] Remove protocol buffers 3.7.1 from requirements in BUILDING.txt (#1558) 
* Removed protobuf 3.7.1 from Dockerfile
 2019-10-03 15:24:47 +09:00
Vinayakumar B
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432) 
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
 2019-09-20 16:08:30 +05:30
Siyao Meng
4fecc2a95e HADOOP-16263. Update BUILDING.txt with macOS native build instructions. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-11 15:04:59 -07:00
Sammi Chen
d1aad44490 HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00
Akira Ajisaka
96a84b61fb
HDFS-14213. Remove Jansson from BUILDING.txt. Contributed by Dinesh Chitlangia. 2019-01-18 15:31:45 +09:00
Akira Ajisaka
10b5da85fa
HADOOP-14739. Update start-build-env.sh and build instruction for docker for Mac instead of docker toolbox. Contributed by Dinesh Chitlangia. 2018-11-20 23:59:49 +09:00
Weiwei Yang
e6444f1c64 HADOOP-15907. Add missing maven modules in BUILDING.txt. Contributed Wanqiang Ji. 2018-11-07 16:45:16 +08:00
James Clampffer
9059376785 HDFS-13376. Specify minimum GCC version to avoid TLS support error in Build of hadoop-hdfs-native-client. Contributed by LiXin Ge. 2018-04-09 13:48:42 -04:00
Haibo Chen
3087e89135 YARN-7946. Update TimelineServerV2 doc as per YARN-7919. (Haibo Chen) 2018-04-04 11:59:31 -07:00
James Clampffer
285bbaa432 HDFS-13338. Update BUILDING.txt for building native libraries. Contributed by Takanobu Asanuma. 2018-03-27 14:21:19 -04:00
Haohui Mai
63eee296c7 HDFS-8745. Use Doxygen to generate documents for libhdfspp. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Chris Douglas
3fc3fa9711 HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-03-19 16:05:55 -07:00
Allen Wittenauer
d4cc50f622 HADOOP-15279. increase maven heap size recommendations 
Signed-off-by: Jason Lowe <jlowe@apache.org>
Signed-off-by: Chris Douglas <cdouglas@apache.org>
 2018-03-01 15:32:43 -08:00
Akira Ajisaka
0ed44f2565
HADOOP-15046. Document Apache Hadoop does not support Java 9 in BUILDING.txt. Contributed by Hanisha Koneru. 2017-11-22 01:07:42 +09:00
Akira Ajisaka
6903cf096e HADOOP-13514. Upgrade maven surefire plugin to 2.20.1 
Signed-off-by: Allen Wittenauer <aw@apache.org>
 2017-11-19 12:39:37 -08:00
Akira Ajisaka
0de40f0e27
Revert "HADOOP-13514. Upgrade maven surefire plugin to 2.19.1." 
This reverts commit 3d04c00aed.
 2017-10-13 18:00:25 +09:00
Akira Ajisaka
3d04c00aed
HADOOP-13514. Upgrade maven surefire plugin to 2.19.1. 2017-10-11 10:42:32 +09:00
Andrew Wang
3cf3540f19 HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew Wang and Sean Mackrory. 2017-09-18 15:13:42 -07:00
Allen Wittenauer
71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms 
Signed-off-by: Chris Douglas <cdouglas@apache.org>
 2017-08-30 21:47:24 -07:00
Akira Ajisaka
e6f6682aa8
HADOOP-14419. Remove findbugs report from docs profile. Contributed by Andras Bokor. 2017-05-17 07:35:41 -04:00
Akira Ajisaka
612578fe67
HADOOP-14285. Update minimum version of Maven from 3.0 to 3.3. 2017-04-08 14:23:26 +09:00
Arpit Agarwal
60865c8ea0 HADOOP-14002. Document -DskipShade property in BUILDING.txt. Contributed by Hanisha Koneru. 2017-01-19 17:32:25 -08:00
Jason Lowe
a0a2761621 HADOOP-13578. Add Codec for ZStandard Compression. Contributed by churro morales 2017-01-04 14:46:25 +00:00
Steve Loughran
f66f61892a HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-12 13:58:27 +00:00
Akira Ajisaka
7d8e440eee Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran" 
This reverts commit c265515725.
 2016-12-09 12:49:27 +09:00
Steve Loughran
c265515725 HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-08 17:57:15 +00:00