hadoop

Go to file

HADOOP-18948. S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete (#6218 )

S3A directory delete and rename will optionally abort all pending multipart uploads
in their under their to-be-deleted paths when.

fs.s3a.directory.operations.purge.upload is true

It is off by default.

The filesystems hasPathCapability("fs.s3a.directory.operations.purge.upload")
probe will return true when this feature is enabled.

Multipart uploads may accrue from interrupted data writes, uncommitted 
staging/magic committer jobs and other operations/applications. On AWS S3
lifecycle rules are the recommended way to clean these; this change improves
support for stores which lack these rules.

Contributed by Steve Loughran

2023-10-25 17:39:16 +01:00

.github

HADOOP-18823. Add Labeler Github Action. (#5874 ). Contributed by Ayush Saxena.

2023-07-25 03:04:49 +05:30

.yetus

Add .yetus/excludes.txt (#4984 )

2022-10-11 09:23:34 -07:00

dev-support

HADOOP-18789. Remove ozone from hadoop dev support. (#5800 ). Contributed by Xiaoqiao He.

2023-07-02 15:23:32 +08:00

hadoop-assemblies

HDFS-15346. FedBalance tool implementation. Contributed by Jinglun.

2020-06-18 13:33:25 +08:00

hadoop-build-tools

HADOOP-17968 Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports (#3584 )

2021-10-28 15:57:15 +09:00

hadoop-client-modules

HADOOP-18916. Exclude all module-info classes from uber jars (#6131 )

2023-10-13 20:01:44 +01:00

hadoop-cloud-storage-project

HADOOP-18890. Remove use of okhttp in runtime code (#6057 )

2023-09-19 12:38:36 +01:00

hadoop-common-project

HADOOP-18948. S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete (#6218 )

2023-10-25 17:39:16 +01:00

hadoop-dist

HADOOP-18718. Fix several maven build warnings (#5592 ). Contributed by Dongjoon Hyun.

2023-06-11 11:38:13 +05:30

hadoop-hdfs-project

HADOOP-18920. RPC Metrics : Optimize logic for log slow RPCs (#6146 )

2023-10-25 13:56:39 +08:00

hadoop-mapreduce-project

HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996 )

2023-10-13 13:48:38 +01:00

hadoop-maven-plugins

HADOOP-18949. upgrade maven dependency plugin due to CVE-2021-26291. (#6219 )

2023-10-24 12:28:40 +01:00

hadoop-minicluster

HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies (#4000 )

2022-03-08 17:27:04 +09:00

hadoop-project

HADOOP-18933. upgrade to netty 4.1.100 due to CVE (#6173 )

2023-10-25 14:06:13 +01:00

hadoop-project-dist

HADOOP-18751. Fix incorrect output path in javadoc build phase (#5688 )

2023-06-26 15:52:17 -07:00

hadoop-tools

HADOOP-18948. S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete (#6218 )

2023-10-25 17:39:16 +01:00

hadoop-yarn-project

YARN-11576. Improve FederationInterceptorREST AuditLog. (#6117 ) Contributed by Shilun Fan.

2023-10-24 09:36:06 +08:00

licenses

HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina.

2020-10-18 18:37:46 +05:30

licenses-binary

HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. (#1796 )

2020-01-09 16:24:58 +09:00

.asf.yaml

HADOOP-18630. Add gh-pages in asf.yaml to deploy the current trunk doc (#5393 ). Contributed by Simhadri Govindappa.

2023-02-14 18:13:29 +05:30

.gitattributes

HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin.

2016-09-14 11:14:31 +09:00

.gitignore

HADOOP-18774. Add .vscode to gitignore. (#5756 ). Contributed by Xiaoqiao He.

2023-06-18 14:08:38 +08:00

BUILDING.txt

HADOOP-18941. Modify HBase version in BUILDING.txt (#6206 )

2023-10-20 16:20:17 +09:00

LICENSE-binary

HADOOP-18933. upgrade to netty 4.1.100 due to CVE (#6173 )

2023-10-25 14:06:13 +01:00

LICENSE.txt

YARN-11356. Upgrade DataTables to 1.11.5 to fix CVEs. Contributed by Bence Kosztolnik.

2022-10-26 22:29:01 +02:00

NOTICE-binary

HADOOP-18890. Remove use of okhttp in runtime code (#6057 )

2023-09-19 12:38:36 +01:00

NOTICE.txt

HADOOP-15958. Revisiting LICENSE and NOTICE files.

2019-08-27 13:47:12 +09:00

pom.xml

HADOOP-18923. Switch to SPDX identifier for license name (#6149 ). Contributed by Colm O hEigeartaigh.

2023-10-07 22:50:38 +05:30

README.txt

HADOOP-15958. Revisiting LICENSE and NOTICE files.

2019-08-27 13:47:12 +09:00

start-build-env.sh

HADOOP-18052. Support Apple Silicon in start-build-env.sh (#3817 )

2021-12-23 18:13:18 +09:00

README.txt

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/

Languages

Java 92.9%

C++ 2.8%

C 1.8%

JavaScript 1.1%

Shell 0.5%

Other 0.6%