big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26,090 Commits 2 Branches 2 Tags 264 MiB
Java 92.9%
C++ 2.8%
C 1.8%
JavaScript 1.1%
Shell 0.5%
Other 0.6%
de37fd37d6
Go to file
Steve Loughran de37fd37d6
MAPREDUCE-7403. manifest-committer dynamic partitioning support. (#4728) 
Declares its compatibility with Spark's dynamic
output partitioning by having the stream capability
"mapreduce.job.committer.dynamic.partitioning"

Requires a Spark release with SPARK-40034, which
does the probing before deciding whether to 
accept/rejecting instantiation with
dynamic partition overwrite set

This feature can be declared as supported by
any other PathOutputCommitter implementations
whose algorithm and destination filesystem
are compatible.

None of the S3A committers are compatible.

The classic FileOutputCommitter is, but it
does not declare itself as such out of our fear
of changing that code. The Spark-side code
will automatically infer compatibility if
the created committer is of that class or
a subclass.

Contributed by Steve Loughran.
2022-08-24 11:18:19 +01:00
.github HADOOP-17799. Improve the GitHub pull request template (#3277) 2021-08-14 21:16:15 +09:00
dev-support HADOOP-11867. Add a high-performance vectored read API. (#3904) 2022-06-22 17:29:32 +01:00
hadoop-assemblies HDFS-15346. FedBalance tool implementation. Contributed by Jinglun. 2020-06-18 13:33:25 +08:00
hadoop-build-tools HADOOP-17968 Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports (#3584) 2021-10-28 15:57:15 +09:00
hadoop-client-modules HADOOP-18332: remove rs-api dependency as it conflicts with jsr311-api (#4547) 2022-07-17 21:37:54 +05:30
hadoop-cloud-storage-project HADOOP-18159. Bump cos_api-bundle to 5.6.69 to update public-suffix-list.txt (#4444) 2022-06-15 20:03:26 +01:00
hadoop-common-project HADOOP-18406: Adds alignment context to call path for creating RPC proxy with multiple connections per user. 2022-08-23 17:00:57 -07:00
hadoop-dist Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-hdfs-project HDFS-16724. RBF should support get the information about ancestor mount points (#4719) 2022-08-23 13:25:42 -07:00
hadoop-mapreduce-project MAPREDUCE-7403. manifest-committer dynamic partitioning support. (#4728) 2022-08-24 11:18:19 +01:00
hadoop-maven-plugins HADOOP-18294. Ensure build folder exists before writing checksum file.ProtocRunner#writeChecksums (#4446) 2022-07-12 20:15:26 +09:00
hadoop-minicluster HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies (#4000) 2022-03-08 17:27:04 +09:00
hadoop-project HADOOP-18417. Upgrade to M7 of surefire plugin (#4795) 2022-08-24 11:04:04 +01:00
hadoop-project-dist HADOOP-18305. Release Hadoop 3.3.4: upstream changelog and jdiff files 2022-08-05 14:06:22 +01:00
hadoop-tools HADOOP-18380. fs.s3a.prefetch.block.size to be read through longBytesOption (#4762) 2022-08-23 10:49:04 +01:00
hadoop-yarn-project YARN-11250. Capture the Performance Metrics of ZookeeperFederationStateStore. (#4738) 2022-08-22 14:09:20 -07:00
licenses HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina. 2020-10-18 18:37:46 +05:30
licenses-binary HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. (#1796) 2020-01-09 16:24:58 +09:00
.asf.yaml HADOOP-17234. Addendum. Add .asf.yaml to allow github and jira integration. (#4686). Contributed by Ayush Saxena. 2022-08-05 08:34:56 +05:30
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore YARN-10407. Add phantomjsdriver.log to gitignore. (#2244) 2020-09-01 10:44:55 +09:00
BUILDING.txt Update BUILDING.txt (#3811) 2021-12-22 13:08:14 +08:00
LICENSE-binary HADOOP-18358. Update commons-math3 from 3.1.1 to 3.6.1. (#4619). Contributed by fanshilun. 2022-08-02 01:48:47 +05:30
LICENSE.txt HADOOP-18044. Hadoop - Upgrade to jQuery 3.6.0 (#3791) 2022-01-12 11:40:32 +08:00
NOTICE-binary HADOOP-18068. upgrade AWS SDK to 1.12.132 (#3864) 2022-01-18 10:31:28 +00:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml HADOOP-18297. Upgrade dependency-check-maven to 7.1.1 (#4449) 2022-07-03 04:04:55 +08:00
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
start-build-env.sh HADOOP-18052. Support Apple Silicon in start-build-env.sh (#3817) 2021-12-23 18:13:18 +09:00

README.txt 

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/