big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26,070 Commits 2 Branches 2 Tags 264 MiB
Java 92.9%
C++ 2.8%
C 1.8%
JavaScript 1.1%
Shell 0.5%
Other 0.6%
682931a6ac
Go to file
Steve Loughran 682931a6ac
HADOOP-18028. High performance S3A input stream (#4752) 
This is the the preview release of the HADOOP-18028 S3A performance input stream.
It is still stabilizing, but ready to test.

Contains

HADOOP-18028. High performance S3A input stream (#4109)
	Contributed by Bhalchandra Pandit.

HADOOP-18180. Replace use of twitter util-core with java futures (#4115)
	Contributed by PJ Fanning.

HADOOP-18177. Document prefetching architecture. (#4205)
	Contributed by Ahmar Suhail

HADOOP-18175. fix test failures with prefetching s3a input stream (#4212)
 Contributed by Monthon Klongklaew

HADOOP-18231.  S3A prefetching: fix failing tests & drain stream async.  (#4386)

	* adds in new test for prefetching input stream
	* creates streamStats before opening stream
	* updates numBlocks calculation method
	* fixes ITestS3AOpenCost.testOpenFileLongerLength
	* drains stream async
	* fixes failing unit test

	Contributed by Ahmar Suhail

HADOOP-18254. Disable S3A prefetching by default. (#4469)
	Contributed by Ahmar Suhail

HADOOP-18190. Collect IOStatistics during S3A prefetching (#4458)

	This adds iOStatisticsConnection to the S3PrefetchingInputStream class, with
	new statistic names in StreamStatistics.

	This stream is not (yet) IOStatisticsContext aware.

	Contributed by Ahmar Suhail

HADOOP-18379 rebase feature/HADOOP-18028-s3a-prefetch to trunk
HADOOP-18187. Convert s3a prefetching to use JavaDoc for fields and enums.
HADOOP-18318. Update class names to be clear they belong to S3A prefetching
	Contributed by Steve Loughran
2022-08-18 13:53:06 +01:00
.github HADOOP-17799. Improve the GitHub pull request template (#3277) 2021-08-14 21:16:15 +09:00
dev-support HADOOP-11867. Add a high-performance vectored read API. (#3904) 2022-06-22 17:29:32 +01:00
hadoop-assemblies HDFS-15346. FedBalance tool implementation. Contributed by Jinglun. 2020-06-18 13:33:25 +08:00
hadoop-build-tools HADOOP-17968 Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports (#3584) 2021-10-28 15:57:15 +09:00
hadoop-client-modules HADOOP-18332: remove rs-api dependency as it conflicts with jsr311-api (#4547) 2022-07-17 21:37:54 +05:30
hadoop-cloud-storage-project HADOOP-18159. Bump cos_api-bundle to 5.6.69 to update public-suffix-list.txt (#4444) 2022-06-15 20:03:26 +01:00
hadoop-common-project HADOOP-18028. High performance S3A input stream (#4752) 2022-08-18 13:53:06 +01:00
hadoop-dist Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-hdfs-project HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously (#4626) 2022-08-18 02:25:09 +08:00
hadoop-mapreduce-project MAPREDUCE-7385. impove JobEndNotifier#httpNotification With recommended methods. (#4403). Contributed by fanshilun. 2022-08-09 00:59:03 +05:30
hadoop-maven-plugins HADOOP-18294. Ensure build folder exists before writing checksum file.ProtocRunner#writeChecksums (#4446) 2022-07-12 20:15:26 +09:00
hadoop-minicluster HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies (#4000) 2022-03-08 17:27:04 +09:00
hadoop-project HADOOP-18301. Upgrade commons-io to 2.11.0 (#4455) 2022-08-03 10:44:39 +09:00
hadoop-project-dist HADOOP-18305. Release Hadoop 3.3.4: upstream changelog and jdiff files 2022-08-05 14:06:22 +01:00
hadoop-tools HADOOP-18028. High performance S3A input stream (#4752) 2022-08-18 13:53:06 +01:00
hadoop-yarn-project YARN-11224. [Federation] Add getAppQueue, updateAppQueue REST APIs for Router. (#4747) 2022-08-17 13:13:07 -07:00
licenses HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina. 2020-10-18 18:37:46 +05:30
licenses-binary HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. (#1796) 2020-01-09 16:24:58 +09:00
.asf.yaml HADOOP-17234. Addendum. Add .asf.yaml to allow github and jira integration. (#4686). Contributed by Ayush Saxena. 2022-08-05 08:34:56 +05:30
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore YARN-10407. Add phantomjsdriver.log to gitignore. (#2244) 2020-09-01 10:44:55 +09:00
BUILDING.txt Update BUILDING.txt (#3811) 2021-12-22 13:08:14 +08:00
LICENSE-binary HADOOP-18358. Update commons-math3 from 3.1.1 to 3.6.1. (#4619). Contributed by fanshilun. 2022-08-02 01:48:47 +05:30
LICENSE.txt HADOOP-18044. Hadoop - Upgrade to jQuery 3.6.0 (#3791) 2022-01-12 11:40:32 +08:00
NOTICE-binary HADOOP-18068. upgrade AWS SDK to 1.12.132 (#3864) 2022-01-18 10:31:28 +00:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml HADOOP-18297. Upgrade dependency-check-maven to 7.1.1 (#4449) 2022-07-03 04:04:55 +08:00
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
start-build-env.sh HADOOP-18052. Support Apple Silicon in start-build-env.sh (#3817) 2021-12-23 18:13:18 +09:00

README.txt 

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/