hadoop/hadoop-common-project
Steve Loughran 1b4dba99b5
HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1)
This defines standard option and values for the
openFile() builder API for opening a file:

fs.option.openfile.read.policy
 A list of the desired read policy, in preferred order.
 standard values are
 adaptive, default, random, sequential, vector, whole-file

fs.option.openfile.length
 How long the file is.

fs.option.openfile.split.start
 start of a task's split

fs.option.openfile.split.end
 end of a task's split

These can be used by filesystem connectors to optimize their
reading of the source file, including but not limited to
* skipping existence/length probes when opening a file
* choosing a policy for prefetching/caching data

The hadoop shell commands which read files all declare "whole-file"
and "sequential", as appropriate.

Contributed by Steve Loughran.

Change-Id: Ia290f79ea7973ce8713d4f90f1315b24d7a23da1
2022-04-24 17:33:04 +01:00
..
hadoop-annotations HADOOP-17947. Additional element types for VisibleForTesting (ADDENDUM) (#3521) 2021-10-06 02:17:05 +09:00
hadoop-auth HADOOP-18006. maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms (#3648) 2021-11-15 22:57:24 +09:00
hadoop-auth-examples Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-common HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1) 2022-04-24 17:33:04 +01:00
hadoop-kms HADOOP-18055. Async Profiler endpoint for Hadoop daemons (#3824) 2022-01-06 17:56:49 +08:00
hadoop-minikdc HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 2021-03-11 10:56:07 +09:00
hadoop-nfs HADOOP-18006. maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms (#3648) 2021-11-15 22:57:24 +09:00
hadoop-registry HADOOP-18040. Use maven.test.failure.ignore instead of ignoreTestFailure (#3774) 2021-12-10 01:36:31 +09:00
pom.xml Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30