hadoop/hadoop-common-project
Steve Loughran 75950e47e7
HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1)
This defines standard option and values for the
openFile() builder API for opening a file:

fs.option.openfile.read.policy
 A list of the desired read policy, in preferred order.
 standard values are
 adaptive, default, random, sequential, vector, whole-file

fs.option.openfile.length
 How long the file is.

fs.option.openfile.split.start
 start of a task's split

fs.option.openfile.split.end
 end of a task's split

These can be used by filesystem connectors to optimize their
reading of the source file, including but not limited to
* skipping existence/length probes when opening a file
* choosing a policy for prefetching/caching data

The hadoop shell commands which read files all declare "whole-file"
and "sequential", as appropriate.

Contributed by Steve Loughran.

Change-Id: Ia290f79ea7973ce8713d4f90f1315b24d7a23da1
2022-04-27 19:23:10 +01:00
..
hadoop-annotations HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-auth HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-auth-examples HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-common HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1) 2022-04-27 19:23:10 +01:00
hadoop-kms HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-minikdc HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-nfs HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-registry HADOOP-17956. Replace all default Charset usage with UTF-8 (#3529) 2022-04-27 10:30:07 +01:00
pom.xml HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00