hadoop/hadoop-common-project/hadoop-common
Steve Loughran 75950e47e7
HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1)
This defines standard option and values for the
openFile() builder API for opening a file:

fs.option.openfile.read.policy
 A list of the desired read policy, in preferred order.
 standard values are
 adaptive, default, random, sequential, vector, whole-file

fs.option.openfile.length
 How long the file is.

fs.option.openfile.split.start
 start of a task's split

fs.option.openfile.split.end
 end of a task's split

These can be used by filesystem connectors to optimize their
reading of the source file, including but not limited to
* skipping existence/length probes when opening a file
* choosing a policy for prefetching/caching data

The hadoop shell commands which read files all declare "whole-file"
and "sequential", as appropriate.

Contributed by Steve Loughran.

Change-Id: Ia290f79ea7973ce8713d4f90f1315b24d7a23da1
2022-04-27 19:23:10 +01:00
..
dev-support Make upstream aware of 3.2.2 release. 2021-01-09 18:07:10 +08:00
src HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1) 2022-04-27 19:23:10 +01:00
HadoopCommon.cmake HADOOP-17196. Fix C/C++ standard warnings (#2208) 2020-08-11 16:35:41 +09:00
HadoopJNI.cmake HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati 2018-08-17 12:25:36 -05:00
pom.xml HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00