hadoop/hadoop-common-project/hadoop-common/src
Steve Loughran 75950e47e7
HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1)
This defines standard option and values for the
openFile() builder API for opening a file:

fs.option.openfile.read.policy
 A list of the desired read policy, in preferred order.
 standard values are
 adaptive, default, random, sequential, vector, whole-file

fs.option.openfile.length
 How long the file is.

fs.option.openfile.split.start
 start of a task's split

fs.option.openfile.split.end
 end of a task's split

These can be used by filesystem connectors to optimize their
reading of the source file, including but not limited to
* skipping existence/length probes when opening a file
* choosing a policy for prefetching/caching data

The hadoop shell commands which read files all declare "whole-file"
and "sequential", as appropriate.

Contributed by Steve Loughran.

Change-Id: Ia290f79ea7973ce8713d4f90f1315b24d7a23da1
2022-04-27 19:23:10 +01:00
..
contrib/bash-tab-completion
main HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1) 2022-04-27 19:23:10 +01:00
site HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1) 2022-04-27 19:23:10 +01:00
test HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1) 2022-04-27 19:23:10 +01:00
CMakeLists.txt HADOOP-17292. Using lz4-java in Lz4Codec (#2350) 2020-12-29 13:17:26 -08:00
config.h.cmake HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00