7f9ca101e2
* HADOOP-18517. ABFS: Add fs.azure.enable.readahead option to disable readahead Adds new config option to turn off readahead * also allows it to be passed in through openFile(), * extends ITestAbfsReadWriteAndSeek to use the option, including one replicated test...that shows that turning it off is slower. Important: this does not address the critical data corruption issue HADOOP-18521. ABFS ReadBufferManager buffer sharing across concurrent HTTP requests What is does do is provide a way to completely bypass the ReadBufferManager. To mitigate the problem, either fs.azure.enable.readahead needs to be set to false, or set "fs.azure.readaheadqueue.depth" to 0 -this still goes near the (broken) ReadBufferManager code, but does't trigger the bug. For safe reading of files through the ABFS connector, readahead MUST be disabled or the followup fix to HADOOP-18521 applied Contributed by Steve Loughran |
||
|---|---|---|
| .github | ||
| .yetus | ||
| dev-support | ||
| hadoop-assemblies | ||
| hadoop-build-tools | ||
| hadoop-client-modules | ||
| hadoop-cloud-storage-project | ||
| hadoop-common-project | ||
| hadoop-dist | ||
| hadoop-hdfs-project | ||
| hadoop-mapreduce-project | ||
| hadoop-maven-plugins | ||
| hadoop-minicluster | ||
| hadoop-project | ||
| hadoop-project-dist | ||
| hadoop-tools | ||
| hadoop-yarn-project | ||
| licenses | ||
| licenses-binary | ||
| .asf.yaml | ||
| .gitattributes | ||
| .gitignore | ||
| BUILDING.txt | ||
| LICENSE-binary | ||
| LICENSE.txt | ||
| NOTICE-binary | ||
| NOTICE.txt | ||
| pom.xml | ||
| README.txt | ||
| start-build-env.sh | ||
For the latest information about Hadoop, please visit our website at: http://hadoop.apache.org/ and our wiki, at: https://cwiki.apache.org/confluence/display/HADOOP/