87fb977777
Clarifies behaviour of VectorIO methods with contract tests as well as specification. * Add precondition range checks to all implementations * Identify and fix bug where direct buffer reads was broken (HADOOP-19101; this surfaced in ABFS contract tests) * Logging in VectoredReadUtils. * TestVectoredReadUtils verifies validation logic. * FileRangeImpl toString() improvements * CombinedFileRange tracks bytes in range which are wanted; toString() output logs this. HDFS * Add test TestHDFSContractVectoredRead ABFS * Add test ITestAbfsFileSystemContractVectoredRead S3A * checks for vector IO being stopped in all iterative vector operations, including draining * maps read() returning -1 to failure * passes in file length to validation * Error reporting to only completeExceptionally() those ranges which had not yet read data in. * Improved logging. readVectored() * made synchronized. This is only for the invocation; the actual async retrieves are unsynchronized. * closes input stream on invocation * switches to random IO, so avoids keeping any long-lived connection around. + AbstractSTestS3AHugeFiles enhancements. + ADDENDUM: test fix in ITestS3AContractVectoredRead Contains: HADOOP-19101. Vectored Read into off-heap buffer broken in fallback implementation Contributed by Steve Loughran Change-Id: Ia4ed71864c595f175c275aad83a2ff5741693432 |
||
|---|---|---|
| .github | ||
| .yetus | ||
| dev-support | ||
| hadoop-assemblies | ||
| hadoop-build-tools | ||
| hadoop-client-modules | ||
| hadoop-cloud-storage-project | ||
| hadoop-common-project | ||
| hadoop-dist | ||
| hadoop-hdfs-project | ||
| hadoop-mapreduce-project | ||
| hadoop-maven-plugins | ||
| hadoop-minicluster | ||
| hadoop-project | ||
| hadoop-project-dist | ||
| hadoop-tools | ||
| hadoop-yarn-project | ||
| licenses | ||
| licenses-binary | ||
| .asf.yaml | ||
| .gitattributes | ||
| .gitignore | ||
| BUILDING.txt | ||
| LICENSE-binary | ||
| LICENSE.txt | ||
| NOTICE-binary | ||
| NOTICE.txt | ||
| pom.xml | ||
| README.txt | ||
| start-build-env.sh | ||
For the latest information about Hadoop, please visit our website at: http://hadoop.apache.org/ and our wiki, at: https://cwiki.apache.org/confluence/display/HADOOP/