big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-15384. Document getLocatedBlocks(String src, long start) of DFSClient only return partial blocks. Contributed by Yang Yun.

Browse Source
This commit is contained in:
He Xiaoqiao 2021-03-08 12:18:46 +08:00
parent 9cb51bf106
commit 4a0b7f7ebe
No known key found for this signature in database
GPG Key ID: A80CC124E9A0FA63
1 changed files with 12 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
View File

@ -861,6 +861,18 @@ public long getRefreshReadBlkLocationsInterval() {
return dfsClientConf.getRefreshReadBlockLocationsMS(); return dfsClientConf.getRefreshReadBlockLocationsMS();
} }
/**
* Get locations of the blocks of the specified file `src` from offset
* `start` within the prefetch size which is related to parameter
* `dfs.client.read.prefetch.size`. DataNode locations for each block are
* sorted by the proximity to the client. Please note that the prefetch size
* is not equal file length generally.
*
* @param src the file path.
* @param start starting offset.
* @return LocatedBlocks
* @throws IOException
*/
public LocatedBlocks getLocatedBlocks(String src, long start) public LocatedBlocks getLocatedBlocks(String src, long start)
throws IOException { throws IOException {
return getLocatedBlocks(src, start, dfsClientConf.getPrefetchSize()); return getLocatedBlocks(src, start, dfsClientConf.getPrefetchSize());