big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-17018. Intermittent failing of ITestAbfsStreamStatistics in ABFS (#1990)

Browse Source 
Contributed by: Mehakmeet Singh

In some cases, ABFS-prefetch thread runs in the background which returns some bytes from the buffer and gives an extra readOp. Thus, making readOps values arbitrary and giving intermittent failures in some cases. Hence, readOps values of 2 or 3 are seen in different setups.
This commit is contained in:
Mehakmeet Singh 2020-05-07 16:45:28 +05:30 committed by GitHub
parent 99840aaba6
commit 192cad9ee2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 14 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsStreamStatistics.java
View File

@ -84,12 +84,21 @@ public void testAbfsStreamOps() throws Exception {
LOG.info("Result of Read operation : {}", result);
/*
Testing if 2 read_ops value is coming after reading full content from a
file (3 if anything to read from Buffer too).
Reason: read() call gives read_ops=1,
reading from AbfsClient(http GET) gives read_ops=2.
* Testing if 2 read_ops value is coming after reading full content
* from a file (3 if anything to read from Buffer too). Reason: read()
* call gives read_ops=1, reading from AbfsClient(http GET) gives
* read_ops=2.
*
* In some cases ABFS-prefetch thread runs in the background which
* returns some bytes from buffer and gives an extra readOp.
* Thus, making readOps values arbitrary and giving intermittent
* failures in some cases. Hence, readOps values of 2 or 3 is seen in
* different setups.
*
*/
assertReadWriteOps("read", 2, statistics.getReadOps());
assertTrue(String.format("The actual value of %d was not equal to the "
+ "expected value of 2 or 3", statistics.getReadOps()),
statistics.getReadOps() == 2 || statistics.getReadOps() == 3);
} finally {
IOUtils.cleanupWithLogger(LOG, inForOneOperation,