big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-6177. FSInputChecker.getPos() would return position greater than the file size. Contributed by Hong Tang.

Browse Source 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@803190 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Hairong Kuang 2009-08-11 16:26:55 +00:00
parent 5fe6906f2a
commit d70dd1a2df
4 changed files with 17 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
CHANGES.txt
View File

@ -502,6 +502,9 @@ Trunk (unreleased changes)
HADOOP-5638. More improvement on block placement performance. (hairong)
HADOOP-6180. NameNode slowed down when many files with same filename
were moved to Trash. (Boris Shkolnik via hairong)
BUG FIXES
HADOOP-5379. CBZip2InputStream to throw IOException on data crc error.
@ -908,6 +911,9 @@ Trunk (unreleased changes)
HADOOP-6124. Fix javac warning detection in test-patch.sh. (Giridharan
Kesavan via szetszwo)
HADOOP-6177. FSInputChecker.getPos() would return position greater
than the file size. (Hong Tang via hairong)
Release 0.20.1 - Unreleased
INCOMPATIBLE CHANGES
@ -4466,9 +4472,6 @@ Release 0.17.0 - 2008-05-18
exponentially increasing number of records (up to 10,000
records/log). (Zheng Shao via omalley)
HADOOP-6180. NameNode slowed down when many files with same filename
were moved to Trash. (Boris Shkolnik via hairong)
BUG FIXES
HADOOP-2195. '-mkdir' behaviour is now closer to Linux shell in case of

1
src/java/org/apache/hadoop/fs/FSInputChecker.java
View File

@ -174,6 +174,7 @@ private void fill( ) throws IOException {
assert(pos>=count);
// fill internal buffer
count = readChecksumChunk(buf, 0, buf.length);
if (count < 0) count = 0;
}
/*

8
src/test/core/org/apache/hadoop/fs/TestChecksumFileSystem.java
View File

@ -54,6 +54,10 @@ public void testVerifyChecksum() throws Exception {
fout.write("testing you".getBytes());
fout.close();
TestLocalFileSystem.readFile(localFs, testPath, 128);
TestLocalFileSystem.readFile(localFs, testPath, 512);
TestLocalFileSystem.readFile(localFs, testPath, 1024);
localFs.delete(localFs.getChecksumFile(testPath), true);
assertTrue("checksum deleted", !localFs.exists(localFs.getChecksumFile(testPath)));
@ -64,7 +68,7 @@ public void testVerifyChecksum() throws Exception {
boolean errorRead = false;
try {
TestLocalFileSystem.readFile(localFs, testPath);
TestLocalFileSystem.readFile(localFs, testPath, 1024);
}catch(ChecksumException ie) {
errorRead = true;
}
@ -72,7 +76,7 @@ public void testVerifyChecksum() throws Exception {
//now setting verify false, the read should succeed
localFs.setVerifyChecksum(false);
String str = TestLocalFileSystem.readFile(localFs, testPath);
String str = TestLocalFileSystem.readFile(localFs, testPath, 1024);
assertTrue("read", "testing".equals(str));
}

5
src/test/core/org/apache/hadoop/fs/TestLocalFileSystem.java
View File

@ -35,13 +35,14 @@ static void writeFile(FileSystem fs, Path name) throws IOException {
stm.close();
}
static String readFile(FileSystem fs, Path name) throws IOException {
byte[] b = new byte[1024];
static String readFile(FileSystem fs, Path name, int buflen) throws IOException {
byte[] b = new byte[buflen];
int offset = 0;
FSDataInputStream in = fs.open(name);
for(int remaining, n;
(remaining = b.length - offset) > 0 && (n = in.read(b, offset, remaining)) != -1;
offset += n);
assertEquals(offset, Math.min(b.length, in.getPos()));
in.close();
String s = new String(b, 0, offset);