big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao.

Browse Source
This commit is contained in:
Jing Zhao 2015-09-04 11:42:22 -07:00
parent d16c4eee18
commit 8928729c80
3 changed files with 12 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -891,6 +891,9 @@ Release 2.8.0 - UNRELEASED
HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to
hadoop-hdfs-client module. (Mingliang Liu via wheat9) hadoop-hdfs-client module. (Mingliang Liu via wheat9)
HDFS-8384. Allow NN to startup if there are files having a lease but are not
under construction. (jing9)
OPTIMIZATIONS OPTIMIZATIONS
HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than

5
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
View File

@ -3184,7 +3184,10 @@ void finalizeINodeFileUnderConstruction(
assert hasWriteLock(); assert hasWriteLock();
FileUnderConstructionFeature uc = pendingFile.getFileUnderConstructionFeature(); FileUnderConstructionFeature uc = pendingFile.getFileUnderConstructionFeature();
Preconditions.checkArgument(uc != null); if (uc == null) {
throw new IOException("Cannot finalize file " + src
+ " because it is not under construction");
}
leaseManager.removeLease(uc.getClientName(), pendingFile); leaseManager.removeLease(uc.getClientName(), pendingFile);
pendingFile.recordModification(latestSnapshot); pendingFile.recordModification(latestSnapshot);

6
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
View File

@ -108,7 +108,11 @@ synchronized long getNumUnderConstructionBlocks() {
long numUCBlocks = 0; long numUCBlocks = 0;
for (Long id : getINodeIdWithLeases()) { for (Long id : getINodeIdWithLeases()) {
final INodeFile cons = fsnamesystem.getFSDirectory().getInode(id).asFile(); final INodeFile cons = fsnamesystem.getFSDirectory().getInode(id).asFile();
Preconditions.checkState(cons.isUnderConstruction()); if (!cons.isUnderConstruction()) {
LOG.warn("The file " + cons.getFullPathName()
+ " is not under construction but has lease.");
continue;
}
BlockInfo[] blocks = cons.getBlocks(); BlockInfo[] blocks = cons.getBlocks();
if(blocks == null) { if(blocks == null) {
continue; continue;