big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke.

Browse Source
This commit is contained in:
Haohui Mai 2015-11-22 16:10:27 -08:00
parent f0cd903866
commit 0c7340f377
2 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -2337,6 +2337,9 @@ Release 2.8.0 - UNRELEASED
HDFS-9443. Disabling HDFS client socket cache causes logging message HDFS-9443. Disabling HDFS client socket cache causes logging message
printed to console for CLI commands. (Chris Nauroth via wheat9) printed to console for CLI commands. (Chris Nauroth via wheat9)
HDFS-8914. Document HA support in the HDFS HdfsDesign.md.
(Lars Francke via wheat9)
Release 2.7.3 - UNRELEASED Release 2.7.3 - UNRELEASED
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

2
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
View File

@ -186,7 +186,7 @@ It is possible that a block of data fetched from a DataNode arrives corrupted. T
The FsImage and the EditLog are central data structures of HDFS. A corruption of these files can cause the HDFS instance to be non-functional. For this reason, the NameNode can be configured to support maintaining multiple copies of the FsImage and EditLog. Any update to either the FsImage or EditLog causes each of the FsImages and EditLogs to get updated synchronously. This synchronous updating of multiple copies of the FsImage and EditLog may degrade the rate of namespace transactions per second that a NameNode can support. However, this degradation is acceptable because even though HDFS applications are very data intensive in nature, they are not metadata intensive. When a NameNode restarts, it selects the latest consistent FsImage and EditLog to use. The FsImage and the EditLog are central data structures of HDFS. A corruption of these files can cause the HDFS instance to be non-functional. For this reason, the NameNode can be configured to support maintaining multiple copies of the FsImage and EditLog. Any update to either the FsImage or EditLog causes each of the FsImages and EditLogs to get updated synchronously. This synchronous updating of multiple copies of the FsImage and EditLog may degrade the rate of namespace transactions per second that a NameNode can support. However, this degradation is acceptable because even though HDFS applications are very data intensive in nature, they are not metadata intensive. When a NameNode restarts, it selects the latest consistent FsImage and EditLog to use.
The NameNode machine is a single point of failure for an HDFS cluster. If the NameNode machine fails, manual intervention is necessary. Currently, automatic restart and failover of the NameNode software to another machine is not supported. Another option to increase resilience against failures is to enable High Availability using multiple NameNodes either with a [shared storage on NFS](./HDFSHighAvailabilityWithNFS.html) or using a [distributed edit log](./HDFSHighAvailabilityWithQJM.html) (called Journal). The latter is the recommended approach.
### Snapshots ### Snapshots