big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-14154. [SBN read] Document dfs.ha.tail-edits.period in user guide. Contributed by Chao Sun.

Browse Source
This commit is contained in:
Chao Sun 2018-12-19 22:20:31 -08:00 committed by Konstantin V Shvachko
parent c9d7737431
commit c546b12e45
2 changed files with 22 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md
View File

@ -27,7 +27,12 @@ This document assumes that the reader has a general understanding of general com
Note: Using the Quorum Journal Manager or Conventional Shared Storage Note: Using the Quorum Journal Manager or Conventional Shared Storage
--------------------------------------------------------------------- ---------------------------------------------------------------------
This guide discusses how to configure and use HDFS HA using the Quorum Journal Manager (QJM) to share edit logs between the Active and Standby NameNodes. For information on how to configure HDFS HA using NFS for shared storage instead of the QJM, please see [this alternative guide.](./HDFSHighAvailabilityWithNFS.html) This guide discusses how to configure and use HDFS HA using the Quorum
Journal Manager (QJM) to share edit logs between the Active and Standby
NameNodes. For information on how to configure HDFS HA using NFS for
shared storage instead of the QJM, please see [this alternative
guide.](./HDFSHighAvailabilityWithNFS.html). For information on how to
configure HDFS HA with Observer NameNode, please see [this guide](./ObserverNameNode.html)
Background Background
---------- ----------

18
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ObserverNameNode.md
View File

@ -23,13 +23,13 @@ Purpose
This guide provides an overview of the HDFS Observer NameNode feature This guide provides an overview of the HDFS Observer NameNode feature
and how to configure/install it in a typical HA-enabled cluster. For a and how to configure/install it in a typical HA-enabled cluster. For a
detailed technical design overview, please check the doc attached to detailed technical design overview, please check the doc attached to
HDFS-12943. [HDFS-12943](https://issues.apache.org/jira/browse/HDFS-12943).
Background Background
----------- -----------
In a HA-enabled HDFS cluster (for more information, check In a HA-enabled HDFS cluster (for more information, check
[HDFSHighAvailabilityWithQJM](./HDFSHighAvailabilityWithQJM.md)), there [HDFSHighAvailabilityWithQJM](./HDFSHighAvailabilityWithQJM.html)), there
is a single Active NameNode and one or more Standby NameNode(s). The is a single Active NameNode and one or more Standby NameNode(s). The
Active NameNode is responsible for serving all client requests, while Active NameNode is responsible for serving all client requests, while
Standby NameNode just keep the up-to-date information regarding the Standby NameNode just keep the up-to-date information regarding the
@ -104,6 +104,20 @@ few configurations to your **hdfs-site.xml**:
<value>true</value> <value>true</value>
</property> </property>
* **dfs.ha.tail-edits.period** - how often Standby/Observer NameNodes
should fetch edits from JournalNodes.
This determines the staleness of Observer NameNode w.r.t the Active.
If too large, RPC time will increase as client requests will wait
longer in the RPC queue before Observer tails edit logs and catches
up the latest state of Active. The default value is 1min. It is
**highly recommend** to configure this to a much lower value.
<property>
<name>dfs.ha.tail-edits.period</name>
<value>0ms</value>
</property>
* **dfs.journalnode.edit-cache-size.bytes** - the in-memory cache size, * **dfs.journalnode.edit-cache-size.bytes** - the in-memory cache size,
in bytes, on the JournalNodes. in bytes, on the JournalNodes.