big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-4618. Default transaction interval for checkpoints is too low. Contributed by Todd Lipcon.

Browse Source 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461783 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Todd Lipcon 2013-03-27 18:46:32 +00:00
parent 81192e4e41
commit 4809bfa6ed
4 changed files with 6 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -373,6 +373,8 @@ Release 2.0.5-beta - UNRELEASED
HDFS-4621. Additional logging to help diagnose slow QJM syncs. (todd) HDFS-4621. Additional logging to help diagnose slow QJM syncs. (todd)
HDFS-4618. Default transaction interval for checkpoints is too low. (todd)
OPTIMIZATIONS OPTIMIZATIONS
BUG FIXES BUG FIXES

2
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
View File

@ -122,7 +122,7 @@ public class DFSConfigKeys extends CommonConfigurationKeys {
public static final String DFS_NAMENODE_CHECKPOINT_PERIOD_KEY = "dfs.namenode.checkpoint.period"; public static final String DFS_NAMENODE_CHECKPOINT_PERIOD_KEY = "dfs.namenode.checkpoint.period";
public static final long DFS_NAMENODE_CHECKPOINT_PERIOD_DEFAULT = 3600; public static final long DFS_NAMENODE_CHECKPOINT_PERIOD_DEFAULT = 3600;
public static final String DFS_NAMENODE_CHECKPOINT_TXNS_KEY = "dfs.namenode.checkpoint.txns"; public static final String DFS_NAMENODE_CHECKPOINT_TXNS_KEY = "dfs.namenode.checkpoint.txns";
public static final long DFS_NAMENODE_CHECKPOINT_TXNS_DEFAULT = 40000; public static final long DFS_NAMENODE_CHECKPOINT_TXNS_DEFAULT = 1000000;
public static final String DFS_NAMENODE_CHECKPOINT_MAX_RETRIES_KEY = "dfs.namenode.checkpoint.max-retries"; public static final String DFS_NAMENODE_CHECKPOINT_MAX_RETRIES_KEY = "dfs.namenode.checkpoint.max-retries";
public static final int DFS_NAMENODE_CHECKPOINT_MAX_RETRIES_DEFAULT = 3; public static final int DFS_NAMENODE_CHECKPOINT_MAX_RETRIES_DEFAULT = 3;
public static final String DFS_NAMENODE_HEARTBEAT_RECHECK_INTERVAL_KEY = "dfs.namenode.heartbeat.recheck-interval"; public static final String DFS_NAMENODE_HEARTBEAT_RECHECK_INTERVAL_KEY = "dfs.namenode.heartbeat.recheck-interval";

2
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
View File

@ -635,7 +635,7 @@
<property> <property>
<name>dfs.namenode.checkpoint.txns</name> <name>dfs.namenode.checkpoint.txns</name>
<value>40000</value> <value>1000000</value>
<description>The Secondary NameNode or CheckpointNode will create a checkpoint <description>The Secondary NameNode or CheckpointNode will create a checkpoint
of the namespace every 'dfs.namenode.checkpoint.txns' transactions, regardless of the namespace every 'dfs.namenode.checkpoint.txns' transactions, regardless
of whether 'dfs.namenode.checkpoint.period' has expired. of whether 'dfs.namenode.checkpoint.period' has expired.

4
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsUserGuide.apt.vm
View File

@ -193,7 +193,7 @@ HDFS Users Guide
* <<<dfs.namenode.checkpoint.period>>>, set to 1 hour by default, specifies * <<<dfs.namenode.checkpoint.period>>>, set to 1 hour by default, specifies
the maximum delay between two consecutive checkpoints, and the maximum delay between two consecutive checkpoints, and
* <<<dfs.namenode.checkpoint.txns>>>, set to 40000 default, defines the * <<<dfs.namenode.checkpoint.txns>>>, set to 1 million by default, defines the
number of uncheckpointed transactions on the NameNode which will number of uncheckpointed transactions on the NameNode which will
force an urgent checkpoint, even if the checkpoint period has not force an urgent checkpoint, even if the checkpoint period has not
been reached. been reached.
@ -232,7 +232,7 @@ HDFS Users Guide
* <<<dfs.namenode.checkpoint.period>>>, set to 1 hour by default, specifies * <<<dfs.namenode.checkpoint.period>>>, set to 1 hour by default, specifies
the maximum delay between two consecutive checkpoints the maximum delay between two consecutive checkpoints
* <<<dfs.namenode.checkpoint.txns>>>, set to 40000 default, defines the * <<<dfs.namenode.checkpoint.txns>>>, set to 1 million by default, defines the
number of uncheckpointed transactions on the NameNode which will number of uncheckpointed transactions on the NameNode which will
force an urgent checkpoint, even if the checkpoint period has not force an urgent checkpoint, even if the checkpoint period has not
been reached. been reached.