big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-16209. Add description for dfs.namenode.caching.enabled (#3378)

Browse Source
This commit is contained in:
litao 2021-09-08 12:38:55 +08:00 committed by GitHub
parent 9512c774c0
commit 5a306955a4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
View File

@ -462,6 +462,7 @@ public class DFSConfigKeys extends CommonConfigurationKeys {
public static final long DFS_NAMENODE_PATH_BASED_CACHE_REFRESH_INTERVAL_MS_DEFAULT = 30000L; public static final long DFS_NAMENODE_PATH_BASED_CACHE_REFRESH_INTERVAL_MS_DEFAULT = 30000L;
public static final String DFS_NAMENODE_CACHING_ENABLED_KEY = public static final String DFS_NAMENODE_CACHING_ENABLED_KEY =
"dfs.namenode.caching.enabled"; "dfs.namenode.caching.enabled";
// TODO: Default value to be set false in 4.0.0 release onwards (HDFS-16209)
public static final boolean DFS_NAMENODE_CACHING_ENABLED_DEFAULT = true; public static final boolean DFS_NAMENODE_CACHING_ENABLED_DEFAULT = true;
/** Pending period of block deletion since NameNode startup */ /** Pending period of block deletion since NameNode startup */

4
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
View File

@ -2868,6 +2868,10 @@
maintain a mapping of cached blocks to DataNodes via processing DataNode maintain a mapping of cached blocks to DataNodes via processing DataNode
cache reports. Based on these reports and addition and removal of caching cache reports. Based on these reports and addition and removal of caching
directives, the NameNode will schedule caching and uncaching work. directives, the NameNode will schedule caching and uncaching work.
In the current implementation, centralized caching introduces additional
write lock overhead (see CacheReplicationMonitor#rescan) even if no path
to cache is specified, so we recommend disabling this feature when not in
use. We will disable centralized caching by default in later versions.
</description> </description>
</property> </property>

2
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/CentralizedCacheManagement.md
View File

@ -256,7 +256,7 @@ The following properties are not required, but may be specified for tuning:
* dfs.namenode.caching.enabled * dfs.namenode.caching.enabled
This parameter can be used to enable/disable the centralized caching in NameNode. When centralized caching is disabled, NameNode will not process cache reports or store information about block cache locations on the cluster. Note that NameNode will continute to store the path based cache locations in the file-system metadata, even though it will not act on this information until the caching is enabled. The default value for this parameter is true (i.e. centralized caching is enabled). This parameter can be used to enable/disable the centralized caching in NameNode. When centralized caching is disabled, NameNode will not process cache reports or store information about block cache locations on the cluster. Note that NameNode will continute to store the path based cache locations in the file-system metadata, even though it will not act on this information until the caching is enabled. The default value for this parameter is true (i.e. centralized caching is enabled). In the current implementation, centralized caching introduces additional write lock overhead (see CacheReplicationMonitor#rescan) even if no path to cache is specified, so we recommend disabling this feature when not in use. We will disable centralized caching by default in later versions.
* dfs.datanode.pmem.cache.recovery * dfs.datanode.pmem.cache.recovery