big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-6530. Fix Balancer documentation.

Browse Source 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603571 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Tsz-wo Sze 2014-06-18 18:44:28 +00:00
parent bae5837d3b
commit 1af69553a1
4 changed files with 15 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
View File

@ -306,9 +306,9 @@ Administration Commands
Runs a cluster balancing utility. An administrator can simply press Ctrl-C Runs a cluster balancing utility. An administrator can simply press Ctrl-C
to stop the rebalancing process. See to stop the rebalancing process. See
{{{../hadoop-hdfs/HdfsUserGuide.html#Rebalancer}Rebalancer}} for more details. {{{../hadoop-hdfs/HdfsUserGuide.html#Balancer}Balancer}} for more details.
Usage: <<<hadoop balancer [-threshold <threshold>]>>> Usage: <<<hadoop balancer [-threshold <threshold>] [-policy <policy>]>>>
*------------------------+-----------------------------------------------------------+ *------------------------+-----------------------------------------------------------+
|| COMMAND_OPTION | Description || COMMAND_OPTION | Description
@ -316,6 +316,11 @@ Administration Commands
| -threshold <threshold> | Percentage of disk capacity. This overwrites the | -threshold <threshold> | Percentage of disk capacity. This overwrites the
| default threshold. | default threshold.
*------------------------+-----------------------------------------------------------+ *------------------------+-----------------------------------------------------------+
| -policy <policy> | <<<datanode>>> (default): Cluster is balanced if each datanode is balanced. \
| <<<blockpool>>>: Cluster is balanced if each block pool in each datanode is balanced.
*------------------------+-----------------------------------------------------------+
Note that the <<<blockpool>>> policy is more strict than the <<<datanode>>> policy.
* <<<daemonlog>>> * <<<daemonlog>>>

2
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -452,6 +452,8 @@ Release 2.5.0 - UNRELEASED
HDFS-6545. Finalizing rolling upgrade can make NN unavailable for a long HDFS-6545. Finalizing rolling upgrade can make NN unavailable for a long
duration. (kihwal) duration. (kihwal)
HDFS-6530. Fix Balancer documentation. (szetszwo)
OPTIMIZATIONS OPTIMIZATIONS
HDFS-6214. Webhdfs has poor throughput for files >2GB (daryn) HDFS-6214. Webhdfs has poor throughput for files >2GB (daryn)

5
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm
View File

@ -287,13 +287,14 @@ HDFS Federation
Policy could be: Policy could be:
* <<<node>>> - this is the <default> policy. This balances the storage at * <<<datanode>>> - this is the <default> policy. This balances the storage at
the datanode level. This is similar to balancing policy from prior releases. the datanode level. This is similar to balancing policy from prior releases.
* <<<blockpool>>> - this balances the storage at the block pool level. * <<<blockpool>>> - this balances the storage at the block pool level.
Balancing at block pool level balances storage at the datanode level also. Balancing at block pool level balances storage at the datanode level also.
Note that Balander only balances the data and does not balance the namespace. Note that Balancer only balances the data and does not balance the namespace.
For the complete command usage, see {{{../hadoop-common/CommandsManual.html#balancer}balancer}}.
** Decommissioning ** Decommissioning

6
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsUserGuide.apt.vm
View File

@ -77,7 +77,7 @@ HDFS Users Guide
* <<<fetchdt>>>: a utility to fetch DelegationToken and store it in a * <<<fetchdt>>>: a utility to fetch DelegationToken and store it in a
file on the local system. file on the local system.
* Rebalancer: tool to balance the cluster when the data is * Balancer: tool to balance the cluster when the data is
unevenly distributed among DataNodes. unevenly distributed among DataNodes.
* Upgrade and rollback: after a software upgrade, it is possible * Upgrade and rollback: after a software upgrade, it is possible
@ -316,7 +316,7 @@ HDFS Users Guide
For command usage, see {{{../hadoop-common/CommandsManual.html#namenode}namenode}}. For command usage, see {{{../hadoop-common/CommandsManual.html#namenode}namenode}}.
* Rebalancer * Balancer
HDFS data might not always be be placed uniformly across the DataNode. HDFS data might not always be be placed uniformly across the DataNode.
One common reason is addition of new DataNodes to an existing cluster. One common reason is addition of new DataNodes to an existing cluster.
@ -338,7 +338,7 @@ HDFS Users Guide
Due to multiple competing considerations, data might not be uniformly Due to multiple competing considerations, data might not be uniformly
placed across the DataNodes. HDFS provides a tool for administrators placed across the DataNodes. HDFS provides a tool for administrators
that analyzes block placement and rebalanaces data across the DataNode. that analyzes block placement and rebalanaces data across the DataNode.
A brief administrator's guide for rebalancer as a PDF is attached to A brief administrator's guide for balancer is available at
{{{https://issues.apache.org/jira/browse/HADOOP-1652}HADOOP-1652}}. {{{https://issues.apache.org/jira/browse/HADOOP-1652}HADOOP-1652}}.
For command usage, see {{{../hadoop-common/CommandsManual.html#balancer}balancer}}. For command usage, see {{{../hadoop-common/CommandsManual.html#balancer}balancer}}.