big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-12351. Explicitly describe the minimal number of DataNodes required to support an EC policy in EC document.. Contributed by Hanisha Koneru.

Browse Source
This commit is contained in:
Arpit Agarwal 2018-01-02 12:54:40 -08:00
parent dfe0cd8655
commit b4d11337c9
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
View File

@ -99,6 +99,10 @@ Deployment
Encoding and decoding work consumes additional CPU on both HDFS clients and DataNodes. Encoding and decoding work consumes additional CPU on both HDFS clients and DataNodes.
Erasure coding requires a minimum of as many DataNodes in the cluster as
the configured EC stripe width. For EC policy RS (6,3), this means
a minimum of 9 DataNodes.
Erasure coded files are also spread across racks for rack fault-tolerance. Erasure coded files are also spread across racks for rack fault-tolerance.
This means that when reading and writing striped files, most operations are off-rack. This means that when reading and writing striped files, most operations are off-rack.
Network bisection bandwidth is thus very important. Network bisection bandwidth is thus very important.