big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-13628. Update Archival Storage doc for Provided Storage

Browse Source 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
This commit is contained in:
Takanobu Asanuma 2018-05-28 19:04:36 +09:00 committed by Akira Ajisaka
parent 88cbe57c06
commit 04757e5864
No known key found for this signature in database
GPG Key ID: C1EDBB9CA400FD50
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md
View File

@ -35,7 +35,7 @@ A new storage type *ARCHIVE*, which has high storage density (petabyte of storag
Another new storage type *RAM\_DISK* is added for supporting writing single replica files in memory. Another new storage type *RAM\_DISK* is added for supporting writing single replica files in memory.
### Storage Policies: Hot, Warm, Cold, All\_SSD, One\_SSD and Lazy\_Persist ### Storage Policies: Hot, Warm, Cold, All\_SSD, One\_SSD, Lazy\_Persist and Provided
A new concept of storage policies is introduced in order to allow files to be stored in different storage types according to the storage policy. A new concept of storage policies is introduced in order to allow files to be stored in different storage types according to the storage policy.
@ -47,6 +47,7 @@ We have the following storage policies:
* **All\_SSD** - for storing all replicas in SSD. * **All\_SSD** - for storing all replicas in SSD.
* **One\_SSD** - for storing one of the replicas in SSD. The remaining replicas are stored in DISK. * **One\_SSD** - for storing one of the replicas in SSD. The remaining replicas are stored in DISK.
* **Lazy\_Persist** - for writing blocks with single replica in memory. The replica is first written in RAM\_DISK and then it is lazily persisted in DISK. * **Lazy\_Persist** - for writing blocks with single replica in memory. The replica is first written in RAM\_DISK and then it is lazily persisted in DISK.
* **Provided** - for storing data outside HDFS. See also [HDFS Provided Storage](./HdfsProvidedStorage.html).
More formally, a storage policy consists of the following fields: More formally, a storage policy consists of the following fields:
@ -68,6 +69,7 @@ The following is a typical storage policy table.
| 7 | Hot (default) | DISK: *n* | \<none\> | ARCHIVE | | 7 | Hot (default) | DISK: *n* | \<none\> | ARCHIVE |
| 5 | Warm | DISK: 1, ARCHIVE: *n*-1 | ARCHIVE, DISK | ARCHIVE, DISK | | 5 | Warm | DISK: 1, ARCHIVE: *n*-1 | ARCHIVE, DISK | ARCHIVE, DISK |
| 2 | Cold | ARCHIVE: *n* | \<none\> | \<none\> | | 2 | Cold | ARCHIVE: *n* | \<none\> | \<none\> |
| 1 | Provided | PROVIDED: 1, DISK: *n*-1 | PROVIDED, DISK | PROVIDED, DISK |
Note 1: The Lazy\_Persist policy is useful only for single replica blocks. For blocks with more than one replicas, all the replicas will be written to DISK since writing only one of the replicas to RAM\_DISK does not improve the overall performance. Note 1: The Lazy\_Persist policy is useful only for single replica blocks. For blocks with more than one replicas, all the replicas will be written to DISK since writing only one of the replicas to RAM\_DISK does not improve the overall performance.