big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty.

Browse Source
This commit is contained in:
Chris Nauroth 2016-03-01 13:18:42 -08:00
parent 5c465df904
commit f98dff329b
2 changed files with 6 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-common-project/hadoop-common/CHANGES.txt
View File

@ -1218,6 +1218,9 @@ Release 2.8.0 - UNRELEASED
HADOOP-10865. Add a Crc32 chunked verification benchmark for both directly HADOOP-10865. Add a Crc32 chunked verification benchmark for both directly
and non-directly buffer cases. (szetszwo) and non-directly buffer cases. (szetszwo)
HADOOP-12853. Change WASB documentation regarding page blob support.
(Madhumita Chakraborty via cnauroth)
BUG FIXES BUG FIXES
HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals

6
hadoop-tools/hadoop-azure/src/site/markdown/index.md
View File

@ -40,8 +40,8 @@ on the additional artifacts it requires, notably the
* Present a hierarchical file system view by implementing the standard Hadoop * Present a hierarchical file system view by implementing the standard Hadoop
[`FileSystem`](../api/org/apache/hadoop/fs/FileSystem.html) interface. [`FileSystem`](../api/org/apache/hadoop/fs/FileSystem.html) interface.
* Supports configuration of multiple Azure Blob Storage accounts. * Supports configuration of multiple Azure Blob Storage accounts.
* Supports both page blobs (suitable for most use cases, such as MapReduce) and * Supports both block blobs (suitable for most use cases, such as MapReduce) and
block blobs (suitable for continuous write use cases, such as an HBase page blobs (suitable for continuous write use cases, such as an HBase
write-ahead log). write-ahead log).
* Reference file system paths using URLs using the `wasb` scheme. * Reference file system paths using URLs using the `wasb` scheme.
* Also reference file system paths using URLs with the `wasbs` scheme for SSL * Also reference file system paths using URLs with the `wasbs` scheme for SSL
@ -168,9 +168,9 @@ can be written any number of times, whereas block blobs can only be appended to
work for HBase logs, so page blob support was introduced to overcome this work for HBase logs, so page blob support was introduced to overcome this
limitation. limitation.
Page blobs can be used for other purposes beyond just HBase log files though.
Page blobs can be up to 1TB in size, larger than the maximum 200GB size for block Page blobs can be up to 1TB in size, larger than the maximum 200GB size for block
blobs. blobs.
You should stick to block blobs for most usage, and page blobs are only tested in context of HBase write-ahead logs.
In order to have the files you create be page blobs, you must set the In order to have the files you create be page blobs, you must set the
configuration variable `fs.azure.page.blob.dir` to a comma-separated list of configuration variable `fs.azure.page.blob.dir` to a comma-separated list of