diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Ozonemetrics.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Ozonemetrics.md new file mode 100644 index 0000000000..4822607504 --- /dev/null +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Ozonemetrics.md @@ -0,0 +1,89 @@ + + + + +HDFS Ozone Metrics +=============== + + + +Overview +-------- + +The container metrics that is used in HDFS Ozone. + +### Storage Container Metrics + +The metrics for various storage container operations in HDFS Ozone. + +Storage container is an optional service that can be enabled by setting +'ozone.enabled' to true. +These metrics are only available when ozone is enabled. + +Storage Container Metrics maintains a set of generic metrics for all +container RPC calls that can be made to a datandoe/container. + +Along with the total number of RPC calls containers maintain a set of metrics +for each RPC call. Following is the set of counters maintained for each RPC +operation. + +*Total number of operation* - We maintain an array which counts how +many times a specific operation has been performed. +Eg.`NumCreateContainer` tells us how many times create container has been +invoked on this datanode. + +*Number of bytes involved in a specific command* - This is an array that is +maintained for all operations, but makes sense only for read and write +operations. + +While it is possible to read the bytes in update container, it really makes +no sense, since no data stream involved. Users are advised to use this +metric only when it makes sense. Eg. `BytesReadChunk` -- Tells us how +many bytes have been read from this data using Read Chunk operation. + +*Average Latency of each operation* - The average latency of the operation. +Eg. `LatencyCreateContainerAvgTime` - This tells us the average latency of +Create Container. + +*Quantiles for each of these operations* - The 50/75/90/95/99th percentile +of these operations. Eg. `CreateContainerNanos60s50thPercentileLatency` -- +gives latency of the create container operations at the 50th percentile latency +(1 minute granularity). We report 50th, 75th, 90th, 95th and 99th percentile +for all RPCs. + +So this leads to the containers reporting these counters for each of these +RPC operations. + +| Name | Description | +|:---- |:---- | +| `NumOps` | Total number of container operations | +| `CreateContainer` | Create container operation | +| `ReadContainer` | Read container operation | +| `UpdateContainer` | Update container operations | +| `DeleteContainer` | Delete container operations | +| `ListContainer` | List container operations | +| `PutKey` | Put key operations | +| `GetKey` | Get key operations | +| `DeleteKey` | Delete key operations | +| `ListKey` | List key operations | +| `ReadChunk` | Read chunk operations | +| `DeleteChunk` | Delete chunk operations | +| `WriteChunk` | Write chunk operations| +| `ListChunk` | List chunk operations | +| `CompactChunk` | Compact chunk operations | +| `PutSmallFile` | Put small file operations | +| `GetSmallFile` | Get small file operations | +| `CloseContainer` | Close container operations | + diff --git a/hadoop-project/src/site/site.xml b/hadoop-project/src/site/site.xml index eda7183d34..1195862adc 100644 --- a/hadoop-project/src/site/site.xml +++ b/hadoop-project/src/site/site.xml @@ -203,6 +203,7 @@ +