big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-8719. Erasure Coding: client generates too many small packets when writing parity data. Contributed by Li Bo.

Browse Source
This commit is contained in:
Walter Su 2015-07-09 16:18:05 +08:00
parent 42a1d0dbde
commit 48f3830f21
3 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
View File

@ -332,3 +332,6 @@
HDFS-8461. Erasure coding: fix priority level of UnderReplicatedBlocks for
striped block. (Walter Su via jing9)
HDFS-8719. Erasure Coding: client generates too many small packets when
writing parity data. (Li Bo via waltersu4549)

2
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
View File

@ -459,7 +459,7 @@ void setCurrentPacket2Empty() throws InterruptedIOException {
* write filled up its partial chunk. Tell the summer to generate full
* crc chunks from now on.
*/
private void adjustChunkBoundary() {
protected void adjustChunkBoundary() {
if (streamer.getAppendChunk() &&
streamer.getBytesCurBlock() % bytesPerChecksum == 0) {
streamer.setAppendChunk(false);

1
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
View File

@ -318,6 +318,7 @@ private synchronized StripedDataStreamer getCurrentStreamer() {
private synchronized StripedDataStreamer setCurrentStreamer(int i) {
streamer = streamers.get(i);
adjustChunkBoundary();
return getCurrentStreamer();
}