big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,989 Commits 2 Branches 2 Tags 264 MiB
5db371f52f
Commit Graph

137 Commits

Author SHA1 Message Date
Jing Zhao
ea6c66ed57 HDFS-8316. Erasure coding: refactor EC constants to be consistent with HDFS-8249. Contributed by Zhe Zhang. 2015-05-26 11:59:57 -07:00
Zhe Zhang
77d94dd5ec HDFS-7949. WebImageViewer need support file size calculation with striped blocks. Contributed by Rakesh R. 2015-05-26 11:59:56 -07:00
Jing Zhao
5059958bf4 HDFS-8308. Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog. Contributed by Jing Zhao. 2015-05-26 11:59:56 -07:00
Zhe Zhang
1a31f1c303 HDFS-8183. Erasure Coding: Improve DFSStripedOutputStream closing of datastreamer threads. Contributed by Rakesh R. 2015-05-26 11:59:56 -07:00
Zhe Zhang
f0628280c3 HDFS-8282. Erasure coding: move striped reading logic to StripedBlockUtil. Contributed by Zhe Zhang. 2015-05-26 11:59:56 -07:00
Zhe Zhang
b00c663877 HDFS-8272. Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read). Contributed by Jing Zhao 2015-05-26 11:59:55 -07:00
Jing Zhao
9593776e34 HDFS-8235. Erasure Coding: Create DFSStripedInputStream in DFSClient#open. Contributed by Kai Sasaki. 2015-05-26 11:59:55 -07:00
Vinayakumar B
5eb17e5726 HDFS-8189. ClientProtocol#createErasureCodingZone API was wrongly annotated as Idempotent (Contributed by Vinayakumar B) 2015-05-26 11:59:55 -07:00
Vinayakumar B
f9eb95c31d HDFS-8230. Erasure Coding: Ignore DatanodeProtocol#DNA_ERASURE_CODING_RECOVERY commands from standbynode if any (Contributed by Vinayakumar B) 2015-05-26 11:59:54 -07:00
Zhe Zhang
89d3378578 HDFS-8033. Erasure coding: stateful (non-positional) read from files in striped layout. Contributed by Zhe Zhang. 2015-05-26 11:59:54 -07:00
Zhe Zhang
f5d4a95ef5 HDFS-8228. Erasure Coding: SequentialBlockGroupIdGenerator#nextValue may cause block id conflicts. Contributed by Jing Zhao. 2015-05-26 11:59:54 -07:00
Jing Zhao
3f2c6938f1 HDFS-8223. Should calculate checksum for parity blocks in DFSStripedOutputStream. Contributed by Yi Liu. 2015-05-26 11:59:54 -07:00
Jing Zhao
eb612b0b70 HDFS-8233. Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last stripe is at the block group boundary. Contributed by Jing Zhao. 2015-05-26 11:59:53 -07:00
Kai Zheng
fcd54ecce2 HDFS-8136. Client gets and uses EC schema when reads and writes a stripping file. Contributed by Kai Sasaki 2015-05-26 11:59:53 -07:00
Zhe Zhang
e8df2581c3 HDFS-8156. Add/implement necessary APIs even we just have the system default schema. Contributed by Kai Zheng. 2015-05-26 11:59:53 -07:00
Uma Maheswara Rao G
014d8675c5 HDFS-8024. Erasure Coding: ECworker frame, basics, bootstraping and configuration. (Contributed by Uma Maheswara Rao G) 2015-05-26 11:59:52 -07:00
Zhe Zhang
2f11109bb0 HDFS-8212. DistributedFileSystem.createErasureCodingZone should pass schema in FileSystemLinkResolver. Contributed by Tsz Wo Nicholas Sze. 2015-05-26 11:59:52 -07:00
Zhe Zhang
e9d85bbf30 HDFS-8216. TestDFSStripedOutputStream should use BlockReaderTestUtil to create BlockReader. Contributed by Tsz Wo Nicholas Sze. 2015-05-26 11:59:52 -07:00
Tsz-Wo Nicholas Sze
e107886d6f HDFS-8190. StripedBlockUtil.getInternalBlockLength may have overflow error. 2015-05-26 11:59:52 -07:00
Vinayakumar B
dfba46ab57 HDFS-8181. createErasureCodingZone sets retryCache state as false always (Contributed by Uma Maheswara Rao G) 2015-05-26 11:59:51 -07:00
Vinayakumar B
a1c9425265 HDFS-8146. Protobuf changes for BlockECRecoveryCommand and its fields for making it ready for transfer to DN (Contributed by Uma Maheswara Rao G) 2015-05-26 11:59:51 -07:00
Zhe Zhang
64f8f0a145 HDFS-8167. BlockManager.addBlockCollectionWithCheck should check if the block is a striped block. Contributed by Hui Zheng. 2015-05-26 11:59:12 -07:00
Tsz-Wo Nicholas Sze
4c039b0876 HDFS-7994. Detect if resevered EC Block ID is already used during namenode startup. Contributed by Hui Zheng 2015-05-26 11:59:12 -07:00
Vinayakumar B
ceb3d1c170 HDFS-7349. Support DFS command for the EC encoding (Contributed by Vinayakumar B) 2015-05-26 11:59:11 -07:00
Vinayakumar B
2c277802c1 HDFS-8027. Erasure Coding: Update CHANGES-HDFS-7285.txt with branch commits (Vinayakumar B) 2015-05-26 11:59:11 -07:00
Vinayakumar B
04c23d95d0 HDFS-8123. Erasure Coding: Better to move EC related proto messages to a separate erasurecoding proto file (Contrubuted by Rakesh R) 2015-05-26 11:59:11 -07:00
Vinayakumar B
f53e402635 HDFS-8090. Erasure Coding: Add RPC to client-namenode to list all ECSchemas loaded in Namenode. (Contributed by Vinayakumar B) 2015-05-26 11:58:40 -07:00
Kai Zheng
4970f2a2ab HDFS-7889 Subclass DFSOutputStream to support writing striping layout files. Contributed by Li Bo 2015-05-26 11:58:40 -07:00
Kai Zheng
c4a0d88366 HDFS-8104 Make hard-coded values consistent with the system default schema first before remove them. Contributed by Kai Zheng 2015-05-26 11:55:17 -07:00
Kai Zheng
1a37b0fe9a HDFS-8074 Define a system-wide default EC schema. Contributed by Kai Zheng 2015-05-26 11:55:16 -07:00
Vinayakumar B
16484f0030 HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B) 2015-05-26 11:55:16 -07:00
Vinayakumar B
aac73c21c3 HADOOP-11740. Combine erasure encoder and decoder interfaces. Contributed by Zhe Zhang. 
Updated CHANGES-HDFS-EC-7285.txt
 2015-05-26 11:55:15 -07:00
Vinayakumar B
d25ca09cf6 HADOOP-11782 Correct two thrown messages in ECSchema class. Contributed by Xinwei Qin 
Updated CHANGES-HDFS-EC-7285.txt
 2015-05-26 11:55:14 -07:00
Zhe Zhang
e54a74b566 HADOOP-11740. Combine erasure encoder and decoder interfaces. Contributed by Zhe Zhang. 2015-05-26 11:55:13 -07:00
Vinayakumar B
af8eaacdb4 HDFS-8027. Erasure Coding: Update CHANGES-HDFS-7285.txt with branch commits (Vinayakumar B) 2015-05-26 11:44:46 -07:00
Tsz-Wo Nicholas Sze
9d1175b8fb HDFS-7716. Add a test for BlockGroup support in FSImage. Contributed by Takuya Fukudome 2015-05-26 11:43:43 -07:00
Vinayakumar B
f166e67a23 HDFS-7347. Configurable erasure coding policy for individual files and directories ( Contributed by Zhe Zhang ) 2015-05-26 11:03:34 -07:00