Zhe Zhang
|
b00c663877
|
HDFS-8272. Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read). Contributed by Jing Zhao
|
2015-05-26 11:59:55 -07:00 |
|
Jing Zhao
|
1a8139e6ad
|
Fix merge conflicts.
|
2015-05-26 11:59:55 -07:00 |
|
Jing Zhao
|
9593776e34
|
HDFS-8235. Erasure Coding: Create DFSStripedInputStream in DFSClient#open. Contributed by Kai Sasaki.
|
2015-05-26 11:59:55 -07:00 |
|
Vinayakumar B
|
5eb17e5726
|
HDFS-8189. ClientProtocol#createErasureCodingZone API was wrongly annotated as Idempotent (Contributed by Vinayakumar B)
|
2015-05-26 11:59:55 -07:00 |
|
Vinayakumar B
|
f9eb95c31d
|
HDFS-8230. Erasure Coding: Ignore DatanodeProtocol#DNA_ERASURE_CODING_RECOVERY commands from standbynode if any (Contributed by Vinayakumar B)
|
2015-05-26 11:59:54 -07:00 |
|
Zhe Zhang
|
89d3378578
|
HDFS-8033. Erasure coding: stateful (non-positional) read from files in striped layout. Contributed by Zhe Zhang.
|
2015-05-26 11:59:54 -07:00 |
|
Zhe Zhang
|
f5d4a95ef5
|
HDFS-8228. Erasure Coding: SequentialBlockGroupIdGenerator#nextValue may cause block id conflicts. Contributed by Jing Zhao.
|
2015-05-26 11:59:54 -07:00 |
|
Jing Zhao
|
3f2c6938f1
|
HDFS-8223. Should calculate checksum for parity blocks in DFSStripedOutputStream. Contributed by Yi Liu.
|
2015-05-26 11:59:54 -07:00 |
|
Jing Zhao
|
eb612b0b70
|
HDFS-8233. Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last stripe is at the block group boundary. Contributed by Jing Zhao.
|
2015-05-26 11:59:53 -07:00 |
|
Kai Zheng
|
fcd54ecce2
|
HDFS-8136. Client gets and uses EC schema when reads and writes a stripping file. Contributed by Kai Sasaki
|
2015-05-26 11:59:53 -07:00 |
|
Zhe Zhang
|
e8df2581c3
|
HDFS-8156. Add/implement necessary APIs even we just have the system default schema. Contributed by Kai Zheng.
|
2015-05-26 11:59:53 -07:00 |
|
Uma Maheswara Rao G
|
014d8675c5
|
HDFS-8024. Erasure Coding: ECworker frame, basics, bootstraping and configuration. (Contributed by Uma Maheswara Rao G)
|
2015-05-26 11:59:52 -07:00 |
|
Zhe Zhang
|
2f11109bb0
|
HDFS-8212. DistributedFileSystem.createErasureCodingZone should pass schema in FileSystemLinkResolver. Contributed by Tsz Wo Nicholas Sze.
|
2015-05-26 11:59:52 -07:00 |
|
Tsz-Wo Nicholas Sze
|
e107886d6f
|
HDFS-8190. StripedBlockUtil.getInternalBlockLength may have overflow error.
|
2015-05-26 11:59:52 -07:00 |
|
Zhe Zhang
|
922631f04f
|
HDFS-8188. Erasure coding: refactor client-related code to sync with HDFS-8082 and HDFS-8169. Contributed by Zhe Zhang.
|
2015-05-26 11:59:51 -07:00 |
|
Vinayakumar B
|
dfba46ab57
|
HDFS-8181. createErasureCodingZone sets retryCache state as false always (Contributed by Uma Maheswara Rao G)
|
2015-05-26 11:59:51 -07:00 |
|
Vinayakumar B
|
a1c9425265
|
HDFS-8146. Protobuf changes for BlockECRecoveryCommand and its fields for making it ready for transfer to DN (Contributed by Uma Maheswara Rao G)
|
2015-05-26 11:59:51 -07:00 |
|
Jing Zhao
|
f6e1160ef1
|
HDFS-8145. Fix the editlog corruption exposed by failed TestAddStripedBlocks. Contributed by Jing Zhao.
|
2015-05-26 11:59:46 -07:00 |
|
Jing Zhao
|
ab76e1fe36
|
HDFS-7937. Erasure Coding: INodeFile quota computation unit tests. Contributed by Kai Sasaki.
|
2015-05-26 11:59:13 -07:00 |
|
Jing Zhao
|
cd458c38a0
|
HDFS-8166. DFSStripedOutputStream should not create empty blocks. Contributed by Jing Zhao.
|
2015-05-26 11:59:13 -07:00 |
|
Zhe Zhang
|
64f8f0a145
|
HDFS-8167. BlockManager.addBlockCollectionWithCheck should check if the block is a striped block. Contributed by Hui Zheng.
|
2015-05-26 11:59:12 -07:00 |
|
Tsz-Wo Nicholas Sze
|
4c039b0876
|
HDFS-7994. Detect if resevered EC Block ID is already used during namenode startup. Contributed by Hui Zheng
|
2015-05-26 11:59:12 -07:00 |
|
Jing Zhao
|
5e8837dd6c
|
HDFS-8120. Erasure coding: created util class to analyze striped block groups. Contributed by Zhe Zhang and Li Bo.
|
2015-05-26 11:59:12 -07:00 |
|
Vinayakumar B
|
ceb3d1c170
|
HDFS-7349. Support DFS command for the EC encoding (Contributed by Vinayakumar B)
|
2015-05-26 11:59:11 -07:00 |
|
Vinayakumar B
|
04c23d95d0
|
HDFS-8123. Erasure Coding: Better to move EC related proto messages to a separate erasurecoding proto file (Contrubuted by Rakesh R)
|
2015-05-26 11:59:11 -07:00 |
|
Zhe Zhang
|
3b56a5aa9e
|
HDFS-8114. Erasure coding: Add auditlog FSNamesystem#createErasureCodingZone if this operation fails. Contributed by Rakesh R.
|
2015-05-26 11:59:10 -07:00 |
|
Zhe Zhang
|
a32c4dc38a
|
HDFS-8122. Erasure Coding: Support specifying ECSchema during creation of ECZone. Contributed by Vinayakumar B.
|
2015-05-26 11:59:06 -07:00 |
|
Zhe Zhang
|
35797b0889
|
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit mainly addresses HDFS-8081 and HDFS-8048. Contributed by Zhe Zhang.
|
2015-05-26 11:58:40 -07:00 |
|
Vinayakumar B
|
f53e402635
|
HDFS-8090. Erasure Coding: Add RPC to client-namenode to list all ECSchemas loaded in Namenode. (Contributed by Vinayakumar B)
|
2015-05-26 11:58:40 -07:00 |
|
Kai Zheng
|
4970f2a2ab
|
HDFS-7889 Subclass DFSOutputStream to support writing striping layout files. Contributed by Li Bo
|
2015-05-26 11:58:40 -07:00 |
|
Jing Zhao
|
4d0bc724f2
|
HDFS-8077. Erasure coding: fix bugs in EC zone and symlinks. Contributed by Jing Zhao and Zhe Zhang.
|
2015-05-26 11:58:32 -07:00 |
|
Kai Zheng
|
1a37b0fe9a
|
HDFS-8074 Define a system-wide default EC schema. Contributed by Kai Zheng
|
2015-05-26 11:55:16 -07:00 |
|
Vinayakumar B
|
2ca3e38feb
|
HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B)
Added missed file
|
2015-05-26 11:55:16 -07:00 |
|
Vinayakumar B
|
16484f0030
|
HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B)
|
2015-05-26 11:55:16 -07:00 |
|
Zhe Zhang
|
bff59392ee
|
HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao
|
2015-05-26 11:55:15 -07:00 |
|
Zhe Zhang
|
91c741a2a1
|
HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao
|
2015-05-26 11:55:15 -07:00 |
|
Zhe Zhang
|
146ce7a978
|
HDFS-7969. Erasure coding: NameNode support for lease recovery of striped block groups. Contributed by Zhe Zhang.
|
2015-05-26 11:55:14 -07:00 |
|
Zhe Zhang
|
c243319eab
|
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit is for HDFS-8035). Contributed by Zhe Zhang
|
2015-05-26 11:55:13 -07:00 |
|
Zhe Zhang
|
1af8c14862
|
HDFS-7839. Erasure coding: implement facilities in NameNode to create and manage EC zones. Contributed by Zhe Zhang
|
2015-05-26 11:55:07 -07:00 |
|
Zhe Zhang
|
578019d6a2
|
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging (this commit is for conflicts from HDFS-6945). Contributed by Zhe Zhang.
|
2015-05-26 11:44:47 -07:00 |
|
Jing Zhao
|
cd655ee817
|
HDFS-8005. Erasure Coding: simplify striped block recovery work computation and add tests. Contributed by Jing Zhao.
|
2015-05-26 11:44:42 -07:00 |
|
Jing Zhao
|
abf833a7b2
|
HDFS-7907. Erasure Coding: track invalid, corrupt, and under-recovery striped blocks in NameNode. Contributed by Jing Zhao.
|
2015-05-26 11:43:44 -07:00 |
|
Zhe Zhang
|
ea2e60fbcc
|
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903, HDFS-7435, HDFS-7930, HDFS-7960 (this commit is for HDFS-7960)
|
2015-05-26 11:43:42 -07:00 |
|
Jing Zhao
|
d0d75a8339
|
HDFS-7827. Erasure Coding: support striped blocks in non-protobuf fsimage. Contributed by Hui Zheng.
|
2015-05-26 11:43:42 -07:00 |
|
Jing Zhao
|
46dac3595f
|
HDFS-7864. Erasure Coding: Update safemode calculation for striped blocks. Contributed by GAO Rui.
|
2015-05-26 11:43:42 -07:00 |
|
Zhe Zhang
|
544f75d651
|
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903, HDFS-7435 and HDFS-7930 (this commit is for HDFS-7930 only)
|
2015-05-26 11:43:42 -07:00 |
|
Zhe Zhang
|
57a84c0d14
|
HDFS-7369. Erasure coding: distribute recovery work for striped blocks to DataNode. Contributed by Zhe Zhang.
|
2015-05-26 11:43:37 -07:00 |
|
Jing Zhao
|
a38a37c634
|
HDFS-7912. Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks. Contributed by Jing Zhao.
|
2015-05-26 11:41:59 -07:00 |
|
Jing Zhao
|
26773d9d6c
|
HDFS-7826. Erasure Coding: Update INodeFile quota computation for striped blocks. Contributed by Kai Sasaki.
|
2015-05-26 11:32:26 -07:00 |
|
Jing Zhao
|
11585883a9
|
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903 and HDFS-7435. Contributed by Zhe Zhang.
|
2015-05-26 11:32:26 -07:00 |
|
Jing Zhao
|
f05c21285e
|
HDFS-7853. Erasure coding: extend LocatedBlocks to support reading from striped files. Contributed by Jing Zhao.
|
2015-05-26 11:32:25 -07:00 |
|
Jing Zhao
|
39a0a85fb7
|
HDFS-7872. Erasure Coding: INodeFile.dumpTreeRecursively() supports to print striped blocks. Contributed by Takuya Fukudome.
|
2015-05-26 11:32:25 -07:00 |
|
Jing Zhao
|
1e1e930407
|
HDFS-7837. Erasure Coding: allocate and persist striped blocks in NameNode. Contributed by Jing Zhao.
|
2015-05-26 11:32:20 -07:00 |
|
Zhe Zhang
|
9af1f4779b
|
Addendum fix for HDFS-7749 to be compatible with HDFS-7993
|
2015-05-26 11:07:16 -07:00 |
|
Jing Zhao
|
9f2f583f40
|
HDFS-7749. Erasure Coding: Add striped block support in INodeFile. Contributed by Jing Zhao.
|
2015-05-26 11:07:15 -07:00 |
|
Jing Zhao
|
ba93714920
|
HDFS-7716. Erasure Coding: extend BlockInfo to handle EC info. Contributed by Jing Zhao.
|
2015-05-26 11:07:11 -07:00 |
|
Zhe Zhang
|
0c1da5a030
|
HDFS-7652. Process block reports for erasure coded blocks. Contributed by Zhe Zhang
|
2015-05-26 11:03:34 -07:00 |
|
Zhe Zhang
|
bc2833b1c9
|
HDFS-7339. Allocating and persisting block groups in NameNode. Contributed by Zhe Zhang
|
2015-05-26 11:03:34 -07:00 |
|
Haohui Mai
|
ada233b7cd
|
HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang.
|
2015-05-24 22:30:32 -07:00 |
|
Vinayakumar B
|
0c6638c2ea
|
HDFS-8268. Port conflict log for data node server is not sufficient (Contributed by Mohammad Shahid Khan)
|
2015-05-22 16:15:15 +05:30 |
|
Xiaoyu Yao
|
05e04f34f2
|
HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran.
|
2015-05-21 11:58:00 -07:00 |
|
Haohui Mai
|
2b6bcfdafa
|
HDFS-8421. Move startFile() and related functions into FSDirWriteFileOp. Contributed by Haohui Mai.
|
2015-05-21 08:08:28 -07:00 |
|
Akira Ajisaka
|
dc8434ab2b
|
HDFS-4383. Document the lease limits. Contributed by Arshad Mohammad.
|
2015-05-21 17:30:43 +09:00 |
|
Kihwal Lee
|
8860e352c3
|
HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts.
|
2015-05-19 13:05:15 -05:00 |
|
Colin Patrick Mccabe
|
470c87dbc6
|
HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe)
|
2015-05-19 10:50:15 -07:00 |
|
Kihwal Lee
|
de30d66b26
|
HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui.
|
2015-05-19 08:04:38 -05:00 |
|
Vinayakumar B
|
93972a332a
|
HDFS-6348. SecondaryNameNode not terminating properly on runtime exceptions (Contributed by Rakesh R)
|
2015-05-19 12:24:25 +05:30 |
|
Ravi Prakash
|
cdfae446ad
|
HDFS-4185. Add a metric for number of active leases (Rakesh R via raviprak)
|
2015-05-18 12:37:21 -07:00 |
|
Tsz-Wo Nicholas Sze
|
0c590e1c09
|
HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma
|
2015-05-19 02:57:54 +08:00 |
|
Arpit Agarwal
|
a2190bf15d
|
HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal)
|
2015-05-18 11:36:29 -07:00 |
|
Arpit Agarwal
|
e453989a57
|
HDFS-8157. Writes to RAM DISK reserve locked memory for block files. (Arpit Agarwal)
|
2015-05-16 09:05:35 -07:00 |
|
Haohui Mai
|
ac742c762d
|
HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal.
|
2015-05-15 19:12:40 -07:00 |
|
Haohui Mai
|
e5afac5896
|
HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai.
|
2015-05-15 19:09:59 -07:00 |
|
Jing Zhao
|
8f37873342
|
HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze.
|
2015-05-15 16:14:54 -07:00 |
|
Vinayakumar B
|
3bef7c80a9
|
HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He)
|
2015-05-15 11:07:51 +05:30 |
|
Vinayakumar B
|
def9136e02
|
HDFS-8150. Make getFileChecksum fail for blocks under construction (Contributed by J.Andreina)
|
2015-05-14 15:54:51 +05:30 |
|
Haohui Mai
|
b2c85db86c
|
HDFS-7728. Avoid updating quota usage while loading edits. Contributed by Jing Zhao.
|
2015-05-13 21:50:35 -07:00 |
|
Colin Patrick Mccabe
|
281d47a969
|
HDFS-8380. Always call addStoredBlock on blocks which have been shifted from one storage to another (cmccabe)
|
2015-05-13 14:29:05 -07:00 |
|
Tsz-Wo Nicholas Sze
|
cdec12d1b8
|
HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore
|
2015-05-13 11:57:49 -07:00 |
|
Vinayakumar B
|
065d8f2a34
|
HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R)
|
2015-05-13 17:27:34 +05:30 |
|
Akira Ajisaka
|
2463666ecb
|
HDFS-6184. Capture NN's thread dump when it fails over. Contributed by Ming Ma.
|
2015-05-13 11:37:22 +09:00 |
|
yliu
|
6d5da94841
|
HDFS-8255. Rename getBlockReplication to getPreferredBlockReplication. (Contributed by Zhe Zhang)
|
2015-05-12 21:29:22 +08:00 |
|
Kihwal Lee
|
ea11590aad
|
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah.
|
2015-05-11 14:31:58 -05:00 |
|
Akira Ajisaka
|
1dd79ffaca
|
HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula.
|
2015-05-12 00:18:18 +09:00 |
|
Junping Du
|
7e543c27fa
|
YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak.
|
2015-05-11 05:27:07 -07:00 |
|
Haohui Mai
|
4536399d47
|
HDFS-8357. Consolidate parameters of INode.CleanSubtree() into a parameter objects. Contributed by Li Lu.
|
2015-05-09 22:51:58 -07:00 |
|
Haohui Mai
|
02a4a22b9c
|
HDFS-8327. Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate(). Contributed by Haohui Mai.
|
2015-05-08 23:09:25 -07:00 |
|
Haohui Mai
|
00fe1ed3a4
|
HDFS-6757. Simplify lease manager with INodeID. Contributed by Haohui Mai.
|
2015-05-08 23:04:31 -07:00 |
|
Kihwal Lee
|
2d4ae3d18b
|
HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah.
|
2015-05-08 16:36:41 -05:00 |
|
Colin Patrick Mccabe
|
f523e963e4
|
HDFS-8113. Add check for null BlockCollection pointers in BlockInfoContiguous structures (Chengbing Liu via Colin P. McCabe)
|
2015-05-08 13:50:51 -07:00 |
|
Kihwal Lee
|
7a7960be41
|
HDFS-7433. Optimize performance of DatanodeManager's node map. Contributed by Daryn Sharp.
|
2015-05-08 15:18:29 -05:00 |
|
Yongjun Zhang
|
730f9930a4
|
HDFS-8311. DataStreamer.transfer() should timeout the socket InputStream. (Esteban Gutierrez via Yongjun Zhang)
|
2015-05-08 00:11:46 -07:00 |
|
cnauroth
|
26f61d41df
|
HDFS-5640. Add snapshot methods to FileContext. Contributed by Rakesh R.
|
2015-05-08 11:20:55 -07:00 |
|
Kihwal Lee
|
6f622672b6
|
HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula.
|
2015-05-08 09:30:38 -05:00 |
|
Uma Maheswara Rao G
|
7b1ea9c481
|
HDFS-6285. tidy an error log inside BlockReceiver. Contributed by Liang Xie.
|
2015-05-08 17:22:52 +05:30 |
|
Vinayakumar B
|
c648317a68
|
HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G)
|
2015-05-08 17:18:14 +05:30 |
|
Tsuyoshi Ozawa
|
888e630990
|
HDFS-8207. Improper log message when blockreport interval compared with initial delay. Contributed by Brahma Reddy Battula and Ashish Singhi.
|
2015-05-08 18:13:09 +09:00 |
|
Uma Maheswara Rao G
|
4b00ae5c11
|
HDFS-8108. Fsck should provide the info on mandatory option to be used along with -blocks ,-locations and -racks. Contributed by J.Andreina.
|
2015-05-08 14:00:45 +05:30 |
|
Uma Maheswara Rao G
|
e16f4b7f70
|
HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R.
|
2015-05-08 12:26:47 +05:30 |
|
Vinayakumar B
|
e5e492a963
|
HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun() ( Contributed by Sanghyun Yun)
|
2015-05-08 11:25:24 +05:30 |
|