big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
19,342 Commits 2 Branches 2 Tags 264 MiB
32f867a6a9
Commit Graph

4392 Commits

Author SHA1 Message Date
Inigo Goiri
2d6195c0af HDFS-13548. TestResolveHdfsSymlink#testFcResolveAfs fails on Windows. Contributed by Anbang Hu. 2018-05-15 11:20:32 -07:00
Inigo Goiri
eaca7980ad HDFS-11700. TestHDFSServerPorts#testBackupNodePorts doesn't pass on Windows. Contributed by Anbang Hu. 2018-05-15 10:27:36 -07:00
Inigo Goiri
92ebd466c7 HDFS-13551. TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster. Contributed by Anbang Hu. 2018-05-15 10:21:42 -07:00
Xiao Chen
960940e0e0 HDFS-13539. DFSStripedInputStream NPE when reportCheckSumFailure. 2018-05-14 09:28:39 -07:00
Inigo Goiri
d50c4d71dc HDFS-13542. TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows. Contributed by Anbang Hu. 2018-05-11 09:47:57 -07:00
Xiao Chen
696a4be0da HDFS-12981. renameSnapshot a Non-Existent snapshot to itself should throw error. Contributed by Kitti Nanasi. 2018-05-07 15:36:29 -07:00
Xiao Chen
118bd75805 Revert "HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445." 
This reverts commit 6503593711.
 2018-05-07 13:32:27 -07:00
Inigo Goiri
9e2cfb2d3f HDFS-13503. Fix TestFsck test failures on Windows. Contributed by Xiao Liang. 2018-05-01 08:12:46 -07:00
Inigo Goiri
fc074a359c HDFS-13283. Percentage based Reserved Space Calculation for DataNode. Contributed by Lukas Majercak. 2018-04-30 13:28:33 -07:00
Inigo Goiri
eb7fe1d588 HDFS-13509. Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows. Contributed by Xiao Liang. 2018-04-28 09:05:30 -07:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Lei Xu
a8e428b2dc HDFS-13468. Add erasure coding metrics into ReadStatistics. (Contributed by Lei (Eddy) Xu) 2018-04-26 13:54:40 -07:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Anu Engineer
3d18ca4926 HDFS-13459. Ozone: Clean-up of ozone related change from MiniDFSCluste. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
979bbb4019 HDFS-13423. Ozone: Clean-up of ozone related change from hadoop-hdfs-project. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Nanda kumar
9a914126a7 HDFS-13008. Ozone: Add DN container open/close state to container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Nanda kumar
1094af072c HDFS-11699. Ozone:SCM: Add support for close containers in SCM. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
b4a3cf1476 HDFS-13070. Ozone: SCM: Support for container replica reconciliation - 1. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
4c10a849e8 HDFS-13078. Ozone: Update Ratis on Ozone to 0.1.1-alpha-8fd74ed-SNAPSHOT. 
To fix large chunk reads (>4M) from Datanodes. Contributed by Mukul Kumar Singh.
 2018-04-26 05:36:04 -07:00
Anu Engineer
136b419594 HDFS-13149. Ozone: Rename Corona to Freon. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
9272e1021d HDFS-12636. Ozone: OzoneFileSystem: Implement seek functionality for rpc client. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
377b31ffa1 HDFS-13022. Block Storage: Kubernetes dynamic persistent volume provisioner. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
ee5495456e HDFS-13116. Ozone: Refactor Pipeline to have transport and container specific information. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
3cf07b43bc HDFS-13127. Fix TestContainerStateManager and TestOzoneConfigurationFields. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
443425a5d9 HDFS-12522. Ozone: Remove the Priority Queues used in the Container State Manager. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
6b03fca37d HDFS-12868. Ozone: Service Discovery API. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
4bb9ad8e8f HDFS-13017. Block Storage: implement simple iscsi discovery in jscsi server. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
4a051ba494 HDFS-12940. Ozone: KSM: TestKeySpaceManager#testExpiredOpenKey fails occasionally. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
916f31f7d1 HDFS-13026. Ozone: TestContainerPersistence is failing becaue of container data mismatch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
c4b88454a7 HDFS-12843. Ozone: Client: TestOzoneRpcClient#testPutKeyRatisThreeNodes is failing. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
98d62e55c4 Revert "HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee." 
This reverts commit 6ce5ec676164b84a9e2f8dc65b5f2199a141506d.
 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
98eeabbdb3 HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
0c48cb2cf2 HFDS-13007. Ozone: Fix TestKeySpaceManager#testKSMInitializationFailure. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
98973cd0c4 HDFS-12966. Ozone: owner name should be set properly when the container allocation happens. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5e31b920f0 HDFS-12871. Ozone: Service Discovery: Adding REST server details in ServiceList. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
3965f1ec99 HDFS-12853. Ozone: Optimize chunk writes for Ratis by avoiding double writes. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Anu Engineer
5766792cad HDFS-12870. Ozone: Service Discovery: REST endpoint in KSM for getServiceList. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
5f10c2d8ce HDFS-12980. Ozone: SCM: Restructuring container state transition and event. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
ce19f09bd0 HDFS-12908. Ozone: write chunk call fails because of Metrics registry exception. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
4b9f66a19e HDFS-12869. Ozone: Service Discovery: RPC endpoint in KSM for getServiceList. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
4dae68eebf HDFS-12741. Ozone: KSM: ADD support for KSM --createObjectStore command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Chen Liang
caeeb78ca4 HDFS-12799. Ozone: SCM: Close containers: extend SCMCommandResponseProto with SCMCloseContainerCmdResponseProto. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
966853894f HDFS-12795. Ozone: SCM: Support for Container LifeCycleState PENDING_CLOSE and LifeCycleEvent FULL_CONTAINER. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
26e270b908 HDFS-12751. Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2f3068bc72 HDFS-12698. Ozone: Use time units in the Ozone configuration values. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
8ff98e2af3 HDFS-12000. Ozone: Container : Add key versioning support-1. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c0c87dea9b HDFS-12626. Ozone : delete open key entries that will no longer be closed. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Nanda kumar
11473b8204 HDFS-12898. Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Nanda kumar
7bba3943f0 HDFS-12876. Ozone: moving NodeType from OzoneConsts to Ozone.proto. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
538e17824d HDFS-12850. Ozone: TestContainerPersistence#testListContainer fails frequently due to timed out. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
8023ba50b5 HDFS-12807. Ozone: Expose RockDB stats via JMX for Ozone metadata stores. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
ea86945754 HDFS-12842. Ozone: SCM: During startup SCM should check if the cluster is INITIALIZED. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
f9c11d952c HDFS-12740. SCM should support a RPC to share the cluster Id with KSM and DataNodes. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c8d8270f72 HDFS-12787. Ozone: SCM: Aggregate the metrics from all the container reports. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
8e8ff85b16 HDFS-12830. Ozone: TestOzoneRpcClient#testPutKeyRatisThreeNodes fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
4460ac9c57 HDFS-12756. Ozone: Add datanodeID to heartbeat responses and container protocol. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
446e84357d HDFS-12796. SCM should not start if Cluster Version file does not exist. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
740a06cdd7 HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
e395244e95 HDFS-12758. Ozone: Correcting assertEquals argument order in test cases. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
cec96b296f HDFS-12549. Ozone: OzoneClient: Support for REST protocol. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
9734f505ea HDFS-12739. Add Support for SCM --init command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
709d56fdc9 HDFS-12474. Ozone: SCM: Handling container report with key count and container usage. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2ec24a7542 HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Yiqun Lin
132f30c8ea HDFS-12443. Ozone: Improve SCM block deletion throttling algorithm. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
7ebe79e879 HDFS-12720. Ozone: Ratis options are not passed from KSM Client protobuf helper correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0559265822 HDFS-12750. Ozone: Fix TestStorageContainerManager#testBlockDeletionTransactions. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
76d34bca62 HDFS-12521. Ozone: SCM should read all Container info into memory when booting up. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
6291ca1e32 HDFS-12663. Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b56a73a4f HDFS-12680. Ozone: SCM: Lease support for container creation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6783dad766 HDFS-11468. Ozone: SCM: Add Node Metrics for SCM. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
393a02d8e1 HDFS-12692. Ozone: fix javadoc/unit test for calling MetadataStore.getRangeKVs with non existing key. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang
6d9f914b55 HDFS-12684. Ozone: SCMMXBean NodeCount is overlapping with NodeManagerMXBean. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Nanda kumar
cc86719451 HDFS-12675. Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
a467017068 HDFS-12411. Ozone: Add container usage information to DN container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Nanda kumar
9bb516118d HDFS-12490. Ozone: OzoneClient: Add creation/modification time information in OzoneVolume/OzoneBucket/OzoneKey. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nandakumar
8881309002 HDFS-12519. Ozone: Lease Manager framework. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
c85d3b2d25 HDFS-12593. Ozone: update Ratis to the latest snapshot. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
9445a9267f HDFS-12572. Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
yuanbo
e76e10fd7a HDFS-12609. Ozone: SCM: Refactoring of chill mode logic in NodeManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
e3b51d9074 HDFS-12598. Ozone: Fix 3 node ratis replication in Ozone. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nandakumar
3504af9411 HDFS-12537. Ozone: Reduce key creation overhead in Corona. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Nandakumar
9630621be7 HDFS-12610. Ozone: OzoneClient: RpcClient list calls throw NPE when iterating over empty list. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nandakumar
c3ef381011 HDFS-12543. Ozone : allow create key without specifying size. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c17521b1bd HDFS-12387. Ozone: Support Ratis as a first class replication mechanism. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer
0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
ceec14bff5 HDFS-12467. Ozone: SCM: NodeManager should log when it comes out of chill mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
6a754caca0 HDFS-12568. Ozone: Cleanup the ozone-default.xml. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
05246e2b32 HDFS-12060. Ozone: OzoneClient: Add list calls in RpcClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
aaec2689f4 HDFS-12511. Ozone: Add tags to config. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7363c8fc09 HDFS-12540. Ozone: node status text reported by SCM is a bit confusing. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang
9530153f33 HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
a44f3a777d HDFS-12501. Ozone: Cleanup javac issues. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6b10723ec3 HDFS-12525. Ozone: OzoneClient: Verify bucket/volume name in create calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
a1a3ba6529 HDFS-12454. Ozone : the sample ozone-site.xml in OzoneGettingStarted does not work. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2804435386 HDFS-12017. Ozone: Container: Move IPC port to 98xx range. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang
9ff136bb02 HDFS-12210. Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
dddded0016 Revert "Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh." to fix commit message. 
This reverts commit 087c69ba2434813044089f733581607d172efbd1.
 2018-04-26 05:36:04 -07:00
Weiwei Yang
d303b7f40f HDFS-12524. Ozone: Record number of keys scanned and hinted for getRangeKVs call. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Yiqun Lin
712bd70ac7 HDFS-12205. Ozone: List Key on an empty ozone bucket fails with command failed error. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00