big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,734 Commits 2 Branches 2 Tags 264 MiB
5e773efd78
Commit Graph

400 Commits

Author SHA1 Message Date
Shashikant Banerjee
671fd6524b HDDS-675. Add blocking buffer and use watchApi for flush/close in OzoneClient. Contributed by Shashikant Banerjee. 2018-11-13 23:39:14 +05:30
Jitendra Pandey
f944f33832 HDDS-709. Modify Close Container handling sequence on datanodes. Contributed by Shashikant Banerjee. 2018-11-12 14:08:39 -08:00
Nanda kumar
18fe65d756 HDDS-576. Move ContainerWithPipeline creation to RPC endpoint. 
Contributed by Nanda kumar.
 2018-11-12 23:33:11 +05:30
Tsz Wo Nicholas Sze
298d2502b0 HDDS-826. Update Ratis to 0.3.0-6f3419a-SNAPSHOT. 2018-11-09 18:35:40 -08:00
Nanda kumar
9fe50b4991 HDDS-733. Create container if not exist, as part of chunk write. 
Contributed by Lokesh Jain.
 2018-11-10 00:08:17 +05:30
Bharat Viswanadham
66bf62404a HDDS-812. TestEndPoint#testCheckVersionResponse is failing. Contributed by Hanisha Koneru. 2018-11-08 17:52:09 -08:00
Shashikant Banerjee
31614bcc7c HDDS-806. Update Ratis to latest snapshot version in ozone. Contributed by Tsz Wo Nicholas Sze and Mukul Kumar Singh. 2018-11-09 00:05:45 +05:30
Yiqun Lin
9317a61f3c HDDS-802. Container State Manager should get open pipelines for allocating container. Contributed by Lokesh Jain. 2018-11-08 23:41:43 +08:00
Nanda kumar
c80f753b0e HDDS-737. Introduce Incremental Container Report. 
Contributed by Nanda kumar.
 2018-11-08 18:33:38 +05:30
Yiqun Lin
addec29297 HDDS-809. Refactor SCMChillModeManager. 2018-11-07 13:54:08 +08:00
Akira Ajisaka
49412a1289
HDDS-811. [JDK 11] Fix compilation failures with jdk 11. Contributed by Lokesh Jain. 2018-11-06 21:26:43 +09:00
Shashikant Banerjee
50f40e0536 HDDS-794. addendum patch to fix compilation failure. Contributed by Shashikant Banerjee. 2018-11-06 00:20:57 +05:30
Arpit Agarwal
5ddefdd507 HDDS-794. Add configs to set StateMachineData write timeout in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-11-05 10:41:28 -08:00
Shashikant Banerjee
942693bddd HDDS-799. Avoid ByteString to byte array conversion cost by using ByteBuffer in Datanode. Contributed by Mukul Kumar Singh. 2018-11-05 23:43:22 +05:30
Arpit Agarwal
c8ca1747c0 HDDS-797. If DN is started before SCM, it does not register. Contributed by Hanisha Koneru. 2018-11-05 09:40:00 -08:00
Yiqun Lin
15df2e7a75 HDDS-796. Fix failed test TestStorageContainerManagerHttpServer#testHttpPolicy. 2018-11-05 17:31:06 +08:00
Yiqun Lin
d16d5f730e HDDS-751. Replace usage of Guava Optional with Java Optional. 2018-11-02 10:50:32 +08:00
Yiqun Lin
2e8ac14dcb HDDS-786. Fix the findbugs for SCMClientProtocolServer#getContainerWithPipeline. 2018-11-01 14:10:17 +08:00
Shashikant Banerjee
b13c56742a HDDS-697. update and validate the BCSID for PutSmallFile/GetSmallFile command. Contributed by Shashikant Banerjee. 2018-11-01 10:21:39 +05:30
Arpit Agarwal
08bb0362e0 HDDS-759. Create config settings for SCM and OM DB directories. Contributed by Arpit Agarwal. 2018-10-31 11:23:15 -07:00
Nanda kumar
e4f22b08e0 HDDS-755. ContainerInfo and ContainerReplica protobuf changes. 
Contributed by Nanda kumar.
 2018-10-31 10:29:35 +05:30
Anu Engineer
773f0d1519 HDDS-754. VolumeInfo#getScmUsed throws NPE. 
Contributed by Hanisha Koneru.
 2018-10-30 19:17:57 -07:00
Anu Engineer
e33b61f335 HDDS-762. Fix unit test failure for TestContainerSQLCli & TestSCMMetrics. 
Contributed by Mukul Kumar Singh.
 2018-10-30 19:16:52 -07:00
Shashikant Banerjee
7757331dbc HDDS-749. Restructure BlockId class in Ozone. Contributed by Shashikant Banerjee. 2018-10-30 14:15:27 +05:30
Arpit Agarwal
496f0ffe90 HDDS-620. ozone.scm.client.address should be an optional setting. Contributed by chencan and Arpit Agarwal. 2018-10-29 17:14:18 -07:00
Arpit Agarwal
a58048e8d5 HDDS-727. ozone.log is not getting created in logs directory. Contributed by Nilotpal Nandi. 2018-10-29 09:35:18 -07:00
Nanda kumar
902345de66 HDDS-728. Datanodes should use different ContainerStateMachine for each pipeline. 
Contributed by Mukul Kumar Singh.
 2018-10-29 19:53:52 +05:30
Nanda kumar
2fa01f823c HDDS-744. Fix ASF license warning in PipelineNotFoundException class. 
Contributed by Lokesh Jain.
 2018-10-27 16:51:08 +05:30
Nanda kumar
dce4ebe814 HDDS-694. Plugin new Pipeline management code in SCM. 
Contributed by Lokesh Jain.
 2018-10-26 17:55:04 +05:30
Hanisha Koneru
38a65e3b7c HDDS-682. Unified o3 address parsing for ozonen sh. Contributed by Elek, Marton. 2018-10-25 17:04:16 -07:00
Bharat Viswanadham
a4b9b7c130 HDDS-702:Used fixed/external version from hadoop jars in hdds/ozone projects. Contributed by Elek Marton. 2018-10-25 16:26:48 -07:00
Arpit Agarwal
244afaba4a HDDS-719. Remove Ozone dependencies on Apache Hadoop 3.2.0. Contributed by Arpit Agarwal. 2018-10-24 16:17:05 -07:00
Shashikant Banerjee
0891cdda79 HDDS-716. Update ozone to latest ratis snapshot build(0.3.0-aa38160-SNAPSHOT). Contributed by Mukul Kumar Singh. 2018-10-24 16:05:08 +05:30
Shashikant Banerjee
b61846392e HDDS-708. Validate BCSID while reading blocks from containers in datanodes. Contributed by Shashikant Banerjee. 2018-10-23 16:52:17 +05:30
Anu Engineer
e3cca12048 HDDS-676. Enable Read from open Containers via Standalone Protocol. 
Contributed by Shashikant Banerjee.
 2018-10-22 15:57:01 -07:00
Mukul Kumar Singh
82919a1e7a HDDS-638. Enable ratis snapshots for HDDS datanodes. Contributed by Mukul Kumar Singh. 2018-10-22 19:59:35 +05:30
Nanda kumar
c696419f3e Revert "HDDS-705. OS3Exception resource name should be the actual resource name." 
This reverts commit 977c6f6470.

There was spurious edit in this commit.
 2018-10-22 15:37:40 +05:30
Anu Engineer
7db3bb3ac1 HDDS-544. Unconditional wait findbug warning from ReplicationSupervisor. 
Contributed by Arpit Agarwal.
 2018-10-21 23:18:38 -07:00
Anu Engineer
c9077a9f5a HDDS-707. Allow registering MBeans without additional jmx properties. 
Contributed by Arpit Agarwal.
 2018-10-21 22:50:28 -07:00
Anu Engineer
977c6f6470 HDDS-705. OS3Exception resource name should be the actual resource name. 
Contributed by Bharat Viswanadham.
 2018-10-21 22:49:16 -07:00
Arpit Agarwal
dc27408043 HDDS-612. Even after setting hdds.scm.chillmode.enabled to false, SCM allocateblock fails with ChillModePrecheck exception. Contributed by Hanisha Koneru. 2018-10-19 11:36:28 -07:00
Jitendra Pandey
d93d515af5 HDDS-651. Rename o3 to o3fs for Filesystem. 2018-10-17 14:19:17 -07:00
Nandakumar
d54f5598f4 HDDS-661. When a volume fails in datanode, VersionEndpointTask#call ends up in dead lock. Contributed by Hanisha Koneru. 2018-10-17 18:44:05 +05:30
Nandakumar
50715c0699 HDDS-662. Introduce ContainerReplicaState in StorageContainerManager. Contributed by Nanda kumar. 2018-10-17 17:45:35 +05:30
Márton Elek
a9a63ae4a8 HDDS-527. Show SCM chill mode status in SCM UI. Contributed by Yiqun Lin. 2018-10-17 12:54:01 +02:00
Nandakumar
64a43c92c2 HDDS-656. Add logic for pipeline report and action processing in new pipeline code. Contributed by Lokesh Jain. 2018-10-17 13:57:38 +05:30
Arpit Agarwal
5f0b43fa93 HDDS-378. Remove dependencies between hdds/ozone and hdfs proto files. Contributed by Elek, Marton. 2018-10-16 10:40:00 -07:00
Jitendra Pandey
53e5173bd1 HDDS-667. Fix TestOzoneFileInterfaces. Contributed by Mukul Kumar Singh. 2018-10-16 10:34:51 -07:00
Arpit Agarwal
2614078b21 HDDS-490. Improve om and scm start up options . Contributed by Namit Maheshwari. 2018-10-15 16:45:08 -07:00
Jitendra Pandey
e13a38f4bc HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee. 2018-10-15 11:52:38 -07:00
Jitendra Pandey
603649d3a9 HDDS-579. ContainerStateMachine should fail subsequent transactions per container in case one fails. Contributed by Shashikant Banerjee. 2018-10-13 19:15:01 -07:00
Jitendra Pandey
5209c7503b HDDS-603. Add BlockCommitSequenceId field per Container and expose it in Container Reports. Contributed by Shashikant Banerjee. 2018-10-13 12:15:42 -07:00
Jitendra Pandey
22f37af935 Revert "HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee." 
This reverts commit 0473b68000.
 2018-10-13 12:14:39 -07:00
Jitendra Pandey
0473b68000 HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee. 2018-10-13 12:12:08 -07:00
Anu Engineer
9227f3d224 HDDS-653. TestMetadataStore#testIterator fails on Windows. 
Contributed by Yiqun Lin.
 2018-10-13 10:41:06 -07:00
Nandakumar
5c8e023ba3 HDDS-587. Add new classes for pipeline management. Contributed by Lokesh Jain. 2018-10-13 03:22:20 +05:30
Arpit Agarwal
3a684a2b23 HDDS-644. Rename dfs.container.ratis.num.container.op.threads. Contributed by Bharat Viswanadham. 2018-10-12 14:44:51 -07:00
Anu Engineer
02e1ef5e07 HDDS-624.PutBlock fails with Unexpected Storage Container Exception. 
Contributed by  Arpit Agarwal.
 2018-10-12 14:22:46 -07:00
Xiaoyu Yao
e12edb3d8f HDDS-555. RandomKeyGenerator runs not closing the XceiverClient properly. Contributed by Mukul Kumar Singh. 2018-10-12 10:36:57 -07:00
Bharat Viswanadham
74db993a61 HDDS-609. On restart, SCM does not exit chill mode as it expects DNs to report containers in ALLOCATED state. Contributed by Hanisha Koneru. 2018-10-11 21:03:01 -07:00
Tsz Wo Nicholas Sze
fb18cc5eaa HDDS-625. putKey hangs for a long time after completion, sometimes forever. Contributed by Arpit Agarwal 2018-10-12 10:37:37 +08:00
Arpit Agarwal
50552479c4 HDDS-636. Turn off ozone.metastore.rocksdb.statistics for now. Contributed by Arpit Agarwal. 2018-10-11 17:21:27 -07:00
Jitendra Pandey
0bc6d0484a HDDS-550. Serialize ApplyTransaction calls per Container in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-10-11 17:12:47 -07:00
Hanisha Koneru
eb34b5f8af HDDS-601. On restart, SCM throws 'No such datanode' exception. 2018-10-11 13:20:32 -07:00
Arpit Agarwal
2bd000c851 HDDS-619. hdds.db.profile should not be tagged as a required setting & should default to DISK. Contributed by Dinesh Chitlangia. 2018-10-10 17:37:20 -07:00
Nanda kumar
90552b1ea8 HDDS-608. Access denied exception for user hdfs while making getContainerWithPipeline call. Contributed by Mukul Kumar Singh. 2018-10-10 20:50:49 +05:30
Akira Ajisaka
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Bharat Viswanadham
6a39739316 HDDS-583. SCM returns zero as the return code, even when invalid options are passed. Contributed by Namit Maheshwari. 2018-10-09 15:21:19 -07:00
Márton Elek
600438bcde HDDS-602. Bump Ozone version to 0.4.0-SNAPSHOT after ozone-0.3 branch cut. 2018-10-09 19:04:25 +02:00
Nanda kumar
9e9915ddab HDDS-599. Fix TestOzoneConfiguration TestOzoneConfigurationFields. Contributed by Sandeep Nemuri. 2018-10-09 18:23:05 +05:30
Nanda kumar
7367ff333b HDDS-450. Generate BlockCommitSequenceId in ContainerStateMachine for every commit operation in Ratis. Contributed by Shashikant Banerjee. 2018-10-09 18:07:01 +05:30
Ajay Kumar
cdf5d58364 HDDS-571. Update SCM chill mode exit criteria to optionally wait for n datanodes. Contributed by Ajay Kumar. 2018-10-05 14:07:16 -07:00
Nanda kumar
f362037527 HDDS-575. LoadExistingContainers should be moved to SCMContainerManager from ContainerStateManager. Contributed by Mukul Kumar Singh. 2018-10-05 16:47:01 +05:30
Bharat Viswanadham
2a07617f85 HDDS-354. VolumeInfo.getScmUsed throws NPE. Contributed by Hanisha Koneru. 2018-10-04 21:46:17 -07:00
Anu Engineer
e6b77ad65f HDDS-572. Support S3 buckets as first class objects in Ozone Manager - 1. 
Contributed by Anu Engineer.
 2018-10-04 21:40:13 -07:00
Nanda kumar
095c269620 HDDS-567. Rename Mapping to ContainerManager in SCM. Contributed by Nanda kumar. 2018-10-03 16:00:39 +05:30
Nanda kumar
7b374482d2 HDDS-526. Clean previous chill mode code from NodeManager. Contributed by Ajay Kumar. 2018-10-03 15:15:43 +05:30
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Nanda kumar
a39296260f HDDS-561. Move Node2ContainerMap and Node2PipelineMap to NodeManager. Contributed by Lokesh Jain. 2018-10-02 19:47:44 +05:30
Bharat Viswanadham
4eff629ab3 HDDS-525. Support virtual-hosted style URLs. Contributed by Bharat Viswanadham 2018-10-01 13:16:08 -07:00
Ajay Kumar
cc80ac2315 HDDS-557. DeadNodeHandler should handle exception from removeContainerHandler api. Contributed by Ajay Kumar. 2018-10-01 11:21:26 -07:00
Nanda kumar
f7ff8c051e HDDS-325. Add event watcher for delete blocks command. Contributed by Lokesh Jain. 2018-10-01 13:50:17 +05:30
Anu Engineer
0da03f8b14 HDDS-391. Simplify Audit Framework to make audit logging easier to use. 
Contributed by Dinesh Chitlangia.
 2018-09-28 14:00:24 -07:00
Ajay Kumar
3c798c1e3c HDDS-448. Move NodeStat to NodeStatemanager from SCMNodeManager. Contributed by LiXin Ge. 2018-09-28 10:33:17 -07:00
Hanisha Koneru
2a5d4315bf HDDS-551. Fix the close container status check in CloseContainerCommandHandler. Contributed by Shashikant Banerjee. 2018-09-27 16:03:49 -07:00
Ajay Kumar
184544eff8 HDDS-401. Update storage statistics on dead node. Contributed by LiXin Ge. 2018-09-27 09:56:30 -07:00
Nanda kumar
3f6195045e HDDS-554. In XceiverClientSpi, implement sendCommand(..) using sendCommandAsync(..). Contributed by Tsz Wo Nicholas Sze. 2018-09-26 09:16:23 +05:30
Ajay Kumar
3928af983e HDDS-370. Add and implement following functions in SCMClientProtocolServer. Contributed by Ajay Kumar. 2018-09-25 13:08:44 -07:00
Bharat Viswanadham
9c3fbbc4f6 HDDS-444. Add rest service to the s3gateway. Contributed by Elek Marton. 2018-09-24 22:10:16 -07:00
Bharat Viswanadham
62f817d32e HDDS-447. Separate ozone-dist and hadoop-dist projects with real classpath separation. Contributed by Elek Marton. 2018-09-24 10:10:11 -07:00
Anu Engineer
524f7cd354 HDDS-514. Clean Unregister JMX upon SCMConnectionManager#close. 
Contributed by Xiaoyu Yao.
 2018-09-20 12:21:34 -07:00
Anu Engineer
096a716080 HDDS-394. Rename *Key Apis in DatanodeContainerProtocol to *Block apis. 
Contributed Dinesh Chitlangia.
 2018-09-20 11:51:49 -07:00
Bharat Viswanadham
f6bb1ca3c1 HDDS-513. Check if the EventQueue is not closed before executing handlers. Contributed by Nanda Kumar. 2018-09-19 14:35:29 -07:00
Nanda kumar
236d16e3a5 HDDS-507. EventQueue should be shutdown on SCM shutdown. Contributed by Xiaoyu Yao. 2018-09-20 01:22:18 +05:30
Nanda kumar
042bf74d5e HDDS-460. Replication manager failed to import container data. Contributed by Elek, Marton. 2018-09-19 23:51:50 +05:30
Nanda kumar
61a4b07bda HDDS-461. Container remains in CLOSING state in SCM forever. Contributed by Shashikant Banerjee. 2018-09-19 21:43:44 +05:30
Márton Elek
b3c5221f30 HDDS-458. numberofKeys is 0 for all containers even when keys are present. Contributed by LiXin Ge. 
(cherry picked from commit 8600b049af184af04dfb93b6bc353740e3e821d3)
 2018-09-19 17:11:05 +02:00
Nanda kumar
15ed74fa24 HDDS-502. Exception in OM startup when running unit tests. Contributed by Arpit Agarwal. 2018-09-19 19:46:25 +05:30
Nanda kumar
c0956ee2a8 HDDS-476. Add Pipeline reports to make pipeline active on SCM restart. 
Contributed by Mukul Kumar Singh.
 2018-09-19 18:52:08 +05:30
Xiaoyu Yao
e435e12f1f HDDS-506. Fields in AllocateScmBlockResponseProto should be optional. Contributed by Arpit Agarwal. 2018-09-18 23:21:29 -07:00