big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,947 Commits 2 Branches 2 Tags 264 MiB
6d0bffe17e
Commit Graph

448 Commits

Author SHA1 Message Date
Hanisha Koneru
64a4b6b08b HDDS-284. Checksum for ChunksData. 2018-11-28 10:53:12 -08:00
Anu Engineer
34a914be03 HDDS-846. Exports ozone metrics to prometheus. 
Contributed by Elek, Marton.
 2018-11-27 15:57:24 -08:00
Márton Elek
96c104d365 HDDS-766. Ozone shell create volume fails if volume name does not have a leading slash. Contributed by chencan. 2018-11-27 12:32:54 +01:00
Márton Elek
753f421350 HDDS-820. Use more strict data format for the Last-Modified headers of s3 gateway. Contributed by Elek, Marton. 2018-11-23 10:05:12 +01:00
Shashikant Banerjee
ee44b069c6 HDDS-866. Handle RaftRetryFailureException in OzoneClient. Contributed by Shashikant Banerjee. 2018-11-22 15:02:07 +05:30
Ajay Kumar
8d2789c5eb HDDS-795. RocksDb specific classes leak from DBStore/Table interfaces. Contributed by Márton Elek. 2018-11-21 12:25:41 -08:00
Bharat Viswanadham
c4d0ef6946 HDDS-816. Create OM metrics for bucket, volume, keys. Contributed by Bharat Viswanadham. 2018-11-21 10:35:39 -08:00
Márton Elek
34b6fa7d6b HDDS-732. Add read method which takes offset and length in SignedChunkInputStream. Contributed by chencan. 2018-11-21 17:59:36 +01:00
Márton Elek
22867deffa HDDS-791. Support Range header for ozone s3 object download. Contributed by Bharat Viswanadham. 2018-11-21 17:04:53 +01:00
Shashikant Banerjee
c8b3dfa625 HDDS-860. Fix TestDataValidate unit tests. Contributed by Shashikant Banerjee. 2018-11-21 11:28:20 +05:30
Shashikant Banerjee
14e1a0a3d6 HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee. 2018-11-21 10:51:50 +05:30
Xiaoyu Yao
f994b526a0 HDDS-855. Move OMMetadataManager from hadoop-ozone/ozone-manager to hadoop-ozone/common. Contributed by Ajay Kumar. 2018-11-20 20:31:07 -08:00
Shashikant Banerjee
1734ace35f Revert "HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee." 
This reverts commit c74783036d.
 2018-11-21 01:07:38 +05:30
Shashikant Banerjee
c74783036d HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee. 2018-11-20 23:01:02 +05:30
Márton Elek
4ecdcc9620 HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath. 2018-11-20 15:24:20 +01:00
Márton Elek
c946f1b121 Revert "HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath." 
This reverts commit 2a7f485991.
 2018-11-20 13:35:40 +01:00
Márton Elek
2a7f485991 HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath. 2018-11-20 12:57:27 +01:00
Yiqun Lin
d0cc679441 HDDS-817. Create SCM metrics for disk from node report. Contributed by Bharat Viswanadham. 2018-11-20 14:22:30 +08:00
Mukul Kumar Singh
b5d7b292c9 HDDS-718. Introduce new SCM Commands to list and close Pipelines. Contributed by Lokesh Jain. 2018-11-19 22:59:30 +05:30
Nanda kumar
93666087bc HDDS-847. TestBlockDeletion is failing. 
Contributed by Lokesh Jain.
 2018-11-19 17:04:18 +05:30
Nanda kumar
5a7ca6ac3e HDDS-837. Persist originNodeId as part of .container file in datanode. 
Contributed by Nanda kumar.
 2018-11-19 14:39:55 +05:30
Mukul Kumar Singh
cfb915f3df HDDS-844. Add logic for pipeline teardown after timeout. Contributed by Lokesh Jain. 2018-11-19 12:11:05 +05:30
Xiaoyu Yao
29374999b6 HDDS-120. Adding HDDS datanode Audit Log. Contributed by Dinesh Chitlangia. 2018-11-16 14:59:59 -08:00
Nanda kumar
c4d9640028 HDDS-801. Quasi close the container when close is not executed via Ratis. 
Contributed by Nanda kumar.
 2018-11-16 23:11:33 +05:30
Anu Engineer
a16aa2f60b HDDS-825. Code cleanup based on messages from ErrorProne. 
Contributed by Anu Engineer.
 2018-11-15 17:36:09 -08:00
Bharat Viswanadham
57866b366f HDDS-821. Handle empty x-amz-storage-class header in Ozone S3 gateway. Contributed by Elek Marton. 2018-11-15 14:21:42 -08:00
Márton Elek
d40912b1bd HDDS-223. Create acceptance test for using datanode plugin. Contributed by Sandeep Nemuri. 2018-11-15 15:18:07 +01:00
Márton Elek
b12e69475b HDDS-658. Implement s3 bucket list backend call and use it from rest endpoint. Contributed by Bharat Viswanadham. 2018-11-15 12:22:16 +01:00
Anu Engineer
9b01f039eb HDDS-832. Docs folder is missing from the Ozone distribution package. Contributed by Elek, Marton. 2018-11-15 11:08:48 +01:00
Arpit Agarwal
bac8807c8b HDDS-819. Match OzoneFileSystem behavior with S3AFileSystem. Contributed by Hanisha Koneru. 2018-11-14 16:12:06 -08:00
Shashikant Banerjee
b57cc73f83 HDDS-774. Remove OpenContainerBlockMap from datanode. Contributed by Shashikant Banerjee. 2018-11-14 20:05:56 +05:30
Shashikant Banerjee
671fd6524b HDDS-675. Add blocking buffer and use watchApi for flush/close in OzoneClient. Contributed by Shashikant Banerjee. 2018-11-13 23:39:14 +05:30
Yiqun Lin
f8713f8ade HDDS-831. TestOzoneShell in integration-test is flaky. Contributed by Nanda kumar. 2018-11-13 10:38:27 +08:00
Jitendra Pandey
f944f33832 HDDS-709. Modify Close Container handling sequence on datanodes. Contributed by Shashikant Banerjee. 2018-11-12 14:08:39 -08:00
Nanda kumar
18fe65d756 HDDS-576. Move ContainerWithPipeline creation to RPC endpoint. 
Contributed by Nanda kumar.
 2018-11-12 23:33:11 +05:30
Márton Elek
9c32b50d61 HDDS-767. OM should not search for STDOUT root logger for audit logging. Contributed by Dinesh Chitlangia. 2018-11-12 10:54:41 +01:00
Tsz Wo Nicholas Sze
298d2502b0 HDDS-826. Update Ratis to 0.3.0-6f3419a-SNAPSHOT. 2018-11-09 18:35:40 -08:00
Nanda kumar
9fe50b4991 HDDS-733. Create container if not exist, as part of chunk write. 
Contributed by Lokesh Jain.
 2018-11-10 00:08:17 +05:30
Bharat Viswanadham
47194fefdd HDDS-823. OzoneRestClient is failing with NPE on getKeyDetails call. Contributed by Bharat Viswanadham. 2018-11-08 21:28:04 -08:00
Shashikant Banerjee
31614bcc7c HDDS-806. Update Ratis to latest snapshot version in ozone. Contributed by Tsz Wo Nicholas Sze and Mukul Kumar Singh. 2018-11-09 00:05:45 +05:30
Yiqun Lin
9317a61f3c HDDS-802. Container State Manager should get open pipelines for allocating container. Contributed by Lokesh Jain. 2018-11-08 23:41:43 +08:00
Nanda kumar
c80f753b0e HDDS-737. Introduce Incremental Container Report. 
Contributed by Nanda kumar.
 2018-11-08 18:33:38 +05:30
Yiqun Lin
addec29297 HDDS-809. Refactor SCMChillModeManager. 2018-11-07 13:54:08 +08:00
Anu Engineer
00a67f770e HDDS-810. Move the "\r\n" at the chunk end in TestSignedChunksInputStream#singlechunkwithoutend. 
Contributed by Chencan.
 2018-11-06 10:42:19 -08:00
Márton Elek
3d5cc1138a HDDS-798. Storage-class is showing incorrectly. Contributed by Bharat Viswanadham. 2018-11-06 18:28:54 +01:00
Akira Ajisaka
49412a1289
HDDS-811. [JDK 11] Fix compilation failures with jdk 11. Contributed by Lokesh Jain. 2018-11-06 21:26:43 +09:00
Shashikant Banerjee
942693bddd HDDS-799. Avoid ByteString to byte array conversion cost by using ByteBuffer in Datanode. Contributed by Mukul Kumar Singh. 2018-11-05 23:43:22 +05:30
Arpit Agarwal
c8ca1747c0 HDDS-797. If DN is started before SCM, it does not register. Contributed by Hanisha Koneru. 2018-11-05 09:40:00 -08:00
Bharat Viswanadham
6d9c18cfa9 HDDS-788. Change title page of bucket browser in S3gateway. Contributed by Bharat Viswanadham. 2018-11-02 08:00:18 -07:00
Yiqun Lin
d16d5f730e HDDS-751. Replace usage of Guava Optional with Java Optional. 2018-11-02 10:50:32 +08:00
Shashikant Banerjee
e0ac3081e9 HDDS-771. ChunkGroupOutputStream stream entries need to be properly updated on closed container exception. Contributed by Lokesh Jain. 2018-11-01 15:43:48 +05:30
Shashikant Banerjee
b13c56742a HDDS-697. update and validate the BCSID for PutSmallFile/GetSmallFile command. Contributed by Shashikant Banerjee. 2018-11-01 10:21:39 +05:30
Bharat Viswanadham
c5eb237e3e HDDS-777. Fix missing jenkins issue in s3gateway module. Contributed by Bharat Viswanadham. 2018-10-31 19:11:12 -07:00
Bharat Viswanadham
6668c19daf HDDS-677. Create documentation for s3 gateway to the docs. Contributed by Elek Marton. 2018-10-31 11:48:23 -07:00
Arpit Agarwal
08bb0362e0 HDDS-759. Create config settings for SCM and OM DB directories. Contributed by Arpit Agarwal. 2018-10-31 11:23:15 -07:00
Bharat Viswanadham
478b2cba0d HDDS-773. Loading ozone s3 bucket browser could be failed. Contributed by Elek Marton. 2018-10-31 07:54:23 -07:00
Márton Elek
b519f3f2a0 HDDS-659. Implement pagination in GET bucket (object list) endpoint. Contributed by Bharat Viswanadham. 2018-10-31 13:29:01 +01:00
Márton Elek
ecac351aac HDDS-712. Use x-amz-storage-class to specify replication type and replication factor. Contributed by Bharat Viswanadham. 2018-10-31 13:28:59 +01:00
Nanda kumar
e4f22b08e0 HDDS-755. ContainerInfo and ContainerReplica protobuf changes. 
Contributed by Nanda kumar.
 2018-10-31 10:29:35 +05:30
Anu Engineer
773f0d1519 HDDS-754. VolumeInfo#getScmUsed throws NPE. 
Contributed by Hanisha Koneru.
 2018-10-30 19:17:57 -07:00
Anu Engineer
e33b61f335 HDDS-762. Fix unit test failure for TestContainerSQLCli & TestSCMMetrics. 
Contributed by Mukul Kumar Singh.
 2018-10-30 19:16:52 -07:00
Shashikant Banerjee
7757331dbc HDDS-749. Restructure BlockId class in Ozone. Contributed by Shashikant Banerjee. 2018-10-30 14:15:27 +05:30
Arpit Agarwal
496f0ffe90 HDDS-620. ozone.scm.client.address should be an optional setting. Contributed by chencan and Arpit Agarwal. 2018-10-29 17:14:18 -07:00
Bharat Viswanadham
3655e573e2 HDDS-743. S3 multi delete request should return XML header in quiet mode. Contributed by Elek Marton. 2018-10-29 12:59:41 -07:00
Arpit Agarwal
a58048e8d5 HDDS-727. ozone.log is not getting created in logs directory. Contributed by Nilotpal Nandi. 2018-10-29 09:35:18 -07:00
Nanda kumar
902345de66 HDDS-728. Datanodes should use different ContainerStateMachine for each pipeline. 
Contributed by Mukul Kumar Singh.
 2018-10-29 19:53:52 +05:30
Márton Elek
bfb720ebc8 HDDS-573. Make VirtualHostStyleFilter port agnostic. Contributed by Danilo Perez. 2018-10-29 14:45:01 +01:00
Bharat Viswanadham
199703f985 HDDS-739. Support MultiDeleteRequest without XML namespace. Contributed by Elek Marton. 2018-10-27 09:32:28 -07:00
Nanda kumar
dce4ebe814 HDDS-694. Plugin new Pipeline management code in SCM. 
Contributed by Lokesh Jain.
 2018-10-26 17:55:04 +05:30
Hanisha Koneru
38a65e3b7c HDDS-682. Unified o3 address parsing for ozonen sh. Contributed by Elek, Marton. 2018-10-25 17:04:16 -07:00
Bharat Viswanadham
a4b9b7c130 HDDS-702:Used fixed/external version from hadoop jars in hdds/ozone projects. Contributed by Elek Marton. 2018-10-25 16:26:48 -07:00
Márton Elek
c7f349bc3f HDDS-703. Ozone docs does not render correctly on a Mobile Device. Contributed by Elek, Marton. 2018-10-25 15:00:39 +02:00
Márton Elek
23887129cd HDDS-643. Parse Authorization header in a separate filter. Contributed by Bharat Viswanadham. 2018-10-25 12:45:53 +02:00
Arpit Agarwal
244afaba4a HDDS-719. Remove Ozone dependencies on Apache Hadoop 3.2.0. Contributed by Arpit Agarwal. 2018-10-24 16:17:05 -07:00
Bharat Viswanadham
021caaa55e HDDS-516. Implement CopyObject REST endpoint. Contributed by Bharat Viswanadham. 2018-10-24 15:53:31 -07:00
Bharat Viswanadham
ebf8e1731d HDDS-693. Support multi-chunk signatures in s3g PUT object endpoint. Contributed by Elek Marton. 2018-10-24 12:51:38 -07:00
Shashikant Banerjee
b61846392e HDDS-708. Validate BCSID while reading blocks from containers in datanodes. Contributed by Shashikant Banerjee. 2018-10-23 16:52:17 +05:30
Anu Engineer
e3cca12048 HDDS-676. Enable Read from open Containers via Standalone Protocol. 
Contributed by Shashikant Banerjee.
 2018-10-22 15:57:01 -07:00
Bharat Viswanadham
f6498af0d7 HDDS-705. addendum patch to fix find bug issue. Contributed by Bharat Viswanadham. 2018-10-22 10:17:12 -07:00
Mukul Kumar Singh
82919a1e7a HDDS-638. Enable ratis snapshots for HDDS datanodes. Contributed by Mukul Kumar Singh. 2018-10-22 19:59:35 +05:30
Nanda kumar
97a41b3dbe HDDS-705. OS3Exception resource name should be the actual resource name. 
Contributed by Bharat Viswanadham.

Recommitting after making sure that patch is clean.
 2018-10-22 15:48:36 +05:30
Nanda kumar
c696419f3e Revert "HDDS-705. OS3Exception resource name should be the actual resource name." 
This reverts commit 977c6f6470.

There was spurious edit in this commit.
 2018-10-22 15:37:40 +05:30
Anu Engineer
c9077a9f5a HDDS-707. Allow registering MBeans without additional jmx properties. 
Contributed by Arpit Agarwal.
 2018-10-21 22:50:28 -07:00
Anu Engineer
977c6f6470 HDDS-705. OS3Exception resource name should be the actual resource name. 
Contributed by Bharat Viswanadham.
 2018-10-21 22:49:16 -07:00
Anu Engineer
d7b012e560 HDDS-679. Add query parameter to the constructed query in VirtualHostStyleFilter. 
Contributed by Bharat Viswanadham.
 2018-10-19 11:40:11 -07:00
Arpit Agarwal
c456d6b3a5 HDDS-621. ozone genconf improvements. Contributed by Dinesh Chitlangia. 2018-10-19 11:36:36 -07:00
Arpit Agarwal
dc27408043 HDDS-612. Even after setting hdds.scm.chillmode.enabled to false, SCM allocateblock fails with ChillModePrecheck exception. Contributed by Hanisha Koneru. 2018-10-19 11:36:28 -07:00
Anu Engineer
9aebafd2da HDDS-701. Support key multi-delete. 
Contributed by Elek, Marton.
 2018-10-19 10:13:08 -07:00
Anu Engineer
cd057918d6 HDDS-672. Spark shell throws OzoneFileSystem not found. 
Contributed by Anu Engineer.
 2018-10-19 09:22:46 -07:00
Anu Engineer
b22651e949 HDDS-680. Provide web based bucket browser. 
Contributed by Elek, Marton.
 2018-10-19 08:48:11 -07:00
Anu Engineer
f8d61b9fc2 HDDS-690. Javadoc build fails in hadoop-ozone. 
Contributed by Takanobu Asanuma.
 2018-10-18 12:14:54 -07:00
Arpit Agarwal
c64018026e HDDS-673. Suppress "Key not found" exception log with stack trace in OM. Contributed by Arpit Agarwal. 2018-10-18 11:51:16 -07:00
Bharat Viswanadham
3ed7163302 HDDS-678. Format of Last-Modified header is invalid in HEAD Object call. Contributed by Elek Marton. 2018-10-17 20:44:52 -07:00
Hanisha Koneru
9146d33e18 HDDS-670. Fix OzoneFS directory rename. 2018-10-17 17:51:29 -07:00
Bharat Viswanadham
4c4364ddd0 HDDS-683. Add a shell command to provide ozone mapping for a S3Bucket. Contributed by Bharat Viswanadham. 2018-10-17 16:46:13 -07:00
Jitendra Pandey
d93d515af5 HDDS-651. Rename o3 to o3fs for Filesystem. 2018-10-17 14:19:17 -07:00
Nandakumar
50715c0699 HDDS-662. Introduce ContainerReplicaState in StorageContainerManager. Contributed by Nanda kumar. 2018-10-17 17:45:35 +05:30
Márton Elek
9df1c84be0 HDDS-563. Support hybrid VirtualHost style URL. Contributed by Bharat Viswanadham. 2018-10-17 12:43:44 +02:00
Nandakumar
64a43c92c2 HDDS-656. Add logic for pipeline report and action processing in new pipeline code. Contributed by Lokesh Jain. 2018-10-17 13:57:38 +05:30
Bharat Viswanadham
657032f5dd HDDS-665. Add hdds.datanode.dir to docker-config. Contributed by Bharat Viswanadham. 2018-10-16 15:29:53 -07:00