big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,590 Commits 2 Branches 2 Tags 264 MiB
d6c233fce6
Commit Graph

680 Commits

Author SHA1 Message Date
Bharat Viswanadham
54b11de2c0 HDDS-898. Continue token should contain the previous dir in Ozone s3g object list. Contributed by Elek Marton. 2019-01-16 17:33:10 -08:00
Xiaoyu Yao
140565f4db HDDS-967. Fix failures in TestOzoneConfigurationFields. Contributed by Ajay Kumar. 2019-01-16 16:44:33 -07:00
Ajay Kumar
0e16cf1973 HDDS-970. Fix classnotfound error for bouncy castle classes in OM,SCM init. Contributed by Ajay Kumar. 2019-01-16 16:40:05 -07:00
Ajay Kumar
0faa5701d9 HDDS-964. Fix test failure in TestOmMetrics. Contributed by Ajay Kumar. 2019-01-16 16:40:04 -07:00
Anu Engineer
8978466fca HDDS-945. Fix generics warnings in delegation token. 
Contributed by Ajay Kumar.
 2019-01-16 16:23:26 -07:00
Anu Engineer
1d5734e341 HDDS-938. Add Client APIs for using S3 Auth interface. 
Contributed by  Dinesh Chitlangia.
 2019-01-16 16:18:39 -07:00
Anu Engineer
30bfc9cbd0 HDDS-955. SCM CA: Add CA to SCM. 
Contributed by Anu Engineer.
 2019-01-16 13:43:31 -07:00
Bharat Viswanadham
ddaef67181 HDDS-937. Create an S3 Auth Table. Contributed by Dinesh Chitlangia. 2019-01-16 13:43:30 -07:00
Ajay Kumar
50c4045fde HDDS-805. Block token: Client api changes for block token. Contributed by Ajay Kumar. 2019-01-16 13:38:27 -07:00
Xiaoyu Yao
2b115222cd HDDS-929. Remove ozone.max.key.len property. Contributed by Ajay Kumar. 2019-01-16 01:42:08 -07:00
Ajay Kumar
f894d86b2f HDDS-115. GRPC: Support secure gRPC endpoint with mTLS. Contributed by Xiaoyu Yao. 2019-01-16 01:42:08 -07:00
Xiaoyu Yao
417951ab58 HDDS-884. Fix merge issue that causes NPE OzoneManager#httpServer. Contributed by Xiaoyu Yao. 2019-01-16 01:22:44 -07:00
Ajay Kumar
6d522dc05d HDDS-804. Block token: Add secret token manager. Contributed by Ajay Kumar. 2019-01-16 01:22:43 -07:00
Anu Engineer
0c8829a9a1 HDDS-696. Bootstrap genesis SCM(CA) with self-signed certificate. 
Contributed by Anu Engineer.
 2019-01-16 01:16:25 -07:00
Ajay Kumar
825310608b HDDS-873. Fix TestSecureOzoneContainer NPE after HDDS-837. Contributed by Xiaoyu Yao. 2019-01-16 01:16:25 -07:00
Xiaoyu Yao
7e2770699c HDDS-9. Add GRPC protocol interceptors for Ozone Block Token. Contributed by Xiaoyu Yao. 2019-01-16 01:10:08 -07:00
Ajay Kumar
bb4a26ca32 HDDS-8. Add OzoneManager Delegation Token support. Contributed by Ajay Kumar. 2019-01-16 01:06:59 -07:00
Xiaoyu Yao
6d6b1a00c2 HDDS-836. Add TokenIdentifier Ozone for delegation token and block token. Contributed by Ajay Kumar. 2019-01-15 23:01:04 -07:00
Xiaoyu Yao
53120e2e6c HDDS-592. Fix ozone-secure.robot test. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar
a28ad7ad81 HDDS-753. SCM security protocol server is not starting. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Xiaoyu Yao
61e85d7cd1 HDDS-684. Fix HDDS-4 branch after HDDS-490 and HADOOP-15832. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Xiaoyu Yao
16e0bb874d HDDS-591. Adding ASF license header to kadm5.acl. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Xiaoyu Yao
0aab74026e HDDS-10. Add kdc docker image for secure ozone cluster. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar
d4511889fd HDDS-566. Move OzoneSecure docker-compose after HDDS-447. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Ajay Kumar
9920506b3d HDDS-547. Fix secure docker and configs. Contributed by Xiaoyu Yao. 2019-01-15 23:01:02 -07:00
Xiaoyu Yao
570b503e3e Fix merge conflicts 2019-01-15 22:54:50 -07:00
Xiaoyu Yao
faf53f8262 HDDS-100. SCM CA: generate public/private key pair for SCM/OM/DNs. Contributed by Ajay Kumar. 2019-01-15 22:53:56 -07:00
Xiaoyu Yao
e47135d9d9 HDDS-70. Fix config names for secure ksm and scm. Contributed by Ajay Kumar. 2019-01-15 22:53:55 -07:00
Xiaoyu Yao
914e93beeb HDDS-7. Enable kerberos auth for Ozone client in hadoop rpc. Contributed by Ajay Kumar. 2019-01-15 22:44:45 -07:00
Xiaoyu Yao
d3920ecbb7 HDDS-5. Enable OzoneManager kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:40:08 -07:00
Xiaoyu Yao
ff61931f91 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:16:58 -07:00
Márton Elek
05c84ab01c HDDS-977. Exclude dependency-reduced-pom.xml from ozone rat check. Contributed by Elek, Marton. 2019-01-14 16:30:21 -08:00
Hanisha Koneru
a4eefe5765 HDDS-947. Implement OzoneManager State Machine. 2019-01-11 10:06:05 -08:00
Bharat Viswanadham
2091d1a4af HDDS-941. Rename ChunkGroupInputStream to keyInputStream and ChunkInputStream to BlockInputStream. Contributed by Shashikant Banerjee. 2019-01-09 20:02:36 -08:00
Hanisha Koneru
f4617c6239 Revert "HDDS-947. Implement OzoneManager State Machine." 
This reverts commit 8dd11a1577.
 2019-01-09 11:20:57 -08:00
Hanisha Koneru
8dd11a1577 HDDS-947. Implement OzoneManager State Machine. 2019-01-09 10:57:24 -08:00
Márton Elek
32cf0412ce HDDS-965. Ozone: checkstyle improvements and code quality scripts. Contributed by Elek, Marton. 2019-01-08 22:27:19 +01:00
Márton Elek
999f31fd20 HDDS-924. MultipartUpload: S3 APi for complete Multipart Upload. Contributed by Bharat Viswanadham. 2019-01-08 15:48:48 +01:00
Márton Elek
695e93c2bb HDDS-969. Fix TestOzoneManagerRatisServer test failure. Contributed by Bharat Viswanadham. 2019-01-08 15:12:08 +01:00
Márton Elek
396ffba1aa HDDS-968. Fix TestObjectPut failures. Contributed by Bharat Viswanadham. 2019-01-08 14:54:38 +01:00
Bharat Viswanadham
4297e20c8b HDDS-926. Use Timeout rule for the the test methods in TestOzoneManager. Contributed by Dinesh Chitlangia. 2019-01-07 22:04:06 -08:00
Anu Engineer
06279ecc55 HDDS-946. AuditParser - insert audit to database in batches. 
contributed by Dinesh Chitlangia.
 2019-01-07 11:19:42 -08:00
Bharat Viswanadham
cdfbec47ce HDDS-930. Multipart Upload: Abort multiupload request. Contributed by Bharat Viswanadham. 2019-01-07 11:09:14 -08:00
Márton Elek
d66925a315 HDDS-902. MultipartUpload: S3 API for uploading a part file. Contributed by Bharat Viswanadham. 2019-01-07 12:39:46 +01:00
Márton Elek
992dd9d189 HDDS-901. MultipartUpload: S3 API for Initiate multipart upload. Contributed by Bharat Viswanadham. 2019-01-07 12:06:22 +01:00
Márton Elek
d14c56d150 HDDS-916. MultipartUpload: Complete Multipart upload request. Contributed by Bharat Viswanadham. 2019-01-07 11:27:36 +01:00
Anu Engineer
999da98d67 HDDS-915. Submit client request to OM Ratis server. 
Contributed by Hanisha Koneru.
 2019-01-04 15:25:26 -08:00
Anu Engineer
ddc0a40507 HDDS-896. Handle over replicated containers in SCM. 
Contributed by Nandakumar.
 2019-01-04 09:04:10 -08:00
Anu Engineer
14d232c0fe HDDS-957. Replace incorrect use of system property user.name. 
Contributed by Dinesh Chitlangia.
 2019-01-03 09:43:26 -08:00
Anu Engineer
996ab4874a HDDS-393. Audit Parser tool for processing ozone audit logs. 
Contributed by Dinesh Chitlangia.
 2019-01-02 11:52:12 -08:00
Nanda kumar
4d50ad030c HDDS-949. Update Ratis to 0.4.0-a8c4ca0-SNAPSHOT. 
Contributed by Xiaoyu Yao.
 2019-01-02 20:00:40 +05:30
Bharat Viswanadham
26e4be7022 HDDS-921. Add JVM pause monitor to Ozone Daemons (OM, SCM and Datanodes). Contributed by Bharat Viswanadham. 2018-12-23 17:33:25 -08:00
Bharat Viswanadham
13d3f99b37 HDDS-881.009. Encapsulate all client to OM requests into one request message. Contributed by Hanisha Koneru. 2018-12-20 09:12:42 -08:00
Nanda kumar
ea621fa087 HDDS-934. Freon run hung on non-functional pipeline. 
Contributed by Nanda kumar.
 2018-12-20 11:13:15 +05:30
Arpit Agarwal
b1ce9aa3b3 HDDS-940. Remove dead store to local variable in OmMetadataManagerImpl. Contributed by Dinesh Chitlangia. 2018-12-19 12:30:40 +05:30
Bharat Viswanadham
2f6bc25044 HDDS-911. Make TestOzoneManager unit tests independent. Contributed by Elek Marton. 2018-12-18 10:52:29 -08:00
Bharat Viswanadham
99b5061265 HDDS-914. Add Grafana support to ozoneperf docker container. Contributed by Dinesh Chitlangia. 2018-12-18 09:59:13 -08:00
Shashikant Banerjee
b51e9e4317 HDDS-912. Update ozone to latest ratis snapshot build (0.4.0-3b0be02-SNAPSHOT). Contributed by Lokesh Jain. 2018-12-18 18:21:17 +05:30
Shashikant Banerjee
4ff1c46d95 HDDS-925. Rename ChunkGroupOutputStream to KeyOutputStream and ChunkOutputStream to BlockOutputStream. Contributed by Shashikant Banerjee. 2018-12-18 18:03:46 +05:30
Shashikant Banerjee
ee10ba26dd HDDS-539. Ozone datanode command ignores the invalid options. Contributed by Vinicius Higa Murakami. 2018-12-18 16:10:27 +05:30
Xiaoyu Yao
94b368f29f HDDS-99. Adding SCM Audit log. Contributed by Dinesh Chitlangia. 2018-12-17 15:40:22 -08:00
Anu Engineer
5426653819 HDDS-908: NPE in TestOzoneRpcClient. 
Contributed by Ajay Kumar.
 2018-12-17 12:18:47 -08:00
Mukul Kumar Singh
92d44b2ad0 HDDS-876. Add blockade tests for flaky network. Contributed by Nilotpal Nandi. 2018-12-13 13:35:40 +05:30
Márton Elek
d44b37d7ac HDDS-889. MultipartUpload: Support uploading a part file in ozone. Contributed by Bharat Viswanadham. 2018-12-12 14:16:21 +01:00
Márton Elek
21c9080121 HDDS-872. Add Dockerfile and skaffold config to deploy ozone dev build to k8s. Contributed by Elek, Marton. 2018-12-11 16:31:33 +01:00
Márton Elek
17a8708039 HDDS-879. MultipartUpload: Add InitiateMultipartUpload in ozone. Contributed by Bharat Viswanadham. 2018-12-10 10:05:38 +01:00
Jitendra Pandey
1afba83f2c HDDS-870. Avoid creating block sized buffer in ChunkGroupOutputStream. Contributed by Shashikant Banerjee. 2018-12-08 09:23:10 -08:00
Anu Engineer
5e773efd78 HDDS-909 : Default implementation for Ozone acls. 
Contributed by Ajay Kumar.
 2018-12-07 23:20:19 -08:00
Bharat Viswanadham
6c852f2a37 HDDS-892. Parse aws v2 headers without spaces in Ozone s3 gateway. Contributed by Elek Marton. 2018-12-06 15:37:34 -08:00
Hanisha Koneru
019836b113 HDDS-858. Start a Standalone Ratis Server on OM 2018-12-06 13:33:52 -08:00
Ajay Kumar
8d882c3786 HDDS-880. Create api for ACL handling in Ozone. (Contributed by Ajay Kumar) 2018-12-06 13:27:28 -08:00
Bharat Viswanadham
343aaea2d1 HDDS-864. Use strongly typed codec implementations for the tables of the OmMetadataManager. Contributed by Elek Marton. 2018-12-06 11:39:59 -08:00
Mukul Kumar Singh
7274115d57 HDDS-890. Handle OverlappingFileLockException during writeStateMachineData in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-12-04 20:12:35 +05:30
Bharat Viswanadham
de4255509a HDDS-894. Content-length should be set for ozone s3 ranged download. Contributed by Elek Marton. 2018-12-03 15:17:44 -08:00
Bharat Viswanadham
ef3b03b75a HDDS-885. Fix test failures due to ChecksumData. Contributed by Hanisha Koneru. 2018-12-03 10:52:49 -08:00
Yiqun Lin
3044b78bd0 HDDS-848. Create SCM metrics related to container state. Contributed by Bharat Viswanadham. 2018-12-03 17:16:34 +08:00
Shashikant Banerjee
8f3e12ff07 HDDS-882. Provide a config to optionally turn on/off the sync flag during chunk writes. Contributed by Shashikant Banerjee. 2018-12-02 08:06:24 +05:30
Shashikant Banerjee
5a3c7714c4 HDDS-887. Add DispatcherContext info to Dispatcher from containerStateMachine. Contributed by Shashikant Banerjee. 2018-12-02 08:00:35 +05:30
Bharat Viswanadham
d15dc43659 HDDS-748. Use strongly typed metadata Table implementation. Contributed by Elek Marton. 2018-12-01 16:52:23 -08:00
Xiaoyu Yao
ae5fbdd9ed HDDS-877. Ensure correct surefire version for Ozone test. Contributed by Xiaoyu Yao. 2018-11-29 11:37:36 -08:00
Shashikant Banerjee
5e102f9aa5 HDDS-850. ReadStateMachineData hits OverlappingFileLockException in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-11-29 22:20:08 +05:30
Ajay Kumar
184cced513 HDDS-808. Simplify OMAction and DNAction classes used for AuditLogging. Contributed by Dinesh Chitlangia. 2018-11-29 08:35:20 -08:00
Hanisha Koneru
64a4b6b08b HDDS-284. Checksum for ChunksData. 2018-11-28 10:53:12 -08:00
Anu Engineer
34a914be03 HDDS-846. Exports ozone metrics to prometheus. 
Contributed by Elek, Marton.
 2018-11-27 15:57:24 -08:00
Márton Elek
96c104d365 HDDS-766. Ozone shell create volume fails if volume name does not have a leading slash. Contributed by chencan. 2018-11-27 12:32:54 +01:00
Márton Elek
753f421350 HDDS-820. Use more strict data format for the Last-Modified headers of s3 gateway. Contributed by Elek, Marton. 2018-11-23 10:05:12 +01:00
Shashikant Banerjee
ee44b069c6 HDDS-866. Handle RaftRetryFailureException in OzoneClient. Contributed by Shashikant Banerjee. 2018-11-22 15:02:07 +05:30
Ajay Kumar
8d2789c5eb HDDS-795. RocksDb specific classes leak from DBStore/Table interfaces. Contributed by Márton Elek. 2018-11-21 12:25:41 -08:00
Bharat Viswanadham
c4d0ef6946 HDDS-816. Create OM metrics for bucket, volume, keys. Contributed by Bharat Viswanadham. 2018-11-21 10:35:39 -08:00
Márton Elek
34b6fa7d6b HDDS-732. Add read method which takes offset and length in SignedChunkInputStream. Contributed by chencan. 2018-11-21 17:59:36 +01:00
Márton Elek
22867deffa HDDS-791. Support Range header for ozone s3 object download. Contributed by Bharat Viswanadham. 2018-11-21 17:04:53 +01:00
Shashikant Banerjee
c8b3dfa625 HDDS-860. Fix TestDataValidate unit tests. Contributed by Shashikant Banerjee. 2018-11-21 11:28:20 +05:30
Shashikant Banerjee
14e1a0a3d6 HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee. 2018-11-21 10:51:50 +05:30
Xiaoyu Yao
f994b526a0 HDDS-855. Move OMMetadataManager from hadoop-ozone/ozone-manager to hadoop-ozone/common. Contributed by Ajay Kumar. 2018-11-20 20:31:07 -08:00
Shashikant Banerjee
1734ace35f Revert "HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee." 
This reverts commit c74783036d.
 2018-11-21 01:07:38 +05:30
Shashikant Banerjee
c74783036d HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee. 2018-11-20 23:01:02 +05:30
Márton Elek
4ecdcc9620 HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath. 2018-11-20 15:24:20 +01:00
Márton Elek
c946f1b121 Revert "HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath." 
This reverts commit 2a7f485991.
 2018-11-20 13:35:40 +01:00
Márton Elek
2a7f485991 HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath. 2018-11-20 12:57:27 +01:00
Yiqun Lin
d0cc679441 HDDS-817. Create SCM metrics for disk from node report. Contributed by Bharat Viswanadham. 2018-11-20 14:22:30 +08:00
Mukul Kumar Singh
b5d7b292c9 HDDS-718. Introduce new SCM Commands to list and close Pipelines. Contributed by Lokesh Jain. 2018-11-19 22:59:30 +05:30
Nanda kumar
93666087bc HDDS-847. TestBlockDeletion is failing. 
Contributed by Lokesh Jain.
 2018-11-19 17:04:18 +05:30
Nanda kumar
5a7ca6ac3e HDDS-837. Persist originNodeId as part of .container file in datanode. 
Contributed by Nanda kumar.
 2018-11-19 14:39:55 +05:30
Mukul Kumar Singh
cfb915f3df HDDS-844. Add logic for pipeline teardown after timeout. Contributed by Lokesh Jain. 2018-11-19 12:11:05 +05:30
Xiaoyu Yao
29374999b6 HDDS-120. Adding HDDS datanode Audit Log. Contributed by Dinesh Chitlangia. 2018-11-16 14:59:59 -08:00
Nanda kumar
c4d9640028 HDDS-801. Quasi close the container when close is not executed via Ratis. 
Contributed by Nanda kumar.
 2018-11-16 23:11:33 +05:30
Anu Engineer
a16aa2f60b HDDS-825. Code cleanup based on messages from ErrorProne. 
Contributed by Anu Engineer.
 2018-11-15 17:36:09 -08:00
Bharat Viswanadham
57866b366f HDDS-821. Handle empty x-amz-storage-class header in Ozone S3 gateway. Contributed by Elek Marton. 2018-11-15 14:21:42 -08:00
Márton Elek
d40912b1bd HDDS-223. Create acceptance test for using datanode plugin. Contributed by Sandeep Nemuri. 2018-11-15 15:18:07 +01:00
Márton Elek
b12e69475b HDDS-658. Implement s3 bucket list backend call and use it from rest endpoint. Contributed by Bharat Viswanadham. 2018-11-15 12:22:16 +01:00
Anu Engineer
9b01f039eb HDDS-832. Docs folder is missing from the Ozone distribution package. Contributed by Elek, Marton. 2018-11-15 11:08:48 +01:00
Arpit Agarwal
bac8807c8b HDDS-819. Match OzoneFileSystem behavior with S3AFileSystem. Contributed by Hanisha Koneru. 2018-11-14 16:12:06 -08:00
Shashikant Banerjee
b57cc73f83 HDDS-774. Remove OpenContainerBlockMap from datanode. Contributed by Shashikant Banerjee. 2018-11-14 20:05:56 +05:30
Shashikant Banerjee
671fd6524b HDDS-675. Add blocking buffer and use watchApi for flush/close in OzoneClient. Contributed by Shashikant Banerjee. 2018-11-13 23:39:14 +05:30
Yiqun Lin
f8713f8ade HDDS-831. TestOzoneShell in integration-test is flaky. Contributed by Nanda kumar. 2018-11-13 10:38:27 +08:00
Jitendra Pandey
f944f33832 HDDS-709. Modify Close Container handling sequence on datanodes. Contributed by Shashikant Banerjee. 2018-11-12 14:08:39 -08:00
Nanda kumar
18fe65d756 HDDS-576. Move ContainerWithPipeline creation to RPC endpoint. 
Contributed by Nanda kumar.
 2018-11-12 23:33:11 +05:30
Márton Elek
9c32b50d61 HDDS-767. OM should not search for STDOUT root logger for audit logging. Contributed by Dinesh Chitlangia. 2018-11-12 10:54:41 +01:00
Tsz Wo Nicholas Sze
298d2502b0 HDDS-826. Update Ratis to 0.3.0-6f3419a-SNAPSHOT. 2018-11-09 18:35:40 -08:00
Nanda kumar
9fe50b4991 HDDS-733. Create container if not exist, as part of chunk write. 
Contributed by Lokesh Jain.
 2018-11-10 00:08:17 +05:30
Bharat Viswanadham
47194fefdd HDDS-823. OzoneRestClient is failing with NPE on getKeyDetails call. Contributed by Bharat Viswanadham. 2018-11-08 21:28:04 -08:00
Shashikant Banerjee
31614bcc7c HDDS-806. Update Ratis to latest snapshot version in ozone. Contributed by Tsz Wo Nicholas Sze and Mukul Kumar Singh. 2018-11-09 00:05:45 +05:30
Yiqun Lin
9317a61f3c HDDS-802. Container State Manager should get open pipelines for allocating container. Contributed by Lokesh Jain. 2018-11-08 23:41:43 +08:00
Nanda kumar
c80f753b0e HDDS-737. Introduce Incremental Container Report. 
Contributed by Nanda kumar.
 2018-11-08 18:33:38 +05:30
Yiqun Lin
addec29297 HDDS-809. Refactor SCMChillModeManager. 2018-11-07 13:54:08 +08:00
Anu Engineer
00a67f770e HDDS-810. Move the "\r\n" at the chunk end in TestSignedChunksInputStream#singlechunkwithoutend. 
Contributed by Chencan.
 2018-11-06 10:42:19 -08:00
Márton Elek
3d5cc1138a HDDS-798. Storage-class is showing incorrectly. Contributed by Bharat Viswanadham. 2018-11-06 18:28:54 +01:00
Akira Ajisaka
49412a1289
HDDS-811. [JDK 11] Fix compilation failures with jdk 11. Contributed by Lokesh Jain. 2018-11-06 21:26:43 +09:00
Shashikant Banerjee
942693bddd HDDS-799. Avoid ByteString to byte array conversion cost by using ByteBuffer in Datanode. Contributed by Mukul Kumar Singh. 2018-11-05 23:43:22 +05:30
Arpit Agarwal
c8ca1747c0 HDDS-797. If DN is started before SCM, it does not register. Contributed by Hanisha Koneru. 2018-11-05 09:40:00 -08:00
Bharat Viswanadham
6d9c18cfa9 HDDS-788. Change title page of bucket browser in S3gateway. Contributed by Bharat Viswanadham. 2018-11-02 08:00:18 -07:00
Yiqun Lin
d16d5f730e HDDS-751. Replace usage of Guava Optional with Java Optional. 2018-11-02 10:50:32 +08:00
Shashikant Banerjee
e0ac3081e9 HDDS-771. ChunkGroupOutputStream stream entries need to be properly updated on closed container exception. Contributed by Lokesh Jain. 2018-11-01 15:43:48 +05:30
Shashikant Banerjee
b13c56742a HDDS-697. update and validate the BCSID for PutSmallFile/GetSmallFile command. Contributed by Shashikant Banerjee. 2018-11-01 10:21:39 +05:30
Bharat Viswanadham
c5eb237e3e HDDS-777. Fix missing jenkins issue in s3gateway module. Contributed by Bharat Viswanadham. 2018-10-31 19:11:12 -07:00
Bharat Viswanadham
6668c19daf HDDS-677. Create documentation for s3 gateway to the docs. Contributed by Elek Marton. 2018-10-31 11:48:23 -07:00
Arpit Agarwal
08bb0362e0 HDDS-759. Create config settings for SCM and OM DB directories. Contributed by Arpit Agarwal. 2018-10-31 11:23:15 -07:00
Bharat Viswanadham
478b2cba0d HDDS-773. Loading ozone s3 bucket browser could be failed. Contributed by Elek Marton. 2018-10-31 07:54:23 -07:00
Márton Elek
b519f3f2a0 HDDS-659. Implement pagination in GET bucket (object list) endpoint. Contributed by Bharat Viswanadham. 2018-10-31 13:29:01 +01:00
Márton Elek
ecac351aac HDDS-712. Use x-amz-storage-class to specify replication type and replication factor. Contributed by Bharat Viswanadham. 2018-10-31 13:28:59 +01:00
Nanda kumar
e4f22b08e0 HDDS-755. ContainerInfo and ContainerReplica protobuf changes. 
Contributed by Nanda kumar.
 2018-10-31 10:29:35 +05:30
Anu Engineer
773f0d1519 HDDS-754. VolumeInfo#getScmUsed throws NPE. 
Contributed by Hanisha Koneru.
 2018-10-30 19:17:57 -07:00
Anu Engineer
e33b61f335 HDDS-762. Fix unit test failure for TestContainerSQLCli & TestSCMMetrics. 
Contributed by Mukul Kumar Singh.
 2018-10-30 19:16:52 -07:00
Shashikant Banerjee
7757331dbc HDDS-749. Restructure BlockId class in Ozone. Contributed by Shashikant Banerjee. 2018-10-30 14:15:27 +05:30
Arpit Agarwal
496f0ffe90 HDDS-620. ozone.scm.client.address should be an optional setting. Contributed by chencan and Arpit Agarwal. 2018-10-29 17:14:18 -07:00
Bharat Viswanadham
3655e573e2 HDDS-743. S3 multi delete request should return XML header in quiet mode. Contributed by Elek Marton. 2018-10-29 12:59:41 -07:00
Arpit Agarwal
a58048e8d5 HDDS-727. ozone.log is not getting created in logs directory. Contributed by Nilotpal Nandi. 2018-10-29 09:35:18 -07:00
Nanda kumar
902345de66 HDDS-728. Datanodes should use different ContainerStateMachine for each pipeline. 
Contributed by Mukul Kumar Singh.
 2018-10-29 19:53:52 +05:30
Márton Elek
bfb720ebc8 HDDS-573. Make VirtualHostStyleFilter port agnostic. Contributed by Danilo Perez. 2018-10-29 14:45:01 +01:00
Bharat Viswanadham
199703f985 HDDS-739. Support MultiDeleteRequest without XML namespace. Contributed by Elek Marton. 2018-10-27 09:32:28 -07:00
Nanda kumar
dce4ebe814 HDDS-694. Plugin new Pipeline management code in SCM. 
Contributed by Lokesh Jain.
 2018-10-26 17:55:04 +05:30
Hanisha Koneru
38a65e3b7c HDDS-682. Unified o3 address parsing for ozonen sh. Contributed by Elek, Marton. 2018-10-25 17:04:16 -07:00
Bharat Viswanadham
a4b9b7c130 HDDS-702:Used fixed/external version from hadoop jars in hdds/ozone projects. Contributed by Elek Marton. 2018-10-25 16:26:48 -07:00
Márton Elek
c7f349bc3f HDDS-703. Ozone docs does not render correctly on a Mobile Device. Contributed by Elek, Marton. 2018-10-25 15:00:39 +02:00
Márton Elek
23887129cd HDDS-643. Parse Authorization header in a separate filter. Contributed by Bharat Viswanadham. 2018-10-25 12:45:53 +02:00
Arpit Agarwal
244afaba4a HDDS-719. Remove Ozone dependencies on Apache Hadoop 3.2.0. Contributed by Arpit Agarwal. 2018-10-24 16:17:05 -07:00
Bharat Viswanadham
021caaa55e HDDS-516. Implement CopyObject REST endpoint. Contributed by Bharat Viswanadham. 2018-10-24 15:53:31 -07:00
Bharat Viswanadham
ebf8e1731d HDDS-693. Support multi-chunk signatures in s3g PUT object endpoint. Contributed by Elek Marton. 2018-10-24 12:51:38 -07:00
Shashikant Banerjee
b61846392e HDDS-708. Validate BCSID while reading blocks from containers in datanodes. Contributed by Shashikant Banerjee. 2018-10-23 16:52:17 +05:30
Anu Engineer
e3cca12048 HDDS-676. Enable Read from open Containers via Standalone Protocol. 
Contributed by Shashikant Banerjee.
 2018-10-22 15:57:01 -07:00
Bharat Viswanadham
f6498af0d7 HDDS-705. addendum patch to fix find bug issue. Contributed by Bharat Viswanadham. 2018-10-22 10:17:12 -07:00
Mukul Kumar Singh
82919a1e7a HDDS-638. Enable ratis snapshots for HDDS datanodes. Contributed by Mukul Kumar Singh. 2018-10-22 19:59:35 +05:30
Nanda kumar
97a41b3dbe HDDS-705. OS3Exception resource name should be the actual resource name. 
Contributed by Bharat Viswanadham.

Recommitting after making sure that patch is clean.
 2018-10-22 15:48:36 +05:30
Nanda kumar
c696419f3e Revert "HDDS-705. OS3Exception resource name should be the actual resource name." 
This reverts commit 977c6f6470.

There was spurious edit in this commit.
 2018-10-22 15:37:40 +05:30
Anu Engineer
c9077a9f5a HDDS-707. Allow registering MBeans without additional jmx properties. 
Contributed by Arpit Agarwal.
 2018-10-21 22:50:28 -07:00
Anu Engineer
977c6f6470 HDDS-705. OS3Exception resource name should be the actual resource name. 
Contributed by Bharat Viswanadham.
 2018-10-21 22:49:16 -07:00
Anu Engineer
d7b012e560 HDDS-679. Add query parameter to the constructed query in VirtualHostStyleFilter. 
Contributed by Bharat Viswanadham.
 2018-10-19 11:40:11 -07:00
Arpit Agarwal
c456d6b3a5 HDDS-621. ozone genconf improvements. Contributed by Dinesh Chitlangia. 2018-10-19 11:36:36 -07:00
Arpit Agarwal
dc27408043 HDDS-612. Even after setting hdds.scm.chillmode.enabled to false, SCM allocateblock fails with ChillModePrecheck exception. Contributed by Hanisha Koneru. 2018-10-19 11:36:28 -07:00
Anu Engineer
9aebafd2da HDDS-701. Support key multi-delete. 
Contributed by Elek, Marton.
 2018-10-19 10:13:08 -07:00
Anu Engineer
cd057918d6 HDDS-672. Spark shell throws OzoneFileSystem not found. 
Contributed by Anu Engineer.
 2018-10-19 09:22:46 -07:00
Anu Engineer
b22651e949 HDDS-680. Provide web based bucket browser. 
Contributed by Elek, Marton.
 2018-10-19 08:48:11 -07:00
Anu Engineer
f8d61b9fc2 HDDS-690. Javadoc build fails in hadoop-ozone. 
Contributed by Takanobu Asanuma.
 2018-10-18 12:14:54 -07:00
Arpit Agarwal
c64018026e HDDS-673. Suppress "Key not found" exception log with stack trace in OM. Contributed by Arpit Agarwal. 2018-10-18 11:51:16 -07:00
Bharat Viswanadham
3ed7163302 HDDS-678. Format of Last-Modified header is invalid in HEAD Object call. Contributed by Elek Marton. 2018-10-17 20:44:52 -07:00
Hanisha Koneru
9146d33e18 HDDS-670. Fix OzoneFS directory rename. 2018-10-17 17:51:29 -07:00
Bharat Viswanadham
4c4364ddd0 HDDS-683. Add a shell command to provide ozone mapping for a S3Bucket. Contributed by Bharat Viswanadham. 2018-10-17 16:46:13 -07:00
Jitendra Pandey
d93d515af5 HDDS-651. Rename o3 to o3fs for Filesystem. 2018-10-17 14:19:17 -07:00
Nandakumar
50715c0699 HDDS-662. Introduce ContainerReplicaState in StorageContainerManager. Contributed by Nanda kumar. 2018-10-17 17:45:35 +05:30
Márton Elek
9df1c84be0 HDDS-563. Support hybrid VirtualHost style URL. Contributed by Bharat Viswanadham. 2018-10-17 12:43:44 +02:00
Nandakumar
64a43c92c2 HDDS-656. Add logic for pipeline report and action processing in new pipeline code. Contributed by Lokesh Jain. 2018-10-17 13:57:38 +05:30
Bharat Viswanadham
657032f5dd HDDS-665. Add hdds.datanode.dir to docker-config. Contributed by Bharat Viswanadham. 2018-10-16 15:29:53 -07:00
Arpit Agarwal
5f0b43fa93 HDDS-378. Remove dependencies between hdds/ozone and hdfs proto files. Contributed by Elek, Marton. 2018-10-16 10:40:00 -07:00
Jitendra Pandey
53e5173bd1 HDDS-667. Fix TestOzoneFileInterfaces. Contributed by Mukul Kumar Singh. 2018-10-16 10:34:51 -07:00
Márton Elek
0c2914e582 HDDS-657. Remove {volume} path segments from all the remaining rest endpoints. Contributed by Elek, Marton. 2018-10-16 17:30:53 +02:00
Arpit Agarwal
0bf8a110a5 HDDS-439. 'ozone oz volume create' should default to current logged in user. Contributed by Dinesh Chitlangia. 2018-10-15 19:30:40 -07:00
Arpit Agarwal
2614078b21 HDDS-490. Improve om and scm start up options . Contributed by Namit Maheshwari. 2018-10-15 16:45:08 -07:00
Jitendra Pandey
e13a38f4bc HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee. 2018-10-15 11:52:38 -07:00
Márton Elek
5033deb13b HDDS-519. Implement ListBucket REST endpoint. Contributed by LiXin Ge. 2018-10-14 11:30:18 +02:00
Jitendra Pandey
603649d3a9 HDDS-579. ContainerStateMachine should fail subsequent transactions per container in case one fails. Contributed by Shashikant Banerjee. 2018-10-13 19:15:01 -07:00
Jitendra Pandey
5209c7503b HDDS-603. Add BlockCommitSequenceId field per Container and expose it in Container Reports. Contributed by Shashikant Banerjee. 2018-10-13 12:15:42 -07:00
Jitendra Pandey
22f37af935 Revert "HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee." 
This reverts commit 0473b68000.
 2018-10-13 12:14:39 -07:00
Jitendra Pandey
0473b68000 HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee. 2018-10-13 12:12:08 -07:00
Arpit Agarwal
addb84600f HDDS-654. Add ServiceLoader resource for OzoneFileSystem. Contributed by Jitendra Nath Pandey. 2018-10-13 09:19:45 -07:00
Márton Elek
ae268b4894 HDDS-518. Implement PutObject Rest endpoint. Contributed by chencan. 2018-10-13 15:55:57 +02:00
Márton Elek
cdad91c039 HDDS-616. Collect all the robot test outputs and return with the right exit code. Contributed by Elek, Marton. 2018-10-13 08:40:49 +02:00
Bharat Viswanadham
28ca5c9d16 HDDS-613. Update HeadBucket, DeleteBucket to not to have volume in path. Contributed by Bharat Viswanadham. 2018-10-12 16:59:11 -07:00
Anu Engineer
3c1fe073d2 HDDS-445. Create a logger to print out all of the incoming requests. 
Contributed by Bharat Viswanadham.
 2018-10-12 16:27:54 -07:00
Arpit Agarwal
ddc9649328 HDDS-646. TestChunkStreams.testErrorReadGroupInputStream fails. Contributed by Nanda kumar. 2018-10-12 15:06:42 -07:00
Nandakumar
5c8e023ba3 HDDS-587. Add new classes for pipeline management. Contributed by Lokesh Jain. 2018-10-13 03:22:20 +05:30
Anu Engineer
c07b95bdfc HDDS-645. Enable OzoneFS contract tests by default. 
Contributed by Arpit Agarwal.
 2018-10-12 14:46:04 -07:00
Bharat Viswanadham
8ae8a5004f HDDS-606. Create delete s3Bucket. Contributed by Bharat Viswanadham. 2018-10-12 13:58:53 -07:00
Arpit Agarwal
56b18b9df1 HDDS-639. ChunkGroupInputStream gets into infinite loop after reading a block. Contributed by Mukul Kumar Singh. 2018-10-12 13:33:38 -07:00
Arpit Agarwal
85ccab7d3f HDDS-641. Fix ozone filesystem robot test. Contributed by Mukul Kumar Singh. 2018-10-12 11:30:57 -07:00
Xiaoyu Yao
07d55292dc HDDS-555. RandomKeyGenerator runs not closing the XceiverClient properly(addendum). Contributed by Mukul Kumar Singh. 2018-10-12 10:51:50 -07:00
Xiaoyu Yao
e12edb3d8f HDDS-555. RandomKeyGenerator runs not closing the XceiverClient properly. Contributed by Mukul Kumar Singh. 2018-10-12 10:36:57 -07:00
Bharat Viswanadham
38fe447d93 HDDS-524. Implement PutBucket REST endpoint. Contributed by Bharat Viswanadham. 2018-10-12 09:58:16 -07:00
Yiqun Lin
5da042227c HDDS-628. Fix outdated names used in HDDS documentations. 2018-10-12 14:00:13 +08:00
Anu Engineer
604c27c33a HDDS-634. OzoneFS: support basic user group and permission for file/dir. 
Contributed by Xiaoyu Yao.
 2018-10-11 17:25:25 -07:00
Jitendra Pandey
0bc6d0484a HDDS-550. Serialize ApplyTransaction calls per Container in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-10-11 17:12:47 -07:00
Xiaoyu Yao
c05b260069 HDDS-627. OzoneFS read from an MR Job throws java.lang.ArrayIndexOutOfBoundsException. Contributed by Mukul Kumar Singh. 2018-10-11 15:08:22 -07:00
Arpit Agarwal
47eeb3eb7b HDDS-630. Rename KSM to OM in Hdds.md. Contributed by Takanobu Asanuma. 2018-10-11 09:20:11 -07:00
Márton Elek
cd280514b8 HDDS-517. Implement HeadObject REST endpoint. Contributed by LiXin Ge. 2018-10-10 15:28:41 +02:00
Anu Engineer
f068296f8a HDDS-443. Create reusable ProgressBar utility for freon tests. 
Contributed by Zsolt Horvath.
 2018-10-09 18:19:00 -07:00
Akira Ajisaka
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Anu Engineer
794c0451cf HDDS-604. Correct Ozone getOzoneConf description. 
Contributed by Dinesh Chitlangia.
 2018-10-09 17:52:06 -07:00
Anu Engineer
4de2dc2699 HDDS-568. ozone sh volume info, update, delete operations fail when volume name is not prefixed by /. 
Contributed by Dinesh Chitlangia.
 2018-10-09 17:32:04 -07:00
Anu Engineer
6a06bc309d HDDS-559. fs.default.name is deprecated. 
Contributed by  Dinesh Chitlangia.
 2018-10-09 16:57:39 -07:00
Anu Engineer
c1fe657a10 HDDS-478. Log files related to each daemon doesn't have proper startup and shutdown logs. 
Contributed by Dinesh Chitlangia.
 2018-10-09 16:44:32 -07:00
Anu Engineer
5b7ba48ced HDDS-577. Support S3 buckets as first class objects in Ozone Manager - 2. 
Contributed by Bharat Viswanadham.
 2018-10-09 13:37:42 -07:00
Bharat Viswanadham
d5dd6f31fc HDDS-585. Handle common request identifiers in a transparent way. Contributed by Elek Marton. 2018-10-09 12:43:04 -07:00
Arpit Agarwal
03d66b1f5d HDDS-590. Add unit test for HDDS-583. Contributed by Namit Maheshwari. 2018-10-09 11:04:43 -07:00
Márton Elek
600438bcde HDDS-602. Bump Ozone version to 0.4.0-SNAPSHOT after ozone-0.3 branch cut. 2018-10-09 19:04:25 +02:00
Nanda kumar
7367ff333b HDDS-450. Generate BlockCommitSequenceId in ContainerStateMachine for every commit operation in Ratis. Contributed by Shashikant Banerjee. 2018-10-09 18:07:01 +05:30
Márton Elek
d7c7f68c26 HDDS-521. Implement DeleteBucket REST endpoint. Contributed by Bharat Viswanadham. 2018-10-08 16:17:42 +02:00
Márton Elek
25b1e8a48e HDDS-512. Addendum: update test.sh to remove robot framework & python-pip installation. Contributed by Dinesh Chitlangia. 2018-10-05 13:32:47 +02:00
Márton Elek
0e28ef35be HDDS-512. update test.sh to remove robot framework & python-pip installation. Contributed by Dinesh Chitlangia. 2018-10-05 13:20:27 +02:00
Márton Elek
9bc8b1cfd3 HDDS-578. om-audit-log4j2.properties must be packaged in ozone-dist . Contributed by Dinesh Chitlangia. 2018-10-05 04:33:19 -04:00
Anu Engineer
153941b236 HDDS-479. Add more ozone fs tests in the robot integration framework. 
Contributed by  Nilotpal Nandi.
 2018-10-04 22:53:51 -07:00
Anu Engineer
e6b77ad65f HDDS-572. Support S3 buckets as first class objects in Ozone Manager - 1. 
Contributed by Anu Engineer.
 2018-10-04 21:40:13 -07:00
Bharat Viswanadham
7fb91b8a53 HDDS-565. TestContainerPersistence fails regularly in Jenkins. Contributed by Dinesh Chitlangia. 2018-10-04 21:27:12 -07:00
Nanda kumar
095c269620 HDDS-567. Rename Mapping to ContainerManager in SCM. Contributed by Nanda kumar. 2018-10-03 16:00:39 +05:30
Nanda kumar
7b374482d2 HDDS-526. Clean previous chill mode code from NodeManager. Contributed by Ajay Kumar. 2018-10-03 15:15:43 +05:30
Bharat Viswanadham
ec075791da HDDS-520. Implement HeadBucket REST endpoint. Contributed by Bharat Viswanadham. 2018-10-02 20:13:59 -07:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Nanda kumar
a39296260f HDDS-561. Move Node2ContainerMap and Node2PipelineMap to NodeManager. Contributed by Lokesh Jain. 2018-10-02 19:47:44 +05:30
Márton Elek
81072d5e3d HDDS-560. Create Generic exception class to be used by S3 rest services. Contributed by Bharat Viswanadham. 2018-10-02 10:00:10 -04:00
Bharat Viswanadham
7d082193d2 HDDS-562. Create acceptance test to test aws cli with the s3 gateway. Contributed by Elek Marton. 2018-10-01 15:04:20 -07:00
Bharat Viswanadham
4eff629ab3 HDDS-525. Support virtual-hosted style URLs. Contributed by Bharat Viswanadham 2018-10-01 13:16:08 -07:00
Nanda kumar
f7ff8c051e HDDS-325. Add event watcher for delete blocks command. Contributed by Lokesh Jain. 2018-10-01 13:50:17 +05:30
Anu Engineer
0da03f8b14 HDDS-391. Simplify Audit Framework to make audit logging easier to use. 
Contributed by Dinesh Chitlangia.
 2018-09-28 14:00:24 -07:00
Hanisha Koneru
35c7351f4b HDDS-455. Ozone genconf tool must use picocli. Contributed by Dinesh Chitlangia. 2018-09-28 13:06:50 -07:00
Hanisha Koneru
72891fc9be HDDS-289. Volume and Bucket name should not contain a delimiter ('/'). Contributed by chencan. 2018-09-28 11:24:42 -07:00
Ajay Kumar
3928af983e HDDS-370. Add and implement following functions in SCMClientProtocolServer. Contributed by Ajay Kumar. 2018-09-25 13:08:44 -07:00
Bharat Viswanadham
9c3fbbc4f6 HDDS-444. Add rest service to the s3gateway. Contributed by Elek Marton. 2018-09-24 22:10:16 -07:00
Bharat Viswanadham
67173b4847 HDDS-441. Create new s3gateway daemon. Contributed by Elek Marton. 2018-09-24 13:42:55 -07:00
Bharat Viswanadham
62f817d32e HDDS-447. Separate ozone-dist and hadoop-dist projects with real classpath separation. Contributed by Elek Marton. 2018-09-24 10:10:11 -07:00
Anu Engineer
096a716080 HDDS-394. Rename *Key Apis in DatanodeContainerProtocol to *Block apis. 
Contributed Dinesh Chitlangia.
 2018-09-20 11:51:49 -07:00
Anu Engineer
98c9bc4aa0 HDDS-509. TestStorageContainerManager is flaky. 
Contributed by Xiaoyu Yao and Ajay Kumar.
 2018-09-19 13:03:19 -07:00
Nanda kumar
61a4b07bda HDDS-461. Container remains in CLOSING state in SCM forever. Contributed by Shashikant Banerjee. 2018-09-19 21:43:44 +05:30