big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,359 Commits 2 Branches 2 Tags 264 MiB
296259e9b3
Commit Graph

598 Commits

Author SHA1 Message Date
Anu Engineer
5cb67cf044 HDDS-1097. Add genesis benchmark for BlockManager#allocateBlock. 
Contributed by Lokesh Jain.
 2019-02-14 23:24:58 -08:00
Anu Engineer
506bd02c63 HDDS-905. Create informative landing page for Ozone S3 gateway. 
Contributed by Elek, Marton.
 2019-02-14 23:16:23 -08:00
Anu Engineer
0395f22145 HDDS-1068. Improve the error propagation for ozone sh. 
Contributed by Elek, Marton.
 2019-02-14 22:51:46 -08:00
Yiqun Lin
084b6a6751 HDDS-1099. Genesis benchmark for ozone key creation in OM. Contributed by Bharat Viswanadham. 2019-02-15 14:23:34 +08:00
Yiqun Lin
492e49e7ca Revert "HDDS-1099. Genesis benchmark for ozone key creation in OM. Contributed by Bharat Viswanadham." 
This reverts commit 5656409327.
 2019-02-15 14:21:29 +08:00
Yiqun Lin
5656409327 HDDS-1099. Genesis benchmark for ozone key creation in OM. Contributed by Bharat Viswanadham. 2019-02-15 14:07:15 +08:00
Anu Engineer
2d83b24994 HDDS-1108. Check s3bucket exists or not before MPU operations. 
Contributed by Bharat Viswanadham.
 2019-02-14 18:10:18 -08:00
Anu Engineer
6c8ffdb958 HDDS-1100. fix asf license errors in newly added files by HDDS-936. 
Contributed by  Dinesh Chitlangia.
 2019-02-14 17:42:59 -08:00
Márton Elek
7a57974f1f HDDS-1096. OzoneManager#loadOMHAConfigs should use default ports in case port is not defined. Contributed by Hanisha Koneru. 2019-02-14 14:02:16 +01:00
Jitendra Pandey
fa067aa157 HDDS-936. Need a tool to map containers to ozone objects. Contributed by Sarun Singla 2019-02-13 15:43:14 -08:00
Bharat Viswanadham
cf4aeccfa0 HDDS-1034. TestOzoneRpcClient and TestOzoneRpcClientWithRatis failure. Contributed by Mukul Kumar Singh. 2019-02-12 22:23:48 -08:00
Bharat Viswanadham
917ac9f108 HDDS-972. Add support for configuring multiple OMs. Contributed by Hanisha Koneru. 2019-02-12 22:07:27 -08:00
Yiqun Lin
06d7890bdd HDDS-1047. Fix TestRatisPipelineProvider#testCreatePipelineWithFactor. Contributed by Nilotpal Nandi. 2019-02-13 10:50:57 +08:00
Mukul Kumar Singh
20b92cdefe HDDS-1050. TestSCMRestart#testPipelineWithScmRestart is failing. Contributed by Supratim Deka. 2019-02-12 16:33:58 +05:30
Nanda kumar
63a9b0d8aa HDDS-1080. Ozonefs Isolated class loader should support FsStorageStatistics. 
Contributed by Elek, Marton.
 2019-02-12 14:49:51 +05:30
Nanda kumar
a536eb5c41 HDDS-360. Use RocksDBStore and TableStore for SCM Metadata. 
Contributed by Anu Engineer.
 2019-02-12 14:26:19 +05:30
Yiqun Lin
26e60135f5 HDDS-1074. Remove dead variable from KeyOutputStream#addKeyLocationInfo. Contributed by Siddharth Wagle. 2019-02-12 16:28:50 +08:00
Ajay Kumar
d48e61dd36 HDDS-1012. Add Default CertificateClient implementation. Contributed by Ajay Kumar 2019-02-11 18:59:47 -08:00
Mukul Kumar Singh
73b67b2df5 HDDS-1040. Add blockade Tests for client failures. Contributed by Nilotpal Nandi. 2019-02-11 20:08:25 +05:30
Márton Elek
e7d1ae52d2 HDDS-1017. Use distributed tracing to indentify performance problems in Ozone. Contributed by Elek, Marton. 2019-02-11 12:18:26 +01:00
Bharat Viswanadham
a1414582a8 HDDS-1077. TestSecureOzoneCluster does not config OM HTTP keytab. Contributed by Xiaoyu Yao. 2019-02-10 19:34:53 -08:00
Bharat Viswanadham
ed99da850e HDDS-1078. TestRatisPipelineProvider failing because of node count mismatch. Contributed by Mukul Kumar Singh. 2019-02-10 19:17:08 -08:00
Shashikant Banerjee
965d26c9c7 HDDS-1026. Reads should fail over to alternate replica. Contributed by Shashikant Banerjee. 2019-02-10 10:53:16 +05:30
Xiaoyu Yao
e50dc7ee59 HDDS-1018. Update the base image of krb5 container for the secure ozone cluster. Contributed by Xiaoyu Yao. 2019-02-08 12:45:49 -08:00
Nanda kumar
394a9f7d5c HDDS-1033. Add FSStatistics for OzoneFileSystem. 
Contributed by Mukul Kumar Singh.
 2019-02-08 17:33:37 +05:30
Márton Elek
df7b7dadf9 HDDS-1073. Fix FindBugs issues on OzoneBucketStub#createMultipartKey. Contributed by Aravindan Vijayan. 2019-02-08 11:35:55 +01:00
Nanda kumar
0c1bc4dcee HDDS-981. Block allocation should involve pipeline selection and then container selection. 
Contributed by Lokesh Jain.
 2019-02-08 15:44:41 +05:30
Bharat Viswanadham
a140a890c6 HDDS-1069. Temporarily disable the security acceptance tests by default in Ozone. Contributed by Marton Elek. 2019-02-07 18:05:05 -08:00
Bharat Viswanadham
75e8441c61 HDDS-1071. Make Ozone s3 acceptance test suite centos compatible. Contributed by Elek Marton. 2019-02-07 08:53:11 -08:00
Márton Elek
a65aca2fef HDDS-922. Create isolated classloder to use ozonefs with any older hadoop versions. Contributed by Elek, Marton. 2019-02-07 17:02:03 +01:00
Mukul Kumar Singh
214112b2d7 HDDS-1010. ContainerSet#getContainerMap should be renamed. Contributed by Supratim Deka. 2019-02-07 18:06:23 +05:30
Mukul Kumar Singh
911790cc26 HDDS-1027. Add blockade Tests for datanode isolation and scm failures. Contributed by Nilotpal Nandi. 2019-02-06 11:32:38 +05:30
Márton Elek
194f0b49fb HDDS-631. Ozone classpath shell command is not working. Contributed by Elek, Marton. 2019-02-05 16:20:30 +01:00
Márton Elek
9f2da01591 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila. 2019-02-05 16:11:19 +01:00
Márton Elek
9ace37b119 HDDS-987. MultipartUpload: S3API for list parts of a object. Contributed by Bharat Viswanadham. 2019-02-05 13:08:36 +01:00
Yiqun Lin
3efa168e1f HDDS-1029. Allow option for force in DeleteContainerCommand. Contributed by Bharat Viswanadham. 2019-02-05 10:51:52 +08:00
Mukul Kumar Singh
b6f90d3957 HDDS-1021. Allow ozone datanode to participate in a 1 node as well as 3 node Ratis Pipeline. Contributed by Shashikant Banerjee. 2019-02-03 00:53:06 +05:30
Shashikant Banerjee
4123353151 HDDS-1037. Fix the block discard logic in Ozone client. Contributed by Shashikant Banerjee. 2019-02-01 20:44:48 +05:30
Mukul Kumar Singh
13aa939224 HDDS-997. Add blockade Tests for scm isolation and mixed node isolation. Contributed by Nilotpal Nandi. 2019-02-01 13:10:17 +05:30
Márton Elek
033d97abac HDDS-956. MultipartUpload: List Parts for a Multipart upload key. Contributed by Bharat Viswanadham. 2019-01-31 14:51:31 +01:00
Bharat Viswanadham
53729279c7 HDDS-1035. Intermittent TestRootList failure. Contributed by Doroszlai Attila. 2019-01-30 16:21:42 -08:00
Bharat Viswanadham
945a61c164 HDDS-549. Add support for key rename in Ozone Shell. Contributed by Doroszlai Attila. 2019-01-30 16:00:18 -08:00
Bharat Viswanadham
7456fc99ee HDDS-1031. Update ratis version to fix a DN restart Bug. Contributed by Bharat Viswanadham. 2019-01-30 11:14:02 -08:00
Márton Elek
0e95ae402c HDDS-1030. Move auditparser robot tests under ozone basic. Contributed by Dinesh Chitlangia. 2019-01-30 17:34:32 +01:00
Yiqun Lin
d583cc45c6 HDDS-1024. Handle DeleteContainerCommand in the SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham. 2019-01-30 13:56:28 +08:00
Yiqun Lin
8326450bca HDDS-974. Add getServiceAddress method to ServiceInfo and use it in TestOzoneShell. Contributed by Doroszlai, Attila. 2019-01-28 17:03:12 +08:00
Hanisha Koneru
91649c34aa HDDS-1013. NPE while listing directories. 2019-01-25 21:39:03 -08:00
Bharat Viswanadham
fb69519e68 HDDS-1011. Fix NPE BucketManagerImpl.setBucketProperty. Contributed by Xiaoyu Yao. 2019-01-25 14:46:17 -08:00
Xiaoyu Yao
2ec296e659 HDDS-991. Fix failures in TestSecureOzoneCluster. Contributed by Ajay Kumar. 2019-01-25 10:50:23 -08:00
Xiaoyu Yao
9fc7df8afb HDDS-793. Support custom key/value annotations on volume/bucket/key. Contributed by Elek, Marton. 2019-01-24 23:24:09 -08:00
Nanda kumar
a448b05287 HDDS-1009. TestAbortMultipartUpload is missing the apache license text. 
Contributed by Dinesh Chitlangia.
 2019-01-25 12:20:12 +05:30
Nanda kumar
45c4cfe790 HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version. 
Contributed by Doroszlai, Attila.
 2019-01-25 12:16:29 +05:30
Nanda kumar
8ff9578126 HDDS-1007. Add robot test for AuditParser. 
Contributed by Dinesh Chitlangia.
 2019-01-25 12:14:10 +05:30
Nanda kumar
c6d901af77 HDDS-1006. AuditParser assumes incorrect log format. 
Contributed by Dinesh Chitlangia.
 2019-01-25 12:11:58 +05:30
Márton Elek
4e0aa2ceac HDDS-948. MultipartUpload: S3 API for Abort Multipart Upload. Contributed by Bharat Viswanadham. 2019-01-24 20:24:19 +01:00
Márton Elek
3c7d700b65 HDDS-1002. ozonesecure compose incompatible with smoke test. Contributed by Doroszlai, Attila. 2019-01-24 17:17:40 +01:00
Shashikant Banerjee
f3d8265582 HDDS-996. Incorrect data length gets updated in OM by client in case it hits exception in multiple successive block writes. Contributed by Shashikant Banerjee. 2019-01-24 16:37:05 +05:30
Márton Elek
09a5859f58 HDDS-993. Update hadoop version to 3.2.0. Contributed by Supratim Deka. 2019-01-24 11:32:58 +01:00
Ajay Kumar
dcbc8b86ed HDDS-975. Manage ozone security tokens with ozone shell cli. Contributed by Ajay Kumar. 2019-01-23 15:34:20 -08:00
Elek, Márton
0b91329ed6 HDDS-764. Run S3 smoke tests with replication STANDARD. (#462) 
HDDS-764. Run S3 smoke tests with replication STANDARD. Contributed by Elek, Marton.
 2019-01-23 11:37:49 -08:00
Shashikant Banerjee
e72e27edd8 HDDS-932. Add blockade Tests for Network partition. Contributed by Nilotpal Nandi. 2019-01-23 14:05:49 +05:30
Xiaoyu Yao
751bc62df7 Merge branch 'HDDS-4' into trunk 2019-01-18 14:44:24 -08:00
Bharat Viswanadham
c26d354e7d HDDS-983. Rename S3Utils to avoid conflict with HDFS classes. Contributed by Bharat Viswanadham. 2019-01-18 13:16:38 -08:00
Xiaoyu Yao
2aaaf12f9e HDDS-943. Add block token validation in HddsDispatcher/XceiverServer. Contributed by Ajay Kumar. 2019-01-18 12:11:34 -08:00
Mukul Kumar Singh
4ac0404fe0 HDDS-959. KeyOutputStream should handle retry failures. Contributed by Lokesh Jain. 2019-01-18 11:08:43 -08:00
Xiaoyu Yao
06c83d3b8a HDDS-984. Fix TestOzoneManagerRatisServer.testIsReadOnlyCapturesAllCmdTypeEnums. Contributed by Xiaoyu Yao. 2019-01-17 14:03:46 -08:00
Bharat Viswanadham
c0683ed253 HDDS-960. Add cli command option for getS3Secret. Contributed by Dinesh Chitlangia. 2019-01-17 10:30:45 -08:00
Ajay Kumar
01a7f9ed35 HDDS-597. Ratis: Support secure gRPC endpoint with mTLS for Ratis. Contributed by Ajay Kumar. 2019-01-17 10:30:44 -08:00
Bharat Viswanadham
54b11de2c0 HDDS-898. Continue token should contain the previous dir in Ozone s3g object list. Contributed by Elek Marton. 2019-01-16 17:33:10 -08:00
Xiaoyu Yao
140565f4db HDDS-967. Fix failures in TestOzoneConfigurationFields. Contributed by Ajay Kumar. 2019-01-16 16:44:33 -07:00
Ajay Kumar
0e16cf1973 HDDS-970. Fix classnotfound error for bouncy castle classes in OM,SCM init. Contributed by Ajay Kumar. 2019-01-16 16:40:05 -07:00
Ajay Kumar
0faa5701d9 HDDS-964. Fix test failure in TestOmMetrics. Contributed by Ajay Kumar. 2019-01-16 16:40:04 -07:00
Anu Engineer
8978466fca HDDS-945. Fix generics warnings in delegation token. 
Contributed by Ajay Kumar.
 2019-01-16 16:23:26 -07:00
Anu Engineer
1d5734e341 HDDS-938. Add Client APIs for using S3 Auth interface. 
Contributed by  Dinesh Chitlangia.
 2019-01-16 16:18:39 -07:00
Anu Engineer
30bfc9cbd0 HDDS-955. SCM CA: Add CA to SCM. 
Contributed by Anu Engineer.
 2019-01-16 13:43:31 -07:00
Bharat Viswanadham
ddaef67181 HDDS-937. Create an S3 Auth Table. Contributed by Dinesh Chitlangia. 2019-01-16 13:43:30 -07:00
Ajay Kumar
50c4045fde HDDS-805. Block token: Client api changes for block token. Contributed by Ajay Kumar. 2019-01-16 13:38:27 -07:00
Xiaoyu Yao
2b115222cd HDDS-929. Remove ozone.max.key.len property. Contributed by Ajay Kumar. 2019-01-16 01:42:08 -07:00
Ajay Kumar
f894d86b2f HDDS-115. GRPC: Support secure gRPC endpoint with mTLS. Contributed by Xiaoyu Yao. 2019-01-16 01:42:08 -07:00
Xiaoyu Yao
417951ab58 HDDS-884. Fix merge issue that causes NPE OzoneManager#httpServer. Contributed by Xiaoyu Yao. 2019-01-16 01:22:44 -07:00
Ajay Kumar
6d522dc05d HDDS-804. Block token: Add secret token manager. Contributed by Ajay Kumar. 2019-01-16 01:22:43 -07:00
Anu Engineer
0c8829a9a1 HDDS-696. Bootstrap genesis SCM(CA) with self-signed certificate. 
Contributed by Anu Engineer.
 2019-01-16 01:16:25 -07:00
Ajay Kumar
825310608b HDDS-873. Fix TestSecureOzoneContainer NPE after HDDS-837. Contributed by Xiaoyu Yao. 2019-01-16 01:16:25 -07:00
Xiaoyu Yao
7e2770699c HDDS-9. Add GRPC protocol interceptors for Ozone Block Token. Contributed by Xiaoyu Yao. 2019-01-16 01:10:08 -07:00
Ajay Kumar
bb4a26ca32 HDDS-8. Add OzoneManager Delegation Token support. Contributed by Ajay Kumar. 2019-01-16 01:06:59 -07:00
Xiaoyu Yao
6d6b1a00c2 HDDS-836. Add TokenIdentifier Ozone for delegation token and block token. Contributed by Ajay Kumar. 2019-01-15 23:01:04 -07:00
Xiaoyu Yao
53120e2e6c HDDS-592. Fix ozone-secure.robot test. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar
a28ad7ad81 HDDS-753. SCM security protocol server is not starting. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Xiaoyu Yao
61e85d7cd1 HDDS-684. Fix HDDS-4 branch after HDDS-490 and HADOOP-15832. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Xiaoyu Yao
16e0bb874d HDDS-591. Adding ASF license header to kadm5.acl. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Xiaoyu Yao
0aab74026e HDDS-10. Add kdc docker image for secure ozone cluster. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar
d4511889fd HDDS-566. Move OzoneSecure docker-compose after HDDS-447. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Ajay Kumar
9920506b3d HDDS-547. Fix secure docker and configs. Contributed by Xiaoyu Yao. 2019-01-15 23:01:02 -07:00
Xiaoyu Yao
570b503e3e Fix merge conflicts 2019-01-15 22:54:50 -07:00
Xiaoyu Yao
faf53f8262 HDDS-100. SCM CA: generate public/private key pair for SCM/OM/DNs. Contributed by Ajay Kumar. 2019-01-15 22:53:56 -07:00
Xiaoyu Yao
e47135d9d9 HDDS-70. Fix config names for secure ksm and scm. Contributed by Ajay Kumar. 2019-01-15 22:53:55 -07:00
Xiaoyu Yao
914e93beeb HDDS-7. Enable kerberos auth for Ozone client in hadoop rpc. Contributed by Ajay Kumar. 2019-01-15 22:44:45 -07:00
Xiaoyu Yao
d3920ecbb7 HDDS-5. Enable OzoneManager kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:40:08 -07:00
Xiaoyu Yao
ff61931f91 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:16:58 -07:00
Márton Elek
05c84ab01c HDDS-977. Exclude dependency-reduced-pom.xml from ozone rat check. Contributed by Elek, Marton. 2019-01-14 16:30:21 -08:00