big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,340 Commits 2 Branches 2 Tags 264 MiB
78c8a3323e
Commit Graph

533 Commits

Author SHA1 Message Date
Márton Elek
a65aca2fef HDDS-922. Create isolated classloder to use ozonefs with any older hadoop versions. Contributed by Elek, Marton. 2019-02-07 17:02:03 +01:00
Mukul Kumar Singh
214112b2d7 HDDS-1010. ContainerSet#getContainerMap should be renamed. Contributed by Supratim Deka. 2019-02-07 18:06:23 +05:30
Bharat Viswanadham
912d9f790d HDDS-1044. Client doesn't propogate correct error code to client on out of disk space. Contributed by Yiqun Lin. 2019-02-06 20:02:17 -08:00
Márton Elek
9f2da01591 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila. 2019-02-05 16:11:19 +01:00
Yiqun Lin
3efa168e1f HDDS-1029. Allow option for force in DeleteContainerCommand. Contributed by Bharat Viswanadham. 2019-02-05 10:51:52 +08:00
Ajay Kumar
9a19d6d48b HDDS-1039. OzoneManager fails to connect with secure SCM. Contributed by Ajay Kumar 2019-02-04 16:30:07 -08:00
Mukul Kumar Singh
b6f90d3957 HDDS-1021. Allow ozone datanode to participate in a 1 node as well as 3 node Ratis Pipeline. Contributed by Shashikant Banerjee. 2019-02-03 00:53:06 +05:30
Yiqun Lin
16195eaee1 HDDS-1025. Handle replication of closed containers in DeadNodeHanlder. Contributed by Bharat Viswanadham. 2019-02-01 11:34:51 +08:00
Márton Elek
033d97abac HDDS-956. MultipartUpload: List Parts for a Multipart upload key. Contributed by Bharat Viswanadham. 2019-01-31 14:51:31 +01:00
Bharat Viswanadham
945a61c164 HDDS-549. Add support for key rename in Ozone Shell. Contributed by Doroszlai Attila. 2019-01-30 16:00:18 -08:00
Arpit Agarwal
c35419579b HDDS-1016. Allow marking containers as unhealthy. Contributed by Arpit Agarwal. 2019-01-30 11:40:50 -08:00
Bharat Viswanadham
7456fc99ee HDDS-1031. Update ratis version to fix a DN restart Bug. Contributed by Bharat Viswanadham. 2019-01-30 11:14:02 -08:00
Márton Elek
14441ccbc6 HDDS-1032. Package builds are failing with missing org.mockito:mockito-core dependency version. Contributed by Doroszlai, Attila. 2019-01-30 13:40:45 +01:00
Yiqun Lin
d583cc45c6 HDDS-1024. Handle DeleteContainerCommand in the SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham. 2019-01-30 13:56:28 +08:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Shashikant Banerjee
5d578d0c4a HDDS-1004. SCMContainerManager#updateContainerStateInternal fails for QUASI_CLOSE and FORCE_CLOSE events. Contributed by Lokesh Jain. 2019-01-29 14:11:56 +05:30
Yiqun Lin
2d06112b74 HDDS-1022. Add cmd type in getCommandResponse in SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham. 2019-01-29 11:26:41 +08:00
Arpit Agarwal
3b49d7aeae HDDS-989. Check Hdds Volumes for errors. Contributed by Arpit Agarwal. 2019-01-27 11:18:30 -08:00
Nanda kumar
5dae1a0c66 HDDS-973. HDDS/Ozone fail to build on Windows. 
Contributed by Xiaoyu Yao.
 2019-01-25 12:19:11 +05:30
Nanda kumar
2181b18850 HDDS-990. Typos in Ozone doc. 
Contributed by Doroszlai, Attila.
 2019-01-25 12:17:36 +05:30
Nanda kumar
45c4cfe790 HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version. 
Contributed by Doroszlai, Attila.
 2019-01-25 12:16:29 +05:30
Márton Elek
09a5859f58 HDDS-993. Update hadoop version to 3.2.0. Contributed by Supratim Deka. 2019-01-24 11:32:58 +01:00
Xiaoyu Yao
e321b91cb5 HDDS-980. Adding getOMCertificate in SCMSecurityProtocol. Contributed by Ajay Kumar. 2019-01-23 19:32:03 -08:00
Márton Elek
2d69a35614 HDDS-982. Fix TestContainerDataYaml#testIncorrectContainerFile. Contributed by Doroszlai, Attila. 2019-01-23 11:31:07 +01:00
Bharat Viswanadham
0ef54f72a6 HDDS-992. ozone-default.xml has invalid text from a stale merge. Contributed by Dinesh Chitlangia. 2019-01-22 13:37:05 -08:00
Bharat Viswanadham
9390a0beef HDDS-913. Ozonefs defaultFs example is wrong in the documentation. Contributed by Supratim Deka. 2019-01-22 09:27:17 -08:00
Xiaoyu Yao
751bc62df7 Merge branch 'HDDS-4' into trunk 2019-01-18 14:44:24 -08:00
Xiaoyu Yao
2aaaf12f9e HDDS-943. Add block token validation in HddsDispatcher/XceiverServer. Contributed by Ajay Kumar. 2019-01-18 12:11:34 -08:00
Mukul Kumar Singh
4ac0404fe0 HDDS-959. KeyOutputStream should handle retry failures. Contributed by Lokesh Jain. 2019-01-18 11:08:43 -08:00
Bharat Viswanadham
c0683ed253 HDDS-960. Add cli command option for getS3Secret. Contributed by Dinesh Chitlangia. 2019-01-17 10:30:45 -08:00
Ajay Kumar
01a7f9ed35 HDDS-597. Ratis: Support secure gRPC endpoint with mTLS for Ratis. Contributed by Ajay Kumar. 2019-01-17 10:30:44 -08:00
Bharat Viswanadham
96ea464aaa HDDS-971. ContainerDataConstructor throws exception on QUASI_CLOSED and UNHEALTHY container state. Contributed by Lokesh Jain. 2019-01-16 17:43:30 -08:00
Xiaoyu Yao
140565f4db HDDS-967. Fix failures in TestOzoneConfigurationFields. Contributed by Ajay Kumar. 2019-01-16 16:44:33 -07:00
Ajay Kumar
0e16cf1973 HDDS-970. Fix classnotfound error for bouncy castle classes in OM,SCM init. Contributed by Ajay Kumar. 2019-01-16 16:40:05 -07:00
Anu Engineer
a5d0fcf704 HDDS-963. Fix failure in TestOzoneShell due to null check in SecurityConfig. 
Contributed by Ajay Kumar.
 2019-01-16 16:18:39 -07:00
Anu Engineer
1d5734e341 HDDS-938. Add Client APIs for using S3 Auth interface. 
Contributed by  Dinesh Chitlangia.
 2019-01-16 16:18:39 -07:00
Anu Engineer
30bfc9cbd0 HDDS-955. SCM CA: Add CA to SCM. 
Contributed by Anu Engineer.
 2019-01-16 13:43:31 -07:00
Xiaoyu Yao
924bea9730 HDDS-102. SCM CA: SCM CA server signs certificate for approved CSR. Contributed by Anu Engineer. 2019-01-16 13:43:30 -07:00
Ajay Kumar
50c4045fde HDDS-805. Block token: Client api changes for block token. Contributed by Ajay Kumar. 2019-01-16 13:38:27 -07:00
Xiaoyu Yao
2b115222cd HDDS-929. Remove ozone.max.key.len property. Contributed by Ajay Kumar. 2019-01-16 01:42:08 -07:00
Ajay Kumar
f894d86b2f HDDS-115. GRPC: Support secure gRPC endpoint with mTLS. Contributed by Xiaoyu Yao. 2019-01-16 01:42:08 -07:00
Ajay Kumar
6d522dc05d HDDS-804. Block token: Add secret token manager. Contributed by Ajay Kumar. 2019-01-16 01:22:43 -07:00
Anu Engineer
0c8829a9a1 HDDS-696. Bootstrap genesis SCM(CA) with self-signed certificate. 
Contributed by Anu Engineer.
 2019-01-16 01:16:25 -07:00
Xiaoyu Yao
7e2770699c HDDS-9. Add GRPC protocol interceptors for Ozone Block Token. Contributed by Xiaoyu Yao. 2019-01-16 01:10:08 -07:00
Ajay Kumar
bb4a26ca32 HDDS-8. Add OzoneManager Delegation Token support. Contributed by Ajay Kumar. 2019-01-16 01:06:59 -07:00
Xiaoyu Yao
6d6b1a00c2 HDDS-836. Add TokenIdentifier Ozone for delegation token and block token. Contributed by Ajay Kumar. 2019-01-15 23:01:04 -07:00
Anu Engineer
6ad794b1b6 HDDS-778. Add an interface for CA and Clients for Certificate operations 
Contributed by Anu Engineer.
 2019-01-15 23:01:03 -07:00
Ajay Kumar
a28ad7ad81 HDDS-753. SCM security protocol server is not starting. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar
8b8a3f5435 HDDS-760. Add asf license to TestCertificateSignRequest. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar
33c274ea2e HDDS-103. SCM CA: Add new security protocol for SCM to expose security related functions. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Anu Engineer
c260c19d18 HDDS-101. SCM CA: generate CSR for SCM CA clients. 
Contributed by Xiaoyu Yao.
 2019-01-15 23:01:03 -07:00
Xiaoyu Yao
e89c35a50b HDDS-704. Fix the Dependency convergence issue on HDDS-4. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Ajay Kumar
8d7c5f4ee7 HDDS-588. SelfSignedCertificate#generateCertificate should sign the certificate the configured security provider. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Ajay Kumar
9920506b3d HDDS-547. Fix secure docker and configs. Contributed by Xiaoyu Yao. 2019-01-15 23:01:02 -07:00
Ajay Kumar
2d269440b0 HDDS-548. Create a Self-Signed Certificate. Contributed by Anu Engineer. 2019-01-15 22:54:51 -07:00
Ajay Kumar
0b034b7005 HDDS-546. Resolve bouncy castle dependency for hadoop-hdds-common. Contributed by Ajay Kumar. 2019-01-15 22:54:51 -07:00
Xiaoyu Yao
570b503e3e Fix merge conflicts 2019-01-15 22:54:50 -07:00
Xiaoyu Yao
faf53f8262 HDDS-100. SCM CA: generate public/private key pair for SCM/OM/DNs. Contributed by Ajay Kumar. 2019-01-15 22:53:56 -07:00
Xiaoyu Yao
e47135d9d9 HDDS-70. Fix config names for secure ksm and scm. Contributed by Ajay Kumar. 2019-01-15 22:53:55 -07:00
Xiaoyu Yao
bfa4929fdb Revert "Bad merge with 996a627b289947af3894bf83e7b63ec702a665cd" 
This reverts commit 996a627b289947af3894bf83e7b63ec702a665cd.
 2019-01-15 22:40:08 -07:00
Xiaoyu Yao
8e6743eb90 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:40:08 -07:00
Xiaoyu Yao
d3920ecbb7 HDDS-5. Enable OzoneManager kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:40:08 -07:00
Xiaoyu Yao
ff61931f91 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:16:58 -07:00
Bharat Viswanadham
f280f52b20 HDDS-978. Fix typo in doc : Client > S3 section. Contributed by Dinesh Chitlangia. 2019-01-15 15:11:16 -08:00
Bharat Viswanadham
2091d1a4af HDDS-941. Rename ChunkGroupInputStream to keyInputStream and ChunkInputStream to BlockInputStream. Contributed by Shashikant Banerjee. 2019-01-09 20:02:36 -08:00
Márton Elek
32cf0412ce HDDS-965. Ozone: checkstyle improvements and code quality scripts. Contributed by Elek, Marton. 2019-01-08 22:27:19 +01:00
Anu Engineer
0f26b5efaa HDDS-931. Add documentation for ozone shell command providing ozone mapping for a S3Bucket. 
Contributed by Bharat Viswanadham.
 2019-01-07 13:06:45 -08:00
Márton Elek
d14c56d150 HDDS-916. MultipartUpload: Complete Multipart upload request. Contributed by Bharat Viswanadham. 2019-01-07 11:27:36 +01:00
Anu Engineer
999da98d67 HDDS-915. Submit client request to OM Ratis server. 
Contributed by Hanisha Koneru.
 2019-01-04 15:25:26 -08:00
Anu Engineer
ddc0a40507 HDDS-896. Handle over replicated containers in SCM. 
Contributed by Nandakumar.
 2019-01-04 09:04:10 -08:00
Anu Engineer
996ab4874a HDDS-393. Audit Parser tool for processing ozone audit logs. 
Contributed by Dinesh Chitlangia.
 2019-01-02 11:52:12 -08:00
Anu Engineer
0cb3316cec HDDS-482. NullPointer exception thrown on console when cli operation failed. 
Contributed by Nandakumar.
 2019-01-02 11:30:54 -08:00
Nanda kumar
4d50ad030c HDDS-949. Update Ratis to 0.4.0-a8c4ca0-SNAPSHOT. 
Contributed by Xiaoyu Yao.
 2019-01-02 20:00:40 +05:30
Akira Ajisaka
dea2c1b485
HDDS-951. Replace the source url of Ozone logo. Contributed by Takanobu Asanuma. 2018-12-27 11:33:25 +09:00
Bharat Viswanadham
26e4be7022 HDDS-921. Add JVM pause monitor to Ozone Daemons (OM, SCM and Datanodes). Contributed by Bharat Viswanadham. 2018-12-23 17:33:25 -08:00
Arpit Agarwal
cf571133b8 HDDS-893. pipeline status is ALLOCATED in scmcli listPipelines command. Contributed by Lokesh Jain. 2018-12-19 13:42:06 +05:30
Bharat Viswanadham
12c139d7de HDDS-933. Add documentation for genconf tool under Tools section. Contributed by Dinesh Chitlangia. 2018-12-18 11:40:32 -08:00
Shashikant Banerjee
b51e9e4317 HDDS-912. Update ozone to latest ratis snapshot build (0.4.0-3b0be02-SNAPSHOT). Contributed by Lokesh Jain. 2018-12-18 18:21:17 +05:30
Shashikant Banerjee
4ff1c46d95 HDDS-925. Rename ChunkGroupOutputStream to KeyOutputStream and ChunkOutputStream to BlockOutputStream. Contributed by Shashikant Banerjee. 2018-12-18 18:03:46 +05:30
Shashikant Banerjee
ee10ba26dd HDDS-539. Ozone datanode command ignores the invalid options. Contributed by Vinicius Higa Murakami. 2018-12-18 16:10:27 +05:30
Xiaoyu Yao
94b368f29f HDDS-99. Adding SCM Audit log. Contributed by Dinesh Chitlangia. 2018-12-17 15:40:22 -08:00
Márton Elek
17a8708039 HDDS-879. MultipartUpload: Add InitiateMultipartUpload in ozone. Contributed by Bharat Viswanadham. 2018-12-10 10:05:38 +01:00
Jitendra Pandey
1afba83f2c HDDS-870. Avoid creating block sized buffer in ChunkGroupOutputStream. Contributed by Shashikant Banerjee. 2018-12-08 09:23:10 -08:00
Anu Engineer
5e773efd78 HDDS-909 : Default implementation for Ozone acls. 
Contributed by Ajay Kumar.
 2018-12-07 23:20:19 -08:00
Hanisha Koneru
019836b113 HDDS-858. Start a Standalone Ratis Server on OM 2018-12-06 13:33:52 -08:00
Ajay Kumar
8d882c3786 HDDS-880. Create api for ACL handling in Ozone. (Contributed by Ajay Kumar) 2018-12-06 13:27:28 -08:00
Bharat Viswanadham
343aaea2d1 HDDS-864. Use strongly typed codec implementations for the tables of the OmMetadataManager. Contributed by Elek Marton. 2018-12-06 11:39:59 -08:00
Mukul Kumar Singh
7274115d57 HDDS-890. Handle OverlappingFileLockException during writeStateMachineData in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-12-04 20:12:35 +05:30
Bharat Viswanadham
ef3b03b75a HDDS-885. Fix test failures due to ChecksumData. Contributed by Hanisha Koneru. 2018-12-03 10:52:49 -08:00
Yiqun Lin
3044b78bd0 HDDS-848. Create SCM metrics related to container state. Contributed by Bharat Viswanadham. 2018-12-03 17:16:34 +08:00
Shashikant Banerjee
8f3e12ff07 HDDS-882. Provide a config to optionally turn on/off the sync flag during chunk writes. Contributed by Shashikant Banerjee. 2018-12-02 08:06:24 +05:30
Shashikant Banerjee
5a3c7714c4 HDDS-887. Add DispatcherContext info to Dispatcher from containerStateMachine. Contributed by Shashikant Banerjee. 2018-12-02 08:00:35 +05:30
Bharat Viswanadham
d15dc43659 HDDS-748. Use strongly typed metadata Table implementation. Contributed by Elek Marton. 2018-12-01 16:52:23 -08:00
Yiqun Lin
62f821115b HDDS-886. Unnecessary buffer copy in HddsDispatcher#dispatch. Contributed by Lokesh Jain. 2018-11-30 22:57:29 +08:00
Xiaoyu Yao
ae5fbdd9ed HDDS-877. Ensure correct surefire version for Ozone test. Contributed by Xiaoyu Yao. 2018-11-29 11:37:36 -08:00
Shashikant Banerjee
5e102f9aa5 HDDS-850. ReadStateMachineData hits OverlappingFileLockException in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-11-29 22:20:08 +05:30
Ajay Kumar
184cced513 HDDS-808. Simplify OMAction and DNAction classes used for AuditLogging. Contributed by Dinesh Chitlangia. 2018-11-29 08:35:20 -08:00
Ajay Kumar
b71cc7f33e HDDS-642. Add chill mode exit condition for pipeline availability. Contributed by Yiqun Lin. 2018-11-28 17:47:57 -08:00
Hanisha Koneru
64a4b6b08b HDDS-284. Checksum for ChunksData. 2018-11-28 10:53:12 -08:00
Anu Engineer
34a914be03 HDDS-846. Exports ozone metrics to prometheus. 
Contributed by Elek, Marton.
 2018-11-27 15:57:24 -08:00