big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,218 Commits 2 Branches 2 Tags 264 MiB
9df6275954
Commit Graph

244 Commits

Author SHA1 Message Date
Mukul Kumar Singh
67dd45fc25 HDDS-1185. Optimize GetFileStatus in OzoneFileSystem by reducing the number of rpc call to OM. Contributed by Mukul Kumar Singh. 2019-03-25 17:03:30 +05:30
Xiaoyu Yao
6a34c9bb29
HDDS-1299. Support TokenIssuer interface for running jobs with OzoneFileSystem. Contributed by Xiaoyu Yao. 
This closes #627.
 2019-03-23 09:22:34 -07:00
Xiaoyu Yao
dea6f2a065 HDDS-1291. Set OmKeyArgs#refreshPipeline flag properly to avoid reading from stale pipeline. Contributed by Xiaoyu Yao. (#639) 2019-03-22 15:08:28 -07:00
Arpit Agarwal
90afc9ab03
HDDS-1320. Update ozone to latest ratis snapshot build (0.4.0-1fc5ace-SNAPSHOT). Contributed by Mukul Kumar Singh. 2019-03-21 14:24:02 -07:00
Arpit Agarwal
2828f8c339 HDDS-1321. TestOzoneManagerHttpServer depends on hard-coded port numbers. Contributed by Arpit Agarwal. (#633) 
Change-Id: I9656af4a7f41812da9d125c10ae0e8daf3dcf7f5
 2019-03-21 14:18:58 -07:00
Yiqun Lin
60cdd4cac1 HDDS-1233. Create an Ozone Manager Service provider for Recon. Contributed by Aravindan Vijayan. 2019-03-21 11:02:29 +08:00
Bharat Viswanadham
310ebf5dc8 HDDS-1308. Fix asf license errors. (#623) 2019-03-19 14:06:39 -07:00
Ajay Yadav
f10d493325
HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601) 2019-03-18 23:08:17 -07:00
Bharat Viswanadham
93db5da4d9
HDDS-1250. In OM HA AllocateBlock call where connecting to SCM from OM should not happen on Ratis. 2019-03-18 13:51:44 -07:00
Xiaoyu Yao
66a104bc57 HDDS-1296. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#616) 2019-03-18 08:38:55 -07:00
Xiaoyu Yao
d1afa03804
HDDS-1138. Ozone Client should avoid talking to SCM directly. Contributed by Xiaoyu Yao and Mukul Kumar Singh. 
Closes #585
 2019-03-15 10:41:06 -07:00
Vivek Ratnavel Subramanian
9001508485 HDDS-761. Create S3 subcommand to run S3 related operations. 
Closes #579
 2019-03-14 21:16:59 -07:00
Vivek Ratnavel Subramanian
2627dad333 HDDS-1265. ozone sh s3 getsecret throws Null Pointer Exception for unsecured clusters. 
Closes #611
 2019-03-14 17:21:06 -07:00
Márton Elek
3d140a537a
HDDS-1247. Bump trunk ozone version to 0.5.0. Contributed by Elek, Marton. 
Closes #588
 2019-03-14 12:01:25 +01:00
Ajay Yadav
4fa009989b
HDDS-1254. Fix failure in TestOzoneManagerHttpServer & TestStorageContainerManagerHttpServer. Contributed by Ajay Kumar. (#598) 2019-03-13 12:01:08 -07:00
Ajay Yadav
dcb0de848d
HDDS-1043. Enable token based authentication for S3 api 
Closes #561
 2019-03-12 18:47:06 +01:00
Mukul Kumar Singh
daf4660eeb HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh. 2019-03-12 20:02:19 +05:30
Lokesh Jain
ebb5fa115b HDDS-1220. KeyManager#openKey should release the bucket lock before doing an allocateBlock. Contributed by Lokesh Jain. 2019-03-11 14:54:44 +05:30
Hanisha Koneru
bb12e81ec8
HDDS-1175. Serve read requests directly from RocksDB. (#557) 
HDDS-1175. Serve read requests directly from RocksDB.
 2019-03-06 19:44:55 -08:00
Márton Elek
45f976fdcf
HDDS-1113. Remove default dependencies from hadoop-ozone project. Contributed by Elek, Marton. 
Closes #490
 2019-03-06 18:22:22 +01:00
Hanisha Koneru
12402b7a74 HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-05 09:24:22 -08:00
Ajay Yadav
353b90d59e HDDS-1156. testDelegationToken is failing in TestSecureOzoneCluster. Contributed by Ajay Kumar. 2019-03-05 08:37:10 -08:00
Anu Engineer
10b802b84b HDDS-1136 : Add metric counters to capture the RocksDB checkpointing statistics. 
Contributed by Aravindan Vijayan.
 2019-03-04 12:00:16 -08:00
Hanisha Koneru
b18c1c22ea
Revert "HDDS-1072. Implement RetryProxy and FailoverProxy for OM client." 
This reverts commit 8e1225991d.
 2019-03-04 16:58:59 +09:00
Ajay Kumar
4a3cddea70 HDDS-134. SCM CA: OM sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. 2019-03-01 15:41:09 -08:00
Shashikant Banerjee
de1dae64f2 HDDS-726. Ozone Client should update SCM to move the container out of allocation path in case a write transaction fails. Contributed by Shashikant Banerjee. 2019-03-01 23:30:28 +05:30
Hanisha Koneru
8e1225991d HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-01 00:27:39 -08:00
Ajay Kumar
cbf82fabf0 HDDS-1061. DelegationToken: Add certificate serial id to Ozone Delegation Token Identifier. Contributed by Ajay Kumar. 2019-02-27 17:00:42 -08:00
Xiaoyu Yao
9e0f3d1c52 HDDS-1038. Support Service Level Authorization for Ozone. Contributed by Xiaoyu Yao and Ajay Kumar. 2019-02-26 06:02:10 -08:00
Anu Engineer
9de34d2990 HDDS-1151. Propagate the tracing id in ScmBlockLocationProtocol. 
Contributed by Elek, Marton.
 2019-02-25 13:50:37 -08:00
Anu Engineer
8387bbdfb5 HDDS-1154. Add tracing to the client side of StorageContainerLocationProtocol and OzoneManagerProtocol. 
Contributed by Elek, Marton.
 2019-02-24 13:13:37 -08:00
Bharat Viswanadham
106bdc6c04
HDDS-1120. Add a config to disable checksum verification during read. 2019-02-23 21:37:55 -08:00
Anu Engineer
c1e5b19212 HDDS-1147. Fix failing unit tests in TestOzoneManager. 
Contributed by Nandakumar.
 2019-02-23 20:56:24 -08:00
Bharat Viswanadham
d33f0666f6
HDDS-1141. Update DBCheckpointSnapshot to DBCheckpoint. 
* HDDS-1141.Update DBCheckpointSnapshot to DBCheckpoint.

* fix test failures in TestOzoneConfigurationFields
 2019-02-21 13:29:10 -08:00
Márton Elek
7c802c42dc
HDDS-1129. Fix findbug/checkstyle errors hdds projects. Contributed by Elek, Marton. 2019-02-21 14:04:18 +01:00
Arpit Agarwal
676a9cb888 HDDS-1053. Generate RaftGroupId from OMServiceID. Contributed by Aravindan Vijayan. 2019-02-20 12:57:49 -08:00
avijayanhwx
e8d7e3b4e6 HDDS-1139 : Fix findbugs issues caused by HDDS-1085. Contributed by Aravindan Vijayan. 2019-02-19 16:32:47 -08:00
Anu Engineer
588b4c4d78 HDDS-1085. Create an OM API to serve snapshots to Recon server. 
Contributed by Aravindan Vijayan.
 2019-02-18 22:35:36 -08:00
Yiqun Lin
67af509097 HDDS-1122. Fix TestOzoneManagerRatisServer#testSubmitRatisRequest unit test failure. Contributed by Yiqun Lin. 2019-02-19 11:29:52 +08:00
Anu Engineer
7ea91494ce HDDS-1041. Support TDE(Transparent Data Encryption) for Ozone. 
Contributed by Xiaoyu Yao.
 2019-02-16 10:23:10 -08:00
Elek, Márton
afe126d71f HDDS-1114. Fix findbugs/checkstyle/accepteance errors in Ozone. Contributed by Marton Elek. 2019-02-15 11:49:52 -08:00
Márton Elek
e0fe3d1eca HDDS-1110. OzoneManager need to login during init when security is enabled. Contributed by Xiaoyu Yao. 2019-02-15 16:12:24 +01:00
Anu Engineer
75e15cc0c4 HDDS-1103.Fix rat/findbug/checkstyle errors in ozone/hdds projects. 
Contributed by Elek, Marton.
 2019-02-14 23:33:25 -08:00
Anu Engineer
0395f22145 HDDS-1068. Improve the error propagation for ozone sh. 
Contributed by Elek, Marton.
 2019-02-14 22:51:46 -08:00
Anu Engineer
2d83b24994 HDDS-1108. Check s3bucket exists or not before MPU operations. 
Contributed by Bharat Viswanadham.
 2019-02-14 18:10:18 -08:00
Márton Elek
7a57974f1f HDDS-1096. OzoneManager#loadOMHAConfigs should use default ports in case port is not defined. Contributed by Hanisha Koneru. 2019-02-14 14:02:16 +01:00
Bharat Viswanadham
917ac9f108 HDDS-972. Add support for configuring multiple OMs. Contributed by Hanisha Koneru. 2019-02-12 22:07:27 -08:00
Ajay Kumar
d48e61dd36 HDDS-1012. Add Default CertificateClient implementation. Contributed by Ajay Kumar 2019-02-11 18:59:47 -08:00
Márton Elek
e7d1ae52d2 HDDS-1017. Use distributed tracing to indentify performance problems in Ozone. Contributed by Elek, Marton. 2019-02-11 12:18:26 +01:00
Márton Elek
9f2da01591 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila. 2019-02-05 16:11:19 +01:00
Márton Elek
033d97abac HDDS-956. MultipartUpload: List Parts for a Multipart upload key. Contributed by Bharat Viswanadham. 2019-01-31 14:51:31 +01:00
Bharat Viswanadham
945a61c164 HDDS-549. Add support for key rename in Ozone Shell. Contributed by Doroszlai Attila. 2019-01-30 16:00:18 -08:00
Xiaoyu Yao
2ec296e659 HDDS-991. Fix failures in TestSecureOzoneCluster. Contributed by Ajay Kumar. 2019-01-25 10:50:23 -08:00
Xiaoyu Yao
9fc7df8afb HDDS-793. Support custom key/value annotations on volume/bucket/key. Contributed by Elek, Marton. 2019-01-24 23:24:09 -08:00
Nanda kumar
45c4cfe790 HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version. 
Contributed by Doroszlai, Attila.
 2019-01-25 12:16:29 +05:30
Márton Elek
4e0aa2ceac HDDS-948. MultipartUpload: S3 API for Abort Multipart Upload. Contributed by Bharat Viswanadham. 2019-01-24 20:24:19 +01:00
Ajay Kumar
dcbc8b86ed HDDS-975. Manage ozone security tokens with ozone shell cli. Contributed by Ajay Kumar. 2019-01-23 15:34:20 -08:00
Xiaoyu Yao
2aaaf12f9e HDDS-943. Add block token validation in HddsDispatcher/XceiverServer. Contributed by Ajay Kumar. 2019-01-18 12:11:34 -08:00
Bharat Viswanadham
c0683ed253 HDDS-960. Add cli command option for getS3Secret. Contributed by Dinesh Chitlangia. 2019-01-17 10:30:45 -08:00
Ajay Kumar
0e16cf1973 HDDS-970. Fix classnotfound error for bouncy castle classes in OM,SCM init. Contributed by Ajay Kumar. 2019-01-16 16:40:05 -07:00
Ajay Kumar
0faa5701d9 HDDS-964. Fix test failure in TestOmMetrics. Contributed by Ajay Kumar. 2019-01-16 16:40:04 -07:00
Anu Engineer
8978466fca HDDS-945. Fix generics warnings in delegation token. 
Contributed by Ajay Kumar.
 2019-01-16 16:23:26 -07:00
Anu Engineer
1d5734e341 HDDS-938. Add Client APIs for using S3 Auth interface. 
Contributed by  Dinesh Chitlangia.
 2019-01-16 16:18:39 -07:00
Bharat Viswanadham
ddaef67181 HDDS-937. Create an S3 Auth Table. Contributed by Dinesh Chitlangia. 2019-01-16 13:43:30 -07:00
Ajay Kumar
50c4045fde HDDS-805. Block token: Client api changes for block token. Contributed by Ajay Kumar. 2019-01-16 13:38:27 -07:00
Ajay Kumar
f894d86b2f HDDS-115. GRPC: Support secure gRPC endpoint with mTLS. Contributed by Xiaoyu Yao. 2019-01-16 01:42:08 -07:00
Xiaoyu Yao
417951ab58 HDDS-884. Fix merge issue that causes NPE OzoneManager#httpServer. Contributed by Xiaoyu Yao. 2019-01-16 01:22:44 -07:00
Ajay Kumar
6d522dc05d HDDS-804. Block token: Add secret token manager. Contributed by Ajay Kumar. 2019-01-16 01:22:43 -07:00
Xiaoyu Yao
7e2770699c HDDS-9. Add GRPC protocol interceptors for Ozone Block Token. Contributed by Xiaoyu Yao. 2019-01-16 01:10:08 -07:00
Ajay Kumar
bb4a26ca32 HDDS-8. Add OzoneManager Delegation Token support. Contributed by Ajay Kumar. 2019-01-16 01:06:59 -07:00
Xiaoyu Yao
6d6b1a00c2 HDDS-836. Add TokenIdentifier Ozone for delegation token and block token. Contributed by Ajay Kumar. 2019-01-15 23:01:04 -07:00
Xiaoyu Yao
61e85d7cd1 HDDS-684. Fix HDDS-4 branch after HDDS-490 and HADOOP-15832. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Ajay Kumar
9920506b3d HDDS-547. Fix secure docker and configs. Contributed by Xiaoyu Yao. 2019-01-15 23:01:02 -07:00
Xiaoyu Yao
e47135d9d9 HDDS-70. Fix config names for secure ksm and scm. Contributed by Ajay Kumar. 2019-01-15 22:53:55 -07:00
Xiaoyu Yao
d3920ecbb7 HDDS-5. Enable OzoneManager kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:40:08 -07:00
Hanisha Koneru
a4eefe5765 HDDS-947. Implement OzoneManager State Machine. 2019-01-11 10:06:05 -08:00
Bharat Viswanadham
2091d1a4af HDDS-941. Rename ChunkGroupInputStream to keyInputStream and ChunkInputStream to BlockInputStream. Contributed by Shashikant Banerjee. 2019-01-09 20:02:36 -08:00
Hanisha Koneru
f4617c6239 Revert "HDDS-947. Implement OzoneManager State Machine." 
This reverts commit 8dd11a1577.
 2019-01-09 11:20:57 -08:00
Hanisha Koneru
8dd11a1577 HDDS-947. Implement OzoneManager State Machine. 2019-01-09 10:57:24 -08:00
Márton Elek
32cf0412ce HDDS-965. Ozone: checkstyle improvements and code quality scripts. Contributed by Elek, Marton. 2019-01-08 22:27:19 +01:00
Márton Elek
999f31fd20 HDDS-924. MultipartUpload: S3 APi for complete Multipart Upload. Contributed by Bharat Viswanadham. 2019-01-08 15:48:48 +01:00
Márton Elek
695e93c2bb HDDS-969. Fix TestOzoneManagerRatisServer test failure. Contributed by Bharat Viswanadham. 2019-01-08 15:12:08 +01:00
Bharat Viswanadham
cdfbec47ce HDDS-930. Multipart Upload: Abort multiupload request. Contributed by Bharat Viswanadham. 2019-01-07 11:09:14 -08:00
Márton Elek
d14c56d150 HDDS-916. MultipartUpload: Complete Multipart upload request. Contributed by Bharat Viswanadham. 2019-01-07 11:27:36 +01:00
Anu Engineer
999da98d67 HDDS-915. Submit client request to OM Ratis server. 
Contributed by Hanisha Koneru.
 2019-01-04 15:25:26 -08:00
Anu Engineer
14d232c0fe HDDS-957. Replace incorrect use of system property user.name. 
Contributed by Dinesh Chitlangia.
 2019-01-03 09:43:26 -08:00
Bharat Viswanadham
26e4be7022 HDDS-921. Add JVM pause monitor to Ozone Daemons (OM, SCM and Datanodes). Contributed by Bharat Viswanadham. 2018-12-23 17:33:25 -08:00
Bharat Viswanadham
13d3f99b37 HDDS-881.009. Encapsulate all client to OM requests into one request message. Contributed by Hanisha Koneru. 2018-12-20 09:12:42 -08:00
Arpit Agarwal
b1ce9aa3b3 HDDS-940. Remove dead store to local variable in OmMetadataManagerImpl. Contributed by Dinesh Chitlangia. 2018-12-19 12:30:40 +05:30
Shashikant Banerjee
b51e9e4317 HDDS-912. Update ozone to latest ratis snapshot build (0.4.0-3b0be02-SNAPSHOT). Contributed by Lokesh Jain. 2018-12-18 18:21:17 +05:30
Shashikant Banerjee
4ff1c46d95 HDDS-925. Rename ChunkGroupOutputStream to KeyOutputStream and ChunkOutputStream to BlockOutputStream. Contributed by Shashikant Banerjee. 2018-12-18 18:03:46 +05:30
Anu Engineer
5426653819 HDDS-908: NPE in TestOzoneRpcClient. 
Contributed by Ajay Kumar.
 2018-12-17 12:18:47 -08:00
Márton Elek
d44b37d7ac HDDS-889. MultipartUpload: Support uploading a part file in ozone. Contributed by Bharat Viswanadham. 2018-12-12 14:16:21 +01:00
Márton Elek
17a8708039 HDDS-879. MultipartUpload: Add InitiateMultipartUpload in ozone. Contributed by Bharat Viswanadham. 2018-12-10 10:05:38 +01:00
Jitendra Pandey
1afba83f2c HDDS-870. Avoid creating block sized buffer in ChunkGroupOutputStream. Contributed by Shashikant Banerjee. 2018-12-08 09:23:10 -08:00
Anu Engineer
5e773efd78 HDDS-909 : Default implementation for Ozone acls. 
Contributed by Ajay Kumar.
 2018-12-07 23:20:19 -08:00
Hanisha Koneru
019836b113 HDDS-858. Start a Standalone Ratis Server on OM 2018-12-06 13:33:52 -08:00
Bharat Viswanadham
343aaea2d1 HDDS-864. Use strongly typed codec implementations for the tables of the OmMetadataManager. Contributed by Elek Marton. 2018-12-06 11:39:59 -08:00
Bharat Viswanadham
d15dc43659 HDDS-748. Use strongly typed metadata Table implementation. Contributed by Elek Marton. 2018-12-01 16:52:23 -08:00
Márton Elek
96c104d365 HDDS-766. Ozone shell create volume fails if volume name does not have a leading slash. Contributed by chencan. 2018-11-27 12:32:54 +01:00