big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,577 Commits 2 Branches 2 Tags 264 MiB
846848ac4c
Commit Graph

275 Commits

Author SHA1 Message Date
Ajay Kumar
0faa5701d9 HDDS-964. Fix test failure in TestOmMetrics. Contributed by Ajay Kumar. 2019-01-16 16:40:04 -07:00
Anu Engineer
8978466fca HDDS-945. Fix generics warnings in delegation token. 
Contributed by Ajay Kumar.
 2019-01-16 16:23:26 -07:00
Anu Engineer
1d5734e341 HDDS-938. Add Client APIs for using S3 Auth interface. 
Contributed by  Dinesh Chitlangia.
 2019-01-16 16:18:39 -07:00
Bharat Viswanadham
ddaef67181 HDDS-937. Create an S3 Auth Table. Contributed by Dinesh Chitlangia. 2019-01-16 13:43:30 -07:00
Ajay Kumar
50c4045fde HDDS-805. Block token: Client api changes for block token. Contributed by Ajay Kumar. 2019-01-16 13:38:27 -07:00
Ajay Kumar
f894d86b2f HDDS-115. GRPC: Support secure gRPC endpoint with mTLS. Contributed by Xiaoyu Yao. 2019-01-16 01:42:08 -07:00
Xiaoyu Yao
417951ab58 HDDS-884. Fix merge issue that causes NPE OzoneManager#httpServer. Contributed by Xiaoyu Yao. 2019-01-16 01:22:44 -07:00
Ajay Kumar
6d522dc05d HDDS-804. Block token: Add secret token manager. Contributed by Ajay Kumar. 2019-01-16 01:22:43 -07:00
Xiaoyu Yao
7e2770699c HDDS-9. Add GRPC protocol interceptors for Ozone Block Token. Contributed by Xiaoyu Yao. 2019-01-16 01:10:08 -07:00
Ajay Kumar
bb4a26ca32 HDDS-8. Add OzoneManager Delegation Token support. Contributed by Ajay Kumar. 2019-01-16 01:06:59 -07:00
Xiaoyu Yao
6d6b1a00c2 HDDS-836. Add TokenIdentifier Ozone for delegation token and block token. Contributed by Ajay Kumar. 2019-01-15 23:01:04 -07:00
Xiaoyu Yao
61e85d7cd1 HDDS-684. Fix HDDS-4 branch after HDDS-490 and HADOOP-15832. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Ajay Kumar
9920506b3d HDDS-547. Fix secure docker and configs. Contributed by Xiaoyu Yao. 2019-01-15 23:01:02 -07:00
Xiaoyu Yao
e47135d9d9 HDDS-70. Fix config names for secure ksm and scm. Contributed by Ajay Kumar. 2019-01-15 22:53:55 -07:00
Xiaoyu Yao
d3920ecbb7 HDDS-5. Enable OzoneManager kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:40:08 -07:00
Hanisha Koneru
a4eefe5765 HDDS-947. Implement OzoneManager State Machine. 2019-01-11 10:06:05 -08:00
Bharat Viswanadham
2091d1a4af HDDS-941. Rename ChunkGroupInputStream to keyInputStream and ChunkInputStream to BlockInputStream. Contributed by Shashikant Banerjee. 2019-01-09 20:02:36 -08:00
Hanisha Koneru
f4617c6239 Revert "HDDS-947. Implement OzoneManager State Machine." 
This reverts commit 8dd11a1577.
 2019-01-09 11:20:57 -08:00
Hanisha Koneru
8dd11a1577 HDDS-947. Implement OzoneManager State Machine. 2019-01-09 10:57:24 -08:00
Márton Elek
32cf0412ce HDDS-965. Ozone: checkstyle improvements and code quality scripts. Contributed by Elek, Marton. 2019-01-08 22:27:19 +01:00
Márton Elek
999f31fd20 HDDS-924. MultipartUpload: S3 APi for complete Multipart Upload. Contributed by Bharat Viswanadham. 2019-01-08 15:48:48 +01:00
Márton Elek
695e93c2bb HDDS-969. Fix TestOzoneManagerRatisServer test failure. Contributed by Bharat Viswanadham. 2019-01-08 15:12:08 +01:00
Bharat Viswanadham
cdfbec47ce HDDS-930. Multipart Upload: Abort multiupload request. Contributed by Bharat Viswanadham. 2019-01-07 11:09:14 -08:00
Márton Elek
d14c56d150 HDDS-916. MultipartUpload: Complete Multipart upload request. Contributed by Bharat Viswanadham. 2019-01-07 11:27:36 +01:00
Anu Engineer
999da98d67 HDDS-915. Submit client request to OM Ratis server. 
Contributed by Hanisha Koneru.
 2019-01-04 15:25:26 -08:00
Anu Engineer
14d232c0fe HDDS-957. Replace incorrect use of system property user.name. 
Contributed by Dinesh Chitlangia.
 2019-01-03 09:43:26 -08:00
Bharat Viswanadham
26e4be7022 HDDS-921. Add JVM pause monitor to Ozone Daemons (OM, SCM and Datanodes). Contributed by Bharat Viswanadham. 2018-12-23 17:33:25 -08:00
Bharat Viswanadham
13d3f99b37 HDDS-881.009. Encapsulate all client to OM requests into one request message. Contributed by Hanisha Koneru. 2018-12-20 09:12:42 -08:00
Arpit Agarwal
b1ce9aa3b3 HDDS-940. Remove dead store to local variable in OmMetadataManagerImpl. Contributed by Dinesh Chitlangia. 2018-12-19 12:30:40 +05:30
Shashikant Banerjee
b51e9e4317 HDDS-912. Update ozone to latest ratis snapshot build (0.4.0-3b0be02-SNAPSHOT). Contributed by Lokesh Jain. 2018-12-18 18:21:17 +05:30
Shashikant Banerjee
4ff1c46d95 HDDS-925. Rename ChunkGroupOutputStream to KeyOutputStream and ChunkOutputStream to BlockOutputStream. Contributed by Shashikant Banerjee. 2018-12-18 18:03:46 +05:30
Anu Engineer
5426653819 HDDS-908: NPE in TestOzoneRpcClient. 
Contributed by Ajay Kumar.
 2018-12-17 12:18:47 -08:00
Márton Elek
d44b37d7ac HDDS-889. MultipartUpload: Support uploading a part file in ozone. Contributed by Bharat Viswanadham. 2018-12-12 14:16:21 +01:00
Márton Elek
17a8708039 HDDS-879. MultipartUpload: Add InitiateMultipartUpload in ozone. Contributed by Bharat Viswanadham. 2018-12-10 10:05:38 +01:00
Jitendra Pandey
1afba83f2c HDDS-870. Avoid creating block sized buffer in ChunkGroupOutputStream. Contributed by Shashikant Banerjee. 2018-12-08 09:23:10 -08:00
Anu Engineer
5e773efd78 HDDS-909 : Default implementation for Ozone acls. 
Contributed by Ajay Kumar.
 2018-12-07 23:20:19 -08:00
Hanisha Koneru
019836b113 HDDS-858. Start a Standalone Ratis Server on OM 2018-12-06 13:33:52 -08:00
Bharat Viswanadham
343aaea2d1 HDDS-864. Use strongly typed codec implementations for the tables of the OmMetadataManager. Contributed by Elek Marton. 2018-12-06 11:39:59 -08:00
Bharat Viswanadham
d15dc43659 HDDS-748. Use strongly typed metadata Table implementation. Contributed by Elek Marton. 2018-12-01 16:52:23 -08:00
Márton Elek
96c104d365 HDDS-766. Ozone shell create volume fails if volume name does not have a leading slash. Contributed by chencan. 2018-11-27 12:32:54 +01:00
Ajay Kumar
8d2789c5eb HDDS-795. RocksDb specific classes leak from DBStore/Table interfaces. Contributed by Márton Elek. 2018-11-21 12:25:41 -08:00
Bharat Viswanadham
c4d0ef6946 HDDS-816. Create OM metrics for bucket, volume, keys. Contributed by Bharat Viswanadham. 2018-11-21 10:35:39 -08:00
Shashikant Banerjee
14e1a0a3d6 HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee. 2018-11-21 10:51:50 +05:30
Xiaoyu Yao
f994b526a0 HDDS-855. Move OMMetadataManager from hadoop-ozone/ozone-manager to hadoop-ozone/common. Contributed by Ajay Kumar. 2018-11-20 20:31:07 -08:00
Shashikant Banerjee
1734ace35f Revert "HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee." 
This reverts commit c74783036d.
 2018-11-21 01:07:38 +05:30
Shashikant Banerjee
c74783036d HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee. 2018-11-20 23:01:02 +05:30
Anu Engineer
a16aa2f60b HDDS-825. Code cleanup based on messages from ErrorProne. 
Contributed by Anu Engineer.
 2018-11-15 17:36:09 -08:00
Márton Elek
b12e69475b HDDS-658. Implement s3 bucket list backend call and use it from rest endpoint. Contributed by Bharat Viswanadham. 2018-11-15 12:22:16 +01:00
Nanda kumar
9fe50b4991 HDDS-733. Create container if not exist, as part of chunk write. 
Contributed by Lokesh Jain.
 2018-11-10 00:08:17 +05:30
Arpit Agarwal
08bb0362e0 HDDS-759. Create config settings for SCM and OM DB directories. Contributed by Arpit Agarwal. 2018-10-31 11:23:15 -07:00
Shashikant Banerjee
7757331dbc HDDS-749. Restructure BlockId class in Ozone. Contributed by Shashikant Banerjee. 2018-10-30 14:15:27 +05:30
Nanda kumar
dce4ebe814 HDDS-694. Plugin new Pipeline management code in SCM. 
Contributed by Lokesh Jain.
 2018-10-26 17:55:04 +05:30
Hanisha Koneru
38a65e3b7c HDDS-682. Unified o3 address parsing for ozonen sh. Contributed by Elek, Marton. 2018-10-25 17:04:16 -07:00
Anu Engineer
c9077a9f5a HDDS-707. Allow registering MBeans without additional jmx properties. 
Contributed by Arpit Agarwal.
 2018-10-21 22:50:28 -07:00
Anu Engineer
f8d61b9fc2 HDDS-690. Javadoc build fails in hadoop-ozone. 
Contributed by Takanobu Asanuma.
 2018-10-18 12:14:54 -07:00
Arpit Agarwal
c64018026e HDDS-673. Suppress "Key not found" exception log with stack trace in OM. Contributed by Arpit Agarwal. 2018-10-18 11:51:16 -07:00
Bharat Viswanadham
4c4364ddd0 HDDS-683. Add a shell command to provide ozone mapping for a S3Bucket. Contributed by Bharat Viswanadham. 2018-10-17 16:46:13 -07:00
Jitendra Pandey
d93d515af5 HDDS-651. Rename o3 to o3fs for Filesystem. 2018-10-17 14:19:17 -07:00
Nandakumar
50715c0699 HDDS-662. Introduce ContainerReplicaState in StorageContainerManager. Contributed by Nanda kumar. 2018-10-17 17:45:35 +05:30
Arpit Agarwal
5f0b43fa93 HDDS-378. Remove dependencies between hdds/ozone and hdfs proto files. Contributed by Elek, Marton. 2018-10-16 10:40:00 -07:00
Arpit Agarwal
0bf8a110a5 HDDS-439. 'ozone oz volume create' should default to current logged in user. Contributed by Dinesh Chitlangia. 2018-10-15 19:30:40 -07:00
Arpit Agarwal
2614078b21 HDDS-490. Improve om and scm start up options . Contributed by Namit Maheshwari. 2018-10-15 16:45:08 -07:00
Arpit Agarwal
ddc9649328 HDDS-646. TestChunkStreams.testErrorReadGroupInputStream fails. Contributed by Nanda kumar. 2018-10-12 15:06:42 -07:00
Bharat Viswanadham
8ae8a5004f HDDS-606. Create delete s3Bucket. Contributed by Bharat Viswanadham. 2018-10-12 13:58:53 -07:00
Anu Engineer
4de2dc2699 HDDS-568. ozone sh volume info, update, delete operations fail when volume name is not prefixed by /. 
Contributed by Dinesh Chitlangia.
 2018-10-09 17:32:04 -07:00
Anu Engineer
5b7ba48ced HDDS-577. Support S3 buckets as first class objects in Ozone Manager - 2. 
Contributed by Bharat Viswanadham.
 2018-10-09 13:37:42 -07:00
Nanda kumar
7367ff333b HDDS-450. Generate BlockCommitSequenceId in ContainerStateMachine for every commit operation in Ratis. Contributed by Shashikant Banerjee. 2018-10-09 18:07:01 +05:30
Anu Engineer
e6b77ad65f HDDS-572. Support S3 buckets as first class objects in Ozone Manager - 1. 
Contributed by Anu Engineer.
 2018-10-04 21:40:13 -07:00
Anu Engineer
0da03f8b14 HDDS-391. Simplify Audit Framework to make audit logging easier to use. 
Contributed by Dinesh Chitlangia.
 2018-09-28 14:00:24 -07:00
Hanisha Koneru
72891fc9be HDDS-289. Volume and Bucket name should not contain a delimiter ('/'). Contributed by chencan. 2018-09-28 11:24:42 -07:00
Bharat Viswanadham
9c3fbbc4f6 HDDS-444. Add rest service to the s3gateway. Contributed by Elek Marton. 2018-09-24 22:10:16 -07:00
Bharat Viswanadham
62f817d32e HDDS-447. Separate ozone-dist and hadoop-dist projects with real classpath separation. Contributed by Elek Marton. 2018-09-24 10:10:11 -07:00
Anu Engineer
096a716080 HDDS-394. Rename *Key Apis in DatanodeContainerProtocol to *Block apis. 
Contributed Dinesh Chitlangia.
 2018-09-20 11:51:49 -07:00
Nanda kumar
15ed74fa24 HDDS-502. Exception in OM startup when running unit tests. Contributed by Arpit Agarwal. 2018-09-19 19:46:25 +05:30
Xiaoyu Yao
3929653707 HDDS-488. Handle chill mode exception from SCM in OzoneManager. Contributed by Ajay Kumar. 2018-09-18 20:05:25 -07:00
Arpit Agarwal
82fbbd546c HDDS-469. Rename 'ozone oz' to 'ozone sh'. Contributed by Arpit Agarwal. 2018-09-15 22:12:47 -07:00
Nanda kumar
5470de420b HDDS-454. TestChunkStreams#testErrorReadGroupInputStream & TestChunkStreams#testReadGroupInputStream are failing. Contributed by chencan. 2018-09-15 01:38:57 +05:30
Arpit Agarwal
a12f12f1af HDDS-438. 'ozone oz' should print usage when command or sub-command is missing. Contributed by Dinesh Chitlangia. 2018-09-14 07:02:28 -07:00
Arpit Agarwal
76a0fdfed9 HDDS-456. TestOzoneShell#init is breaking due to Null Pointer Exception. Contributed by Dinesh Chitlangia. 2018-09-13 13:00:35 -07:00
Márton Elek
e084627150 HDDS-415. 'ozone om' with incorrect argument first logs all the STARTUP_MSG. Contributed by Namit Maheshwari. 2018-09-13 14:01:53 +02:00
Anu Engineer
6e2129cf4e HDDS-428. OzoneManager lock optimization. 
Contributed by Nanda Kumar.
 2018-09-12 10:38:36 -07:00
Anu Engineer
a406f6f60e HDDS-424. Consolidate ozone oz parameters to use GNU convention. 
Contributed by Elek, Marton.
 2018-09-11 16:47:33 -07:00
Xiaoyu Yao
8fe4062bfc HDDS-403. Fix createdOn and modifiedOn timestamp for volume, bucket, key. Contributed by Dinesh Chitlangia. 2018-09-10 08:45:49 -07:00
Anu Engineer
73fcbdd296 HDDS-190. Improve shell error message for unrecognized option. 
Contributed by Elek, Marton.
 2018-09-07 12:54:32 -07:00
Xiaoyu Yao
396ce7b884 HDDS-397. Handle deletion for keys with no blocks. Contributed by Lokesh Jain. 2018-09-06 16:58:15 -07:00
Anu Engineer
fa2945e7a9 HDDS-405. User/volume mapping is not cleaned up during the deletion of the last volume. 
Contributed by Elek, Marton.
 2018-09-06 12:44:08 -07:00
Nanda kumar
df0d61e3a0 HDDS-358. Use DBStore and TableStore for DeleteKeyService. Contributed by Anu Engineer 2018-09-05 18:35:33 +05:30
Márton Elek
c7403a448d HDDS-315. ozoneShell infoKey does not work for directories created as key and throws 'KEY_NOT_FOUND' error. Contributed by Dinesh Chitlangia. 2018-09-05 12:41:06 +02:00
Nanda kumar
6bbd249011 HDDS-98. Adding Ozone Manager Audit Log. Contributed by Dinesh Chitlangia. 2018-09-05 00:11:07 +05:30
Márton Elek
211034a6c2 HDDS-336. Print out container location information for a specific ozone key . Contributed by LiXin Ge. 2018-09-03 13:32:55 +02:00
Anu Engineer
ff036e49ff HDDS-357. Use DBStore and TableStore for OzoneManager non-background service. 
Contributed by Nandakumar.
 2018-09-02 11:47:32 -07:00
Nanda kumar
6edf3d2ea3 Revert "HDDS-98. Adding Ozone Manager Audit Log." 
This reverts commit 630b64ec7e.
 2018-09-02 00:18:13 +05:30
Anu Engineer
630b64ec7e HDDS-98. Adding Ozone Manager Audit Log. 
Contributed by Dinesh Chitlangia.
 2018-08-31 14:20:56 -07:00
Anu Engineer
c5629d546d HDDS-382. Remove RatisTestHelper#RatisTestSuite constructor argument and fix checkstyle in ContainerTestHelper, GenericTestUtils 
Contributed by Nandakumar.
 2018-08-28 14:06:19 -07:00
Anu Engineer
ab37423ad8 HDDS-355. Disable OpenKeyDeleteService and DeleteKeysService. 
Contributed by Anu Engineer.
 2018-08-17 11:50:46 -07:00
Anu Engineer
11daa010d2 HDDS-346. ozoneShell show the new volume info after updateVolume command like updateBucket command. 
Contributed by chencan.
 2018-08-13 10:08:58 -07:00
Mukul Kumar Singh
900c0e114f HDDS-230. ContainerStateMachine should implement readStateMachineData api to read data from Containers if required during replication. Contributed by Mukul Kumar Singh. 2018-08-07 15:03:14 +05:30
Nanda kumar
e83719c830 HDDS-290. putKey is failing with KEY_ALLOCATION_ERROR. Contributed by Xiaoyu Yao. 2018-08-02 19:02:25 +05:30
Mukul Kumar Singh
f4db753bb6 HDDS-226. Client should update block length in OM while committing the key. Contributed by Shashikant Banerjee. 2018-08-01 09:03:00 +05:30
Xiaoyu Yao
c7ae55675e HDDS-275. Add message output for succeeded -deleteVolume CLI. Contributed by Nilotpal Nandi. 2018-07-20 13:12:53 -07:00
Xiaoyu Yao
129269f989 HDDS-207. ozone listVolume command accepts random values as argument. Contributed by Lokesh Jain. 2018-07-18 11:05:42 -07:00
Xiaoyu Yao
103f2eeb57 HDDS-210. Make "-file" argument optional for ozone getKey command. Contributed by Lokesh Jain. 2018-07-13 11:45:02 -07:00
Xiaoyu Yao
4e59b92784 HDDS-208. ozone createVolume command ignores the first character of the volume name argument. Contributed by Lokesh Jain. 2018-07-10 14:07:23 -07:00
Arpit Agarwal
061b168529 HDDS-167. Rename KeySpaceManager to OzoneManager. Contributed by Arpit Agarwal. 2018-07-06 12:09:05 -07:00
Nanda kumar
71df8c27c9 HDDS-212. Introduce NodeStateManager to manage the state of Datanodes in SCM. Contributed by Nanda kumar. 2018-07-05 02:11:10 +05:30
Anu Engineer
308a1591f9 HDDS-172. The numbers of operation should be integer in KSM UI. 
Contributed by Takanobu Asanuma.
 2018-06-15 14:10:17 -07:00
Jitendra Pandey
f4c7c91123 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-13 00:36:02 -07:00
Xiaoyu Yao
23bfd9f7e4 HDDS-72. Add deleteTransactionId field in ContainerInfo. Contributed by Lokesh Jain. 2018-06-11 16:02:32 -07:00
Anu Engineer
a1272448bf HDDS-157. Upgrade common-langs version to 3.7 in HDDS and Ozone. 
Contributed by Takanobu Asanuma.
 2018-06-08 10:27:01 -07:00
Anu Engineer
950dea86f4 HDDS-112. OzoneShell should support commands with url without scheme. 
Contributed by Lokesh Jain.
 2018-05-31 12:05:50 -07:00
Anu Engineer
3b34148c4f HDDS-88. Create separate message structure to represent ports in DatanodeDetails. 
Contributed by Nanda Kumar.
 2018-05-30 08:52:07 -07:00
Anu Engineer
481bfdb94f HDDS-89. Create ozone specific inline documentation as part of the build. 
Contributed by Elek, Marton.
 2018-05-22 10:49:10 -07:00
Xiaoyu Yao
41ae5c5002 HDDS-77. Key replication factor and type should be stored per key by Ozone Manager. Contributed by Mukul Kumar Singh. 2018-05-17 08:31:44 -07:00
Anu Engineer
46e0f27862 HDDS-18. Ozone Shell should use RestClient and RpcClient. Contributed by Lokesh Jain. 2018-05-09 23:00:13 -07:00
Mukul Kumar Singh
208b97e969 HDDS-20. Ozone: Add support for rename key within a bucket for rpc client. Contributed by Lokesh Jain. 2018-05-09 19:06:07 +05:30
Anu Engineer
3a43ac2851 HDDS-1. Remove SCM Block DB. Contributed by Xiaoyu Yao. 2018-05-07 14:58:52 -07:00
Mukul Kumar Singh
51459d4351 HDFS-13414. Ozone: Update existing Ozone documentation according to the recent changes. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
3779fadc5c HDFS-13324. Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
82eb5da2b6 HDFS-13342. Ozone: Rename and fix ozone CLI scripts. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
8b832f3c35 HDFS-13405. Ozone: Rename HDSL to HDDS. 
Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.
 2018-04-26 05:36:04 -07:00
Nanda kumar
43f7307829 HDFS-13316. Ozone: Update the ksm/scm CLI usage info. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
4f37ff7616 HDFS-13341. Ozone: Move ozone specific ServiceRuntimeInfo utility to the framework. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
498b84155f HDFS-13300. Ozone: Remove DatanodeID dependency from HDSL and Ozone. 
Contributed by Nanda kumar.
 2018-04-26 05:36:04 -07:00
Anu Engineer
1f9a7ae12f HDFS-13304. Document: update the new ozone docker file location. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00