big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,951 Commits 2 Branches 2 Tags 264 MiB
ccc43621ab
Commit Graph

449 Commits

Author SHA1 Message Date
Nanda kumar
095c269620 HDDS-567. Rename Mapping to ContainerManager in SCM. Contributed by Nanda kumar. 2018-10-03 16:00:39 +05:30
Nanda kumar
7b374482d2 HDDS-526. Clean previous chill mode code from NodeManager. Contributed by Ajay Kumar. 2018-10-03 15:15:43 +05:30
Bharat Viswanadham
ec075791da HDDS-520. Implement HeadBucket REST endpoint. Contributed by Bharat Viswanadham. 2018-10-02 20:13:59 -07:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Nanda kumar
a39296260f HDDS-561. Move Node2ContainerMap and Node2PipelineMap to NodeManager. Contributed by Lokesh Jain. 2018-10-02 19:47:44 +05:30
Márton Elek
81072d5e3d HDDS-560. Create Generic exception class to be used by S3 rest services. Contributed by Bharat Viswanadham. 2018-10-02 10:00:10 -04:00
Bharat Viswanadham
7d082193d2 HDDS-562. Create acceptance test to test aws cli with the s3 gateway. Contributed by Elek Marton. 2018-10-01 15:04:20 -07:00
Bharat Viswanadham
4eff629ab3 HDDS-525. Support virtual-hosted style URLs. Contributed by Bharat Viswanadham 2018-10-01 13:16:08 -07:00
Nanda kumar
f7ff8c051e HDDS-325. Add event watcher for delete blocks command. Contributed by Lokesh Jain. 2018-10-01 13:50:17 +05:30
Anu Engineer
0da03f8b14 HDDS-391. Simplify Audit Framework to make audit logging easier to use. 
Contributed by Dinesh Chitlangia.
 2018-09-28 14:00:24 -07:00
Hanisha Koneru
35c7351f4b HDDS-455. Ozone genconf tool must use picocli. Contributed by Dinesh Chitlangia. 2018-09-28 13:06:50 -07:00
Hanisha Koneru
72891fc9be HDDS-289. Volume and Bucket name should not contain a delimiter ('/'). Contributed by chencan. 2018-09-28 11:24:42 -07:00
Ajay Kumar
3928af983e HDDS-370. Add and implement following functions in SCMClientProtocolServer. Contributed by Ajay Kumar. 2018-09-25 13:08:44 -07:00
Bharat Viswanadham
9c3fbbc4f6 HDDS-444. Add rest service to the s3gateway. Contributed by Elek Marton. 2018-09-24 22:10:16 -07:00
Bharat Viswanadham
67173b4847 HDDS-441. Create new s3gateway daemon. Contributed by Elek Marton. 2018-09-24 13:42:55 -07:00
Bharat Viswanadham
62f817d32e HDDS-447. Separate ozone-dist and hadoop-dist projects with real classpath separation. Contributed by Elek Marton. 2018-09-24 10:10:11 -07:00
Anu Engineer
096a716080 HDDS-394. Rename *Key Apis in DatanodeContainerProtocol to *Block apis. 
Contributed Dinesh Chitlangia.
 2018-09-20 11:51:49 -07:00
Anu Engineer
98c9bc4aa0 HDDS-509. TestStorageContainerManager is flaky. 
Contributed by Xiaoyu Yao and Ajay Kumar.
 2018-09-19 13:03:19 -07:00
Nanda kumar
61a4b07bda HDDS-461. Container remains in CLOSING state in SCM forever. Contributed by Shashikant Banerjee. 2018-09-19 21:43:44 +05:30
Nanda kumar
15ed74fa24 HDDS-502. Exception in OM startup when running unit tests. Contributed by Arpit Agarwal. 2018-09-19 19:46:25 +05:30
Nanda kumar
c0956ee2a8 HDDS-476. Add Pipeline reports to make pipeline active on SCM restart. 
Contributed by Mukul Kumar Singh.
 2018-09-19 18:52:08 +05:30
Arpit Agarwal
fb85351dc6 HDDS-497. Suppress license warnings for error log files. Contributed by Arpit Agarwal. 2018-09-18 21:18:15 -07:00
Xiaoyu Yao
3929653707 HDDS-488. Handle chill mode exception from SCM in OzoneManager. Contributed by Ajay Kumar. 2018-09-18 20:05:25 -07:00
Arpit Agarwal
17f5651a51 HDDS-503. Suppress ShellBasedUnixGroupsMapping exception in tests. Contributed by Arpit Agarwal. 2018-09-18 16:06:21 -07:00
Arpit Agarwal
7ff00f5587 HDDS-500. TestErrorCode.java has wrong package name. Contributed by Anu Engineer. 2018-09-18 15:52:00 -07:00
Arpit Agarwal
f176e8a3ae HDDS-501. AllocateBlockResponse.keyLocation must be an optional field. Contributed by Arpit Agarwal. 2018-09-18 15:41:28 -07:00
Bharat Viswanadham
5c2ae7e493 HDDS-496. Ozone tools module is incorrectly classified as 'hdds' component. Contributed by Dinesh Chitlangia. 2018-09-18 14:51:03 -07:00
Bharat Viswanadham
5896372761 HDDS-495. Ozone docs and ozonefs packages have undefined hadoop component. Contributed by Dinesh Chitlangia. 2018-09-18 13:50:39 -07:00
Xiaoyu Yao
295cce39ed HDDS-366. Update functions impacted by SCM chill mode in StorageContainerLocationProtocol. Contributed by Ajay Kumar. 2018-09-18 13:32:48 -07:00
Anu Engineer
f938925bde HDDS-464. Fix TestCloseContainerHandlingByClient. 
Contributed by Lokesh Jain.
 2018-09-18 10:45:51 -07:00
Anu Engineer
48319d6eee HDDS-483. Update ozone Documentation to fix below issues. 
Contributed by  Namit Maheshwari.
 2018-09-18 10:23:36 -07:00
Márton Elek
001611cca3 HDDS-481. Classes are missing from the shaded ozonefs jar. Contributed by Bharat Viswanadham. 
(cherry picked from commit e043783917ed58a1a967ca67b7240b1b3e846a69)
 2018-09-18 10:32:36 +02:00
Bharat Viswanadham
8b2f5e60fa HDDS-352. Separate install and testing phases in acceptance tests. Contributed by Elek Marton. 2018-09-17 17:13:52 -07:00
Arpit Agarwal
0a26c521f0 HDDS-487. Doc files are missing ASF license headers. Contributed by Namit Maheshwari. 2018-09-17 16:21:10 -07:00
Anu Engineer
8af8453589 HDDS-435. Enhance the existing ozone documentation. 
Contributed by Elek, Marton.
 2018-09-17 10:46:28 -07:00
Nanda kumar
846936698b HDDS-399. Persist open pipeline information across SCM restart. Contributed by Mukul Kumar Singh. 2018-09-17 21:51:54 +05:30
Arpit Agarwal
07385f886e HDDS-470. Ozone acceptance tests are failing. Contributed by Elek, Marton. 
(cherry picked from commit dca8d0c2615d142bca55d367a0bc988ce9860368)
 2018-09-16 14:31:29 -07:00
Arpit Agarwal
82fbbd546c HDDS-469. Rename 'ozone oz' to 'ozone sh'. Contributed by Arpit Agarwal. 2018-09-15 22:12:47 -07:00
Nanda kumar
c9fa081897 HDDS-462. Optimize ContainerStateMap#getMatchingContainerIDs in SCM. Contributed by Nanda kumar. 2018-09-15 23:11:39 +05:30
Márton Elek
985f3bf3fb HDDS-409. Ozone acceptance-test and integration-test packages have undefined hadoop component. Contributed by Dinesh Chitlangia. 2018-09-15 13:21:32 +02:00
Bharat Viswanadham
b95aa56062 HDDS-446. Provide shaded artifact to start ozone service as a datanode plugin. Contributed by Elek Marton 2018-09-14 14:12:03 -07:00
Xiaoyu Yao
6f037468bc HDDS-419. ChunkInputStream bulk read api does not read from all the chunks. Contributed by Lokesh Jain and Mukul Kumar. 2018-09-14 13:45:38 -07:00
Nanda kumar
5470de420b HDDS-454. TestChunkStreams#testErrorReadGroupInputStream & TestChunkStreams#testReadGroupInputStream are failing. Contributed by chencan. 2018-09-15 01:38:57 +05:30
Nanda kumar
c1df3084ff HDDS-389. Remove XceiverServer and XceiverClient and related classes. Contributed by chencan. 2018-09-15 00:20:19 +05:30
Arpit Agarwal
a12f12f1af HDDS-438. 'ozone oz' should print usage when command or sub-command is missing. Contributed by Dinesh Chitlangia. 2018-09-14 07:02:28 -07:00
Hanisha Koneru
291dcf2247 HDDS-423. Introduce an ozone specific log4j.properties. Contributed by Elek, Marton. 2018-09-13 16:37:44 -07:00
Hanisha Koneru
4441fe9201 HDDS-414. Fix sbin/stop-ozone.sh to stop Ozone daemons. Contributed by Elek, Marton. 2018-09-13 13:34:22 -07:00
Arpit Agarwal
76a0fdfed9 HDDS-456. TestOzoneShell#init is breaking due to Null Pointer Exception. Contributed by Dinesh Chitlangia. 2018-09-13 13:00:35 -07:00
Mukul Kumar Singh
c9e0b69ab3 HDDS-233. Update ozone to latest ratis snapshot build(0.3.0-50588bd-SNAPSHOT). Contributed by Shashikant Banerjee. 2018-09-13 19:01:07 +05:30
Márton Elek
e084627150 HDDS-415. 'ozone om' with incorrect argument first logs all the STARTUP_MSG. Contributed by Namit Maheshwari. 2018-09-13 14:01:53 +02:00
Anu Engineer
6e2129cf4e HDDS-428. OzoneManager lock optimization. 
Contributed by Nanda Kumar.
 2018-09-12 10:38:36 -07:00
Márton Elek
9011567169 HDDS-425. Move unit test of the genconf tool to hadoop-ozone/tools module. Contributed by Dinesh Chitlangia. 2018-09-12 11:11:12 +02:00
Anu Engineer
a406f6f60e HDDS-424. Consolidate ozone oz parameters to use GNU convention. 
Contributed by Elek, Marton.
 2018-09-11 16:47:33 -07:00
Xiaoyu Yao
1d567c25d0 HDDS-416. Remove currentPosition from ChunkInputStreamEntry. Contributed by Lokesh Jain. 2018-09-11 15:15:24 -07:00
Anu Engineer
7b5886bf78 HDDS-222. Remove hdfs command line from ozone distribution. 
Contributed by Elek, Marton.
 2018-09-11 10:19:50 -07:00
Xiaoyu Yao
8fe4062bfc HDDS-403. Fix createdOn and modifiedOn timestamp for volume, bucket, key. Contributed by Dinesh Chitlangia. 2018-09-10 08:45:49 -07:00
Márton Elek
585a4f96d7 HDDS-417. Ambiguous error message when using genconf tool. Contributed by Dinesh Chitlangia. 2018-09-10 13:37:48 +02:00
Anu Engineer
ca3adf588c HDDS-398. Support multiple tests in freon. 
Contributed by Elek, Marton.
 2018-09-07 13:27:15 -07:00
Anu Engineer
73fcbdd296 HDDS-190. Improve shell error message for unrecognized option. 
Contributed by Elek, Marton.
 2018-09-07 12:54:32 -07:00
Hanisha Koneru
ab90248b30 HDDS-400. Check global replication state for containers of dead node. Contributed by Elek, Marton. 2018-09-07 11:20:25 -07:00
Anu Engineer
ff64d35716 HDDS-351. Add chill mode state to SCM. 
Contributed by Ajay Kumar.
 2018-09-07 10:54:35 -07:00
Márton Elek
94ed5cffd7 HDDS-214. Bump ozone/hdds version after the release branch cut 2018-09-07 17:45:26 +02:00
Anu Engineer
be1ec005f1 HDDS-408. Read (getKey) operation is very slow. 
Contributed by Nandakumar.
 2018-09-07 07:36:24 -07:00
Xiaoyu Yao
396ce7b884 HDDS-397. Handle deletion for keys with no blocks. Contributed by Lokesh Jain. 2018-09-06 16:58:15 -07:00
Anu Engineer
c5bf43a8e8 HDDS-406. Enable acceptace test of the putKey for rpc protocol. 
Contributed by Elek, Marton.
 2018-09-06 15:09:21 -07:00
Tsz Wo Nicholas Sze
b3161c4dd9 HDDS-297. Add pipeline actions in Ozone. Contributed by Mukul Kumar Singh and Shashikant Banerjee 2018-09-06 14:35:07 -07:00
Anu Engineer
fa2945e7a9 HDDS-405. User/volume mapping is not cleaned up during the deletion of the last volume. 
Contributed by Elek, Marton.
 2018-09-06 12:44:08 -07:00
Márton Elek
b6c543fe45 HDDS-404. Implement toString() in OmKeyLocationInfo. Contributed by Dinesh Chitlangia. 2018-09-06 13:16:54 +02:00
Nanda kumar
df0d61e3a0 HDDS-358. Use DBStore and TableStore for DeleteKeyService. Contributed by Anu Engineer 2018-09-05 18:35:33 +05:30
Márton Elek
045270a679 HDDS-333. Create an Ozone Logo. Contributed by Priyanka Nagwekar. 2018-09-05 14:31:36 +02:00
Márton Elek
c7403a448d HDDS-315. ozoneShell infoKey does not work for directories created as key and throws 'KEY_NOT_FOUND' error. Contributed by Dinesh Chitlangia. 2018-09-05 12:41:06 +02:00
Tsz Wo Nicholas Sze
6883fe860f HDDS-383. Ozone Client should discard preallocated blocks from closed containers. Contributed by Shashikant Banerjee 2018-09-04 17:10:44 -07:00
Anu Engineer
6e4c731471 HDDS-396. Remove openContainers.db from SCM. 
Contributed by Dinesh Chitlangia.
 2018-09-04 16:27:31 -07:00
Nanda kumar
6bbd249011 HDDS-98. Adding Ozone Manager Audit Log. Contributed by Dinesh Chitlangia. 2018-09-05 00:11:07 +05:30
Nanda kumar
b9932162e9 HDDS-75. Support for CopyContainer. Contributed by Elek, Marton. 2018-09-04 23:41:50 +05:30
Márton Elek
211034a6c2 HDDS-336. Print out container location information for a specific ozone key . Contributed by LiXin Ge. 2018-09-03 13:32:55 +02:00
Mukul Kumar Singh
873ef8ae81 HDDS-263. Add retries in Ozone Client to handle BlockNotCommitted Exception. Contributed by Shashikant Banerjee. 2018-09-03 12:26:34 +05:30
Anu Engineer
ff036e49ff HDDS-357. Use DBStore and TableStore for OzoneManager non-background service. 
Contributed by Nandakumar.
 2018-09-02 11:47:32 -07:00
Nanda kumar
6edf3d2ea3 Revert "HDDS-98. Adding Ozone Manager Audit Log." 
This reverts commit 630b64ec7e.
 2018-09-02 00:18:13 +05:30
Anu Engineer
76bae4ccb1 HDDS-379. Simplify and improve the cli arg parsing of ozone scmcli. 
Contributed by Elek, Marton.
 2018-08-31 18:11:01 -07:00
Anu Engineer
630b64ec7e HDDS-98. Adding Ozone Manager Audit Log. 
Contributed by Dinesh Chitlangia.
 2018-08-31 14:20:56 -07:00
Mukul Kumar Singh
692736f7cf HDDS-280. Support ozone dist-start-stitching on openbsd/osx. Contributed by Elek, Marton. 2018-08-30 02:21:24 +05:30
Nanda kumar
0bd4217194 HDDS-380. Remove synchronization from ChunkGroupOutputStream and ChunkOutputStream. Contributed by Shashikant Banerjee. 2018-08-29 13:31:19 +05:30
Anu Engineer
c5629d546d HDDS-382. Remove RatisTestHelper#RatisTestSuite constructor argument and fix checkstyle in ContainerTestHelper, GenericTestUtils 
Contributed by Nandakumar.
 2018-08-28 14:06:19 -07:00
Anu Engineer
df21e1b1dd HDDS-332. Remove the ability to configure ozone.handler.type 
Contributed by Nandakumar and Anu Engineer.
 2018-08-28 09:56:02 -07:00
Nanda kumar
2172399c55 HDDS-381. Fix TestKeys#testPutAndGetKeyWithDnRestart. Contributed by Mukul Kumar Singh. 2018-08-28 22:19:52 +05:30
Mukul Kumar Singh
3974427f67 HDDS-247. Handle CLOSED_CONTAINER_IO exception in ozoneClient. Contributed by Shashikant Banerjee. 2018-08-28 07:12:07 +05:30
Márton Elek
a813fd0215 HDDS-227. Use Grpc as the default transport protocol for Standalone pipeline. Contributed by chencan. 2018-08-27 16:07:55 +02:00
Márton Elek
744ce200d2 HDDS-313. Add metrics to containerState Machine. Contributed by chencan. 2018-08-27 15:51:34 +02:00
Nanda kumar
12b2f362cc HDDS-374. Support to configure container size in units lesser than GB. Contributed by Nanda kumar. 2018-08-27 18:29:32 +05:30
Márton Elek
91836f0f81 HDDS-334. Update GettingStarted page to mention details about Ozone GenConf tool. Contributed by Dinesh Chitlangia. 2018-08-27 11:41:08 +02:00
Márton Elek
a5eba25506 HDDS-364. Update open container replica information in SCM during DN register. Contributed by Ajay Kumar. 2018-08-24 22:42:51 +02:00
Márton Elek
55b6931059 HDDS-317. Use new StorageSize API for reading ozone.scm.container.size.gb. Contributed by Junjie Chen. 2018-08-24 13:56:30 +02:00
Hanisha Koneru
5aa15cfaff HDDS-265. Move numPendingDeletionBlocks and deleteTransactionId from ContainerData to KeyValueContainerData. Contributed by LiXin Ge. 2018-08-22 10:04:15 -07:00
Mukul Kumar Singh
e3d73bbc24 HDDS-353. Multiple delete Blocks tests are failing consistently. Contributed by Lokesh Jain. 2018-08-20 13:37:58 +05:30
Anu Engineer
ab37423ad8 HDDS-355. Disable OpenKeyDeleteService and DeleteKeysService. 
Contributed by Anu Engineer.
 2018-08-17 11:50:46 -07:00
Xiaoyu Yao
9dd5d5ba71 HDDS-218. add existing docker-compose files to the ozone release artifact. Contributed by Elek Marton. 2018-08-17 07:58:24 -07:00
Xiaoyu Yao
2d13e410d8 HDDS-119. Skip Apache license header check for some ozone doc scripts. Contributed by Ajay Kumar. 2018-08-16 22:18:52 -07:00
Mukul Kumar Singh
5ef29087ad HDDS-179. CloseContainer/PutKey command should be syncronized with write operations. Contributed by Shashikant Banerjee. 2018-08-16 23:35:19 +05:30
Anu Engineer
d951af22b4 HADOOP-15552. Addendum patch to fix the build break in Ozone File system. 
Contributed by Anu Engineer.
 2018-08-15 10:58:29 -07:00
Anu Engineer
c918d88a9b Revert "HDDS-119:Skip Apache license header check for some ozone doc scripts. Contributed by Ajay Kumar" 
This reverts commit c4bbcd5634.
 2018-08-15 10:06:17 -07:00
Xiaoyu Yao
f2315f2e9c HDDS-347. TestCloseContainerByPipeline#testCloseContainerViaStandaAlone sometimes fails. Contributed by Lixin Ge. 2018-08-15 08:11:17 -07:00
Xiaoyu Yao
b4031a8f1b HDDS-324. Use pipeline name as Ratis groupID to allow datanode to report pipeline info. Contributed by Mukul Kumar Singh. 2018-08-13 12:47:49 -07:00
Anu Engineer
11daa010d2 HDDS-346. ozoneShell show the new volume info after updateVolume command like updateBucket command. 
Contributed by chencan.
 2018-08-13 10:08:58 -07:00
Mukul Kumar Singh
a8dae0047c HDDS-308. SCM should identify a container with pending deletes using container reports. Contributed by Lokesh Jain. 2018-08-13 12:47:52 +05:30
Mukul Kumar Singh
398d895543 HDDS-339. Add block length and blockId in PutKeyResponse. Contributed by Shashikant Banerjee. 2018-08-10 23:45:56 +05:30
Márton Elek
3d96bc6e5f HDDS-219. Genearate version-info.properties for hadoop and ozone. Contributed by Sandeep Nemuri. 2018-08-09 11:06:03 +02:00
Hanisha Koneru
d81cd3611a HDDS-267. Handle consistency issues during container update/close. 2018-08-08 16:47:25 -07:00
Mukul Kumar Singh
6ed8593d18 HDDS-301. ozone command shell does not contain subcommand to run ozoneFS commands. Contributed by Nilotpal Nandi. 2018-08-07 16:09:53 +05:30
Mukul Kumar Singh
900c0e114f HDDS-230. ContainerStateMachine should implement readStateMachineData api to read data from Containers if required during replication. Contributed by Mukul Kumar Singh. 2018-08-07 15:03:14 +05:30
Nanda kumar
e83719c830 HDDS-290. putKey is failing with KEY_ALLOCATION_ERROR. Contributed by Xiaoyu Yao. 2018-08-02 19:02:25 +05:30
Mukul Kumar Singh
f4db753bb6 HDDS-226. Client should update block length in OM while committing the key. Contributed by Shashikant Banerjee. 2018-08-01 09:03:00 +05:30
Mukul Kumar Singh
b28bdc7e8b HDDS-279. DeleteBlocks command should not be sent for open containers. Contributed by Lokesh Jain. 2018-07-31 19:50:40 +05:30
Tsz Wo Nicholas Sze
ee53602a81 HDDS-293. Reduce memory usage and object creation in KeyData. 2018-07-30 15:00:29 -07:00
Nanda kumar
3517a47897 HDDS-287. Add Close ContainerAction to Datanode#StateContext when the container gets full. Contributed by Nanda kumar. 2018-07-30 21:18:42 +05:30
Bharat Viswanadham
007e6f5113 HDDS-248. Refactor DatanodeContainerProtocol.proto Contributed by Hanisha Koneru. 2018-07-28 14:57:11 -07:00
Mukul Kumar Singh
6b038f82da HDDS-246. Datanode should throw BlockNotCommittedException for uncommitted blocks to Ozone Client. Contributed by Shashikant Banerjee. 2018-07-28 22:04:11 +05:30
Anu Engineer
3cc7ce816e HDDS-270. Move generic container util functions to ContianerUtils. 
Contributed by Hanisha Koneru.
 2018-07-27 07:12:21 -07:00
Xiaoyu Yao
fd31cb6cfe HDDS-277. PipelineStateMachine should handle closure of pipelines in SCM. Contributed by Mukul Kumar Singh. 2018-07-26 13:15:55 -07:00
Nanda kumar
3c4fbc635e HDDS-288. Fix bugs in OpenContainerBlockMap. Contributed by Tsz Wo Nicholas Sze. 2018-07-25 20:27:03 +05:30
Nanda kumar
b507f83e15 HDDS-266. Integrate checksum into .container file. Contributed by Hanisha Koneru. 2018-07-25 18:55:26 +05:30
Mukul Kumar Singh
955f795101 HDDS-203. Add getCommittedBlockLength API in datanode. Contributed by Shashikant Banerjee. 2018-07-25 14:15:54 +05:30
Mukul Kumar Singh
773d312f74 HDDS-272. TestBlockDeletingService is failing with DiskOutOfSpaceException. Contributed by Lokesh Jain. 2018-07-24 21:23:20 +05:30
Xiaoyu Yao
2ced3efe94 HDDS-258. Helper methods to generate NodeReport and ContainerReport for testing. Contributed by Nanda Kumar. 2018-07-23 22:04:20 -07:00
Mukul Kumar Singh
bbe2f6225e HDDS-181. CloseContainer should commit all pending open Keys on a datanode. Contributed by Shashikant Banerjee. 2018-07-23 09:13:03 +05:30
Xiaoyu Yao
9be25e3476 HDDS-239. Add PipelineStateManager to track pipeline state transition. Contributed by Mukul Kumar Singh. 2018-07-20 14:22:02 -07:00
Xiaoyu Yao
d2acf8d560 Revert "HDDS-239. Add PipelineStateManager to track pipeline state transition. Contributed by Mukul Kumar Singh." 
This reverts commit 6837121a43231f854b0b22ad20330012439313ce.(Mixed with HDDS-260)
 2018-07-20 14:20:18 -07:00
Xiaoyu Yao
c7ae55675e HDDS-275. Add message output for succeeded -deleteVolume CLI. Contributed by Nilotpal Nandi. 2018-07-20 13:12:53 -07:00
Xiaoyu Yao
6837121a43 HDDS-239. Add PipelineStateManager to track pipeline state transition. Contributed by Mukul Kumar Singh. 2018-07-20 13:03:35 -07:00
Hanisha Koneru
de894d34f6 HDDS-250. Cleanup ContainerData. 2018-07-20 11:36:42 -07:00
Mukul Kumar Singh
5c19ee3994 HDDS-264. 'oz' subcommand reference is not present in 'ozone' command help. Contributed by Sandeep Nemuri. 2018-07-20 22:13:09 +05:30
Xiaoyu Yao
68b57ad32c HDDS-259. Implement ContainerReportPublisher and NodeReportPublisher. Contributed by Nanda kumar. 2018-07-20 09:12:48 -07:00
Xiaoyu Yao
c492eaccc2 HDDS-255. Fix TestOzoneConfigurationFields for missing hdds.command.status.report.interval in config classes. Contributed by Sandeep Nemuri. 2018-07-18 11:46:26 -07:00
Xiaoyu Yao
129269f989 HDDS-207. ozone listVolume command accepts random values as argument. Contributed by Lokesh Jain. 2018-07-18 11:05:42 -07:00
Bharat Viswanadham
5074ca93af HDDS-254. Fix TestStorageContainerManager#testBlockDeletingThrottling. Contributed by Lokesh Jain 2018-07-15 10:47:20 -07:00
Bharat Viswanadham
0927bc4f76 HDDS-251. Integrate BlockDeletingService in KeyValueHandler. Contributed by Lokesh Jain 2018-07-15 10:34:00 -07:00
Xiaoyu Yao
103f2eeb57 HDDS-210. Make "-file" argument optional for ozone getKey command. Contributed by Lokesh Jain. 2018-07-13 11:45:02 -07:00
Nanda kumar
d18507209e HDDS-232. Parallel unit test execution for HDDS/Ozone. Contributed by Arpit Agarwal. 2018-07-13 19:50:52 +05:30
Nanda kumar
1fe5b93843 HDDS-253. SCMBlockDeletingService should publish events for delete blocks to EventQueue. Contributed by Lokesh Jain. 2018-07-13 17:18:42 +05:30
Xiaoyu Yao
3f3f72221f HDDS-238. Add Node2Pipeline Map in SCM to track ratis/standalone pipelines. Contributed by Mukul Kumar Singh. 2018-07-12 22:14:03 -07:00
Anu Engineer
5ee90efed3 HDDS-228. Add the ReplicaMaps to ContainerStateManager. 
Contributed by Ajay Kumar.
 2018-07-12 10:43:24 -07:00
Anu Engineer
b56785873a Revert "HDDS-242. Introduce NEW_NODE, STALE_NODE and DEAD_NODE event" 
This reverts commit a47ec5dac4.
There was a spurious file in this commit. Revert to clean it.
 2018-07-11 12:03:42 -07:00
Anu Engineer
a47ec5dac4 HDDS-242. Introduce NEW_NODE, STALE_NODE and DEAD_NODE event 
and corresponding event handlers in SCM.
Contributed by Nanda Kumar.
 2018-07-10 15:58:47 -07:00
Xiaoyu Yao
4e59b92784 HDDS-208. ozone createVolume command ignores the first character of the volume name argument. Contributed by Lokesh Jain. 2018-07-10 14:07:23 -07:00
Bharat Viswanadham
3584baf264 HDDS-48. Fix branch after merging from trunk. 2018-07-09 13:22:30 -07:00
Bharat Viswanadham
c275a9a6a0 Merge trunk into HDDS-48 2018-07-09 12:13:03 -07:00
Bharat Viswanadham
e899c4cc01 HDDS-211. Add a create container Lock. Contributed by Bharat Viswanadham 2018-07-06 15:02:40 -07:00
Arpit Agarwal
061b168529 HDDS-167. Rename KeySpaceManager to OzoneManager. Contributed by Arpit Agarwal. 2018-07-06 12:09:05 -07:00
Bharat Viswanadham
7dcf5878a4 HDDS-204. Modify Integration tests for new ContainerIO classes. Contributed by Bharat Viswanadham 2018-07-06 11:01:33 -07:00
Bharat Viswanadham
2c2351e87b HDDS-182:CleanUp Reimplemented classes. Contributed by Hansiha Koneru 2018-07-05 15:23:39 -07:00
Nanda kumar
71df8c27c9 HDDS-212. Introduce NodeStateManager to manage the state of Datanodes in SCM. Contributed by Nanda kumar. 2018-07-05 02:11:10 +05:30
Anu Engineer
7ca4f0cefa HDDS-175. Refactor ContainerInfo to remove Pipeline object from it. 
Contributed by Ajay Kumar.
 2018-07-03 14:11:52 -07:00
Anu Engineer
c0ef7e7680 HDDS-198. Create AuditLogger mechanism to be used by OM, SCM and Datanode. 
Contributed by Dinesh Chitlangia.
 2018-07-03 11:39:50 -07:00
Hanisha Koneru
44b091a8d7 HDDS-205. Add metrics to HddsDispatcher. Contributed by Bharat Viswanadham. 2018-07-03 09:50:25 -07:00
Nanda kumar
ab2f8343a9 HDDS-206. Ozone shell command doesn't respect KSM port set in ozone-site.xml. Contributed by Shashikant Banerjee. 2018-07-03 00:51:16 +05:30
Nanda kumar
e4d7227aad HDDS-178: DN should update transactionId on block delete. Contributed by Lokesh Jain. 2018-06-29 04:37:16 +05:30
Nanda kumar
852ca6b432 HDDS-185: TestCloseContainerByPipeline#testCloseContainerViaRatis fail intermittently. Contributed by Shashikant Banerjee. 2018-06-29 04:11:39 +05:30
Bharat Viswanadham
c104525168 Merge remote-tracking branch 'apache-commit/trunk' into HDDS-48 2018-06-28 14:28:23 -07:00
Bharat Viswanadham
52d1d9603e HDDS-183:Integrate Volumeset, ContainerSet and HddsDispatcher. Contributed by Bharat Viswanadham 2018-06-28 14:07:52 -07:00
Hanisha Koneru
13579f9296 HDDS-173. Refactor Dispatcher and implement Handler for new ContainerIO design. 2018-06-28 10:13:30 -07:00
Anu Engineer
8752a48564 HDDS-193. Make Datanode heartbeat dispatcher in SCM event based. 
Contributed by Elek, Marton.
 2018-06-27 14:18:25 -07:00
Anu Engineer
18932717c4 HDDS-94. Change ozone datanode command to start the standalone datanode plugin. 
Contributed by  Sandeep Nemuri.
 2018-06-27 14:15:15 -07:00
Xiaoyu Yao
1e30547642 HDDS-170. Fix TestBlockDeletingService#testBlockDeletionTimeout. Contributed by Lokesh Jain. 2018-06-27 13:56:45 -07:00
Xiaoyu Yao
56a4cdb980 HDDS-194. Remove NodePoolManager and node pool handling from SCM. Contributed by Elek Marton. 2018-06-27 13:28:00 -07:00
Xiaoyu Yao
0d6fe5f36b Revert "HDDS-194. Remove NodePoolManager and node pool handling from SCM. Contributed by Elek Marton" 
This reverts commit aaf03cc459.
 2018-06-27 13:25:45 -07:00
Xiaoyu Yao
aaf03cc459 HDDS-194. Remove NodePoolManager and node pool handling from SCM. Contributed by Elek Marton 2018-06-27 12:39:15 -07:00
Bharat Viswanadham
238fe00ad2 HDDS-192:Create new SCMCommand to request a replication of a container. Contributed by Elek Marton 2018-06-25 21:12:05 -07:00
Anu Engineer
e16e5b307d HDDS-177. Create a releasable ozonefs artifact 
Contributed by Marton, Elek.
 2018-06-24 01:05:04 -07:00
Anu Engineer
ca14fec02f HDDS-184. Upgrade common-langs version to 3.7 in hadoop-tools/hadoop-ozone. 
Contributed by Takanobu Asanuma.
 2018-06-23 15:49:44 -07:00
Mukul Kumar Singh
2a4632d3d7 HDDS-141. Remove PipeLine Class from SCM and move the data field in the Pipeline to ContainerInfo. Contributed by Shashikant Banerjee. 2018-06-17 23:48:49 -07:00
Anu Engineer
308a1591f9 HDDS-172. The numbers of operation should be integer in KSM UI. 
Contributed by Takanobu Asanuma.
 2018-06-15 14:10:17 -07:00
Anu Engineer
020dd61988 HDDS-146. Refactor the structure of the acceptance tests. 
Contributed by  Elek, Marton.
 2018-06-14 14:14:24 -07:00
Anu Engineer
ddd09d59f3 HDDS-163. Add Datanode heartbeat dispatcher in SCM. 
Contributed by Nandakumar.
 2018-06-13 20:18:22 -07:00
Anu Engineer
7547740e5c HDDS-161. Add functionality to queue ContainerClose command from SCM Heartbeat Response to Ratis. 
Contributed by Shashikant Banerjee.
 2018-06-13 18:48:59 -07:00
Xiaoyu Yao
7566e0ec5f HDDS-159. RestClient: Implement list operations for volume, bucket and keys. Contributed by Lokesh Jain. 2018-06-13 11:43:18 -07:00
Anu Engineer
43baa036ae HDDS-109. Add reconnect logic for XceiverClientGrpc. 
Contributed by Lokesh Jain.
 2018-06-13 09:50:10 -07:00
Jitendra Pandey
f4c7c91123 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-13 00:36:02 -07:00
Xiaoyu Yao
e9ea902299 HDDS-130. TestGenerateOzoneRequiredConfigurations should use GenericTestUtils#getTempPath to set output directory. Contributed by Sandeep Nemuri. 2018-06-12 09:03:55 -07:00
Anu Engineer
5c7ad52573 HDDS-111. Include tests for Rest Client in TestVolume and TestBucket. 
Contributed by Lokesh Jain.
 2018-06-12 08:25:18 -07:00
Xiaoyu Yao
23bfd9f7e4 HDDS-72. Add deleteTransactionId field in ContainerInfo. Contributed by Lokesh Jain. 2018-06-11 16:02:32 -07:00
Anu Engineer
a1272448bf HDDS-157. Upgrade common-langs version to 3.7 in HDDS and Ozone. 
Contributed by Takanobu Asanuma.
 2018-06-08 10:27:01 -07:00
Arpit Agarwal
ba303b1f89 HDDS-147. Update Ozone site docs. Contributed by Arpit Agarwal. 2018-06-07 14:11:20 -07:00
Bharat Viswanadham
c4bbcd5634 HDDS-119:Skip Apache license header check for some ozone doc scripts. Contributed by Ajay Kumar 2018-06-07 12:47:58 -07:00
Anu Engineer
30f0582f2b HDDS-107. TestOzoneConfigurationFields is failing. 
Contributed by LiXin Ge & Mukul Kumar Singh.
 2018-06-06 08:39:18 -07:00
Anu Engineer
950dea86f4 HDDS-112. OzoneShell should support commands with url without scheme. 
Contributed by Lokesh Jain.
 2018-05-31 12:05:50 -07:00
Anu Engineer
3b34148c4f HDDS-88. Create separate message structure to represent ports in DatanodeDetails. 
Contributed by Nanda Kumar.
 2018-05-30 08:52:07 -07:00
Anu Engineer
201440b987 HDDS-81. Moving ContainerReport inside Datanode heartbeat. 
Contributed by Nanda Kumar.
 2018-05-29 12:48:50 -07:00
Anu Engineer
2a9652e696 HDDS-113. Rest and Rpc Client should verify resource name using HddsClientUtils. 
Contributed by Lokesh Jain.
 2018-05-25 15:45:50 -07:00
Anu Engineer
8733012ae3 HDDS-96. Add an option in ozone script to generate a site file with minimally required ozone configs. 
Contributed by Dinesh Chitlangia.
 2018-05-25 13:06:14 -07:00
Mukul Kumar Singh
774daa8d53 HDDS-45. Removal of old OzoneRestClient. Contributed by Lokesh Jain. 2018-05-24 15:53:42 +05:30
Anu Engineer
e83b943fed HDDS-44. Ozone: start-ozone.sh fail to start datanode because of incomplete classpaths. 
Contributed by Mukul Kumar Singh.
 2018-05-23 09:29:35 -07:00
Mukul Kumar Singh
fed2bef647 HDDS-85. Send Container State Info while sending the container report from Datanode to SCM. Contributed by Shashikant Banerjee. 2018-05-23 14:15:35 +05:30
Anu Engineer
5a9140690a HDDS-49. Standalone protocol should use grpc in place of netty. 
Contributed by Mukul Kumar Singh.
 2018-05-22 19:56:15 -07:00
Xiaoyu Yao
68c7fd8e60 HDDS-79. Remove ReportState from SCMHeartbeatRequestProto. Contributed by Nanda kumar. 2018-05-22 15:46:59 -07:00
Anu Engineer
43be9ab44f HDDS-89. Addendum Patch-1. Create ozone specific inline documentation as part of the build. 
Contributed by Elek, Marton.
 2018-05-22 14:29:06 -07:00
Anu Engineer
481bfdb94f HDDS-89. Create ozone specific inline documentation as part of the build. 
Contributed by Elek, Marton.
 2018-05-22 10:49:10 -07:00
Xiaoyu Yao
5e88126776 HDDS-82. Merge ContainerData and ContainerStatus classes. Contributed by Bharat Viswanadham. 2018-05-21 16:09:24 -07:00
Anu Engineer
e0367d3b24 HDDS-73. Add acceptance tests for Ozone Shell. 
Contributed by Lokesh Jain.
 2018-05-17 15:21:26 -07:00
Xiaoyu Yao
41ae5c5002 HDDS-77. Key replication factor and type should be stored per key by Ozone Manager. Contributed by Mukul Kumar Singh. 2018-05-17 08:31:44 -07:00
Xiaoyu Yao
fc5d49c202 HDDS-29. Fix TestStorageContainerManager#testRpcPermission. Contributed by Mukul Kumar Singh. 2018-05-14 09:10:03 -07:00
Anu Engineer
53c8ebcd02 HDDS-32. Fix TestContainerDeletionChoosingPolicy#testTopNOrderedChoosingPolicy. 
Contributed by Mukul Kumar Singh.
 2018-05-12 10:24:06 -07:00
Anu Engineer
cd8b9e913e HDDS-53. Fix TestKey#testPutAndGetKeyWithDnRestart. 
Contributed by Mukul Kumar Singh.
 2018-05-12 10:24:05 -07:00
Anu Engineer
7e26e1f216 HDDS-52. Fix TestSCMCli#testInfoContainer. 
Contributed by Mukul Kumar Singh.
 2018-05-12 10:24:05 -07:00
Anu Engineer
1194ec31d7 HDDS-17. Add node to container map class to simplify state in SCM. 
Contributed by Anu Engineer.
 2018-05-12 09:58:20 -07:00
Mukul Kumar Singh
413285569a HDDS-21. Add support for rename key within a bucket for rest client. Contributed by Lokesh Jain. 2018-05-12 17:40:29 +05:30
Anu Engineer
4b4f24ad5f HDDS-40. Separating packaging of Ozone/HDDS from the main Hadoop. 
Contributed by Elek, Marton.
 2018-05-11 14:24:36 -07:00
Anu Engineer
c1d64d60f6 HDDS-39. Ozone: Compile Ozone/HDFS/Cblock protobuf files with proto3 compiler using maven protoc plugin. 
Contributed by Mukul Kumar Singh.
 2018-05-11 11:08:45 -07:00
Anu Engineer
3a93af731e HDDS-47. Add acceptance tests for Ozone Shell. 
Contributed by Lokesh Jain.
 2018-05-11 10:20:04 -07:00
Bharat Viswanadham
84b305f11a HDDS-43: Rename hdsl to hdds in hadoop-ozone/acceptance-test/README.md. Contributed by Sandeep Nemuri 2018-05-10 17:24:40 -07:00
Anu Engineer
30293f6065 HDDS-34. Remove .meta file during creation of container 
Contributed by Bharat Viswanadham.
 2018-05-10 17:08:26 -07:00
Anu Engineer
db1ab0fc16 HDDS-37. Remove dependency of hadoop-hdds-common and hadoop-hdds-server-scm from hadoop-ozone/tools/pom.xml. 
Contributed by Sandeep Nemuri.
 2018-05-10 16:27:21 -07:00
Xiaoyu Yao
7369f41020 HDDS-16. Remove Pipeline from Datanode Container Protocol protobuf definition. Contributed by Mukul Kumar Singh. 2018-05-10 14:49:58 -07:00
Anu Engineer
48d0b54849 HDDS-31. Fix TestSCMCli. Contributed by Lokesh Jain. 2018-05-10 12:43:13 -07:00
Anu Engineer
f8b540049d HDDS-42. Inconsistent module names and descriptions. Contributed by Tsz Wo Nicholas Sze. 2018-05-10 11:44:14 -07:00
Anu Engineer
7482963f1a HDDS-30. Fix TestContainerSQLCli. Contributed by Shashikant Banerjee. 2018-05-10 11:36:25 -07:00
Anu Engineer
46e0f27862 HDDS-18. Ozone Shell should use RestClient and RpcClient. Contributed by Lokesh Jain. 2018-05-09 23:00:13 -07:00
Mukul Kumar Singh
343b51dcf4 HDDS-28. Ozone:Duplicate declaration in hadoop-tools/hadoop-ozone/pom.xml. Contributed by Tsz Wo Nicholas Sze. 2018-05-09 20:20:21 +05:30
Mukul Kumar Singh
208b97e969 HDDS-20. Ozone: Add support for rename key within a bucket for rpc client. Contributed by Lokesh Jain. 2018-05-09 19:06:07 +05:30
Mukul Kumar Singh
809135082a HDDS-33. Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey. Contributed by Shashikant Banerjee. 2018-05-09 10:21:41 +05:30
Xiaoyu Yao
08ea90e1e4 HDDS-27. Fix TestStorageContainerManager#testBlockDeletionTra;nsactions. Contributed by Xiaoyu Yao. 2018-05-07 18:29:50 -07:00
Anu Engineer
3a43ac2851 HDDS-1. Remove SCM Block DB. Contributed by Xiaoyu Yao. 2018-05-07 14:58:52 -07:00
Anu Engineer
6b63a0af9b HDDS-15. Add memory profiler support to Genesis. Contributed by Anu Engineer. 2018-05-02 12:54:49 -07:00
Anu Engineer
f0c3dc4cf4 HDDS-13. Refactor StorageContainerManager into seperate RPC endpoints. Contributed by Anu Engineer. 2018-04-30 21:41:10 -07:00
Anu Engineer
3d43474f75 HDDS-11. Fix findbugs exclude rules for ozone and hdds projects. Contributed by Elek, Marton. 2018-04-30 09:20:58 -07:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Xiaoyu Yao
9da4caee89 HDFS-13422. Ozone: Fix whitespaces and license issues in HDFS-7240 branch. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
20dab05564 HDFS-13407. Ozone: Use separated version schema for Hdds/Ozone projects. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
17974ba3a6 HDFS-13444. Ozone: Fix checkstyle issues in HDFS-7240. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
ae8ac7f082 HDFS-13424. Ozone: Refactor MiniOzoneClassicCluster. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
025058f251 HDFS-13446. Ozone: Fix OzoneFileSystem contract test failures. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
979bbb4019 HDFS-13423. Ozone: Clean-up of ozone related change from hadoop-hdfs-project. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
51459d4351 HDFS-13414. Ozone: Update existing Ozone documentation according to the recent changes. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
ec6c8742e5 HDFS-13415. Ozone: Remove cblock code from HDFS-7240. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
3779fadc5c HDFS-13324. Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
25f2398bbd HDFS-13395. Ozone: Plugins support in HDSL Datanode Service. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
82eb5da2b6 HDFS-13342. Ozone: Rename and fix ozone CLI scripts. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
8b832f3c35 HDFS-13405. Ozone: Rename HDSL to HDDS. 
Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.
 2018-04-26 05:36:04 -07:00
Nanda kumar
43f7307829 HDFS-13316. Ozone: Update the ksm/scm CLI usage info. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Nanda kumar
6cc765fbb6 HDFS-13320. Ozone: Support for MicrobenchMarking Tool. Contributed by Shashikant Banerjee 2018-04-26 05:36:04 -07:00
Nanda kumar
7ae9543a57 HDFS-13391. Ozone: Make dependency of internal sub-module scope as provided in maven. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
4f37ff7616 HDFS-13341. Ozone: Move ozone specific ServiceRuntimeInfo utility to the framework. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
498b84155f HDFS-13300. Ozone: Remove DatanodeID dependency from HDSL and Ozone. 
Contributed by Nanda kumar.
 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
70216e7978 HDFS-13340. Ozone: Fix false positive RAT warning when project built without hds/cblock. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
6d9f069ede HDFS-13343. Ozone: Provide docker based acceptance testing on pseudo cluster. 
Contributed by Elek, Marton.
 2018-04-26 05:36:04 -07:00
Anu Engineer
6eb1dc9e8c HDFS-13319. Ozone: start-ozone.sh/stop-ozone.sh fail because of unresolved path. 
Contributed by Mukul Kumar Singh.
 2018-04-26 05:36:04 -07:00
Anu Engineer
1f9a7ae12f HDFS-13304. Document: update the new ozone docker file location. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
04e1287771 HDFS-13298. Ozone: Make ozone/hdsl/cblock modules turned off by default. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00