big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,711 Commits 2 Branches 2 Tags 264 MiB
b1eee8b52e
Commit Graph

490 Commits

Author SHA1 Message Date
Hrishikesh Gadre
acef5e0cec HDDS-1201. Reporting corrupted containers info to SCM (#1032) 2019-07-11 14:49:58 +05:30
Ajay Yadav
cdb20adfcc HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973) 2019-07-10 11:03:58 -07:00
supratimdeka
ac7a8accdf HDDS-1603. Handle Ratis Append Failure in Container State Machine. Contributed by Supratim Deka (#1019) 2019-07-10 07:49:36 +05:30
Xiaoyu Yao
030307226a
HDDS-1586. Allow Ozone RPC client to read with topology awareness. Contributed by Sammi Chen. 2019-07-09 14:43:55 -07:00
Hanisha Koneru
8861573e8c
HDDS-1717. MR Job fails as OMFailoverProxyProvider has dependency hadoop-3.2 
Closes #1056
 2019-07-09 13:22:00 +02:00
Mukul Kumar Singh
e5ffb88257 HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh. (#1050) 
* HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh.
 2019-07-08 20:24:12 -07:00
Lokesh Jain
b5d30e4914 HDDS-1750. Add block allocation metrics for pipelines in SCM. Contributed by Lokesh Jain. (#1047) 2019-07-09 08:42:55 +05:30
Mukul Kumar Singh
ce91d35b29
HDDS-1728. Add metrics for leader's latency in ContainerStateMachine. Contributed by Mukul Kumar Singh. (#1022) 2019-07-08 12:19:16 +05:30
Mukul Kumar Singh
c9c3429a16
HDDS-1758. Add replication and key deletion tests to MiniOzoneChaosCluster. Contributed by Mukul Kumar Singh. (#1049) 2019-07-05 17:04:40 +05:30
Bharat Viswanadham
8965ddcf59
HDDS-1731. Implement File CreateFile Request to use Cache and DoubleBuffer. (#1044) 2019-07-03 13:15:04 -07:00
Bharat Viswanadham
4848280437
HDDS-1638. Implement Key Write Requests to use Cache and DoubleBuffer. (#956) 2019-06-26 18:04:12 -07:00
avijayanhwx
0a37ccf379 HDDS-1691 : RDBTable#isExist should use Rocksdb#keyMayExist (#1013) 2019-06-26 11:44:49 -07:00
Anu Engineer
4b50981cfa HDDS-1709. TestScmSafeNode is flaky. 
Contributed by Elek, Marton.
 2019-06-25 13:57:01 -07:00
Elek, Márton
004207996c HDDS-1597. Remove hdds-server-scm dependency from ozone-common (#969) 
* HDDS-1597. Remove hdds-server-scm dependency from ozone-common. Contributed by Elek, Marton.

* checkstyle fixes

* revert the import reorder of HddsUtil

* add javadoc

* switch back to the commons-lang2

* fix typo

* fix metrics core classpath problem (+rebase fix)
 2019-06-24 11:56:04 -07:00
Hanisha Koneru
2dfa932818
HDDS-1684. OM should create Ratis related dirs only if ratis is enabled (#965) 2019-06-18 16:08:48 -07:00
S O'Donnell
22b36dd311
HDDS-1660. Use Picocli for Ozone Manager 
Closes #925
 2019-06-17 14:13:50 +02:00
Xiaoyu Yao
a43f4440f7
HDDS-1543. Implement addAcl,removeAcl,setAcl,getAcl for Prefix. Contr… (#927) 2019-06-12 16:25:31 -07:00
Márton Elek
f918e3fe62
HDDS-1669. SCM startup is failing if network-topology-default.xml is part of a jar 
Closes #946
 2019-06-11 19:30:22 +02:00
Sammi Chen
b05747901f
HDDS-1662. Missing test resources of integrataion-test project in target directory after compile 
Closes #933
 2019-06-11 12:25:42 +02:00
Ajay Yadav
9deac3b6bf HDDS-1657. Fix parallelStream usage in volume and key native acl. Contributed by Ajay Kumar. (#926) 2019-06-07 21:47:14 -07:00
Elek, Márton
46b23c11b0 HDDS-1636. Tracing id is not propagated via async datanode grpc call (#895) 2019-06-07 20:40:32 -07:00
S O'Donnell
85479577da
HDDS-1622. Use picocli for StorageContainerManager 
Closes #918
 2019-06-07 17:50:57 +02:00
Hanisha Koneru
0b115b60b0 HDDS-1371. OMSnapshotProvider to download DB checkpoint from leader OM. (#703) 2019-06-06 19:42:34 -07:00
Sammi Chen
8ca58efeec HDDS-1490. Support configurable container placement policy through 'o… (#903) 2019-06-06 12:14:47 -07:00
S O'Donnell
36757ad6a9
HDDS-1645. Change the version of Pico CLI to the latest 3.x release - 3.9.6 
Closes #909
 2019-06-06 16:00:00 +02:00
Ajay Yadav
3b1c2577d7 HDDS-1541. Implement addAcl,removeAcl,setAcl,getAcl for Key. Contributed by Ajay Kumat. (#885) 2019-06-05 14:42:10 -07:00
Bharat Viswanadham
580b639908
HDDS-1600. Add userName and IPAddress as part of OMRequest. (#857) 2019-06-04 12:31:44 -07:00
bshashikant
e4b4fec66f HDDS-1224. Restructure code to validate the response from server in the Read path (#806) 2019-06-04 10:37:02 -07:00
avijayanhwx
21de9af903 HDDS-1625 : ConcurrentModificationException when SCM has containers of different owners. (#883) 2019-06-04 01:15:04 +05:30
Shashikant Banerjee
f3271126fc HDDS-1558. IllegalArgumentException while processing container Reports. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-06-04 00:59:02 +05:30
Márton Elek
2a97a37d9e
Revert "HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860)" 
This reverts commit baee71551d.
 2019-06-03 08:34:19 +02:00
Ajay Yadav
c1d2d92187 HDDS-1540. Implement addAcl,removeAcl,setAcl,getAcl for Bucket. Contributed by Ajay Kumar. (#874) 2019-05-31 14:08:28 -07:00
Márton Elek
1ae062c818
HDDS-1382. Create customized CSI server for Ozone 
Closes #693
 2019-05-31 13:35:58 +02:00
Xiaoyu Yao
219e286722
HDDS-1608. Support Ozone Prefix ACLs in OM metadata table. Contributed by Xiaoyu Yao. (#875) 2019-05-30 16:44:38 -07:00
Bharat Viswanadham
c965f7f499
HDDS-1551. Implement Bucket Write Requests to use Cache and DoubleBuffer. (#850) 2019-05-30 15:28:15 -07:00
Elek, Márton
baee71551d HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860) 2019-05-30 09:20:16 -07:00
Shashikant Banerjee
8c8cb2d6aa HDDS-1502. Fix the compilation issue with commit 18c1eebc08. 2019-05-30 18:44:22 +05:30
Shashikant Banerjee
18c1eebc08 HDDS-1502. Add metrics for Ozone Ratis performance.Contributed by Shashikant Banerjee(#833). 2019-05-30 16:17:45 +05:30
Ajay Yadav
12be6ff2ff HDDS-1539. Implement addAcl,removeAcl,setAcl,getAcl for Volume. Contributed Ajay Kumar. (#847) 2019-05-29 23:18:06 -07:00
Akira Ajisaka
afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Xiaoyu Yao
7f2e87a419
HDDS-1604. ContainerReader#initializeUsedBytes leaks DB reference. Co… (#866) 2019-05-28 18:39:23 -07:00
Elek, Márton
79d14d0d42 HDDS-1341. TestContainerReplication#testContainerReplication fails intermittently. Contributed by Elek, Marton. (#862) 2019-05-28 14:40:45 -07:00
Xiaoyu Yao
fb0b39f4bf
HDDS-1536. testSCMSafeModeRestrictedOp is failing consistently. Contributed by Xiaoyu Yao. (#865) 2019-05-28 14:02:36 -07:00
Shashikant Banerjee
83549dbbea HDDS-1509. TestBlockOutputStreamWithFailures#test2DatanodesFailure fails intermittently. Contributed by Shashikant Banerjee (#805). 2019-05-27 16:31:44 +05:30
Shashikant Banerjee
f0e44b3a3f HDDS-1584. Fix TestFailureHandlingByClient tests. Contributed by Shashikant Banerjee (#845). 2019-05-27 16:26:21 +05:30
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Ajay Yadav
9c61494c02 HDDS-1065. OM and DN should persist SCM certificate as the trust root. Contributed by Ajay Kumar. (#834) 2019-05-22 11:47:32 -07:00
Mukul Kumar Singh
2fc6f8599a HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh. (#825) 2019-05-22 17:18:40 +05:30
Bharat Viswanadham
77c49f2963
HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714) 2019-05-21 18:41:30 -07:00
Ajay Yadav
ef1cc725b8
HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828) 2019-05-21 15:53:40 -07:00
Siddharth
c183bd8e20 HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle. 
This closes #822.
 2019-05-16 15:13:10 -07:00
supratimdeka
9569015802 HDDS-1511. Space tracking for Open Containers in HDDS Volumes. Contributed by Supratim Deka (#812) 2019-05-15 10:26:01 -07:00
Hanisha Koneru
02c9efcb81
HDDS-1491. Ozone KeyInputStream seek() should not read the chunk file. (#795) 2019-05-13 20:49:52 -07:00
avijayanhwx
eb9c8900bc HDDS-1475 : Fix OzoneContainer start method. (#788) 2019-05-07 14:09:41 -07:00
Shashikant Banerjee
7f0e2c67e0 HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749). Contributed by Shashikant Banerjee 2019-05-08 01:00:10 +05:30
Shashikant Banerjee
01451a57de Revert "HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)" 
This reverts commit d525633619.
 2019-05-08 00:49:15 +05:30
bshashikant
d525633619
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749) 
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749).
 2019-05-08 00:38:53 +05:30
Márton Elek
fb7c1cad0e
Revert "HDDS-1384. TestBlockOutputStreamWithFailures is failing" 
This reverts commit dead9b4049.
 2019-05-06 20:17:00 +02:00
avijayanhwx
f194540520 HDDS-1448 : RatisPipelineProvider should only consider open pipeline while excluding dn for pipeline allocation. (#786) 2019-05-04 00:19:00 +05:30
Siddharth
1df679985b HDDS-1473. DataNode ID file should be human readable. (#781) 2019-05-02 13:59:14 -07:00
Márton Elek
e2f0f72677
HDDS-1469. Generate default configuration fragments based on annotations 
Closes #773
 2019-05-02 12:14:43 +02:00
Márton Elek
dead9b4049
HDDS-1384. TestBlockOutputStreamWithFailures is failing 
Closes #750
 2019-04-30 14:27:39 +02:00
Bharat Viswanadham
43b2a4b77b
HDDS-1456. Stop the datanode, when any datanode statemachine state is… (#769) 2019-04-26 14:25:34 -07:00
Kitti Nanasi
3f787cd506
HDDS-1192. Support -conf command line argument in GenericCli 
Closes #713
 2019-04-24 15:34:49 +02:00
Siddharth
59ded7641f HDDS-1411. Add unit test to check if SCM correctly sends close commands for containers in closing state after a restart. (#755) 2019-04-23 21:04:14 +05:30
Márton Elek
b979fdde99
HDDS-1447. Fix CheckStyle warnings. Contributed by Wanqiang Ji. 2019-04-18 08:54:43 +02:00
Shashikant Banerjee
d608be660f HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException.(#729) 2019-04-17 14:49:02 +05:30
Shashikant Banerjee
082f1e0437 Revert "HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)" 
This reverts commit df2ae27f3e.
 2019-04-17 14:38:08 +05:30
Shashikant Banerjee
df2ae27f3e HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729) 2019-04-17 14:25:35 +05:30
Shashikant Banerjee
e5a918cda3 Revert "Merge pull request #729 from bshashikant/HDDS-1373" 
This reverts commit 4ff4314e40, reversing
changes made to 04c0437d13.
 2019-04-17 14:17:54 +05:30
bshashikant
4ff4314e40
Merge pull request #729 from bshashikant/HDDS-1373 
HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee.
 2019-04-17 14:12:52 +05:30
Hanisha Koneru
e543c3b31a
HDDS-1376. Datanode exits while executing client command when scmId is null (#724) 2019-04-16 13:51:39 -07:00
Arpit Agarwal
f2ab2795db
HDDS-1432. Ozone client list command truncates response without any indication. Contributed by Siddharth Wagle. 2019-04-16 12:35:49 -07:00
Shashikant Banerjee
bfcb6534cd HDDS-1380. Add functonality to write from multiple clients in MiniOzoneChaosCluster. Contributed by Shashikant Banerjee. 2019-04-16 18:52:07 +05:30
Arpit Agarwal
ebbda181e4
HDDS-1198. Rename chill mode to safe mode. Contributed by Siddharth Wagle. (#737) 2019-04-13 07:27:03 -07:00
Elek, Márton
5379d85d8e HDDS-1387. ConcurrentModificationException in TestMiniChaosOzoneCluster. Contributed by Marton Elek. (#732) 2019-04-12 15:19:50 -07:00
Bharat Viswanadham
a458c5bd96
HDDS-1285. Implement actions need to be taken after chill mode exit wait time. (#612) 2019-04-12 09:40:27 -07:00
Arpit Agarwal
abace709cc
HDDS-1420. Tracing exception in DataNode HddsDispatcher 
Closes #723
 2019-04-12 12:21:10 +02:00
Doroszlai, Attila
75ba719ddd
HDDS-1419. Fix shellcheck errors in start-chaos.sh 
Closes #728
 2019-04-12 11:59:55 +02:00
Arpit Agarwal
fd676e190a
HDDS-1426. Minor logging improvements for MiniOzoneChaosCluster 
Closes #730
 2019-04-12 11:42:23 +02:00
Nanda kumar
4003849fa4
HDDS-1416. MiniOzoneCluster should set custom value for hdds.datanode.replication.work.dir. 
Contributed by chencan.
 2019-04-12 14:45:57 +05:30
Arpit Agarwal
fcc7f9b32f
HDDS-1410. TestSCMNodeMetrics is flaky. Contributed by Siddharth Wagle. 2019-04-11 15:14:28 -07:00
Shashikant Banerjee
22d0468b39 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee 2019-04-11 22:13:41 +05:30
Shashikant Banerjee
a0468c5756 HDDS-1348. Refactor BlockOutpuStream Class. Contributed by Shashikant Banerjee. 2019-04-11 19:31:26 +05:30
Arpit Agarwal
feaab241e5
HDDS-1418. Move bang line to the start of the start-chaos.sh script. Contributed by Arpit Agarwal. (#720) 
Change-Id: I4fcf39d61a7d4c4ca79cb56a6958db0f691fe971
 2019-04-10 12:44:01 -07:00
Mukul Kumar Singh
df01469141 HDDS-1401. Static ContainerCache in Datanodes can result in overwrite of container db. Contributed by Mukul Kumar Singh. (#708) 2019-04-10 18:00:10 +05:30
Bharat Viswanadham
25c421bcc0
HDDS-1393. Convert all OM Bucket related operations to HA model. (#704) 2019-04-09 14:57:32 -07:00
Mukul Kumar Singh
312d4d4cc5 HDDS-1388. Add a shell script to run MiniOzoneChaosCluster using mvn exec. Contributed by Mukul Kumar Singh. (#709) 2019-04-09 13:08:25 -07:00
Bharat Viswanadham
77fe51e136
HDDS-1379. Convert all OM Volume related operations to HA model. (#689) 2019-04-04 21:09:50 -07:00
avijayanhwx
993f36ee33 HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681) 2019-04-04 19:18:32 +05:30
Hanisha Koneru
f09a78f73f
HDDS-1339. Implement ratis snapshots on OM (#651) 2019-04-03 22:50:28 -07:00
Hanisha Koneru
4e64f8d763 HDDS-1324. TestOzoneManagerHA tests are flaky (#676) 2019-04-03 16:02:19 -07:00
Bharat Viswanadham
358f7f9b99
HDDS-1211. Test SCMChillMode failing randomly in Jenkins run (#543) 2019-04-03 15:02:00 -07:00
Ajay Kumar
bfc90bdc64
HDDS-1355. Only FQDN is accepted for OM rpc address in secure environment. Contributed by Ajay Kumar. 
Closes #677
 2019-04-02 15:43:01 +02:00
Yiqun Lin
d31c86892e HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee. 2019-04-02 16:32:30 +08:00
Shashikant Banerjee
ef5de29243 HDDS-1312. Add more unit tests to verify BlockOutputStream functionalities. Contributed by Shashikant Banerjee. 2019-04-01 16:09:58 +05:30
Shashikant Banerjee
509f31b109 HDDS-1295. Add MiniOzoneChaosCluster to mimic long running workload in a unit test environment. Contributed by Mukul Kumar Singh. 2019-04-01 15:50:03 +05:30
Bharat Viswanadham
bf3b7fd732
HDDS-1288. SCM - Failing test on trunk that waits for HB report processing. Contributed by Siddharth Wagle. 2019-03-30 10:54:45 -07:00
Elek, Márton
ec82e4cab2 HDDS-1357. ozone s3 shell command has confusing subcommands (#663) 2019-03-30 10:13:29 -07:00
Xiaoyu Yao
ca5e4ce036
HDDS-1318. Fix MalformedTracerStateStringException on DN logs. Contributed by Xiaoyu Yao. 
This closes #641
 2019-03-28 12:00:58 -07:00