big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,027 Commits 2 Branches 2 Tags 264 MiB
b7cb8fe07c
Commit Graph

1116 Commits

Author SHA1 Message Date
Nanda kumar
a2cc961086 HDDS-1855. TestStorageContainerManager#testScmProcessDatanodeHeartbeat is failing. (#1153) 2019-07-26 10:54:23 +08:00
Siyao Meng
b7fba78fb6 HDDS-1830 OzoneManagerDoubleBuffer#stop should wait for daemon thread to die (#1156) 2019-07-25 16:14:50 -07:00
Bharat Viswanadham
3c4159ff3d
HDDS-1842. Implement S3 Abort MPU request to use Cache and DoubleBuffer. (#1155) 2019-07-25 08:51:11 -07:00
Bharat Viswanadham
1d98a212cb
HDDS-1819. Implement S3 Commit MPU request to use Cache and DoubleBuffer. (#1140) 2019-07-24 14:03:40 -07:00
Nanda kumar
cb69700ac6
HDDS-1853. Fix failing blockade test-cases. (#1151) 2019-07-25 00:02:28 +05:30
Bharat Viswanadham
cf9ff08f4b
HDDS-1848. Fix TestOzoneManagerHA and TestOzoneManagerSnapShotProvider. (#1148) 2019-07-23 20:23:06 -07:00
Bharat Viswanadham
ecb9f817ac
HDDS-1845. Fix OMVolumeSetQuota|OwnerRequest#validateAndUpdateCache return response. (#1141) 2019-07-23 14:45:47 -07:00
Doroszlai, Attila
278390f5bf HDDS-1710. Fix checkstyle errors 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-23 07:56:30 -07:00
Márton Elek
c533b79c32 HDDS-1710. Publish JVM metrics via Hadoop metrics 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-22 16:27:50 -07:00
Doroszlai, Attila
d59f2711e0 HDDS-1803. shellcheck.sh does not work on Mac 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-22 16:21:42 -07:00
Márton Elek
d70ec4b5fa HDDS-1799. Add goofyfs to the ozone-runner docker image 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-22 15:53:18 -07:00
Vivek Ratnavel Subramanian
f3f2f3419d HDDS-1585. Add LICENSE.txt and NOTICE.txt to Ozone Recon Web 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-22 15:26:23 -07:00
Bharat Viswanadham
2ea71d953b HDDS-1686. Remove check to get from openKeyTable in acl implementatio… (#966) 2019-07-22 15:11:10 -07:00
Hanisha Koneru
cdc36fe286 HDDS-1649. On installSnapshot notification from OM leader, download checkpoint and reload OM state (#948) 2019-07-22 12:06:55 -07:00
Bharat Viswanadham
340bbaf8bf HDDS-1840. Fix TestSecureOzoneContainer. (#1135) 2019-07-22 10:23:48 -07:00
Bharat Viswanadham
acdb0a1276
HDDS-1841. Fix TestSecureContainerServer. (#1136) 2019-07-20 08:14:23 -07:00
Bharat Viswanadham
4aa76e327d
HDDS-1805. Implement S3 Initiate MPU request to use Cache and DoubleBuffer. (#1108) 2019-07-20 08:11:36 -07:00
Bharat Viswanadham
f5b2f7538c
HDDS-1795. Implement S3 Delete Bucket request to use Cache and DoubleBuffer. (#1097) 2019-07-19 17:43:23 -07:00
Sammi Chen
69a46a95bb HDDS-1713. ReplicationManager fail to find proper node topology based… (#1112) 2019-07-19 15:45:26 -07:00
Xiaoyu Yao
4e66cb9333 HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao. (#1067) 
* HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao.

* use ip/hostname instead of network name for -o output and add smoke test
 2019-07-19 11:00:49 +08:00
Bharat Viswanadham
f22818d0d6
HDDS-1820. Fix numKeys metrics in OM HA. (#1116) 2019-07-18 09:36:15 -07:00
Shashikant Banerjee
8ef2365ffd HDDS-1779. TestWatchForCommit tests are flaky.Contributed by Shashikant Banerjee. (#1071) 2019-07-18 19:57:12 +05:30
Shashikant Banerjee
ccceedb432 HDDS-1780. TestFailureHandlingByClient tests are flaky. Contributed by Shashikant Banerjee. (#1073) 2019-07-18 16:01:58 +05:30
Lokesh Jain
53a4c22b40
HDDS-1481: Cleanup BasicOzoneFileSystem#mkdir (#1114) 2019-07-18 14:40:45 +05:30
Bharat Viswanadham
3dc256ef81
HDDS-1689. Implement S3 Create Bucket request to use Cache and DoubleBuffer. (#1088) 2019-07-17 21:46:27 -07:00
Bharat Viswanadham
73e6ffce69
HDDS-1802. Add Eviction policy for table cache. (#1100) 2019-07-17 18:19:06 -07:00
Bharat Viswanadham
0541322c9a
HDDS-1675. Cleanup Volume Request 2 phase old code. (#964) 2019-07-17 14:13:06 -07:00
Xiaoyu Yao
eb0849b547 HDDS-1801. Make Topology Aware Replication/Read non-default for ozone 0.4.1. Contributed by Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-17 10:32:59 -07:00
Doroszlai, Attila
256fcc160e
HDDS-1785. OOM error in Freon due to the concurrency handling 
Closes #1085
 2019-07-17 15:02:15 +02:00
Hanisha Koneru
7adc4a327a
HDDS-1775. Make OM KeyDeletingService compatible with HA model (#1063) 2019-07-16 16:59:57 -07:00
Xiaoyu Yao
71c21f0a9b
HDDS-1813. Fix false warning from ozones3 acceptance test. Contributed by Xiaoyu Yao. (#1107) 2019-07-16 13:47:51 -07:00
Xiaoyu Yao
d429d742f0 HDDS-1544. Support default Acls for volume, bucket, keys and prefix. Contributed by Ajay Kumar, Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-16 12:58:59 -07:00
Doroszlai, Attila
c5e3ab5a4d
HDDS-1793. Acceptance test of ozone-topology cluster is failing 
Closes #1096
 2019-07-16 16:52:14 +02:00
Shashikant Banerjee
d21eccf8ba HDDS-1492. Generated chunk size name too long. Contributed by Shashikannt Banerjee. (#1084) 2019-07-16 18:01:13 +05:30
Bharat Viswanadham
395cb3cfd7
HDDS-1736. Cleanup 2phase old HA code for Key requests. (#1038) 2019-07-15 21:51:59 -07:00
Bharat Viswanadham
ef66e4999f HDDS-1666. Issue in openKey when allocating block. (#943) 2019-07-15 17:54:41 -07:00
Bharat Viswanadham
585f4d5c64
HDDS-1761. Fix class hierarchy for KeyRequest and FileRequest classes. (#1052) 2019-07-15 17:53:19 -07:00
Anu Engineer
141151325b
Merge pull request #944 from elek/HDDS-1668 
HDDS-1668. Add liveness probe to the example k8s resources files
 2019-07-15 13:19:32 -07:00
Doroszlai, Attila
61bbdeee19
HDDS-1800. Result of author check is inverted 
Closes #1092
 2019-07-15 18:00:10 +02:00
Anu Engineer
4a70a0d816
HDDS-1735. Create separate unit and integration test executor dev-support script. Contributed by Elek, Marton. 2019-07-12 09:25:00 -07:00
Nanda kumar
190e4349d7
HDDS-1790. Fix checkstyle issues in TestDataScrubber 
Closes #1082
 2019-07-12 14:42:29 +02:00
Nanda kumar
7b8177ba0f
HDDS-1791. Update network-tests/src/test/blockade/README.md file 
Closes #1083
 2019-07-12 14:38:22 +02:00
Márton Elek
51e7dc3221
make author check yetus safe 2019-07-12 11:59:24 +02:00
Márton Elek
ca7b3e8310
reenable unit test execution 2019-07-12 11:59:20 +02:00
Márton Elek
317f2c896f
fix rat problem 2019-07-12 11:30:56 +02:00
Márton Elek
987bf3f696
fix print0 consequences 2019-07-12 11:30:56 +02:00
Márton Elek
6f25420f57
fixing ALL the checkstule issues under hadoop-ozone/dev-support/checks 2019-07-12 11:30:56 +02:00
Márton Elek
0bae9e8ec8
HDDS-1735. Create separate unit and integration test executor dev-support script 2019-07-12 11:30:55 +02:00
Márton Elek
9119ed07ff
HDDS-1384. TestBlockOutputStreamWithFailures is failing 
Closes #1029
 2019-07-12 11:00:03 +02:00
Hrishikesh Gadre
acef5e0cec HDDS-1201. Reporting corrupted containers info to SCM (#1032) 2019-07-11 14:49:58 +05:30
Anu Engineer
6872efcabf HDDS-1611.[Addendum] Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973) 
Fixes a build break in ozone.
 2019-07-10 11:28:18 -07:00
Ajay Yadav
cdb20adfcc HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973) 2019-07-10 11:03:58 -07:00
Nanda kumar
efb916457f HDDS-1778. Fix existing blockade tests. (#1068) 2019-07-10 09:43:58 -07:00
Márton Elek
bbf5844968
HDDS-1525. Mapreduce failure when using Hadoop 2.7.5 
Closes #1065
 2019-07-10 15:31:28 +02:00
Márton Elek
93824886e9
HDDS-1764. Fix hidden errors in acceptance tests 
Closes #1059
 2019-07-10 13:22:51 +02:00
supratimdeka
ac7a8accdf HDDS-1603. Handle Ratis Append Failure in Container State Machine. Contributed by Supratim Deka (#1019) 2019-07-10 07:49:36 +05:30
Xiaoyu Yao
030307226a
HDDS-1586. Allow Ozone RPC client to read with topology awareness. Contributed by Sammi Chen. 2019-07-09 14:43:55 -07:00
pifta
1dc58c5b04
HDDS-1742. Merge ozone-perf and ozonetrace example clusters 
Closes #1057
 2019-07-09 17:21:16 +02:00
Hanisha Koneru
8861573e8c
HDDS-1717. MR Job fails as OMFailoverProxyProvider has dependency hadoop-3.2 
Closes #1056
 2019-07-09 13:22:00 +02:00
Vivek Ratnavel Subramanian
82d88a8d30 HDDS-1705. Recon: Add estimatedTotalCount to the response of containers and containers/{id} endpoints. Contributed by Vivek Ratnavel Subramanian. 2019-07-08 21:06:50 -07:00
Mukul Kumar Singh
e5ffb88257 HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh. (#1050) 
* HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh.
 2019-07-08 20:24:12 -07:00
Lokesh Jain
b5d30e4914 HDDS-1750. Add block allocation metrics for pipelines in SCM. Contributed by Lokesh Jain. (#1047) 2019-07-09 08:42:55 +05:30
Mukul Kumar Singh
ce91d35b29
HDDS-1728. Add metrics for leader's latency in ContainerStateMachine. Contributed by Mukul Kumar Singh. (#1022) 2019-07-08 12:19:16 +05:30
Mukul Kumar Singh
c9c3429a16
HDDS-1758. Add replication and key deletion tests to MiniOzoneChaosCluster. Contributed by Mukul Kumar Singh. (#1049) 2019-07-05 17:04:40 +05:30
pifta
acd2d524e8
HDDS-1741. Fix prometheus configuration in ozoneperf example cluster 
Closes #1045
 2019-07-03 22:48:40 +02:00
Bharat Viswanadham
8965ddcf59
HDDS-1731. Implement File CreateFile Request to use Cache and DoubleBuffer. (#1044) 2019-07-03 13:15:04 -07:00
Bharat Viswanadham
91cc197227 HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. (#1048) 2019-07-02 16:02:27 -07:00
Márton Elek
75b1e458b1
HDDS-1716. Smoketest results are generated with an internal user 
Closes #1002
 2019-07-02 22:52:06 +02:00
Elek, Márton
564758ae8d HDDS-1698. Switch to use apache/ozone-runner in the compose/Dockerfile (#979) 
* HDDS-1698. Switch to use apache/ozone-runner in the compose/Dockerfile

* docker image is moved to the parent in HDDS-1667
 2019-07-02 09:54:43 -07:00
Márton Elek
d3a5abec71
HDDS-1668. Add liveness probe to the example k8s resources files 2019-07-02 03:12:28 +02:00
Márton Elek
2f4b37b53c
HDDS-1667. Docker compose file may referring to incorrect docker image name. Contributed by Eric Yang. 2019-07-02 02:19:57 +02:00
Vivek Ratnavel Subramanian
db674a0b14 HDDS-1685. Recon: Add support for "start" query param to containers and containers/{id} endpoints. 2019-07-01 12:44:14 -07:00
dineshchitlangia
1f75660350 HDDS-373. Genconf tool must generate ozone-site.xml with sample values (#1025) 2019-07-01 08:58:47 -07:00
Mukul Kumar Singh
9df6275954 HDDS-1555. Disable install snapshot for ContainerStateMachine. Contributed by Siddharth Wagle. (#846) 2019-07-01 20:52:32 +05:30
Bharat Viswanadham
1e727cf2a1
HDDS-1730. Implement File CreateDirectory Request to use Cache and Do… (#1026) 2019-06-30 22:38:53 -07:00
Bharat Viswanadham
49c5e8ac24
HDDS-1672. Improve locking in OzoneManager. (#1016) 2019-06-28 14:05:19 -07:00
Bharat Viswanadham
4848280437
HDDS-1638. Implement Key Write Requests to use Cache and DoubleBuffer. (#956) 2019-06-26 18:04:12 -07:00
avijayanhwx
0a37ccf379 HDDS-1691 : RDBTable#isExist should use Rocksdb#keyMayExist (#1013) 2019-06-26 11:44:49 -07:00
Bharat Viswanadham
a79bdf760e
HDDS-1727. Use generation of resourceName for locks in OzoneManagerLock. (#1014) 2019-06-25 17:12:45 -07:00
Anu Engineer
4b50981cfa HDDS-1709. TestScmSafeNode is flaky. 
Contributed by Elek, Marton.
 2019-06-25 13:57:01 -07:00
Bharat Viswanadham
049bb42760 HDDS-1723. Create new OzoneManagerLock class. (#1006) 
* HDDS-1723. Create new OzoneManagerLock class.

* add logs

* fix exception message

* add license

* fix findbug

* throw exception when acquiring 2nd lock failed.

* fix review comments

* fix missed review comment

* fix checkstyle

* fix review comments
 2019-06-25 08:48:03 -07:00
Elek, Márton
d023f1f864 HDDS-1646. Support real persistence in the k8s example files (#945) 
* HDDS-1646. Support real persistence in the k8s example files

* ephemeral clusters can be scaled up
 2019-06-24 12:02:35 -07:00
Elek, Márton
004207996c HDDS-1597. Remove hdds-server-scm dependency from ozone-common (#969) 
* HDDS-1597. Remove hdds-server-scm dependency from ozone-common. Contributed by Elek, Marton.

* checkstyle fixes

* revert the import reorder of HddsUtil

* add javadoc

* switch back to the commons-lang2

* fix typo

* fix metrics core classpath problem (+rebase fix)
 2019-06-24 11:56:04 -07:00
S O'Donnell
cba13c7f83
HDDS-1715. Update the Intellij runner definitition of SCM to use the new class name 
Closes #1000
 2019-06-21 16:25:10 +02:00
Márton Elek
256bdd6982
HDDS-1678. Default image name for kubernetes examples should be ozone and not hadoop 
Closes #959
 2019-06-21 14:14:06 +02:00
Bharat Viswanadham
d9a9e9913e
HDDS-1579. Create OMDoubleBuffer metrics. (#871) 2019-06-20 09:42:45 -07:00
Márton Elek
9c4b15d2f4
HDDS-1508. Provide example k8s deployment files for the new CSI server 
Closes #905
 2019-06-20 16:36:42 +02:00
Hanisha Koneru
2dfa932818
HDDS-1684. OM should create Ratis related dirs only if ratis is enabled (#965) 2019-06-18 16:08:48 -07:00
Vivek Ratnavel Subramanian
37bd5bb342 HDDS-1702. Optimize Ozone Recon build time (#982) 2019-06-18 14:44:23 -07:00
Vivek Ratnavel Subramanian
fb1ce0d50a HDDS-1670. Add limit support to /api/containers and /api/containers/{id} endpoints (#954) 2019-06-18 08:51:16 -07:00
Xudong Cao
3d020e914f HDDS-1532. Improve the concurrent testing framework of Freon. (#957) 2019-06-17 11:09:05 -07:00
S O'Donnell
22b36dd311
HDDS-1660. Use Picocli for Ozone Manager 
Closes #925
 2019-06-17 14:13:50 +02:00
Bharat Viswanadham
e70aeb4d7e
HDDS-1601. Implement updating lastAppliedIndex after buffer flush to OM DB. (#972) 2019-06-15 13:47:07 -07:00
Elek, Márton
89acbbc0b9 HDDS-1677. Auditparser robot test shold use a world writable working directory (#958) 2019-06-13 16:04:14 -07:00
Chen, Junjie
940bcf01cd HDDS-1587. Support dynamically adding delegated classes from to isolated class loader. Contributed by Junjie Chen. (#942) 2019-06-12 20:08:15 -07:00
Bharat Viswanadham
88c53d516c
HDDS-1620. Implement Volume Write Requests to use Cache and DoubleBuffer. (#884) 2019-06-12 17:45:42 -07:00
Xiaoyu Yao
a43f4440f7
HDDS-1543. Implement addAcl,removeAcl,setAcl,getAcl for Prefix. Contr… (#927) 2019-06-12 16:25:31 -07:00
Sammi Chen
1732312f45 HDDS-1663. Add datanode to network topology cluster during node regis… (#937) 2019-06-12 12:16:42 -07:00
Ajay Yadav
3b31694c35 HDDS-1545. Cli to add,remove,get and delete acls for Ozone objects. Contributed by Ajay Kumar. (#920) 2019-06-12 06:51:34 -07:00
Elek, Márton
96e1e41746 HDDS-1627. Make the version of the used hadoop-runner configurable (#886) 2019-06-11 11:18:12 -07:00
Elek, Márton
3c9a5e7b16 HDDS-1635. Maintain docker entrypoint and envtoconf inside ozone project (#894) 2019-06-11 11:12:05 -07:00
Márton Elek
f918e3fe62
HDDS-1669. SCM startup is failing if network-topology-default.xml is part of a jar 
Closes #946
 2019-06-11 19:30:22 +02:00
Sammi Chen
b05747901f
HDDS-1662. Missing test resources of integrataion-test project in target directory after compile 
Closes #933
 2019-06-11 12:25:42 +02:00
Ajay Yadav
9deac3b6bf HDDS-1657. Fix parallelStream usage in volume and key native acl. Contributed by Ajay Kumar. (#926) 2019-06-07 21:47:14 -07:00
Elek, Márton
46b23c11b0 HDDS-1636. Tracing id is not propagated via async datanode grpc call (#895) 2019-06-07 20:40:32 -07:00
S O'Donnell
85479577da
HDDS-1622. Use picocli for StorageContainerManager 
Closes #918
 2019-06-07 17:50:57 +02:00
Hanisha Koneru
a91d24fea4
HDDS-1496. Support partial chunk reads and checksum verification (#804) 2019-06-06 19:44:40 -07:00
Hanisha Koneru
0b115b60b0 HDDS-1371. OMSnapshotProvider to download DB checkpoint from leader OM. (#703) 2019-06-06 19:42:34 -07:00
Sammi Chen
8ca58efeec HDDS-1490. Support configurable container placement policy through 'o… (#903) 2019-06-06 12:14:47 -07:00
Xiaoyu Yao
76c0183ae3 HDDS-1652. HddsDispatcher should not shutdown volumeSet. Contributed by Xiaoyu Yao. (#916) 2019-06-06 23:47:59 +05:30
Bharat Viswanadham
0b8a2608e0 HDDS-1605. Implement AuditLogging for OM HA Bucket write requests. (#867) 2019-06-06 10:13:36 -07:00
S O'Donnell
36757ad6a9
HDDS-1645. Change the version of Pico CLI to the latest 3.x release - 3.9.6 
Closes #909
 2019-06-06 16:00:00 +02:00
Márton Elek
f7c77b395f
HDDS-1458. Create a maven profile to run fault injection tests. Contributed by Eric Yang. 2019-06-06 12:24:13 +02:00
Ajay Yadav
3b1c2577d7 HDDS-1541. Implement addAcl,removeAcl,setAcl,getAcl for Key. Contributed by Ajay Kumat. (#885) 2019-06-05 14:42:10 -07:00
Márton Elek
42cd861be0
HDDS-1628. Fix the execution and return code of smoketest executor shell script 
Closes #902
 2019-06-05 14:04:17 +02:00
Vivek Ratnavel Subramanian
ef2da40a70
HDDS-1640. Reduce the size of recon jar file 
Closes #907
 2019-06-05 07:51:23 +02:00
avijayanhwx
1a78794227 HDDS-1624 : Refactor operations inside the bucket lock in OM key write. (#882) 2019-06-04 15:41:37 -07:00
Bharat Viswanadham
580b639908
HDDS-1600. Add userName and IPAddress as part of OMRequest. (#857) 2019-06-04 12:31:44 -07:00
Elek, Márton
97607f3dce HDDS-1510. Classpath files are deployed to the maven repository as pom/jar files. Contributed by Elek, Marton. (#900) 2019-06-04 10:38:37 -07:00
bshashikant
e4b4fec66f HDDS-1224. Restructure code to validate the response from server in the Read path (#806) 2019-06-04 10:37:02 -07:00
Elek, Márton
50909a7aa0 HDDS-1641. Csi server fails because transitive Netty dependencies (#906) 2019-06-04 09:18:15 -05:00
Elek, Márton
5d5081eff8 HDDS-1631. Fix auditparser smoketests (#892) 2019-06-04 01:30:43 -05:00
Elek, Márton
e140a45046 HDDS-1629. Tar file creation can be optional for non-dist builds. Contributed by Elek, Marton. (#887) 2019-06-04 01:20:45 -05:00
Elek, Márton
1fc359fc10 HDDS-1607. Create smoketest for non-secure mapreduce example (#869) 
* HDDS-1607. Create smoketest for non-secure mapreduce example.

* remove hardcoded project version
 2019-06-04 01:18:02 -05:00
avijayanhwx
21de9af903 HDDS-1625 : ConcurrentModificationException when SCM has containers of different owners. (#883) 2019-06-04 01:15:04 +05:30
Shashikant Banerjee
f3271126fc HDDS-1558. IllegalArgumentException while processing container Reports. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-06-04 00:59:02 +05:30
Márton Elek
2a97a37d9e
Revert "HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860)" 
This reverts commit baee71551d.
 2019-06-03 08:34:19 +02:00
Ajay Yadav
c1d2d92187 HDDS-1540. Implement addAcl,removeAcl,setAcl,getAcl for Bucket. Contributed by Ajay Kumar. (#874) 2019-05-31 14:08:28 -07:00
Márton Elek
1ae062c818
HDDS-1382. Create customized CSI server for Ozone 
Closes #693
 2019-05-31 13:35:58 +02:00
Xiaoyu Yao
219e286722
HDDS-1608. Support Ozone Prefix ACLs in OM metadata table. Contributed by Xiaoyu Yao. (#875) 2019-05-30 16:44:38 -07:00
Bharat Viswanadham
c965f7f499
HDDS-1551. Implement Bucket Write Requests to use Cache and DoubleBuffer. (#850) 2019-05-30 15:28:15 -07:00
Elek, Márton
baee71551d HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860) 2019-05-30 09:20:16 -07:00
Márton Elek
1b041d4fd4
HDDS-1497. Refactor blockade Tests. Contributed by Nilotpal Nandi. 2019-05-30 16:46:06 +02:00
Shashikant Banerjee
8c8cb2d6aa HDDS-1502. Fix the compilation issue with commit 18c1eebc08. 2019-05-30 18:44:22 +05:30
Shashikant Banerjee
18c1eebc08 HDDS-1502. Add metrics for Ozone Ratis performance.Contributed by Shashikant Banerjee(#833). 2019-05-30 16:17:45 +05:30
Ajay Yadav
12be6ff2ff HDDS-1539. Implement addAcl,removeAcl,setAcl,getAcl for Volume. Contributed Ajay Kumar. (#847) 2019-05-29 23:18:06 -07:00
Xudong Cao
9ad7cad205 HDDS-1530. Freon support big files larger than 2GB and add --bufferSize and --validateWrites options. Contributed by Xudong Cao. (#830) 2019-05-29 15:57:48 -07:00
Xiaoyu Yao
0ead2090a6 HDDS-1542. Create Radix tree to support ozone prefix ACLs. Contributed by Xiaoyu Yao. 2019-05-29 22:39:27 +02:00
Akira Ajisaka
afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Xiaoyu Yao
7f2e87a419
HDDS-1604. ContainerReader#initializeUsedBytes leaks DB reference. Co… (#866) 2019-05-28 18:39:23 -07:00
Elek, Márton
79d14d0d42 HDDS-1341. TestContainerReplication#testContainerReplication fails intermittently. Contributed by Elek, Marton. (#862) 2019-05-28 14:40:45 -07:00
Xiaoyu Yao
fb0b39f4bf
HDDS-1536. testSCMSafeModeRestrictedOp is failing consistently. Contributed by Xiaoyu Yao. (#865) 2019-05-28 14:02:36 -07:00
Márton Elek
06fa4d2764
HDDS-1565. Rename k8s-dev and k8s-dev-push profiles to docker and docker-push. 
Closes: #840
 2019-05-28 16:53:54 +02:00
Mukul Kumar Singh
72dd79015a HDDS-1534. freon should return non-zero exit code on failure. Contributed by Nilotpal Nandi. 2019-05-28 07:52:56 +05:30
Shashikant Banerjee
83549dbbea HDDS-1509. TestBlockOutputStreamWithFailures#test2DatanodesFailure fails intermittently. Contributed by Shashikant Banerjee (#805). 2019-05-27 16:31:44 +05:30
Shashikant Banerjee
f0e44b3a3f HDDS-1584. Fix TestFailureHandlingByClient tests. Contributed by Shashikant Banerjee (#845). 2019-05-27 16:26:21 +05:30
Eric Yang
a3745c59a3
HDDS-1588. Update Ozone dist image to use jdk11. Contributed by Eric Yang. 2019-05-27 10:18:42 +02:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Bharat Viswanadham
6d0e79c121
HDDS-1512. Implement DoubleBuffer in OzoneManager. (#810) 2019-05-24 10:50:54 -07:00
avijayanhwx
4b099b8b89 HDDS-1501 : Create a Recon task interface to update internal DB on updates from OM. (#819) 2019-05-23 15:34:55 -07:00
Ajay Yadav
9c61494c02 HDDS-1065. OM and DN should persist SCM certificate as the trust root. Contributed by Ajay Kumar. (#834) 2019-05-22 11:47:32 -07:00
Mukul Kumar Singh
2fc6f8599a HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh. (#825) 2019-05-22 17:18:40 +05:30
Bharat Viswanadham
77c49f2963
HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714) 2019-05-21 18:41:30 -07:00
Ajay Yadav
ef1cc725b8
HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828) 2019-05-21 15:53:40 -07:00
Ajay Kumar
ff5691ca06
HDDS-1442. add spark container to ozonesecure-mr compose files 
Closes #746
 2019-05-21 17:10:46 +02:00
Lokesh Jain
c1d7d68c78
HDDS-1461. Optimize listStatus api in OzoneFileSystem (#782) 2019-05-21 14:47:48 +05:30
Vivek Ratnavel Subramanian
05db2a598e HDDS-1487. Bootstrap React framework for Recon UI (#831) 
HDDS-1487. Bootstrap React framework for Recon UI.
Contributed by Vivek Ratnavel Subramanian.
 2019-05-20 22:09:58 +02:00
Bharat Viswanadham
0d1d7c86ec HDDS-1499. OzoneManager Cache. (#798) 2019-05-19 19:23:02 -07:00
Siddharth
c183bd8e20 HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle. 
This closes #822.
 2019-05-16 15:13:10 -07:00
Elek, Márton
b7de735159 HDDS-1522. Provide intellij runConfiguration for Ozone components. Contributed by Elek, Marton. 2019-05-16 16:38:50 +02:00
supratimdeka
9569015802 HDDS-1511. Space tracking for Open Containers in HDDS Volumes. Contributed by Supratim Deka (#812) 2019-05-15 10:26:01 -07:00
Lokesh Jain
6bcc1dce52
HDDS-1503. Reduce garbage generated by non-netty threads in datanode ratis server (#813) 2019-05-14 14:49:50 +05:30
Hanisha Koneru
02c9efcb81
HDDS-1491. Ozone KeyInputStream seek() should not read the chunk file. (#795) 2019-05-13 20:49:52 -07:00
Vivek Ratnavel Subramanian
b832e174b0 HDDS-1474. "ozone.scm.datanode.id" config should take path for a dir and not a file (#809) 2019-05-09 17:11:09 -07:00
Hanisha Koneru
2d31ccc1b3
Revert "HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792)" (#808) 
This reverts commit 18cca583ac.
 2019-05-09 11:18:46 -07:00
Vivek Ratnavel Subramanian
18cca583ac HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792) 2019-05-09 10:23:44 -07:00
avijayanhwx
25951255ce HDDS-1500. Allocate block failures in client should print exception trace. Contributed by Aravindan Vijayan (#801). 2019-05-09 15:38:49 +05:30
Siddharth
c4be3ea276 HDDS-1489. Unnecessary log messages on console with Ozone shell. (#797) 2019-05-07 14:48:39 -07:00
avijayanhwx
eb9c8900bc HDDS-1475 : Fix OzoneContainer start method. (#788) 2019-05-07 14:09:41 -07:00
Shashikant Banerjee
7f0e2c67e0 HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749). Contributed by Shashikant Banerjee 2019-05-08 01:00:10 +05:30
Shashikant Banerjee
01451a57de Revert "HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)" 
This reverts commit d525633619.
 2019-05-08 00:49:15 +05:30
bshashikant
d525633619
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749) 
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749).
 2019-05-08 00:38:53 +05:30
Márton Elek
1a696cc761
HDDS-1424. Support multi-container robot test execution 
Closes #726
 2019-05-07 17:48:10 +02:00
Márton Elek
fb7c1cad0e
Revert "HDDS-1384. TestBlockOutputStreamWithFailures is failing" 
This reverts commit dead9b4049.
 2019-05-06 20:17:00 +02:00
Siddharth
1d70c8ca0f HDDS-1464. Client should have different retry policies for different exceptions. (#785) 2019-05-05 09:21:15 -07:00
avijayanhwx
f194540520 HDDS-1448 : RatisPipelineProvider should only consider open pipeline while excluding dn for pipeline allocation. (#786) 2019-05-04 00:19:00 +05:30
Siddharth
1df679985b HDDS-1473. DataNode ID file should be human readable. (#781) 2019-05-02 13:59:14 -07:00
Bharat Viswanadham
4605db369e
HDDS-1482. Use strongly typed codec implementations for the S3Table 
Closes #789
 2019-05-02 13:56:30 +02:00
Márton Elek
e2f0f72677
HDDS-1469. Generate default configuration fragments based on annotations 
Closes #773
 2019-05-02 12:14:43 +02:00
Márton Elek
f1673b0db1
HDDS-1478. Provide k8s resources files for prometheus and performance tests 
Closes #783
 2019-05-02 11:23:44 +02:00
Márton Elek
2ab318b603
HDDS-1412. Provide example k8s deployment files as part of the release package 
Closes #719
 2019-05-02 11:18:54 +02:00
dineshchitlangia
f682a171f5 HDDS-1483. Fix getMultipartKey javadoc. (#790) 2019-05-01 21:51:11 -07:00
Márton Elek
dead9b4049
HDDS-1384. TestBlockOutputStreamWithFailures is failing 
Closes #750
 2019-04-30 14:27:39 +02:00
Ajay Yadav
95790bb7e5
HDDS-1430. NPE if secure ozone if KMS uri is not defined. Contributed by Ajay Kumar. (#752) 2019-04-29 14:07:23 -07:00
Xiaoyu Yao
a96d16a843
HDDS-1455. Inconsistent naming convention with Ozone Kerberos configuration. Contributed by Xiaoyu Yao. (#762) 2019-04-29 11:18:11 -07:00
Ajay Yadav
dbce729b24 HDDS-1472. Add retry to kinit command in smoketests. Contributed by Ajay Kumar. (#778) 2019-04-29 07:46:01 -07:00
Ajay Yadav
b434f558ca
HDDS-1471. Update ratis dependency to 0.3.0. Contributed by Ajay Kumar. (#777) 2019-04-28 20:57:59 -07:00
Bharat Viswanadham
43b2a4b77b
HDDS-1456. Stop the datanode, when any datanode statemachine state is… (#769) 2019-04-26 14:25:34 -07:00
Hanisha Koneru
37582705fa
HDDS-1403. KeyOutputStream writes fails after max retries while writing to a closed container (#753) 2019-04-26 10:39:03 -07:00
Lokesh Jain
556eafd01a
HDDS-1460: Add the optmizations of HDDS-1300 to BasicOzoneFileSystem (#765) 2019-04-26 17:29:50 +05:30
Siddharth Wagle
c35abcd831
HDDS-999. Make the DNS resolution in OzoneManager more resilient 
Closes #758
 2019-04-26 12:38:24 +02:00
Kitti Nanasi
3f787cd506
HDDS-1192. Support -conf command line argument in GenericCli 
Closes #713
 2019-04-24 15:34:49 +02:00
Siddharth
59ded7641f HDDS-1411. Add unit test to check if SCM correctly sends close commands for containers in closing state after a restart. (#755) 2019-04-23 21:04:14 +05:30
Márton Elek
b979fdde99
HDDS-1447. Fix CheckStyle warnings. Contributed by Wanqiang Ji. 2019-04-18 08:54:43 +02:00
Shashikant Banerjee
d608be660f HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException.(#729) 2019-04-17 14:49:02 +05:30
Shashikant Banerjee
082f1e0437 Revert "HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)" 
This reverts commit df2ae27f3e.
 2019-04-17 14:38:08 +05:30
Shashikant Banerjee
df2ae27f3e HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729) 2019-04-17 14:25:35 +05:30
Shashikant Banerjee
e5a918cda3 Revert "Merge pull request #729 from bshashikant/HDDS-1373" 
This reverts commit 4ff4314e40, reversing
changes made to 04c0437d13.
 2019-04-17 14:17:54 +05:30
bshashikant
4ff4314e40
Merge pull request #729 from bshashikant/HDDS-1373 
HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee.
 2019-04-17 14:12:52 +05:30
Elek, Márton
04c0437d13 HDDS-1402. Remove unused ScmBlockLocationProtocol from ObjectStoreHandler (#707) 2019-04-16 19:16:05 -07:00
Hanisha Koneru
e543c3b31a
HDDS-1376. Datanode exits while executing client command when scmId is null (#724) 2019-04-16 13:51:39 -07:00
Arpit Agarwal
f2ab2795db
HDDS-1432. Ozone client list command truncates response without any indication. Contributed by Siddharth Wagle. 2019-04-16 12:35:49 -07:00
Shashikant Banerjee
bfcb6534cd HDDS-1380. Add functonality to write from multiple clients in MiniOzoneChaosCluster. Contributed by Shashikant Banerjee. 2019-04-16 18:52:07 +05:30
avijayanhwx
62e38ea911 HDDS-1396 : Recon start fails due to changes in Aggregate Schema definition. (#700) 2019-04-15 12:58:06 -07:00
Arpit Agarwal
ebbda181e4
HDDS-1198. Rename chill mode to safe mode. Contributed by Siddharth Wagle. (#737) 2019-04-13 07:27:03 -07:00
Elek, Márton
5379d85d8e HDDS-1387. ConcurrentModificationException in TestMiniChaosOzoneCluster. Contributed by Marton Elek. (#732) 2019-04-12 15:19:50 -07:00
Elek, Márton
0c1fec34f4 HDDS-1425. Ozone compose files are not compatible with the latest docker-compose. (#727) 2019-04-12 09:45:36 -07:00
Bharat Viswanadham
a458c5bd96
HDDS-1285. Implement actions need to be taken after chill mode exit wait time. (#612) 2019-04-12 09:40:27 -07:00
Arpit Agarwal
abace709cc
HDDS-1420. Tracing exception in DataNode HddsDispatcher 
Closes #723
 2019-04-12 12:21:10 +02:00
Doroszlai, Attila
75ba719ddd
HDDS-1419. Fix shellcheck errors in start-chaos.sh 
Closes #728
 2019-04-12 11:59:55 +02:00
Arpit Agarwal
fd676e190a
HDDS-1426. Minor logging improvements for MiniOzoneChaosCluster 
Closes #730
 2019-04-12 11:42:23 +02:00
Doroszlai, Attila
3b08ac4659 HDDS-1363. ozone.metadata.dirs doesn't pick multiple dirs (#691) 2019-04-12 15:00:08 +05:30
Nanda kumar
4003849fa4
HDDS-1416. MiniOzoneCluster should set custom value for hdds.datanode.replication.work.dir. 
Contributed by chencan.
 2019-04-12 14:45:57 +05:30
Arpit Agarwal
fcc7f9b32f
HDDS-1410. TestSCMNodeMetrics is flaky. Contributed by Siddharth Wagle. 2019-04-11 15:14:28 -07:00
Shashikant Banerjee
22d0468b39 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee 2019-04-11 22:13:41 +05:30
Shashikant Banerjee
a0468c5756 HDDS-1348. Refactor BlockOutpuStream Class. Contributed by Shashikant Banerjee. 2019-04-11 19:31:26 +05:30
Arpit Agarwal
feaab241e5
HDDS-1418. Move bang line to the start of the start-chaos.sh script. Contributed by Arpit Agarwal. (#720) 
Change-Id: I4fcf39d61a7d4c4ca79cb56a6958db0f691fe971
 2019-04-10 12:44:01 -07:00
Mukul Kumar Singh
df01469141 HDDS-1401. Static ContainerCache in Datanodes can result in overwrite of container db. Contributed by Mukul Kumar Singh. (#708) 2019-04-10 18:00:10 +05:30
Bharat Viswanadham
25c421bcc0
HDDS-1393. Convert all OM Bucket related operations to HA model. (#704) 2019-04-09 14:57:32 -07:00
Mukul Kumar Singh
312d4d4cc5 HDDS-1388. Add a shell script to run MiniOzoneChaosCluster using mvn exec. Contributed by Mukul Kumar Singh. (#709) 2019-04-09 13:08:25 -07:00
Lokesh Jain
a78bdd15fc
HDDS-1405. ITestOzoneContractCreate is failing. (#712) 2019-04-09 23:05:54 +05:30
Márton Elek
8ef3bc8113
HDDS-1383. Create project skeleton with CLI interface for In-place upgrade. 
Closes #694
 2019-04-09 17:35:14 +02:00
Shashikant Banerjee
2d4f6b6daa HDDS-372. There are three buffer copies in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-04-09 10:27:58 +05:30
Vivek Ratnavel Subramanian
69e3745b86 HDDS-1340. Add List Containers API for Recon (#648) 2019-04-08 09:36:06 -07:00
avijayanhwx
26dc940813 HDDS-1390. Fix OzoneS3 Gateway server due to exclusion of hk2-api. (#698) 2019-04-05 12:36:00 -07:00
Elek, Márton
89f9c8604b HDDS-1385. Make the ozonesecure-mr environment definition version independent. Contributed by Elek, Marton. (#695) 
(cherry picked from commit d758c978735fc9c0adb232527b2c3c0bdce1bafa)
 2019-04-05 09:30:40 -07:00
Ajay Kumar
0356b3699d HDDS-1333. OzoneFileSystem can't work with spark/hadoop2.7 because incompatible security classes. Contributed by Elek, Marton. (#653) 
(cherry picked from commit 7ec6a31eb314b9ecc1c2affbcc7d63c07bd33523)
 2019-04-05 09:29:21 -07:00
Bharat Viswanadham
77fe51e136
HDDS-1379. Convert all OM Volume related operations to HA model. (#689) 2019-04-04 21:09:50 -07:00
Bharat Viswanadham
d2141ef886
HDDS-1189. Build failing due to rat check failure introduced by HDDS-1189. Contributed by Siddharth Wagle. 2019-04-04 17:38:02 -07:00
avijayanhwx
993f36ee33 HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681) 2019-04-04 19:18:32 +05:30
Yiqun Lin
a92806d05a HDDS-1189. Recon Aggregate DB schema and ORM. Contributed by Siddharth Wagle. 2019-04-04 17:33:37 +08:00
Hanisha Koneru
f09a78f73f
HDDS-1339. Implement ratis snapshots on OM (#651) 2019-04-03 22:50:28 -07:00
Hanisha Koneru
4e64f8d763 HDDS-1324. TestOzoneManagerHA tests are flaky (#676) 2019-04-03 16:02:19 -07:00
Bharat Viswanadham
358f7f9b99
HDDS-1211. Test SCMChillMode failing randomly in Jenkins run (#543) 2019-04-03 15:02:00 -07:00
avijayanhwx
8ff41d6243 HDDS-1358 : Recon Server REST API not working as expected. (#668) 2019-04-03 14:52:06 -07:00
avijayanhwx
59022b28b7 HDDS-1330 : Add a docker compose for Ozone deployment with Recon. (#669) 2019-04-03 13:20:51 -07:00
Xiaoyu Yao
d6c233fce6 HDDS-1377. OM failed to start with incorrect hostname set as ip address in CSR. (#683) 2019-04-03 11:53:51 -07:00
Shashikant Banerjee
7cd7045eea HDDS-1164. Add New blockade Tests to test Replica Manager. Contributed by Nilotpal Nandi. 2019-04-03 22:19:10 +05:30
Ajay Kumar
bfc90bdc64
HDDS-1355. Only FQDN is accepted for OM rpc address in secure environment. Contributed by Ajay Kumar. 
Closes #677
 2019-04-02 15:43:01 +02:00
Kitti Nanasi
aeb43dfba3
HDDS-1153. Make tracing instrumentation configurable 
Closes #673
 2019-04-02 14:21:47 +02:00
Yiqun Lin
d31c86892e HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee. 2019-04-02 16:32:30 +08:00
Vivek Ratnavel Subramanian
04f1db8936 HDDS-1260. Create Recon Server lifecyle integration with Ozone. (#643) 2019-04-01 09:59:32 -07:00
Ajay Kumar
5f951ea2e3
HDDS-1255. Refactor ozone acceptance test to allow run in secure mode 2019-04-01 17:04:28 +02:00
Shashikant Banerjee
0df949202a HDDS-1067 . freon run on client gets hung when two of the datanodes are down in 3 datanode cluster. Contributed by Nilotpal Nandi. 2019-04-01 19:02:56 +05:30
Shashikant Banerjee
ef5de29243 HDDS-1312. Add more unit tests to verify BlockOutputStream functionalities. Contributed by Shashikant Banerjee. 2019-04-01 16:09:58 +05:30
Shashikant Banerjee
509f31b109 HDDS-1295. Add MiniOzoneChaosCluster to mimic long running workload in a unit test environment. Contributed by Mukul Kumar Singh. 2019-04-01 15:50:03 +05:30
Bharat Viswanadham
bf3b7fd732
HDDS-1288. SCM - Failing test on trunk that waits for HB report processing. Contributed by Siddharth Wagle. 2019-03-30 10:54:45 -07:00
Elek, Márton
ec82e4cab2 HDDS-1357. ozone s3 shell command has confusing subcommands (#663) 2019-03-30 10:13:29 -07:00
Doroszlai, Attila
7dc0ecc446 [HDDS-1351] NoClassDefFoundError when running ozone genconf (). Contributed by Doroszlai Attila. 
This closes #659.
 2019-03-29 09:01:56 -07:00
Lokesh Jain
6186ed94b9 HDDS-1300. Optimize non-recursive ozone filesystem apis. Contributed by Lokesh Jain. 2019-03-29 19:27:29 +05:30