Commit Graph

1143 Commits

Author SHA1 Message Date
dchitlangia
c55ac6a1c7 HDDS-2174. Delete GDPR Encryption Key from metadata when a Key is deleted
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-26 21:10:34 -07:00
Anu Engineer
06998a1126
HDDS-2180. Add Object ID and update ID on VolumeList Object. (#1526) 2019-09-26 09:41:03 -07:00
Vivek Ratnavel Subramanian
7b6219a8d1
HDDS-2182. Fix checkstyle violations introduced by HDDS-1738
Closes #1529
2019-09-26 12:09:28 +02:00
Márton Elek
f647185905 HDDS-2067. Create generic service facade with tracing/metrics/logging support
Signed-off-by: Anu Engineer <aengineer@apache.org>
Co-Authored-By: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
2019-09-25 17:19:22 -07:00
Doroszlai, Attila
16f626f7f0
HDDS-2165. Freon fails if bucket does not exists
Closes #1503
2019-09-25 13:31:35 +02:00
Yi Sheng
e6fb6ee94d
HDDS-1738. Add nullable annotation for OMResponse classes
Closes #1499
2019-09-25 12:14:07 +02:00
Doroszlai, Attila
8baebb54e1
HDDS-2171. Dangling links in test report due to incompatible realpath
Closes #1515
2019-09-25 12:02:42 +02:00
Vivek Ratnavel Subramanian
a3463812ef HDDS-2168. TestOzoneManagerDoubleBufferWithOMResponse sometimes fails with out of memory error (#1509) 2019-09-24 16:28:25 -07:00
Bharat Viswanadham
6917754ba7
HDDS-2172.Ozone shell should remove description about REST protocol support. Contributed by Siddharth Wagle. 2019-09-24 16:06:10 -07:00
Anu Engineer
f16cf877e5
HDDS-2170. Add Object IDs and Update ID to Volume Object (#1510) 2019-09-24 11:01:05 -07:00
Márton Elek
91f50b98ca
HDDS-2167. Hadoop31-mr acceptance test is failing due to the shading
Closes #1507
2019-09-24 17:52:36 +02:00
avijayanhwx
8f1a135118 HDDS-2081. Fix TestRatisPipelineProvider#testCreatePipelinesDnExclude. Contributed by Aravindan Vijayan. (#1506) 2019-09-24 12:59:51 +05:30
Xiaoyu Yao
6cbe5d3809 HDDS-2160. Add acceptance test for ozonesecure-mr compose. Contributed by Xiaoyu Yao. (#1490) 2019-09-23 13:28:38 -07:00
dchitlangia
3fd3d746fc HDDS-2161. Create RepeatedKeyInfo structure to be saved in deletedTable
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-23 13:15:47 -07:00
Vivek Ratnavel Subramanian
dbdc612b3b HDDS-2163. Add 'Replication factor' to the output of list keys (#1493) 2019-09-20 22:59:45 -07:00
Siyao Meng
d7d6ec8fae HDDS-2128. Make ozone sh command work with OM HA service ids (#1445) 2019-09-20 14:22:55 -07:00
Doroszlai, Attila
53637301d1 HDDS-2157. checkstyle: print filenames relative to project root (#1485) 2019-09-20 14:05:35 -07:00
Nanda kumar
b3173e1f58 HDDS-2001. Update Ratis version to 0.4.0.
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-20 10:03:46 -07:00
Anu Engineer
5553887d95 HDDS-1949. Missing or error-prone test cleanup.
Contributed by Doroszlai, Attila.
2019-09-20 09:57:08 -07:00
Xiaoyu Yao
d072d3304c HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao.
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-19 22:43:00 -07:00
Vivek Ratnavel Subramanian
b7ae8a96cd HDDS-2101. Ozone filesystem provider doesn't exist (#1473) 2019-09-19 16:28:29 -07:00
Elek, Márton
126ef77a81 HDDS-2154. Fix Checkstyle issues (#1475) 2019-09-19 11:30:33 -07:00
Elek, Márton
da1c67e0c2 HDDS-1054. List Multipart uploads in a bucket (#1277) 2019-09-19 11:06:02 -07:00
Doroszlai, Attila
c9900a01d1
HDDS-2141. Missing total number of operations
Closes #1462
2019-09-19 14:11:44 +02:00
dchitlangia
28913f733e
HDDS-2148. Remove redundant code in CreateBucketHandler.java
Closes #1471
2019-09-19 12:26:53 +02:00
dchitlangia
5c963a75d6
HDDS-2016. Add option to enforce GDPR in Bucket Create command
Closes #1458
2019-09-19 11:18:16 +02:00
Doroszlai, Attila
1029060e61
HDDS-2147. Include dumpstream in test report
Closes #1470
2019-09-19 10:41:00 +02:00
cxorm
ef478fe73e
HDDS-730. ozone fs cli prints hadoop fs in usage
Closes #1464
2019-09-19 09:16:12 +02:00
Bharat Viswanadham
f16eb093f7
HDDS-2144. MR job failing on secure Ozone cluster. (#1466) 2019-09-18 13:27:08 -07:00
Elek, Márton
53b57fdfa4 HDDS-2121. Create a shaded ozone filesystem (client) jar (#1452) 2019-09-18 13:25:15 -07:00
bshashikant
2c52d00a27
HDDS-2032. Ozone client should retry writes in case of any ratis/stateMachine exceptions. Contributed by Shashikant Banerjee (#1420). 2019-09-18 22:30:41 +05:30
Siyao Meng
8d9e9ec3e5
HDDS-2065. Implement OMNodeDetails#toString
Closes #1387
2019-09-18 17:10:26 +02:00
Márton Elek
c28e731156
Wrong commit message: Revert second "HDDS-2143. Rename classes under package org.apache.hadoop.utils"
This reverts commit 111b08a330.
2019-09-18 17:09:33 +02:00
Siyao Meng
111b08a330
HDDS-2143. Rename classes under package org.apache.hadoop.utils
Closes #1465
2019-09-18 17:00:26 +02:00
Bharat Viswanadham
6d4b20c047
HDDS-2143. Rename classes under package org.apache.hadoop.utils
Closes #1465
2019-09-18 16:51:39 +02:00
Viraj Jasani
285ed0a849
HDDS-2137. HddsClientUtils and OzoneUtils have duplicate verifyResourceName()
Closes #1455
2019-09-18 16:43:24 +02:00
Doroszlai, Attila
087ed86bcc
HDDS-2138. OM bucket operations do not add up
Closes #1468
2019-09-18 15:23:27 +02:00
Doroszlai, Attila
419dd0faf6
HDDS-2134. OM Metrics graphs include empty request type
Closes #1451
2019-09-18 14:09:15 +02:00
Márton Elek
15fded2788
HDDS-2022. Add additional freon tests
Closes #1341
2019-09-18 13:28:43 +02:00
Doroszlai, Attila
b88b6826c9 HDDS-2136. OM block allocation metric not paired with its failures (#1460) 2019-09-17 17:31:08 -07:00
Doroszlai, Attila
a9ba2b6710 HDDS-2142. OM metrics mismatch (abort multipart request) (#1461) 2019-09-17 17:27:08 -07:00
Doroszlai, Attila
f3de141787
HDDS-2135. OM Metric mismatch (MultipartUpload failures)
Closes #1453
2019-09-17 12:41:23 +02:00
Márton Elek
3a549cea19
HDDS-2120. Remove hadoop classes from ozonefs-current jar
Closes #1434
2019-09-17 12:28:40 +02:00
avijayanhwx
4f67081038 HDDS-2098 : Ozone shell command prints out ERROR when the log4j file … (#1411) 2019-09-16 22:36:53 -07:00
Mukul Kumar Singh
292bce7908 HDDS-2114. Rename does not preserve non-explicitly created interim directories. Contributed by Lokesh Jain & Istvan Fajth. 2019-09-17 10:47:12 +05:30
Márton Elek
2358e53e9c HDDS-2111. XSS fragments can be injected to the S3g landing page
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-16 15:48:09 -07:00
Vivek Ratnavel Subramanian
66bd1681f8 HDDS-2107. Datanodes should retry forever to connect to SCM in an unsecure environment (#1424) 2019-09-16 12:58:16 -07:00
Márton Elek
c8d61ffef6 HDDS-2030. Generate simplifed reports by the dev-support/checks/*.sh scripts
Signed-off-by: Anu Engineer <aengineer@apache.org>
Co-Authored-By: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
2019-09-16 12:17:33 -07:00
Xiaoyu Yao
56f042c48f
HDDS-2078. Get/Renew DelegationToken NPE after HDDS-1909
Closes #1444
2019-09-16 16:58:10 +02:00
Doroszlai, Attila
e952ecf807
HDDS-2109. Refactor scm.container.client config
Closes #1426
2019-09-16 15:22:11 +02:00
shwetayakkali@cloudera.com
1e13fe6310
HDDS-2044.Remove 'ozone' from the recon module names. 2019-09-16 14:18:01 +02:00
Arpit Agarwal
6a9f7caef4
Revert "HDDS-2057. Incorrect Default OM Port in Ozone FS URI Error Message."
This reverts commit 95010a41fc.
2019-09-13 11:40:42 -07:00
Siyao Meng
e22a324f87 HDDS-2007. Make ozone fs shell command work with OM HA service ids (#1360) 2019-09-13 11:22:00 -07:00
avijayanhwx
8024818d33 HDDS-1786 : Datanodes takeSnapshot should delete previously created snapshots (#1163) 2019-09-13 23:20:31 +05:30
sdeka
95010a41fc HDDS-2057. Incorrect Default OM Port in Ozone FS URI Error Message.
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-13 07:08:31 -07:00
Shashikant Banerjee
fe8cdf0ab8 HDDS-2076. Read fails because the block cannot be located in the container (#1410)
Signed-off-by: Nanda kumar <nanda@apache.org>
2019-09-12 21:18:14 +05:30
Márton Elek
f537410563
HDDS-2106. Avoid usage of hadoop projects as parent of hdds/ozone
Closes #1423
2019-09-12 02:38:41 +02:00
Doroszlai, Attila
64ed6b177d HDDS-2075. Tracing in OzoneManager call is propagated with wrong parent (#1415) 2019-09-11 11:59:01 -07:00
Doroszlai, Attila
5a381f73e9 HDDS-2103. TestContainerReplication fails due to unhealthy container (#1421) 2019-09-11 19:49:10 +05:30
Xiaoyu Yao
650c4cead5
HDDS-2053. Fix TestOzoneManagerRatisServer failure. Contributed by Xiaoyu Yao. (#1373) 2019-09-09 15:38:11 -07:00
Shashikant Banerjee
469165e6f2 HDDS-1843. Undetectable corruption after restart of a datanode. Contributed by Shashikant Banerjee(#1364). 2019-09-09 22:43:20 +05:30
Vivek Ratnavel Subramanian
3b9584d12b HDDS-2087. Remove the hard coded config key in ChunkManager (#1409) 2019-09-08 20:44:07 -07:00
Bharat Viswanadham
ca32917e0f
HDDS-2100. Ozone TokenRenewer provider is incorrectly configured. Contributed by Jitendra Nath Pandey. 2019-09-07 12:55:35 -07:00
Sammi Chen
c46d43ab13 HDDS-1553. Add metrics in rack aware container placement policy. (#1361) 2019-09-06 17:11:54 -07:00
dchitlangia
b15c116c1e HDDS-2015. Encrypt/decrypt key using symmetric key while writing/reading
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-06 11:35:00 -07:00
Lokesh Jain
6e4cdf89ef HDDS-1561: Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove (#1401) 2019-09-06 13:15:49 +05:30
Anu Engineer
acbea8d976 HDDS-1708. Add container scrubber metrics.
Contributed by Hrishikesh Gadre.
2019-09-05 14:33:06 -07:00
Nanda kumar
2b16d5377c
HDDS-1898. GrpcReplicationService#download cannot replicate the container. (#1326) 2019-09-05 18:12:36 +05:30
Hrishikesh Gadre
f347c348d8 Add support for checksum verification in data scrubber
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-04 20:31:49 -07:00
Xiaoyu Yao
ae287474c0
HDDS-2079. Fix TestSecureOzoneManager. Contributed by Xiaoyu Yao. (#1400) 2019-09-04 14:03:38 -07:00
Bharat Viswanadham
cfa41a49af
HDDS-2018. Handle Set DtService of token for OM HA. (#1371) 2019-09-03 14:06:14 -07:00
Bharat Viswanadham
f25fe92743
HDDS-1909. Use new HA code for Non-HA in OM. (#1225) 2019-09-03 13:24:32 -07:00
Nanda kumar
0b9704f610
HDDS-1810. SCM command to Activate and Deactivate pipelines. (#1224) 2019-09-03 16:50:57 +05:30
avijayanhwx
b53d19a343 HDDS-1783 : Latency metric for applyTransaction in ContainerStateMachine (#1363). 2019-09-03 15:18:50 +05:30
Márton Elek
c187d2cb9f HDDS-2060. Create Ozone specific LICENSE file for the Ozone source package
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-31 15:01:46 -07:00
Doroszlai, Attila
c4411f7fdf HDDS-2042. Avoid log on console with Ozone shell
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-30 17:40:34 -07:00
Doroszlai, Attila
70855126d1 HDDS-2061. Add hdds.container.chunk.persistdata as exception to TestOzoneConfigurationFields (#1382) 2019-08-30 12:31:52 -07:00
Doroszlai, Attila
472a26d2b8 HDDS-2063. Integration tests create untracked file audit.log (#1384) 2019-08-30 12:23:25 -07:00
dineshchitlangia
46696bd9b0 HDDS-2014. Create Symmetric Key for GDPR (#1362) 2019-08-30 09:55:36 -07:00
Xiaoyu Yao
ec34cee5e3
HDDS-2047. Datanodes fail to come up after 10 retries in a secure env… (#1379) 2019-08-30 09:27:37 -07:00
Nanda kumar
22a58615a2
HDDS-2058. Remove hadoop dependencies in ozone build
Closes #1376
2019-08-30 10:21:04 +02:00
Elek, Márton
4f5f46eb4a HDDS-1935. Improve the visibility with Ozone Insight tool (#1255) 2019-08-29 17:07:55 -07:00
Vivek Ratnavel Subramanian
7b3fa4fcaa HDDS-2050. Error while compiling ozone-recon-web (#1374) 2019-08-29 16:57:02 -07:00
Doroszlai, Attila
c749f62470
HDDS-2045. Partially started compose cluster left running
Closes #1358
2019-08-29 09:46:50 +02:00
Márton Elek
371c9eb6a6
Revert "HDDS-1596. Create service endpoint to download configuration from SCM."
This reverts commit c0499bd704.
2019-08-29 09:25:03 +02:00
Márton Elek
aef6a4fe0d HDDS-1950. S3 MPU part-list call fails if there are no parts
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-28 12:08:16 -07:00
Márton Elek
2fcd0da7dc HDDS-1942. Support copy during S3 multipart upload part creation
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-28 11:56:50 -07:00
Márton Elek
addfb7ff7d HDDS-1937. Acceptance tests fail if scm webui shows invalid json
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-28 11:40:37 -07:00
supratimdeka
1407414a52 HDDS-1094. Performance test infrastructure : skip writing user data on Datanode. Contributed by Supratim Deka (#1323) 2019-08-28 10:05:20 -07:00
Anu Engineer
dc72782008 HDDS-738. Removing REST protocol support from OzoneClient.
Contributed by Elek, Marton.
2019-08-28 09:53:31 -07:00
Márton Elek
c0499bd704 HDDS-1596. Create service endpoint to download configuration from SCM.
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-28 09:13:46 -07:00
bshashikant
5d31a4eff7 HDDS-1753. Datanode unable to find chunk while replication data using ratis. (#1318) 2019-08-28 10:54:44 +05:30
Vivek Ratnavel Subramanian
b1eee8b52e HDDS-1946. CertificateClient should not persist keys/certs to ozone.m… (#1311) 2019-08-27 17:29:27 -07:00
Nanda kumar
2b9cc7eb95 HDDS-2037. Fix hadoop version in pom.ozone.xml.
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-27 16:14:01 -07:00
Shashikant Banerjee
66cfa482c4 HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee(#1226). 2019-08-27 23:38:43 +05:30
Shashikant Banerjee
ce8eb1283a Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee."
This reverts commit 62445021d5 as it has unintended changes in DirectoryWithSnapshotFeature class..
2019-08-27 23:23:44 +05:30
HUAN-PING SU
3329257d99 HDDS-1998. TestSecureContainerServer#testClientServerRatisGrpc is failing (#1328) 2019-08-27 11:58:23 +05:30
Bharat Viswanadham
d1aa8596e0
HDDS-1975. Implement default acls for bucket/volume/key for OM HA code. (#1315) 2019-08-26 11:05:40 -07:00
Hanisha Koneru
3f887f3b92
HDDS-1827. Load Snapshot info when OM Ratis server starts. (#1130) 2019-08-23 13:19:13 -07:00
Nanda kumar
20064b69a8
HDDS-1978. Create helper script to run blockade tests. (#1310) 2019-08-23 22:26:30 +05:30