big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,047 Commits 2 Branches 2 Tags 264 MiB
aa24add8f0
Commit Graph

1122 Commits

Author SHA1 Message Date
Elek, Márton
126ef77a81 HDDS-2154. Fix Checkstyle issues (#1475) 2019-09-19 11:30:33 -07:00
Elek, Márton
da1c67e0c2 HDDS-1054. List Multipart uploads in a bucket (#1277) 2019-09-19 11:06:02 -07:00
Doroszlai, Attila
c9900a01d1
HDDS-2141. Missing total number of operations 
Closes #1462
 2019-09-19 14:11:44 +02:00
dchitlangia
28913f733e
HDDS-2148. Remove redundant code in CreateBucketHandler.java 
Closes #1471
 2019-09-19 12:26:53 +02:00
dchitlangia
5c963a75d6
HDDS-2016. Add option to enforce GDPR in Bucket Create command 
Closes #1458
 2019-09-19 11:18:16 +02:00
Doroszlai, Attila
1029060e61
HDDS-2147. Include dumpstream in test report 
Closes #1470
 2019-09-19 10:41:00 +02:00
cxorm
ef478fe73e
HDDS-730. ozone fs cli prints hadoop fs in usage 
Closes #1464
 2019-09-19 09:16:12 +02:00
Bharat Viswanadham
f16eb093f7
HDDS-2144. MR job failing on secure Ozone cluster. (#1466) 2019-09-18 13:27:08 -07:00
Elek, Márton
53b57fdfa4 HDDS-2121. Create a shaded ozone filesystem (client) jar (#1452) 2019-09-18 13:25:15 -07:00
bshashikant
2c52d00a27
HDDS-2032. Ozone client should retry writes in case of any ratis/stateMachine exceptions. Contributed by Shashikant Banerjee (#1420). 2019-09-18 22:30:41 +05:30
Siyao Meng
8d9e9ec3e5
HDDS-2065. Implement OMNodeDetails#toString 
Closes #1387
 2019-09-18 17:10:26 +02:00
Márton Elek
c28e731156
Wrong commit message: Revert second "HDDS-2143. Rename classes under package org.apache.hadoop.utils" 
This reverts commit 111b08a330.
 2019-09-18 17:09:33 +02:00
Siyao Meng
111b08a330
HDDS-2143. Rename classes under package org.apache.hadoop.utils 
Closes #1465
 2019-09-18 17:00:26 +02:00
Bharat Viswanadham
6d4b20c047
HDDS-2143. Rename classes under package org.apache.hadoop.utils 
Closes #1465
 2019-09-18 16:51:39 +02:00
Viraj Jasani
285ed0a849
HDDS-2137. HddsClientUtils and OzoneUtils have duplicate verifyResourceName() 
Closes #1455
 2019-09-18 16:43:24 +02:00
Doroszlai, Attila
087ed86bcc
HDDS-2138. OM bucket operations do not add up 
Closes #1468
 2019-09-18 15:23:27 +02:00
Doroszlai, Attila
419dd0faf6
HDDS-2134. OM Metrics graphs include empty request type 
Closes #1451
 2019-09-18 14:09:15 +02:00
Márton Elek
15fded2788
HDDS-2022. Add additional freon tests 
Closes #1341
 2019-09-18 13:28:43 +02:00
Doroszlai, Attila
b88b6826c9 HDDS-2136. OM block allocation metric not paired with its failures (#1460) 2019-09-17 17:31:08 -07:00
Doroszlai, Attila
a9ba2b6710 HDDS-2142. OM metrics mismatch (abort multipart request) (#1461) 2019-09-17 17:27:08 -07:00
Doroszlai, Attila
f3de141787
HDDS-2135. OM Metric mismatch (MultipartUpload failures) 
Closes #1453
 2019-09-17 12:41:23 +02:00
Márton Elek
3a549cea19
HDDS-2120. Remove hadoop classes from ozonefs-current jar 
Closes #1434
 2019-09-17 12:28:40 +02:00
avijayanhwx
4f67081038 HDDS-2098 : Ozone shell command prints out ERROR when the log4j file … (#1411) 2019-09-16 22:36:53 -07:00
Mukul Kumar Singh
292bce7908 HDDS-2114. Rename does not preserve non-explicitly created interim directories. Contributed by Lokesh Jain & Istvan Fajth. 2019-09-17 10:47:12 +05:30
Márton Elek
2358e53e9c HDDS-2111. XSS fragments can be injected to the S3g landing page 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-16 15:48:09 -07:00
Vivek Ratnavel Subramanian
66bd1681f8 HDDS-2107. Datanodes should retry forever to connect to SCM in an unsecure environment (#1424) 2019-09-16 12:58:16 -07:00
Márton Elek
c8d61ffef6 HDDS-2030. Generate simplifed reports by the dev-support/checks/*.sh scripts 
Signed-off-by: Anu Engineer <aengineer@apache.org>
Co-Authored-By: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
 2019-09-16 12:17:33 -07:00
Xiaoyu Yao
56f042c48f
HDDS-2078. Get/Renew DelegationToken NPE after HDDS-1909 
Closes #1444
 2019-09-16 16:58:10 +02:00
Doroszlai, Attila
e952ecf807
HDDS-2109. Refactor scm.container.client config 
Closes #1426
 2019-09-16 15:22:11 +02:00
shwetayakkali@cloudera.com
1e13fe6310
HDDS-2044.Remove 'ozone' from the recon module names. 2019-09-16 14:18:01 +02:00
Arpit Agarwal
6a9f7caef4
Revert "HDDS-2057. Incorrect Default OM Port in Ozone FS URI Error Message." 
This reverts commit 95010a41fc.
 2019-09-13 11:40:42 -07:00
Siyao Meng
e22a324f87 HDDS-2007. Make ozone fs shell command work with OM HA service ids (#1360) 2019-09-13 11:22:00 -07:00
avijayanhwx
8024818d33 HDDS-1786 : Datanodes takeSnapshot should delete previously created snapshots (#1163) 2019-09-13 23:20:31 +05:30
sdeka
95010a41fc HDDS-2057. Incorrect Default OM Port in Ozone FS URI Error Message. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-13 07:08:31 -07:00
Shashikant Banerjee
fe8cdf0ab8 HDDS-2076. Read fails because the block cannot be located in the container (#1410) 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-09-12 21:18:14 +05:30
Márton Elek
f537410563
HDDS-2106. Avoid usage of hadoop projects as parent of hdds/ozone 
Closes #1423
 2019-09-12 02:38:41 +02:00
Doroszlai, Attila
64ed6b177d HDDS-2075. Tracing in OzoneManager call is propagated with wrong parent (#1415) 2019-09-11 11:59:01 -07:00
Doroszlai, Attila
5a381f73e9 HDDS-2103. TestContainerReplication fails due to unhealthy container (#1421) 2019-09-11 19:49:10 +05:30
Xiaoyu Yao
650c4cead5
HDDS-2053. Fix TestOzoneManagerRatisServer failure. Contributed by Xiaoyu Yao. (#1373) 2019-09-09 15:38:11 -07:00
Shashikant Banerjee
469165e6f2 HDDS-1843. Undetectable corruption after restart of a datanode. Contributed by Shashikant Banerjee(#1364). 2019-09-09 22:43:20 +05:30
Vivek Ratnavel Subramanian
3b9584d12b HDDS-2087. Remove the hard coded config key in ChunkManager (#1409) 2019-09-08 20:44:07 -07:00
Bharat Viswanadham
ca32917e0f
HDDS-2100. Ozone TokenRenewer provider is incorrectly configured. Contributed by Jitendra Nath Pandey. 2019-09-07 12:55:35 -07:00
Sammi Chen
c46d43ab13 HDDS-1553. Add metrics in rack aware container placement policy. (#1361) 2019-09-06 17:11:54 -07:00
dchitlangia
b15c116c1e HDDS-2015. Encrypt/decrypt key using symmetric key while writing/reading 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-06 11:35:00 -07:00
Lokesh Jain
6e4cdf89ef HDDS-1561: Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove (#1401) 2019-09-06 13:15:49 +05:30
Anu Engineer
acbea8d976 HDDS-1708. Add container scrubber metrics. 
Contributed by Hrishikesh Gadre.
 2019-09-05 14:33:06 -07:00
Nanda kumar
2b16d5377c
HDDS-1898. GrpcReplicationService#download cannot replicate the container. (#1326) 2019-09-05 18:12:36 +05:30
Hrishikesh Gadre
f347c348d8 Add support for checksum verification in data scrubber 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-04 20:31:49 -07:00
Xiaoyu Yao
ae287474c0
HDDS-2079. Fix TestSecureOzoneManager. Contributed by Xiaoyu Yao. (#1400) 2019-09-04 14:03:38 -07:00
Bharat Viswanadham
cfa41a49af
HDDS-2018. Handle Set DtService of token for OM HA. (#1371) 2019-09-03 14:06:14 -07:00
Bharat Viswanadham
f25fe92743
HDDS-1909. Use new HA code for Non-HA in OM. (#1225) 2019-09-03 13:24:32 -07:00
Nanda kumar
0b9704f610
HDDS-1810. SCM command to Activate and Deactivate pipelines. (#1224) 2019-09-03 16:50:57 +05:30
avijayanhwx
b53d19a343 HDDS-1783 : Latency metric for applyTransaction in ContainerStateMachine (#1363). 2019-09-03 15:18:50 +05:30
Márton Elek
c187d2cb9f HDDS-2060. Create Ozone specific LICENSE file for the Ozone source package 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-31 15:01:46 -07:00
Doroszlai, Attila
c4411f7fdf HDDS-2042. Avoid log on console with Ozone shell 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-30 17:40:34 -07:00
Doroszlai, Attila
70855126d1 HDDS-2061. Add hdds.container.chunk.persistdata as exception to TestOzoneConfigurationFields (#1382) 2019-08-30 12:31:52 -07:00
Doroszlai, Attila
472a26d2b8 HDDS-2063. Integration tests create untracked file audit.log (#1384) 2019-08-30 12:23:25 -07:00
dineshchitlangia
46696bd9b0 HDDS-2014. Create Symmetric Key for GDPR (#1362) 2019-08-30 09:55:36 -07:00
Xiaoyu Yao
ec34cee5e3
HDDS-2047. Datanodes fail to come up after 10 retries in a secure env… (#1379) 2019-08-30 09:27:37 -07:00
Nanda kumar
22a58615a2
HDDS-2058. Remove hadoop dependencies in ozone build 
Closes #1376
 2019-08-30 10:21:04 +02:00
Elek, Márton
4f5f46eb4a HDDS-1935. Improve the visibility with Ozone Insight tool (#1255) 2019-08-29 17:07:55 -07:00
Vivek Ratnavel Subramanian
7b3fa4fcaa HDDS-2050. Error while compiling ozone-recon-web (#1374) 2019-08-29 16:57:02 -07:00
Doroszlai, Attila
c749f62470
HDDS-2045. Partially started compose cluster left running 
Closes #1358
 2019-08-29 09:46:50 +02:00
Márton Elek
371c9eb6a6
Revert "HDDS-1596. Create service endpoint to download configuration from SCM." 
This reverts commit c0499bd704.
 2019-08-29 09:25:03 +02:00
Márton Elek
aef6a4fe0d HDDS-1950. S3 MPU part-list call fails if there are no parts 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-28 12:08:16 -07:00
Márton Elek
2fcd0da7dc HDDS-1942. Support copy during S3 multipart upload part creation 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-28 11:56:50 -07:00
Márton Elek
addfb7ff7d HDDS-1937. Acceptance tests fail if scm webui shows invalid json 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-28 11:40:37 -07:00
supratimdeka
1407414a52 HDDS-1094. Performance test infrastructure : skip writing user data on Datanode. Contributed by Supratim Deka (#1323) 2019-08-28 10:05:20 -07:00
Anu Engineer
dc72782008 HDDS-738. Removing REST protocol support from OzoneClient. 
Contributed by Elek, Marton.
 2019-08-28 09:53:31 -07:00
Márton Elek
c0499bd704 HDDS-1596. Create service endpoint to download configuration from SCM. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-28 09:13:46 -07:00
bshashikant
5d31a4eff7 HDDS-1753. Datanode unable to find chunk while replication data using ratis. (#1318) 2019-08-28 10:54:44 +05:30
Vivek Ratnavel Subramanian
b1eee8b52e HDDS-1946. CertificateClient should not persist keys/certs to ozone.m… (#1311) 2019-08-27 17:29:27 -07:00
Nanda kumar
2b9cc7eb95 HDDS-2037. Fix hadoop version in pom.ozone.xml. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-27 16:14:01 -07:00
Shashikant Banerjee
66cfa482c4 HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee(#1226). 2019-08-27 23:38:43 +05:30
Shashikant Banerjee
ce8eb1283a Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee." 
This reverts commit 62445021d5 as it has unintended changes in DirectoryWithSnapshotFeature class..
 2019-08-27 23:23:44 +05:30
HUAN-PING SU
3329257d99 HDDS-1998. TestSecureContainerServer#testClientServerRatisGrpc is failing (#1328) 2019-08-27 11:58:23 +05:30
Bharat Viswanadham
d1aa8596e0
HDDS-1975. Implement default acls for bucket/volume/key for OM HA code. (#1315) 2019-08-26 11:05:40 -07:00
Hanisha Koneru
3f887f3b92
HDDS-1827. Load Snapshot info when OM Ratis server starts. (#1130) 2019-08-23 13:19:13 -07:00
Nanda kumar
20064b69a8
HDDS-1978. Create helper script to run blockade tests. (#1310) 2019-08-23 22:26:30 +05:30
Doroszlai, Attila
75bf090990
HDDS-2024. rat.sh: grep: warning: recursive search of stdin 
Closes #1343
 2019-08-23 12:32:40 +02:00
Vivek Ratnavel Subramanian
e2a55482ee
HDDS-2023. Fix rat check failures in trunk 
Closes #1342
 2019-08-23 12:02:45 +02:00
Vivek Ratnavel Subramanian
b4a95a2b00
HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trunk snapshot 
Closes #1339
 2019-08-23 10:56:00 +02:00
Elek, Márton
edd708527d HDDS-1948. S3 MPU can't be created with octet-stream content-type (#1266) 2019-08-22 23:15:43 -07:00
Doroszlai, Attila
f6af7d0fd7 HDDS-1808. TestRatisPipelineCreateAndDestory times out (#1338) 2019-08-22 17:55:42 -07:00
Bharat Viswanadham
4028cac56d
HDDS-1347. In OM HA getS3Secret call Should happen only leader OM. (#670) 2019-08-22 15:00:17 -07:00
avijayanhwx
28fb4b527a HDDS-2008 : Wrong package for RatisHelper class in hadoop-hdds/common module. (#1333) 2019-08-22 10:57:22 -07:00
Xiaoyu Yao
d58eba8672 HDDS-1927. Consolidate add/remove Acl into OzoneAclUtil class. Contributed by Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-21 16:16:18 -07:00
Márton Elek
8fc6567b94 HDDS-1871. Remove anti-affinity rules from k8s minkube example 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-08-21 12:45:13 -07:00
Doroszlai, Attila
2ae7f444bd HDDS-1999. Basic acceptance test and SCM/OM web UI broken by Bootstrap upgrade (#1327) 2019-08-21 12:16:14 -07:00
Bharat Viswanadham
217e74816c
HDDS-1973. Implement OM RenewDelegationToken request to use Cache and DoubleBuffer. (#1316) 2019-08-21 12:01:27 -07:00
Xiaoyu Yao
10b4997b42
HDDS-1965. Compile error due to leftover ScmBlockLocationTestIngClient file. (fix build) 2019-08-20 16:46:53 -07:00
Xiaoyu Yao
7653ebdbb2
Revert "HDDS-1965. Compile error due to leftover ScmBlockLocationTestIngClient file (#1293)" 
This reverts commit 83e452ecea.
 2019-08-20 16:46:52 -07:00
Sunil G
bd92462321 HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297 2019-08-20 18:42:19 +05:30
Shashikant Banerjee
62445021d5 HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee. 2019-08-20 14:37:01 +05:30
Bharat Viswanadham
4f925afa82
HDDS-1972. Provide example ha proxy with multiple s3 servers back end. (#1304) 2019-08-19 13:59:58 -07:00
avijayanhwx
d69a1a0aa4 HDDS-1105 : Add mechanism in Recon to obtain DB snapshot 'delta' updates from Ozone Manager (#1259) 2019-08-19 10:47:49 -07:00
Bharat Viswanadham
b83eae7bdb
HDDS-1974. Implement OM CancelDelegationToken request to use Cache and DoubleBuffer. (#1308) 2019-08-18 10:06:00 -07:00
Siyao Meng
3bba8086e0 HDDS-1938. Change omPort parameter type from String to int in BasicOzoneFileSystem#createAdapter (#1305) 2019-08-17 21:54:48 -07:00
Bharat Viswanadham
a46ba03d15
HDDS-1913. Fix OzoneBucket and RpcClient APIS for acl. (#1257) 2019-08-16 16:39:49 -07:00
Bharat Viswanadham
c8675ec42e
HDDS-1911. Support Prefix ACL operations for OM HA. (#1275) 2019-08-16 15:11:11 -07:00