big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,699 Commits 2 Branches 2 Tags 264 MiB
3329257d99
Commit Graph

997 Commits

Author SHA1 Message Date
Bharat Viswanadham
91f41b7d88
HDDS-1884. Support Bucket ACL operations for OM HA. (#1202) 2019-08-08 21:29:00 -07:00
Bharat Viswanadham
aa5f445fb9
HDDS-1863. Freon RandomKeyGenerator even if keySize is set to 0, it returns some random data to key. (#1167) 2019-08-08 15:40:19 -07:00
Siyao Meng
14a4ce3cee HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated. Contributed by Siyao Meng. 2019-08-08 13:38:10 -07:00
Xiaoyu Yao
3ac0f3a0c1
HDDS-1619. Support volume acl operations for OM HA. Contributed by… (#1147) 2019-08-08 09:55:46 -07:00
Bharat Viswanadham
63161cf590
HDDS-1926. The new caching layer is used for old OM requests but not updated 
Closes #1247
 2019-08-08 15:52:04 +02:00
Bharat Viswanadham
70b4617cfe
HDDS-1900. Remove UpdateBucket handler which supports add/remove Acl. (#1219) 2019-08-07 17:07:40 -07:00
Doroszlai, Attila
ab6a5c9d07 HDDS-1925. ozonesecure acceptance test broken by HTTP auth requirement (#1248) 2019-08-07 17:07:15 -07:00
Sammi Chen
8f9245bc2d HDDS-1865. Use "ozone.network.topology.aware.read" to control both RPC client and server side logic (#1184) 2019-08-07 16:59:13 -07:00
Xiaoyu Yao
70f46746b1
HDDS-1907. TestOzoneRpcClientWithRatis is failing with ACL errors. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-08-07 18:34:05 +05:30
Doroszlai, Attila
9cd211ac86 HDDS-1921. TestOzoneManagerDoubleBufferWithOMResponse is flaky (#1238) 2019-08-06 21:14:41 -07:00
HUAN-PING SU
38e6968647 HDDS-1919. Fix Javadoc in TestAuditParser (#1240) 2019-08-06 18:00:34 -07:00
Doroszlai, Attila
c4d97ae500 HDDS-1918. hadoop-ozone-tools has integration tests run as unit (#1236) 2019-08-06 17:00:54 -07:00
Xiaoyu Yao
a63023f261
HDDS-1901. Fix Ozone HTTP WebConsole Authentication. Contributed by Xiaoyu Yao. (#1228) 2019-08-06 12:07:53 -07:00
Bharat Viswanadham
c589983e9c HDDS-1893. Fix bug in removeAcl in Bucket. (#1216) 2019-08-05 10:25:18 -07:00
supratimdeka
f8ea6e1ce1 HDDS-1798. Propagate failure in writeStateMachineData to Ratis. Contributed by Supratim Deka (#1113) 2019-08-05 13:43:41 +05:30
Vivek Ratnavel Subramanian
ec1d453846 HDDS-1788. Fix kerberos principal error in Ozone Recon. (#1201) 2019-08-03 10:49:07 -07:00
Nanda kumar
eaf350c414 HDDS-1887. Enable all the blockade test-cases. Contibuted by NandaKumar(#1206). 2019-08-02 23:23:11 +05:30
Mukul Kumar Singh
b5c74d4ab8 HDDS-1782. Add an option to MiniOzoneChaosCluster to read files multiple times. Contributed by Mukul Kumar Singh. (#1076) 2019-08-02 21:50:39 +05:30
Bharat Viswanadham
c1f74405d7
HDDS-1885. Fix bug in checkAcls in OzoneManager. (#1199) 2019-07-31 19:00:37 -07:00
Bharat Viswanadham
a6f47b5876 HDDS-1875. Fix failures in TestS3MultipartUploadAbortResponse. (#1188) 2019-07-31 09:37:53 -07:00
Bharat Viswanadham
90e5eb0a48
HDDS-1849. Implement S3 Complete MPU request to use Cache and DoubleBuffer. (#1181) 2019-07-31 09:18:40 -07:00
Doroszlai, Attila
ac8ed7b5db
HDDS-1877. hadoop31-mapreduce fails due to wrong HADOOP_VERSION 
Closes #1193
 2019-07-31 16:07:27 +02:00
Bharat Viswanadham
e5c4131485
HDDS-1856. Make required changes for Non-HA to use new HA code in OM. (#1174) 2019-07-30 22:39:03 -07:00
Doroszlai, Attila
e68d8446c4 HDDS-1834. parent directories not found in secure setup due to ACL check. Contributed by Doroszlai, Attila. 
This closes #1171.
 2019-07-30 13:41:16 -07:00
Bharat Viswanadham
88daa730a3
HDDS-1872. Fix entry clean up from openKeyTable during complete MPU. (#1182) 2019-07-30 09:28:26 -07:00
Bharat Viswanadham
68f9f2d833
Revert HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated. (#1183) 2019-07-29 16:44:48 -07:00
Siyao Meng
4d07134b13 HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated (#1164) 2019-07-29 16:42:00 -07:00
avijayanhwx
60325c9611 HDDS-1391 : Add ability in OM to serve delta updates through an API. (#1033) 2019-07-29 09:39:40 -07:00
Márton Elek
90afb7bf8c
HDDS-1725. pv-test example to test csi is not working 2019-07-29 11:54:36 +02:00
Doroszlai, Attila
61ec03c966
HDDS-1852. Fix typo in TestOmAcls 
Closes #1173
 2019-07-29 10:46:11 +02:00
Sammi Chen
c01e137273 HDDS-1864. Turn on topology aware read in TestFailureHandlingByClient. (#1168) 2019-07-26 23:11:34 +05:30
Nanda kumar
a2cc961086 HDDS-1855. TestStorageContainerManager#testScmProcessDatanodeHeartbeat is failing. (#1153) 2019-07-26 10:54:23 +08:00
Siyao Meng
b7fba78fb6 HDDS-1830 OzoneManagerDoubleBuffer#stop should wait for daemon thread to die (#1156) 2019-07-25 16:14:50 -07:00
Bharat Viswanadham
3c4159ff3d
HDDS-1842. Implement S3 Abort MPU request to use Cache and DoubleBuffer. (#1155) 2019-07-25 08:51:11 -07:00
Bharat Viswanadham
1d98a212cb
HDDS-1819. Implement S3 Commit MPU request to use Cache and DoubleBuffer. (#1140) 2019-07-24 14:03:40 -07:00
Nanda kumar
cb69700ac6
HDDS-1853. Fix failing blockade test-cases. (#1151) 2019-07-25 00:02:28 +05:30
Bharat Viswanadham
cf9ff08f4b
HDDS-1848. Fix TestOzoneManagerHA and TestOzoneManagerSnapShotProvider. (#1148) 2019-07-23 20:23:06 -07:00
Bharat Viswanadham
ecb9f817ac
HDDS-1845. Fix OMVolumeSetQuota|OwnerRequest#validateAndUpdateCache return response. (#1141) 2019-07-23 14:45:47 -07:00
Doroszlai, Attila
278390f5bf HDDS-1710. Fix checkstyle errors 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-23 07:56:30 -07:00
Márton Elek
c533b79c32 HDDS-1710. Publish JVM metrics via Hadoop metrics 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-22 16:27:50 -07:00
Doroszlai, Attila
d59f2711e0 HDDS-1803. shellcheck.sh does not work on Mac 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-22 16:21:42 -07:00
Márton Elek
d70ec4b5fa HDDS-1799. Add goofyfs to the ozone-runner docker image 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-22 15:53:18 -07:00
Vivek Ratnavel Subramanian
f3f2f3419d HDDS-1585. Add LICENSE.txt and NOTICE.txt to Ozone Recon Web 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-22 15:26:23 -07:00
Bharat Viswanadham
2ea71d953b HDDS-1686. Remove check to get from openKeyTable in acl implementatio… (#966) 2019-07-22 15:11:10 -07:00
Hanisha Koneru
cdc36fe286 HDDS-1649. On installSnapshot notification from OM leader, download checkpoint and reload OM state (#948) 2019-07-22 12:06:55 -07:00
Bharat Viswanadham
340bbaf8bf HDDS-1840. Fix TestSecureOzoneContainer. (#1135) 2019-07-22 10:23:48 -07:00
Bharat Viswanadham
acdb0a1276
HDDS-1841. Fix TestSecureContainerServer. (#1136) 2019-07-20 08:14:23 -07:00
Bharat Viswanadham
4aa76e327d
HDDS-1805. Implement S3 Initiate MPU request to use Cache and DoubleBuffer. (#1108) 2019-07-20 08:11:36 -07:00
Bharat Viswanadham
f5b2f7538c
HDDS-1795. Implement S3 Delete Bucket request to use Cache and DoubleBuffer. (#1097) 2019-07-19 17:43:23 -07:00
Sammi Chen
69a46a95bb HDDS-1713. ReplicationManager fail to find proper node topology based… (#1112) 2019-07-19 15:45:26 -07:00
Xiaoyu Yao
4e66cb9333 HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao. (#1067) 
* HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao.

* use ip/hostname instead of network name for -o output and add smoke test
 2019-07-19 11:00:49 +08:00
Bharat Viswanadham
f22818d0d6
HDDS-1820. Fix numKeys metrics in OM HA. (#1116) 2019-07-18 09:36:15 -07:00
Shashikant Banerjee
8ef2365ffd HDDS-1779. TestWatchForCommit tests are flaky.Contributed by Shashikant Banerjee. (#1071) 2019-07-18 19:57:12 +05:30
Shashikant Banerjee
ccceedb432 HDDS-1780. TestFailureHandlingByClient tests are flaky. Contributed by Shashikant Banerjee. (#1073) 2019-07-18 16:01:58 +05:30
Lokesh Jain
53a4c22b40
HDDS-1481: Cleanup BasicOzoneFileSystem#mkdir (#1114) 2019-07-18 14:40:45 +05:30
Bharat Viswanadham
3dc256ef81
HDDS-1689. Implement S3 Create Bucket request to use Cache and DoubleBuffer. (#1088) 2019-07-17 21:46:27 -07:00
Bharat Viswanadham
73e6ffce69
HDDS-1802. Add Eviction policy for table cache. (#1100) 2019-07-17 18:19:06 -07:00
Bharat Viswanadham
0541322c9a
HDDS-1675. Cleanup Volume Request 2 phase old code. (#964) 2019-07-17 14:13:06 -07:00
Xiaoyu Yao
eb0849b547 HDDS-1801. Make Topology Aware Replication/Read non-default for ozone 0.4.1. Contributed by Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-17 10:32:59 -07:00
Doroszlai, Attila
256fcc160e
HDDS-1785. OOM error in Freon due to the concurrency handling 
Closes #1085
 2019-07-17 15:02:15 +02:00
Hanisha Koneru
7adc4a327a
HDDS-1775. Make OM KeyDeletingService compatible with HA model (#1063) 2019-07-16 16:59:57 -07:00
Xiaoyu Yao
71c21f0a9b
HDDS-1813. Fix false warning from ozones3 acceptance test. Contributed by Xiaoyu Yao. (#1107) 2019-07-16 13:47:51 -07:00
Xiaoyu Yao
d429d742f0 HDDS-1544. Support default Acls for volume, bucket, keys and prefix. Contributed by Ajay Kumar, Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-16 12:58:59 -07:00
Doroszlai, Attila
c5e3ab5a4d
HDDS-1793. Acceptance test of ozone-topology cluster is failing 
Closes #1096
 2019-07-16 16:52:14 +02:00
Shashikant Banerjee
d21eccf8ba HDDS-1492. Generated chunk size name too long. Contributed by Shashikannt Banerjee. (#1084) 2019-07-16 18:01:13 +05:30
Bharat Viswanadham
395cb3cfd7
HDDS-1736. Cleanup 2phase old HA code for Key requests. (#1038) 2019-07-15 21:51:59 -07:00
Bharat Viswanadham
ef66e4999f HDDS-1666. Issue in openKey when allocating block. (#943) 2019-07-15 17:54:41 -07:00
Bharat Viswanadham
585f4d5c64
HDDS-1761. Fix class hierarchy for KeyRequest and FileRequest classes. (#1052) 2019-07-15 17:53:19 -07:00
Anu Engineer
141151325b
Merge pull request #944 from elek/HDDS-1668 
HDDS-1668. Add liveness probe to the example k8s resources files
 2019-07-15 13:19:32 -07:00
Doroszlai, Attila
61bbdeee19
HDDS-1800. Result of author check is inverted 
Closes #1092
 2019-07-15 18:00:10 +02:00
Anu Engineer
4a70a0d816
HDDS-1735. Create separate unit and integration test executor dev-support script. Contributed by Elek, Marton. 2019-07-12 09:25:00 -07:00
Nanda kumar
190e4349d7
HDDS-1790. Fix checkstyle issues in TestDataScrubber 
Closes #1082
 2019-07-12 14:42:29 +02:00
Nanda kumar
7b8177ba0f
HDDS-1791. Update network-tests/src/test/blockade/README.md file 
Closes #1083
 2019-07-12 14:38:22 +02:00
Márton Elek
51e7dc3221
make author check yetus safe 2019-07-12 11:59:24 +02:00
Márton Elek
ca7b3e8310
reenable unit test execution 2019-07-12 11:59:20 +02:00
Márton Elek
317f2c896f
fix rat problem 2019-07-12 11:30:56 +02:00
Márton Elek
987bf3f696
fix print0 consequences 2019-07-12 11:30:56 +02:00
Márton Elek
6f25420f57
fixing ALL the checkstule issues under hadoop-ozone/dev-support/checks 2019-07-12 11:30:56 +02:00
Márton Elek
0bae9e8ec8
HDDS-1735. Create separate unit and integration test executor dev-support script 2019-07-12 11:30:55 +02:00
Márton Elek
9119ed07ff
HDDS-1384. TestBlockOutputStreamWithFailures is failing 
Closes #1029
 2019-07-12 11:00:03 +02:00
Hrishikesh Gadre
acef5e0cec HDDS-1201. Reporting corrupted containers info to SCM (#1032) 2019-07-11 14:49:58 +05:30
Anu Engineer
6872efcabf HDDS-1611.[Addendum] Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973) 
Fixes a build break in ozone.
 2019-07-10 11:28:18 -07:00
Ajay Yadav
cdb20adfcc HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973) 2019-07-10 11:03:58 -07:00
Nanda kumar
efb916457f HDDS-1778. Fix existing blockade tests. (#1068) 2019-07-10 09:43:58 -07:00
Márton Elek
bbf5844968
HDDS-1525. Mapreduce failure when using Hadoop 2.7.5 
Closes #1065
 2019-07-10 15:31:28 +02:00
Márton Elek
93824886e9
HDDS-1764. Fix hidden errors in acceptance tests 
Closes #1059
 2019-07-10 13:22:51 +02:00
supratimdeka
ac7a8accdf HDDS-1603. Handle Ratis Append Failure in Container State Machine. Contributed by Supratim Deka (#1019) 2019-07-10 07:49:36 +05:30
Xiaoyu Yao
030307226a
HDDS-1586. Allow Ozone RPC client to read with topology awareness. Contributed by Sammi Chen. 2019-07-09 14:43:55 -07:00
pifta
1dc58c5b04
HDDS-1742. Merge ozone-perf and ozonetrace example clusters 
Closes #1057
 2019-07-09 17:21:16 +02:00
Hanisha Koneru
8861573e8c
HDDS-1717. MR Job fails as OMFailoverProxyProvider has dependency hadoop-3.2 
Closes #1056
 2019-07-09 13:22:00 +02:00
Vivek Ratnavel Subramanian
82d88a8d30 HDDS-1705. Recon: Add estimatedTotalCount to the response of containers and containers/{id} endpoints. Contributed by Vivek Ratnavel Subramanian. 2019-07-08 21:06:50 -07:00
Mukul Kumar Singh
e5ffb88257 HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh. (#1050) 
* HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh.
 2019-07-08 20:24:12 -07:00
Lokesh Jain
b5d30e4914 HDDS-1750. Add block allocation metrics for pipelines in SCM. Contributed by Lokesh Jain. (#1047) 2019-07-09 08:42:55 +05:30
Mukul Kumar Singh
ce91d35b29
HDDS-1728. Add metrics for leader's latency in ContainerStateMachine. Contributed by Mukul Kumar Singh. (#1022) 2019-07-08 12:19:16 +05:30
Mukul Kumar Singh
c9c3429a16
HDDS-1758. Add replication and key deletion tests to MiniOzoneChaosCluster. Contributed by Mukul Kumar Singh. (#1049) 2019-07-05 17:04:40 +05:30
pifta
acd2d524e8
HDDS-1741. Fix prometheus configuration in ozoneperf example cluster 
Closes #1045
 2019-07-03 22:48:40 +02:00
Bharat Viswanadham
8965ddcf59
HDDS-1731. Implement File CreateFile Request to use Cache and DoubleBuffer. (#1044) 2019-07-03 13:15:04 -07:00
Bharat Viswanadham
91cc197227 HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. (#1048) 2019-07-02 16:02:27 -07:00
Márton Elek
75b1e458b1
HDDS-1716. Smoketest results are generated with an internal user 
Closes #1002
 2019-07-02 22:52:06 +02:00
Elek, Márton
564758ae8d HDDS-1698. Switch to use apache/ozone-runner in the compose/Dockerfile (#979) 
* HDDS-1698. Switch to use apache/ozone-runner in the compose/Dockerfile

* docker image is moved to the parent in HDDS-1667
 2019-07-02 09:54:43 -07:00