big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,897 Commits 2 Branches 2 Tags 264 MiB
9f0d3412f8
Commit Graph

694 Commits

Author SHA1 Message Date
Siddharth
59ded7641f HDDS-1411. Add unit test to check if SCM correctly sends close commands for containers in closing state after a restart. (#755) 2019-04-23 21:04:14 +05:30
Nanda kumar
7e1f8d3a1b
HDDS-1368. Cleanup old ReplicationManager code from SCM. 2019-04-23 17:35:39 +05:30
Chen, Junjie
518f47bf9b HDDS-976: Parse network topology from yaml file. Contributed by Junjie Chen. (#661) 
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
 2019-04-18 16:53:25 -07:00
Márton Elek
8aea5e4cb6
HDDS-1433. Rename GetScmInfoRespsonseProto to GetScmInfoResponseProto due to typos. Contributed by Wanqiang Ji. 2019-04-18 08:53:08 +02:00
Shashikant Banerjee
d608be660f HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException.(#729) 2019-04-17 14:49:02 +05:30
Shashikant Banerjee
082f1e0437 Revert "HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)" 
This reverts commit df2ae27f3e.
 2019-04-17 14:38:08 +05:30
Shashikant Banerjee
df2ae27f3e HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729) 2019-04-17 14:25:35 +05:30
Shashikant Banerjee
e5a918cda3 Revert "Merge pull request #729 from bshashikant/HDDS-1373" 
This reverts commit 4ff4314e40, reversing
changes made to 04c0437d13.
 2019-04-17 14:17:54 +05:30
bshashikant
4ff4314e40
Merge pull request #729 from bshashikant/HDDS-1373 
HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee.
 2019-04-17 14:12:52 +05:30
Doroszlai, Attila
0a87fd1464 HDDS-1434. TestDatanodeStateMachine is flaky (#740) 2019-04-16 19:13:01 -07:00
Hanisha Koneru
e543c3b31a
HDDS-1376. Datanode exits while executing client command when scmId is null (#724) 2019-04-16 13:51:39 -07:00
Bharat Viswanadham
3ac3e50b20
HDDS-1374. ContainerStateMap cannot find container while allocating blocks. (#735) 2019-04-16 13:49:29 -07:00
Arpit Agarwal
ebbda181e4
HDDS-1198. Rename chill mode to safe mode. Contributed by Siddharth Wagle. (#737) 2019-04-13 07:27:03 -07:00
Arpit Agarwal
62f4808617
HDDS-1428. Remove benign warning in handleCreateContainer. Contributed by Siddharth Wagle. 2019-04-12 15:08:01 -07:00
Bharat Viswanadham
a458c5bd96
HDDS-1285. Implement actions need to be taken after chill mode exit wait time. (#612) 2019-04-12 09:40:27 -07:00
Arpit Agarwal
abace709cc
HDDS-1420. Tracing exception in DataNode HddsDispatcher 
Closes #723
 2019-04-12 12:21:10 +02:00
Arpit Agarwal
a9b46c58fe
HDDS-1421. Avoid unnecessary object allocations in TracingUtil 
Closes #722
 2019-04-12 11:52:31 +02:00
Doroszlai, Attila
3b08ac4659 HDDS-1363. ozone.metadata.dirs doesn't pick multiple dirs (#691) 2019-04-12 15:00:08 +05:30
Nanda kumar
c298415eb5
HDDS-1404. Fix typos in HDDS. Contributed by bianqi. 2019-04-12 14:12:10 +05:30
Shashikant Banerjee
22d0468b39 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee 2019-04-11 22:13:41 +05:30
Shashikant Banerjee
a0468c5756 HDDS-1348. Refactor BlockOutpuStream Class. Contributed by Shashikant Banerjee. 2019-04-11 19:31:26 +05:30
Nanda kumar
e9c4109004 HDDS-1417. After successfully importing a container, datanode should delete the container tar.gz file from working directory. (#721) 2019-04-10 16:55:37 -07:00
Bharat Viswanadham
0e770a6539
HDDS-1370. Command Execution in Datanode fails because of NPE (#715) 2019-04-10 10:25:28 -07:00
Márton Elek
dfb518bbf5
HDDS-1397. Avoid the usage of signal handlers in datanodes of the MiniOzoneClusters. 
Closes #701
 2019-04-10 18:35:36 +02:00
Mukul Kumar Singh
df01469141 HDDS-1401. Static ContainerCache in Datanodes can result in overwrite of container db. Contributed by Mukul Kumar Singh. (#708) 2019-04-10 18:00:10 +05:30
Mukul Kumar Singh
312d4d4cc5 HDDS-1388. Add a shell script to run MiniOzoneChaosCluster using mvn exec. Contributed by Mukul Kumar Singh. (#709) 2019-04-09 13:08:25 -07:00
Nanda kumar
73f43ac2dc
HDDS-1372. getContainerWithPipeline for a standalone pipeline fails with ConcurrentModificationException. (#682) 2019-04-09 15:10:07 +05:30
Shashikant Banerjee
2d4f6b6daa HDDS-372. There are three buffer copies in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-04-09 10:27:58 +05:30
Anu Engineer
dc013f7a35 HDDS-1329. Update documentation for Ozone-0.4.0 release. 
Contributed By Ajay Kumar.
 2019-04-05 14:14:42 -07:00
Doroszlai, Attila
274ab4e86c HDDS-1332. Attempt to fix flaky test testStartStopDatanodeStateMachine (#697) 2019-04-05 13:21:58 -07:00
Nanda kumar
5750bb94ed HDDS-1389. Fix testSCMChillModeRestrictedOp. (#696) 2019-04-05 10:42:00 -07:00
Ajay Kumar
0356b3699d HDDS-1333. OzoneFileSystem can't work with spark/hadoop2.7 because incompatible security classes. Contributed by Elek, Marton. (#653) 
(cherry picked from commit 7ec6a31eb314b9ecc1c2affbcc7d63c07bd33523)
 2019-04-05 09:29:21 -07:00
avijayanhwx
993f36ee33 HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681) 2019-04-04 19:18:32 +05:30
Shashikant Banerjee
33e159bf8d HDDS-1349. Remove watchClient from XceiverClientRatis. Contributed by Shashikant Banerjee. 2019-04-04 16:35:37 +05:30
Nanda kumar
48a58bce37
HDDS-1207. Refactor Container Report Processing logic and plugin new Replication Manager. (#662) 2019-04-04 16:32:59 +05:30
Yiqun Lin
a92806d05a HDDS-1189. Recon Aggregate DB schema and ORM. Contributed by Siddharth Wagle. 2019-04-04 17:33:37 +08:00
Hanisha Koneru
f09a78f73f
HDDS-1339. Implement ratis snapshots on OM (#651) 2019-04-03 22:50:28 -07:00
Yiqun Lin
f96fb05a2b HDDS-1365. Fix error handling in KeyValueContainerCheck. Contributed by Supratim Deka. 2019-04-03 14:01:30 +08:00
Kitti Nanasi
aeb43dfba3
HDDS-1153. Make tracing instrumentation configurable 
Closes #673
 2019-04-02 14:21:47 +02:00
Yiqun Lin
d31c86892e HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee. 2019-04-02 16:32:30 +08:00
Doroszlai, Attila
36756703f0 HDDS-1322. Hugo errors when building Ozone (#671) 2019-04-01 08:52:05 -07:00
Yiqun Lin
8bfef21efa HDDS-1360. Invalid metric type due to fully qualified class name. Contributed by Doroszlai, Attila. 2019-04-01 19:26:44 +08:00
Shweta Yakkali
82769ba1b8 HDDS-1352. Remove unused call in TestStorageContainerManagerHttpServer. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-29 17:26:48 +05:30
Xiaoyu Yao
d7a2f9456b HDDS-1350. Fix checkstyle issue in TestDatanodeStateMachine. Contributed by Xiaoyu Yao. (#656) 2019-03-28 14:50:34 -07:00
nilotpalnandi
4cceeb2682 HDDS-1309 . change logging from warn to debug in XceiverClient. Contributed by Nilotpal Nandi. (#638) 2019-03-28 12:22:44 -07:00
Shashikant Banerjee
ac4010bb22 HDDS-1293. ExcludeList#getProtoBuf throws ArrayIndexOutOfBoundsException. Contributed by Shashikant Banerjee. 2019-03-29 00:43:28 +05:30
Xiaoyu Yao
ca5e4ce036
HDDS-1318. Fix MalformedTracerStateStringException on DN logs. Contributed by Xiaoyu Yao. 
This closes #641
 2019-03-28 12:00:58 -07:00
Arpit Agarwal
b4ed81c4e6 HDDS-1332. Add some logging for flaky test testStartStopDatanodeStateMachine. Contributed by Arpit Agarwal. (#649) 2019-03-27 20:53:30 +05:30
Sandeep Nemuri
5c0a81ad3c HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-26 15:44:18 +05:30
Bharat Viswanadham
8739693514 HDDS-1217. Refactor ChillMode rules and chillmode manager. (#558) 2019-03-26 01:41:03 +05:30
Yiqun Lin
e5d72f504e HDDS-1234. Iterate the OM DB snapshot and populate the recon container DB. Contributed by Aravindan Vijayan. 2019-03-25 22:52:02 +08:00
Shashikant Banerjee
d4e4a7d456 HDDS-1317. KeyOutputStream#write throws ArrayIndexOutOfBoundsException when running RandomWrite MR examples. Contributed by Shashikant Banerjee. 2019-03-25 15:41:20 +05:30
Bharat Viswanadham
128dd91e10
HDDS-1331. In DatanodeStateMachine join check for not null. 2019-03-23 09:43:49 -07:00
Bharat Viswanadham
43e421afef
HDDS-918. Expose SCMMXBean as a MetricsSource. Contributed by Siddharth Wagle. 2019-03-22 16:44:12 -07:00
Ajay Kumar
771ea6b5e7 Revert "HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri." 
This reverts commit efad5717ec.
 2019-03-22 16:20:44 -07:00
Ajay Kumar
efad5717ec HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri. 2019-03-22 15:53:12 -07:00
Vivek Ratnavel Subramanian
73f7b04e2b HDDS-1302. Fix SCM CLI does not list container with id 1. 2019-03-22 15:15:01 -07:00
Arpit Agarwal
f854a89190
HDDS-1205. Refactor ReplicationManager to handle QUASI_CLOSED containers. Contributed by Nanda kumar. (#620) 2019-03-22 14:36:29 -07:00
Arpit Agarwal
90afc9ab03
HDDS-1320. Update ozone to latest ratis snapshot build (0.4.0-1fc5ace-SNAPSHOT). Contributed by Mukul Kumar Singh. 2019-03-21 14:24:02 -07:00
Yiqun Lin
60cdd4cac1 HDDS-1233. Create an Ozone Manager Service provider for Recon. Contributed by Aravindan Vijayan. 2019-03-21 11:02:29 +08:00
Mukul Kumar Singh
072750c00a HDDS-1306. TestContainerStateManagerIntegration fails in Ratis shutdown. Contributed by Lokesh Jain. 2019-03-20 17:14:19 +05:30
Ajay Yadav
f10d493325
HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601) 2019-03-18 23:08:17 -07:00
Arpit Agarwal
ae3a2c3851
Revert "HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)" 
This reverts commit 44b8451821.
 2019-03-18 14:21:57 -07:00
Sammi Chen
4d2a116d6e HDDS-699. Detect Ozone Network topology. Contributed by Sammi Chen. 2019-03-17 17:08:29 +08:00
Bharat Viswanadham
926d548caa
HDDS-1281. Fix the findbug issue caused by HDDS-1163. Contributed by Aravindan Vijayan. 2019-03-15 21:06:44 -07:00
Vivek Ratnavel Subramanian
af2dfc9f3d HDDS-1263. SCM CLI does not list container with id 1. 2019-03-15 15:39:30 -07:00
Elek, Márton
44b8451821 HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608) 2019-03-15 14:51:37 -07:00
Shashikant Banerjee
155ab6d5d8 HDDS-1098. Introduce Retry Policy in Ozone Client. Contributed by Shashikant Banerjee. 2019-03-15 15:40:59 +05:30
Vivek Ratnavel Subramanian
9001508485 HDDS-761. Create S3 subcommand to run S3 related operations. 
Closes #579
 2019-03-14 21:16:59 -07:00
Bharat Viswanadham
091a664977
HDDS-917. Expose NodeManagerMXBean as a MetricsSource. Contributed by Siddharth Wagle. 2019-03-14 12:41:52 -07:00
Shashikant Banerjee
d60673c470 HDDS-1257. Incorrect object because of mismatch in block lengths. Contributed by Shashikant Banerjee. 2019-03-14 19:32:36 +05:30
Márton Elek
3d140a537a
HDDS-1247. Bump trunk ozone version to 0.5.0. Contributed by Elek, Marton. 
Closes #588
 2019-03-14 12:01:25 +01:00
Mukul Kumar Singh
ea9fb07fd2 HDDS-1241. Update ozone to latest ratis snapshot build (0.4.0-5680cf5-SNAPSHOT). Contributed by Mukul Kumar Singh. 2019-03-14 15:18:06 +05:30
Ajay Yadav
4fa009989b
HDDS-1254. Fix failure in TestOzoneManagerHttpServer & TestStorageContainerManagerHttpServer. Contributed by Ajay Kumar. (#598) 2019-03-13 12:01:08 -07:00
Xiaoyu Yao
86d508c7c7 HDDS-1087. Fix TestDefaultCertificateClient#testSignDataStream. Contributed by Xiaoyu Yao. (#596) 2019-03-13 11:50:53 -07:00
Aravindan Vijayan
1f47fb7a2f HDDS-1209. Fix the block allocation logic in SCM when client wants to exclude all available open containers in a chosen pipeline. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-13 17:12:43 +05:30
Vivek Ratnavel Subramanian
c1649bf1e1
HDDS-1226. Addendum. ozone-filesystem jar missing in hadoop classpath 
Closes #560
 2019-03-13 12:08:45 +01:00
Lokesh Jain
0d62753da9 HDDS-1128. Create stateful manager class for the pipeline creation scheduling. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-13 16:30:06 +05:30
Mukul Kumar Singh
d602d462a6 HDDS-1256. Fix failure in TestKeyValueContainerCheck. Contributed by Supratim Deka. 2019-03-13 16:22:19 +05:30
Arpit Agarwal
24793d2d97
HDDS-1163. Basic framework for Ozone Data Scrubber. Contributed by Supratim Deka. 2019-03-13 04:32:39 +09:00
Márton Elek
c449cdebe6
HDDS-807. Period should be an invalid character in bucket names. Contributed by Siddharth Wagle. 2019-03-12 19:14:55 +01:00
Ajay Yadav
dcb0de848d
HDDS-1043. Enable token based authentication for S3 api 
Closes #561
 2019-03-12 18:47:06 +01:00
Xiaoyu Yao
2778259b88 HDDS-1253. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#593) 
Closes PR #593
 2019-03-12 09:34:18 -07:00
Mukul Kumar Singh
daf4660eeb HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh. 2019-03-12 20:02:19 +05:30
Márton Elek
d17e31e062
HDDS-1214. Enable tracing for the datanode read/write path. Contributed by Elek, Marton. 
Closes #550.
 2019-03-12 11:34:56 +01:00
Ajay Yadav
599e926d46 HDDS-596. Add robot test for OM Block Token. Contributed by Ajay Kumar. 
This closes (#581)
 2019-03-11 15:01:04 -07:00
Shashikant Banerjee
b4aa24d3c5 HDDS-1173. Fix a data corruption bug in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-03-11 23:15:49 +05:30
Bharat Viswanadham
68aa2ccd89
HDDS-1240. Fix check style issues caused by HDDS-1196. 2019-03-09 21:50:46 -08:00
Xiaoyu Yao
11db46956c
HDDS-1235. BaseHttpServer NPE is HTTP policy is HTTPS_ONLY. Contributed by Xiaoyu Yao. 
Closes #572
 2019-03-09 09:38:01 -08:00
Mukul Kumar Singh
8458ced11e HDDS-1238. Fix Ratis Snapshot creation error if the snapshot file already exists in ContainerStateMachine. Contributed by Shashikant Banerjee. 2019-03-09 22:46:20 +05:30
Bharat Viswanadham
341c076f05 HDDS-1196. Add a ReplicationStartTimer class. (#567) 2019-03-08 20:55:33 +05:30
Yiqun Lin
fb851c9481 HDDS-1232. Recon Container DB service definition. Contributed by Aravindan Vijayan. 2019-03-08 16:59:41 +08:00
Ajay Yadav
064f38b3a5 HDDS-594. SCM CA: DN sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. (#547) 2019-03-07 14:41:52 -08:00
Vivek Ratnavel Subramanian
475011bbf8
HDDS-1226. ozone-filesystem jar missing in hadoop classpath 
Closes #560
 2019-03-07 14:09:13 +01:00
Hanisha Koneru
bb12e81ec8
HDDS-1175. Serve read requests directly from RocksDB. (#557) 
HDDS-1175. Serve read requests directly from RocksDB.
 2019-03-06 19:44:55 -08:00
Vivek Ratnavel Subramanian
a55fc36299 HDDS-1093. Configuration tab in OM/SCM ui is not displaying the correct values. 2019-03-06 17:43:57 -08:00
Mukul Kumar Singh
2c3ec37738 HDDS-1210. Ratis pipeline creation doesn't check raft client reply status during initialization. Contributed by Mukul Kumar Singh. 2019-03-06 21:38:21 +05:30
Lokesh Jain
129fd5dd18 HDDS-1208. ContainerStateMachine should set chunk data as state machine data for ratis. Contributed by Lokesh Jain. 2019-03-06 17:00:37 +05:30
Shashikant Banerjee
62e89dc275 HDDS-1184. Parallelization of write chunks in datanodes is broken. Contributed by Shashikant Banerjee. 2019-03-06 10:00:16 +05:30
Arpit Agarwal
1ad5bfc53f
HDDS-1188. Implement a skeleton patch for Recon server with initial set of interfaces. Contributed by Siddharth Wagle. 2019-03-05 15:54:29 -08:00