big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,424 Commits 2 Branches 2 Tags 264 MiB
7849bdcf70
Commit Graph

5563 Commits

Author SHA1 Message Date
Uma Maheswara Rao G
6215e35bb6 HDFS-11150: [SPS]: Provide persistence when satisfying storage policy. Contributed by Yuanbo Liu 2018-08-12 03:05:57 -07:00
Uma Maheswara Rao G
df2b551e79 HDFS-11293: [SPS]: Local DN should be given preference as source node, when target available in same node. Contributed by Yuanbo Liu and Uma Maheswara Rao G 2018-08-12 03:05:56 -07:00
Rakesh Radhakrishnan
422f870607 HDFS-11289. [SPS]: Make SPS movement monitor timeouts configurable. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:56 -07:00
Uma Maheswara Rao G
81ed364b63 HDFS-11193 : [SPS]: Erasure coded files should be considered for satisfying storage policy. Contributed by Rakesh R 2018-08-12 03:05:56 -07:00
Uma Maheswara Rao G
b7bed9f00a HDFS-11248: [SPS]: Handle partial block location movements. Contributed by Rakesh R 2018-08-12 03:05:55 -07:00
Uma Maheswara Rao G
d81611fe55 HDFS-11032: [SPS]: Handling of block movement failure at the coordinator datanode. Contributed by Rakesh R 2018-08-12 03:05:55 -07:00
Rakesh Radhakrishnan
5179d99b7e HDFS-11123. [SPS] Make storage policy satisfier daemon work on/off dynamically. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:55 -07:00
Rakesh Radhakrishnan
cd5262aba0 HDFS-10885. [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on. Contributed by Wei Zhou 2018-08-12 03:05:55 -07:00
Rakesh Radhakrishnan
b07291e176 HDFS-11151. [SPS]: StoragePolicySatisfier should gracefully handle when there is no target node with the required storage type. Contributed by Rakesh R 2018-08-12 03:05:54 -07:00
Rakesh Radhakrishnan
b67ae6d9d7 HDFS-10802. [SPS]: Add satisfyStoragePolicy API in HdfsAdmin. Contributed by Yuanbo Liu 2018-08-12 03:05:54 -07:00
Uma Maheswara Rao G
19b5aee3e4 HDFS-11068: [SPS]: Provide unique trackID to track the block movement sends to coordinator. Contributed by Rakesh R 2018-08-12 03:05:54 -07:00
Rakesh Radhakrishnan
047526b4c2 HDFS-11029. [SPS]:Provide retry mechanism for the blocks which were failed while moving its storage at DNs. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:53 -07:00
Rakesh Radhakrishnan
0f2d1ddc2c HDFS-10954. [SPS]: Provide mechanism to send blocks movement result back to NN from coordinator DN. Contributed by Rakesh R 2018-08-12 03:05:53 -07:00
Uma Maheswara Rao G
24add8c2f8 HDFS-10884: [SPS]: Add block movement tracker to track the completion of block movement future tasks at DN. Contributed by Rakesh R 2018-08-12 03:05:53 -07:00
Rakesh Radhakrishnan
e2a15d18bb HDFS-10801. [SPS]: Protocol buffer changes for sending storage movement commands from NN to DN. Contributed by Rakesh R 2018-08-12 03:05:53 -07:00
Uma Maheswara Rao G
1438da4944 HDFS-10800: [SPS]: Daemon thread in Namenode to find blocks placed in other storage than what the policy specifies. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:53 -07:00
Kai Zheng
5692887395 HDFS-10794. [SPS]: Provide storage policy satisfy worker at DN for co-ordinating the block storage movement work. Contributed by Rakesh R 2018-08-12 03:05:52 -07:00
Chao Sun
a2a8c48699 Remove super user privilege requirement for NameNode.getServiceStatus. Contributed by Chao Sun. 2018-08-10 15:59:39 -07:00
Virajith Jalaparti
15241c6349 HDFS-13795. Fix potential NPE in InMemoryLevelDBAliasMapServer. 2018-08-10 09:38:40 -07:00
Chen Liang
5326a7906d HDFS-13735. Make QJM HTTP URL connection timeout configurable. Contributed by Chao Sun. 2018-08-09 10:11:47 -07:00
Xiao Chen
9499df7b81 HDFS-13658. Expose HighestPriorityLowRedundancy blocks statistics. Contributed by Kitti Nanasi. 2018-08-08 10:40:20 -07:00
Márton Elek
36c0d742d4 HDFS-13447. Fix Typos - Node Not Chosen. Contributed by Beluga Behr. 2018-08-08 17:31:55 +02:00
Vinayakumar B
4203bc738c HDFS-13785. EC: 'removePolicy' is not working for built-in/system Erasure Code policies. Contributed by Ayush Saxena 2018-08-08 12:42:20 +05:30
Xiao Chen
6677717c68 HDFS-13728. Disk Balancer should not fail if volume usage is greater than capacity. Contributed by Stephen O'Donnell. 2018-08-07 22:05:17 -07:00
Vinayakumar B
2b0f977241 HDFS-13786. EC: Display erasure coding policy for sub-directories is not working. Contributed by Ayush Saxena. 2018-08-08 07:53:17 +05:30
Xiao Chen
0f8cb127cd HDFS-13799. TestEditLogTailer#testTriggersLogRollsForAllStandbyNN fails due to missing synchronization 
between rollEditsRpcExecutor and tailerThread shutdown. Contributed by Hrishikesh Gadre.
 2018-08-07 16:13:41 -07:00
Virajith Jalaparti
b1a59b1644 HDFS-13796. Allow verbosity of InMemoryLevelDBAliasMapServer to be configurable. 2018-08-07 10:15:28 -07:00
Brahma Reddy Battula
3108d27edd HDFS-12716. 'dfs.datanode.failed.volumes.tolerated' to support minimum number of volumes to be available. Contributed by Ranith Sardar and usharani 2018-07-30 15:50:04 +05:30
Arpit Agarwal
1c40bc2836 HDFS-13765. Fix javadoc for FSDirMkdirOp#createParentDirectories. Contributed by Lokesh Jain. 2018-07-27 10:14:01 -07:00
Anu Engineer
64e739e344 HDFS-13727. Log full stack trace if DiskBalancer exits with an unhandled exception. 
Contributed by Gabor Bota.
 2018-07-27 06:11:56 -07:00
Wei-Chiu Chuang
e95c5e9f62 HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 2018-07-25 11:04:18 -07:00
Xiao Chen
26864471c2 HDFS-13761. Add toString Method to AclFeature Class. Contributed by Shweta. 2018-07-24 21:42:47 -07:00
Daniel Templeton
849c45db18 HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica 
(Contributed by BELUGA BEHR via Daniel Templeton)

Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
 2018-07-24 16:05:27 -07:00
Konstantin V Shvachko
0a1e922f3d Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 2018-07-16 18:24:18 -07:00
Akira Ajisaka
88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Xiao Chen
87eeb26e72 HDFS-13663. Should throw exception when incorrect block size is set. Contributed by Shweta. 2018-07-12 20:24:11 -07:00
Xiao Chen
b37074be5a HDFS-12837. Intermittent failure in TestReencryptionWithKMS. 2018-07-11 21:03:19 -07:00
Bharat Viswanadham
c275a9a6a0 Merge trunk into HDDS-48 2018-07-09 12:13:03 -07:00
Andrew Wang
eecb5baaaa HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 2018-07-09 15:17:21 +02:00
Xiao Chen
936e0df0d3 HDFS-13721. NPE in DataNode due to uninitialized DiskBalancer. 2018-07-06 21:00:32 -07:00
Bharat Viswanadham
59a3038bc3 HDFS-13715:diskbalancer does not work if one of the blockpools are empty on a Federated cluster. Contributed by Bharat Viswanadham 2018-07-02 21:43:18 -07:00
Inigo Goiri
1804a31515 HDFS-13536. [PROVIDED Storage] HA for InMemoryAliasMap. Contributed by Virajith Jalaparti. 2018-07-02 10:48:20 -07:00
Andrew Wang
5d748bd056 HDFS-13702. Remove HTrace hooks from DFSClient to reduce CPU usage. Contributed by Todd Lipcon. 2018-07-02 12:11:06 +02:00
Andrew Wang
6ba9974108 HDFS-13703. Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit. Contributed by Todd Lipcon. 2018-07-02 12:02:19 +02:00
Inigo Goiri
73746c5da7 HDFS-13707. [PROVIDED Storage] Fix failing integration tests in ITestProvidedImplementation. Contributed by Virajith Jalaparti. 2018-06-29 09:56:13 -07:00
Bharat Viswanadham
c104525168 Merge remote-tracking branch 'apache-commit/trunk' into HDDS-48 2018-06-28 14:28:23 -07:00
Akira Ajisaka
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Yiqun Lin
30728aced4 HDFS-13692. StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet. Contributed by Bharat Viswanadham. 2018-06-22 10:50:54 +08:00
Akira Ajisaka
fba9d7cd74
HDFS-13621. Upgrade commons-lang version to 3.7 in hadoop-hdfs-project. Contributed by Takanobu Asanuma. 2018-06-18 10:17:07 -07:00
Inigo Goiri
d31a3ce767 HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 2018-06-15 15:59:19 -07:00
Wei-Chiu Chuang
c966a3837a HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 2018-06-15 13:36:46 -07:00
Mukul Kumar Singh
3e37a9a70b HDFS-13679. Fix Typo in javadoc for ScanInfoPerBlockPool#addAll. Contributed by Shashikant Banerjee. 2018-06-15 14:58:20 +05:30
Arpit Agarwal
418cff4820 Merge remote-tracking branch 'apache-commit/trunk' into HDDS-48 2018-06-14 09:38:20 -07:00
Yiqun Lin
8e7548d33b HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 2018-06-13 20:05:55 +08:00
Jitendra Pandey
f4c7c91123 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-13 00:36:02 -07:00
Inigo Goiri
2e5cfe6df3 HDFS-13653. Make dfs.client.failover.random.order a per nameservice configuration. Contributed by Ekanth Sethuramalingam. 2018-06-11 11:20:29 -07:00
Bharat Viswanadham
ccfb816d39 HDFS-13667:Typo: Marking all datandoes as stale. Contributed by Nanda Kumar 2018-06-09 16:43:03 -07:00
Arpit Agarwal
000a678396 HDFS-12670. can't renew HDFS tokens with only the hdfs client jar. Contributed by Arpit Agarwal. 2018-06-08 17:57:12 -07:00
Xiao Chen
cf4108313d HDFS-13642. Creating a file with block size smaller than EC policy's cell size should fail. 2018-06-08 15:14:11 -07:00
Brahma Reddy Battula
9654dd1f47 HDFS-12950. [oiv] ls will fail in secure cluster. Contributed by Wei-Chiu Chuang. 2018-06-07 10:16:02 +05:30
Hanisha Koneru
772c95395b HDDS-140. Add DU usage to VolumeInfo. 2018-06-06 16:21:21 -07:00
Akira Ajisaka
0e3c31579d
HDFS-13545. "guarded" is misspelled as "gaurded" in FSPermissionChecker.java. Contributed by Jianchao Jia. 2018-06-05 17:53:24 +09:00
Rushabh Shah
e2289c8d14 HDFS-13281 Namenode#createFile should be /.reserved/raw/ aware.. Contributed by Rushabh S Shah 2018-06-04 09:19:03 -05:00
Wei-Chiu Chuang
bccdfeee0a HDFS-13155. BlockPlacementPolicyDefault.chooseTargetInOrder Not Checking Return Value for NULL. Contributed by Zsolt Venczel. 2018-06-04 07:03:05 -07:00
Xiao Chen
9efb4b7db0 HDFS-13339. Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume. Contributed by liaoyuxiangqin and Zsolt Venczel. 2018-06-03 22:33:34 -07:00
Chao Sun
ff013d2c95 HDFS-13602. Add checkOperation(WRITE) checks in FSNamesystem. Contributed by Chao Sun. 2018-05-31 17:37:44 -07:00
Wei-Chiu Chuang
3f4a29813b HDFS-13646. DFSAdmin doesn't display specialized help for triggerBlockReport. Contributed by Takanobu Asanuma. 2018-05-31 15:26:08 -07:00
Konstantin V Shvachko
ebe5853a45 HDFS-12978. Fine-grained locking while consuming journal stream. Contributed by Konstantin Shvachko. 2018-05-31 14:56:32 -07:00
Yiqun Lin
b24098bc8f HDFS-13626. Fix incorrect username when deny the setOwner operation. Contributed by Zsolt Venczel. 2018-05-30 16:52:21 +08:00
Yiqun Lin
7a87add4ea HDFS-13598. Reduce unnecessary byte-to-string transform operation in INodesInPath#toString. Contributed by Gabor Bota. 2018-05-24 10:57:35 +08:00
Inigo Goiri
cddbbe5f69 HDFS-13493. Reduce the HttpServer2 thread count on DataNodes. Contributed by Erik Krogen. 2018-05-23 12:12:08 -07:00
Bharat Viswanadham
bc6d9d4c79 HDFS-13589: Add dfsAdmin command to query if upgrade is finalized. Contributed by Hanisha Koneru 2018-05-23 10:15:40 -07:00
Yiqun Lin
f749517cc7 HDFS-13573. Javadoc for BlockPlacementPolicyDefault is inaccurate. Contributed by Zsolt Venczel. 2018-05-18 10:43:53 +08:00
Kihwal Lee
0ce6290de6 HDFS-13581. DN UI logs link is broken when https is enabled. Contributed by Shashikant Banerjee. 2018-05-17 09:23:24 -05:00
Arpit Agarwal
0fc988e6a3 HDFS-13512. WebHdfs getFileStatus doesn't return ecPolicy. Contributed by Ajay Kumar. 2018-05-16 11:28:39 -07:00
Inigo Goiri
55d5549087 HDFS-13550.TestDebugAdmin#testComputeMetaCommand fails on Windows. Contributed by Anbang Hu. 2018-05-16 10:08:49 -07:00
Hanisha Koneru
6beb25ab7e HDFS-13544. Improve logging for JournalNode in federated cluster. 2018-05-14 10:12:08 -07:00
Anu Engineer
4b4f24ad5f HDDS-40. Separating packaging of Ozone/HDDS from the main Hadoop. 
Contributed by Elek, Marton.
 2018-05-11 14:24:36 -07:00
Xiao Chen
696a4be0da HDFS-12981. renameSnapshot a Non-Existent snapshot to itself should throw error. Contributed by Kitti Nanasi. 2018-05-07 15:36:29 -07:00
Aaron Fabbri
a732acd873
HDFS-5926 Documentation should clarify dfs.datanode.du.reserved impact from reserved disk capacity (Gabor Bota) 2018-05-04 16:38:30 -07:00
Inigo Goiri
fc074a359c HDFS-13283. Percentage based Reserved Space Calculation for DataNode. Contributed by Lukas Majercak. 2018-04-30 13:28:33 -07:00
Inigo Goiri
eb7fe1d588 HDFS-13509. Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows. Contributed by Xiao Liang. 2018-04-28 09:05:30 -07:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Mukul Kumar Singh
979bbb4019 HDFS-13423. Ozone: Clean-up of ozone related change from hadoop-hdfs-project. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
25f2398bbd HDFS-13395. Ozone: Plugins support in HDSL Datanode Service. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Anu Engineer
5e7164c614 HDFS-13137. Ozone: Ozonefs read fails because ChunkGroupInputStream#read does not iterate through all the blocks in the key. 
Contributed by Mukul Kumar Singh.
 2018-04-26 05:36:04 -07:00
Anu Engineer
61651dcf5c HDFS-13108. Ozone: OzoneFileSystem: Simplified url schema for Ozone File System. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
9a914126a7 HDFS-13008. Ozone: Add DN container open/close state to container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Nanda kumar
1094af072c HDFS-11699. Ozone:SCM: Add support for close containers in SCM. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
b4a3cf1476 HDFS-13070. Ozone: SCM: Support for container replica reconciliation - 1. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
4c10a849e8 HDFS-13078. Ozone: Update Ratis on Ozone to 0.1.1-alpha-8fd74ed-SNAPSHOT. 
To fix large chunk reads (>4M) from Datanodes. Contributed by Mukul Kumar Singh.
 2018-04-26 05:36:04 -07:00
Anu Engineer
136b419594 HDFS-13149. Ozone: Rename Corona to Freon. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
377b31ffa1 HDFS-13022. Block Storage: Kubernetes dynamic persistent volume provisioner. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
ee5495456e HDFS-13116. Ozone: Refactor Pipeline to have transport and container specific information. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
3cf07b43bc HDFS-13127. Fix TestContainerStateManager and TestOzoneConfigurationFields. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
443425a5d9 HDFS-12522. Ozone: Remove the Priority Queues used in the Container State Manager. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
32245c78e2 HDFS-13080. Ozone: Make finalhash in ContainerInfo of StorageContainerDatanodeProtocol.proto optional. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
7b3179f551 HDFS-13072. Ozone: DatanodeStateMachine: Handling Uncaught Exception in command handler thread. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
6b03fca37d HDFS-12868. Ozone: Service Discovery API. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
fb09d75379 HDFS-13018. Block Storage: make the iscsi target addres configurable for discovery. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
4bb9ad8e8f HDFS-13017. Block Storage: implement simple iscsi discovery in jscsi server. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
ba4d5a52a8 HDFS-12986. Ozone: Update ozone to latest ratis snapshot build (0.1.1-alpha-0f7169d-SNAPSHOT). Contributed by Lokesh Jain 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
94c0346f35 HDFS-13024. Ozone: ContainerStateMachine should synchronize operations between createContainer and writeChunk. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
4a051ba494 HDFS-12940. Ozone: KSM: TestKeySpaceManager#testExpiredOpenKey fails occasionally. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
1fb1ce107f HDFS-13012. TestOzoneConfigurationFields fails due to missing configs in ozone-default.xml. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
916f31f7d1 HDFS-13026. Ozone: TestContainerPersistence is failing becaue of container data mismatch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
28b87af51b HDFS-13013. Fix closeContainer API with the right container state change. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
98d62e55c4 Revert "HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee." 
This reverts commit 6ce5ec676164b84a9e2f8dc65b5f2199a141506d.
 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
98eeabbdb3 HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
98973cd0c4 HDFS-12966. Ozone: owner name should be set properly when the container allocation happens. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5e31b920f0 HDFS-12871. Ozone: Service Discovery: Adding REST server details in ServiceList. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
3965f1ec99 HDFS-12853. Ozone: Optimize chunk writes for Ratis by avoiding double writes. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5cc5149bd8 HDFS-12961. Ozone: Use relative resource urls in KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
5766792cad HDFS-12870. Ozone: Service Discovery: REST endpoint in KSM for getServiceList. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
5f10c2d8ce HDFS-12980. Ozone: SCM: Restructuring container state transition and event. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
87b7774488 HDFS-12958. Ozone: remove setAllocatedBytes method in ContainerInfo. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Nanda kumar
ce19f09bd0 HDFS-12908. Ozone: write chunk call fails because of Metrics registry exception. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
4b9f66a19e HDFS-12869. Ozone: Service Discovery: RPC endpoint in KSM for getServiceList. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
79432ee6cf HDFS-12863. Ozone: Avoid configuration.get() call for each container report. Contributed by Akira Ajisaka. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
0380dbb928 HDFS-12941. Ozone: ConfServlet does not trim values during the description parsing. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
4dae68eebf HDFS-12741. Ozone: KSM: ADD support for KSM --createObjectStore command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Chen Liang
caeeb78ca4 HDFS-12799. Ozone: SCM: Close containers: extend SCMCommandResponseProto with SCMCloseContainerCmdResponseProto. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
966853894f HDFS-12795. Ozone: SCM: Support for Container LifeCycleState PENDING_CLOSE and LifeCycleEvent FULL_CONTAINER. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
26e270b908 HDFS-12751. Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fd09c2ce5b HDFS-12890 . Ozone: XceiverClient should have upper bound on async requests. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
2f3068bc72 HDFS-12698. Ozone: Use time units in the Ozone configuration values. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
8ff98e2af3 HDFS-12000. Ozone: Container : Add key versioning support-1. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c0c87dea9b HDFS-12626. Ozone : delete open key entries that will no longer be closed. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6c630d0b09 HDFS-12899. Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
74362cb503 HDFS-12901. Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
207523915d HDFS-12588. Use GenericOptionsParser for scm and ksm daemon. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
7bba3943f0 HDFS-12876. Ozone: moving NodeType from OzoneConsts to Ozone.proto. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
85ec48a646 HDFS-12867. Ozone: TestOzoneConfigurationFields fails consistently. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
e32e187448 HDFS-12864. Ozone: Turn off full synced write in RocksDB MetadataStore. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
8023ba50b5 HDFS-12807. Ozone: Expose RockDB stats via JMX for Ozone metadata stores. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
86802a461c HDFS-12851. Ozone: Upgrade to latest ratis build. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nanda kumar
ea86945754 HDFS-12842. Ozone: SCM: During startup SCM should check if the cluster is INITIALIZED. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
73ca8608da HDFS-12786. Ozone: add port/service names to the ksm/scm web ui. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
f9c11d952c HDFS-12740. SCM should support a RPC to share the cluster Id with KSM and DataNodes. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c8d8270f72 HDFS-12787. Ozone: SCM: Aggregate the metrics from all the container reports. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
cbe4f314b3 HDFS-12731. Ozone: Fix inconsistent default value in ozone settings. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Anu Engineer
4deb845d7c HDFS-12759. Ozone: web: integrate configuration reader page to the SCM/KSM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Yiqun Lin
214b52dfce HDFS-12500. Ozone: add logger for oz shell commands and move error stack traces to DEBUG level. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
419e86d580 HDFS-12805. Ozone: Redundant characters printed in exception log. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
93f1b04949 HDFS-12798. Ozone: scm web: fix the node status table. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
4460ac9c57 HDFS-12756. Ozone: Add datanodeID to heartbeat responses and container protocol. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
446e84357d HDFS-12796. SCM should not start if Cluster Version file does not exist. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
740a06cdd7 HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Yiqun Lin
761926db2c HDFS-12793. Ozone : TestSCMCli is failing consistently. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
cec96b296f HDFS-12549. Ozone: OzoneClient: Support for REST protocol. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
9734f505ea HDFS-12739. Add Support for SCM --init command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
709d56fdc9 HDFS-12474. Ozone: SCM: Handling container report with key count and container usage. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2ec24a7542 HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Yiqun Lin
132f30c8ea HDFS-12443. Ozone: Improve SCM block deletion throttling algorithm. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
7ebe79e879 HDFS-12720. Ozone: Ratis options are not passed from KSM Client protobuf helper correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
0760418bec HDFS-12697. Ozone services must stay disabled in secure setup for alpha. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
b687902590 HDFS-12718. Ozone: fix thread number calculation in CBlockManager. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
76d34bca62 HDFS-12521. Ozone: SCM should read all Container info into memory when booting up. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
6291ca1e32 HDFS-12663. Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
80357c0eee HDFS-12678. Ozone: Corona: Add statistical information to json output. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b56a73a4f HDFS-12680. Ozone: SCM: Lease support for container creation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
338c909ac6 HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
333dae888c HDFS-12690. Ozone: generate swagger descriptor for the Ozone REST Api. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
0582d8b1b1 Revert "HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton." 
This reverts commit 18c0cdcec9ff5676b10ba711039143d07dd4a510.
 2018-04-26 05:36:04 -07:00
Anu Engineer
402a1c1ba3 HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
731a69faa1 HDFS-12693. Ozone: Enable XFrame options for KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6783dad766 HDFS-11468. Ozone: SCM: Add Node Metrics for SCM. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
393a02d8e1 HDFS-12692. Ozone: fix javadoc/unit test for calling MetadataStore.getRangeKVs with non existing key. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang
6d9f914b55 HDFS-12684. Ozone: SCMMXBean NodeCount is overlapping with NodeManagerMXBean. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0f0417a12c HDFS-12673. Ozone: Log elapsed time for block deletion tasks. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
06c6582694 HDFS-12585. Ozone: KSM UI: Add description for configs in UI. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
763484e596 HDFS-12661. Ozone: Support optional documentation link in KSM/SCM webui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
a467017068 HDFS-12411. Ozone: Add container usage information to DN container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
fb545e4291 HDFS-12415. Ozone: TestXceiverClientManager and TestAllocateContainer occasionally fails. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
895a4cc60a HDFS-12632. Ozone: OzoneFileSystem: Add contract tests to OzoneFileSystem. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
9bb516118d HDFS-12490. Ozone: OzoneClient: Add creation/modification time information in OzoneVolume/OzoneBucket/OzoneKey. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nandakumar
8881309002 HDFS-12519. Ozone: Lease Manager framework. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
c85d3b2d25 HDFS-12593. Ozone: update Ratis to the latest snapshot. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Nandakumar
5ed663b32f HDFS-12631. Ozone: ContainerStorageLocation#scmUsage should count only SCM usage. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Yiqun Lin
d19b4c8763 HDFS-12583. Ozone: Fix swallow exceptions which makes hard to debug failures. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
9445a9267f HDFS-12572. Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
ae2b77a0e5 HDFS-12546. Ozone: DB listing operation performance improvement. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang
b05ad0b315 HDFS-12611. Ozone: SCM Cli: Use the hostname of SCM server instead of the bind address. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
yuanbo
e76e10fd7a HDFS-12609. Ozone: SCM: Refactoring of chill mode logic in NodeManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
e3b51d9074 HDFS-12598. Ozone: Fix 3 node ratis replication in Ozone. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nandakumar
3504af9411 HDFS-12537. Ozone: Reduce key creation overhead in Corona. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Nandakumar
c3ef381011 HDFS-12543. Ozone : allow create key without specifying size. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c97ce55d06 HDFS-12602. Add Ozone config UI to KSM and SCM index page. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
303ecae5d5 HDFS-12557. Ozone: Improve the formatting of the RPC stats on web UI. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
c17521b1bd HDFS-12387. Ozone: Support Ratis as a first class replication mechanism. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer
0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
c52019c291 HDFS-12539. Ozone: refactor some functions in KSMMetadataManagerImpl to be more readable and reusable. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
ceec14bff5 HDFS-12467. Ozone: SCM: NodeManager should log when it comes out of chill mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
d7a94a21ae HDFS-12471. Ozone: Reduce some KSM/SCM deletion log messages from INFO to DEBUG. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6a754caca0 HDFS-12568. Ozone: Cleanup the ozone-default.xml. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer
aaec2689f4 HDFS-12511. Ozone: Add tags to config. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7363c8fc09 HDFS-12540. Ozone: node status text reported by SCM is a bit confusing. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
13fdb58490 HDFS-12554. Ozone: Fix TestDatanodeStateMachine#testDatanodeStateMachineWithInvalidConfiguration. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
a44f3a777d HDFS-12501. Ozone: Cleanup javac issues. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6b10723ec3 HDFS-12525. Ozone: OzoneClient: Verify bucket/volume name in create calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5e4a6b686c HDFS-12425. Ozone: OzoneFileSystem: read/write/create/open/getFileInfo APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
a1a3ba6529 HDFS-12454. Ozone : the sample ozone-site.xml in OzoneGettingStarted does not work. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2804435386 HDFS-12017. Ozone: Container: Move IPC port to 98xx range. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
b2447f8dd9 HDFS-11614. Ozone: Cleanup javadoc issues. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
9ff136bb02 HDFS-12210. Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
dddded0016 Revert "Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh." to fix commit message. 
This reverts commit 087c69ba2434813044089f733581607d172efbd1.
 2018-04-26 05:36:04 -07:00
Weiwei Yang
d303b7f40f HDFS-12524. Ozone: Record number of keys scanned and hinted for getRangeKVs call. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Yiqun Lin
712bd70ac7 HDFS-12205. Ozone: List Key on an empty ozone bucket fails with command failed error. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
8052374e79 Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
fd1564b87e HDFS-12506. Ozone: ListBucket is too slow. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
46b65ec4fb HDFS-12503. Ozone: some UX improvements to oz_debug. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1c354e51ac HDFS-12477. Ozone: Some minor text improvement in SCM web UI. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nandakumar
0313bdf886 HDFS-11897. Ozone: KSM: Changing log level for client calls in KSM. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Weiwei Yang
3b290fe3bf HDFS-12466. Ozone: KSM: Make ozone.ksm.address as mandatory property for client. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
8ddf75da17 HDFS-12489. Ozone: OzoneRestClientException swallows exceptions which makes client hard to debug failures. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c329d3b4b4 HDFS-12329. Ozone: Ratis: Readonly calls in XceiverClientRatis should use sendReadOnly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
607d1fcd31 HDFS-12256. Ozone : handle inactive containers on DataNode. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
77d4b18700 HDFS-12385. Ozone: OzoneClient: Refactoring OzoneClient API. Contributed by Nadakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
b984e903f3 HDFS-12481. Ozone: Corona: Support for variable key length in offline mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
d9c3d9a5ef HDFS-11873. Ozone: Object store handler supports reusing http client for multiple requests. Contributed by Xiaoyu Yao and Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fe4139bdc3 HDFS-11612. Ozone: Cleanup Checkstyle issues. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Weiwei Yang
eaf3732217 HDFS-12465. Ozone: fix a KeySpaceManager startup message typo. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
cf841286b8 HDFS-12426. Ozone: BlockManager MBean unregister failure upon shutdown. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Weiwei Yang
efa25373d7 HDFS-12019. Ozone: Misc : Make sure that ozone-site.xml is in etc/hadoop in tarball created from mvn package. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
80c50a1eac HDFS-12423. Ozone: TopN container choosing policy should ignore containers that has no pending deletion blocks. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1087ce4cd2 HDFS-11676. Ozone: SCM CLI: Implement close container command. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0c17776646 HDFS-12389. Ozone: oz commandline list calls should return valid JSON format output. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
fec04f1158 HDFS-12370. Ozone: Implement TopN container choosing policy for BlockDeletionService. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
743be0d7c0 HDFS-12235. Ozone: DeleteKey-3: KSM SCM block deletion message and ACK interactions. Contributed by Weiwei Yang and Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Weiwei Yang
3aa846412d HDFS-12397. Ozone: KSM: multiple delete methods in KSMMetadataManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang
e06a9a5bb1 HDFS-12391. Ozone: TestKSMSQLCli is not working as expected. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
1a21136322 HDFS-12382. Ozone: SCM: BlockManager creates a new container for each allocateBlock call. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
2e2e30373c HDFS-12333. Ozone: Extend Datanode web interface with SCM information. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
ca72c2342c HDFS-12366. Ozone: Refactor KSM metadata class names to avoid confusion. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7b3e7eb565 HDFS-12354. Ozone: Shuffle container list for datanode BlockDeletingService. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
eef437d5e2 HDFS-12282. Ozone: DeleteKey-4: Block delete via HB between SCM and DN. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2f4dfbc8fb HDFS-12275. Ozone: Corona: Support for random validation of writes. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
4d3b8d332d HDFS-12283. Ozone: DeleteKey-5: Implement SCM DeletedBlockLog. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
02f8c2224f HDFS-11888. Ozone: SCM: use state machine for open containers allocated for key/blocks. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
f352b104f7 HDFS-12337. Ozone: Concurrent RocksDB open calls fail because of "No locks available". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
9e733215a8 HDFS-12327. Ozone: support setting timeout in background service. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
38bc1d1097 HDFS-12216. Ozone: TestKeys is failing consistently. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
bd33038bf4 HDFS-12159. Ozone: SCM: Add create replication pipeline RPC. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Chen Liang
f9bce29dcc HDFS-12290. Block Storage: Change dfs.cblock.jscsi.server.address default bind address to 0.0.0.0. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
6a16d7c7ab HDFS-12255. Block Storage: Cblock should generated unique trace ID for the ops. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
e61530f5d9 HDFS-12298. Ozone: Block deletion service floods the log when deleting large number of block files. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
25586607a1 HDFS-12230. Ozone: KSM: Add creation time field in bucket info. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
dcc21a4f93 HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
c9ea431267 HDFS-12281. Ozone: Ozone-default.xml has 3 properties that do not match the default Config value. Contributed by Ajay Yadav. 2018-04-26 05:36:04 -07:00
Weiwei Yang
639b4fb8a9 HDFS-12196. Ozone: DeleteKey-2: Implement block deleting service to delete stale blocks at background. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
285d61cd8d HDFS-12274. Ozone: Corona: move corona from test to tools package. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
bb9a4793d3 HDFS-12259. Ozone: OzoneClient: Refactor move ozone client from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
b4aecc1009 HDFS-12262. Ozone: KSM: Reduce default handler thread count from 200. Contributed by Ajay Yadav. 2018-04-26 05:36:04 -07:00
Anu Engineer
bb5f1c4228 HDFS-12231. Ozone: KSM: Add creation time field in volume info. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
89d8d20c73 HDFS-12246. Ozone: potential thread leaks. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
23dee0f123 HDFS-11935. Ozone: TestStorageContainerManager#testRpcPermission fails when ipv6 address used. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
ccc1607424 HDFS-12178. Ozone: OzoneClient: Handling SCM container creationFlag at client side. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
37a0ab5ca3 HDFS-12247. Ozone: KeySpaceManager should unregister KSMMetrics upon stop. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1ad95cf2a9 HDFS-11984. Ozone: Ensures listKey lists all required key fields. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
acfae2716a HDFS-12034. Ozone: Web interface for KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Chen Liang
070f7e2e59 HDFS-11920. Ozone : add key partition. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
bc413a69cc HDFS-12195. Ozone: DeleteKey-1: KSM replies delete key request asynchronously. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
86b42907cf HDFS-12183. Ozone: KSM : Use proper defaults for block client address. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
7907fe1c80 HDFS-12059. Ozone: OzoneClient: OzoneClientImpl Add setBucketProperty and delete calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
fcd4537e63 HDFS-12180. Ozone: Corona: Add stats and progress bar to corona. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
a4b3160efb HDFS-12145. Ozone: OzoneFileSystem: Ozone & KSM should support / delimited key names. Contributed by Mukul Kumar Singh. 
(cherry picked from commit fe0a957e92d506330ba005abba18f665eb5ac1cc)
 2018-04-26 05:36:04 -07:00
Yiqun Lin
8cf0c864c4 HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d50a743015 HDFS-12155. Ozone : add RocksDB support to DEBUG CLI. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
0cc166c053 HDFS-12115. Ozone: SCM: Add queryNode RPC Call. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Weiwei Yang
539842ed8b HDFS-12187. Ozone : add support to DEBUG CLI for ksm.db. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
eda559ffd2 HDFS-12181. Ozone: Fix TestContainerReplicationManager by setting proper log level for LogCapturer. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
f1e2592bf8 HDFS-12127. Ozone: Ozone shell: Add more testing for key shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
bbbfc885e7 HDFS-12149. Ozone: RocksDB implementation of ozone metadata store. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6225622dee HDFS-12071. Ozone: Corona: Implementation of Corona. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d2ef3e40bc HDFS-12126. Ozone: Ozone shell: Add more testing for bucket shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
4d333f8db1 HDFS-12058. Ozone: OzoneClient: OzoneClientImpl add getDetails calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang
27262d8889 HDFS-12118. Ozone: Ozone shell: Add more testing for volume shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
db859e8172 HDFS-12148. Ozone: TestOzoneConfigurationFields is failing because ozone-default.xml has some missing properties. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
9522356b46 HDFS-12069. Ozone: Create a general abstraction for metadata store. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
b94b5021b8 HDFS-11989. Ozone: add TestKeysRatis, TestBucketsRatis and TestVolumeRatis. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
ab3d510c1d HDFS-12123. Ozone: OzoneClient: Abstraction of OzoneClient and default implementation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
3d430a6076 HDFS-11493. Ozone: SCM: Add the ability to handle container reports. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Yiqun Lin
58e850f262 HDFS-12083. Ozone: KSM: previous key has to be excluded from result in listVolumes, listBuckets and listKeys. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
a4cbd33dd5 HDFS-12129. Ozone: SCM http server is not stopped with SCM#stop(). Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
ea5e24c925 HDFS-12105. Ozone: listVolumes doesn't work from ozone commandline. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7afddca9d5 HDFS-12037. Ozone: Improvement rest API output format for better looking. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
25c1b296e6 HDFS-11679. Ozone: SCM CLI: Implement list container command. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Weiwei Yang
eb18b86e00 HDFS-12080. Ozone: Fix UT failure in TestOzoneConfigurationFields. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
eab407beac HDFS-12057. Ozone: OzoneClient: Implementation of OzoneClient. Contributed by Nandakumar 2018-04-26 05:36:04 -07:00
Anu Engineer
a353094ba4 HDFS-12076. Ozone: Review all cases where we are returning FAILED_INTERNAL_ERROR. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
8caa5d1ac0 HDFS-12086. Ozone: Add the unit test for KSMMetrics. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
63ce883746 HDFS-12035. Ozone: listKey doesn't work from ozone commandline. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
5379ffa8de HDFS-12030. Ozone: CLI: support infoKey command. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
4b63e3e504 HDFS-12002. Ozone : SCM cli misc fixes/improvements. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c567e6766b HDFS-11955. Ozone: Set proper parameter default values for listBuckets http request. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
b7b8511bae HDFS-11773. Ozone: KSM : add listVolumes. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang
b49c165d28 HDFS-12063. Ozone: Ozone shell: Multiple RPC calls for put/get key. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Chen Liang
0a8c903d3f HDFS-12050. Ozone: StorageHandler: Implementation of close for releasing resources during shutdown. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
8ef1163eca HDFS-12031. Ozone: Rename OzoneClient to OzoneRestClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7e834c839d HDFS-12018. Ozone: Misc: Add CBlocks defaults to ozone-defaults.xml. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fea1ab6192 HDFS-12007. Ozone: Enable HttpServer2 for SCM and KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
723da57644 HDFS-12016. Ozone: SCM: Container metadata are not loaded properly after datanode restart. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
77a4b248e4 HDFS-11990. Ozone: Add all configurable entries into ozone-default.xml. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
3b0996e2a1 HDFS-11585. Ozone: Support force update a container. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
e73d285567 HDFS-11782. Ozone: KSM: Add listKey. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
03606dd1a8 HDFS-11962. Ozone: Add stop-ozone.sh script. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fc48e18151 HDFS-11961. Ozone: Add start-ozone.sh to quickly start ozone. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
8916c70d90 HDFS-11969. Block Storage: Convert unnecessary info log levels to debug. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
863e4cfb56 HDFS-11958. Ozone: Ensure KSM is initiated using ProtobufRpcEngine. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fa26dea89d HDFS-11185. Ozone: remove disabled tests. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
e3591aeec0 HDFS-11953. Remove Guava v21 usage from HDFS-7240. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Rakesh Radhakrishnan
665f7c8b69 HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen. 2018-04-26 05:36:04 -07:00
Arpit Agarwal
d18909d338 HDFS-11866. JournalNode Sync should be off by default in hdfs-default.xml. Contributed by Hanisha Koneru. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
ddc88eaf88 HDFS-11952. Ozone: Fix TestContainerSQLCli#testConvertContainerDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0c37e05b10 HDFS-11779. Ozone: KSM: add listBuckets. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
bacd1188f1 HDFS-11880. Ozone: KSM: Remove protobuf formats from KSM wrappers. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2007e85d5b HDFS-11777. Ozone: KSM: add deleteBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Anu Engineer
3c0ed97df9 HDFS-11926. Ozone: Implement a common helper to return a range of KVs in levelDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
f617b2dddb HDFS-11568. Ozone: Create metadata path automatically after null checking. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d6dd557b24 HDFS-11887. Shared XceiverClient should be closed if there is no open clients to avoid resource leak. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
d67542c115 HDFS-11822. Block Storage: Fix TestCBlockCLI, failing because of " Address already in use". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
bf8c168a80 HDFS-11781. Ozone: KSM: Add deleteKey. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
42eaecae55 HDFS-11771. Ozone: KSM: Add checkVolumeAccess. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b2d0b4fec HDFS-11796. Ozone: MiniOzoneCluster should set "ozone.handler.type" key correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
aa5722dc44 HDFS-11913. Ozone: TestKeySpaceManager#testDeleteVolume fails. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4893692972 HDFS-11597. Ozone: Add Ratis management API. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
a8c0976fb4 HDFS-11894. Ozone: Cleanup imports. Contributed by Weiwei Yang. 2018-04-26 05:35:59 -07:00
Xiaoyu Yao
6b9915fcbd HDFS-11774. Ozone: KSM: add deleteVolume. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
236c410881 HDFS-11776. Ozone: KSM: add SetBucketProperty. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
72b228a9e6 HDFS-11853. Ozone: KSM: Add getKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
59d273b175 HDFS-11655. Ozone: CLI: Guarantees user runs SCM commands has appropriate permission. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
b71efcf1b0 HDFS-11865. Ozone: Do not initialize Ratis cluster during datanode startup. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
e3c8f6a24d HDFS-11780. Ozone: KSM: Add putKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ff5dbeec07 HDFS-11846. Ozone: Fix Http connection leaks in ozone clients. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
84294de9a2 HDFS-11778. Ozone: KSM: add getBucketInfo. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
47c4867765 HDFS-11859. Ozone: SCM: Separate BlockLocationProtocol from ContainerLocationProtocol. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
0753e094d7 HDFS-11727. Block Storage: Retry Blocks should be requeued when cblock is restarted. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
37642c12df HDFS-11770. Ozone: KSM: Add setVolumeProperty. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
56966a0535 HDFS-11860. Ozone: SCM: SCMContainerPlacementCapacity#chooseNode sometimes does not remove chosen node from healthy list. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Weiwei Yang
ae5242accb HDFS-11830. Ozone: Datanode needs to re-register to SCM if SCM is restarted. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ca70300eea HDFS-11775. Ozone: KSM: add createBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Weiwei Yang
e9d09c209e HDFS-11824. Ozone: Fix javac warnings. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
c9e6d4378d HDFS-11769. Ozone: KSM: Add createVolume API. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Chen Liang
dfb0dab966 HDFS-11694. Block Storage: Add Support for 2 BlockIDBuffers and also for periodic flush of BlockIDBuffer. Contributed by Mukul Kumar Singh 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
821740e4d3 HDFS-11815. CBlockManager#main should join() after start() service. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
a6013e2b5b HDFS-11805. Ensure LevelDB DBIterator is closed. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
b592df6413 HDFS-11811. Ozone: SCM: Support Delete Block. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
9bc494b909 HDFS-11802. Ozone : add DEBUG CLI support for open container db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
edff6c6a24 HDFS-11768. Ozone: KSM: Create Key Space manager service. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00