Commit Graph

7380 Commits

Author SHA1 Message Date
Chen Liang
0a8c903d3f HDFS-12050. Ozone: StorageHandler: Implementation of close for releasing resources during shutdown. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
8ef1163eca HDFS-12031. Ozone: Rename OzoneClient to OzoneRestClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7e834c839d HDFS-12018. Ozone: Misc: Add CBlocks defaults to ozone-defaults.xml. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fea1ab6192 HDFS-12007. Ozone: Enable HttpServer2 for SCM and KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
723da57644 HDFS-12016. Ozone: SCM: Container metadata are not loaded properly after datanode restart. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
77a4b248e4 HDFS-11990. Ozone: Add all configurable entries into ozone-default.xml. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
77e3bb783b HDFS-11970. Ozone: TestXceiverClientManager.testFreeByEviction fails occasionally. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Yiqun Lin
52aa60ed2f HDFS-11963. Ozone: Documentation: Add getting started page (addendum). Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
3d2339de38 HDFS-11983. Ozone: Add documentation for metrics in KSMMetrics to OzoneMetrics.md. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
3b0996e2a1 HDFS-11585. Ozone: Support force update a container. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
8d289df627 HDFS-11963. Ozone: Documentation: Add getting started page. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
e73d285567 HDFS-11782. Ozone: KSM: Add listKey. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
721e7a6418 HDFS-11979. Ozone: TestContainerPersistence never uses MiniOzoneCluster. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
03606dd1a8 HDFS-11962. Ozone: Add stop-ozone.sh script. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fc48e18151 HDFS-11961. Ozone: Add start-ozone.sh to quickly start ozone. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
18ddec6144 HDFS-11977. Ozone: cannot enable test debug/trace log. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
8916c70d90 HDFS-11969. Block Storage: Convert unnecessary info log levels to debug. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
15cc4007b0 HDFS-11946. Ozone: Containers in different datanodes are mapped to the same location. Contributed by Nandakumar 2018-04-26 05:36:04 -07:00
Anu Engineer
938728744c HDFS-11845. Ozone: Output error when DN handshakes with SCM. Contributed by Weiwei Yang 2018-04-26 05:36:04 -07:00
Anu Engineer
863e4cfb56 HDFS-11958. Ozone: Ensure KSM is initiated using ProtobufRpcEngine. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fa26dea89d HDFS-11185. Ozone: remove disabled tests. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
e3591aeec0 HDFS-11953. Remove Guava v21 usage from HDFS-7240. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Lei Xu
08460f1559 HDFS-11659. TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Contributed by Lei (Eddy) Xu. 2018-04-26 05:36:04 -07:00
Rakesh Radhakrishnan
665f7c8b69 HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen. 2018-04-26 05:36:04 -07:00
Arpit Agarwal
d18909d338 HDFS-11866. JournalNode Sync should be off by default in hdfs-default.xml. Contributed by Hanisha Koneru. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
ddc88eaf88 HDFS-11952. Ozone: Fix TestContainerSQLCli#testConvertContainerDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0c37e05b10 HDFS-11779. Ozone: KSM: add listBuckets. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
bacd1188f1 HDFS-11880. Ozone: KSM: Remove protobuf formats from KSM wrappers. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2007e85d5b HDFS-11777. Ozone: KSM: add deleteBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Anu Engineer
3c0ed97df9 HDFS-11926. Ozone: Implement a common helper to return a range of KVs in levelDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
f617b2dddb HDFS-11568. Ozone: Create metadata path automatically after null checking. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d6dd557b24 HDFS-11887. Shared XceiverClient should be closed if there is no open clients to avoid resource leak. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
f5d17b8f7c HDFS-11919. Ozone: SCM: TestNodeManager takes too long to execute. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Chen Liang
d67542c115 HDFS-11822. Block Storage: Fix TestCBlockCLI, failing because of " Address already in use". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
bf8c168a80 HDFS-11781. Ozone: KSM: Add deleteKey. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
42eaecae55 HDFS-11771. Ozone: KSM: Add checkVolumeAccess. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b2d0b4fec HDFS-11796. Ozone: MiniOzoneCluster should set "ozone.handler.type" key correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
aa5722dc44 HDFS-11913. Ozone: TestKeySpaceManager#testDeleteVolume fails. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4893692972 HDFS-11597. Ozone: Add Ratis management API. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
a8c0976fb4 HDFS-11894. Ozone: Cleanup imports. Contributed by Weiwei Yang. 2018-04-26 05:35:59 -07:00
Mingliang Liu
677dcf529c HDFS-11903. Ozone: Cleaning up local storage when closing MiniOzoneCluster. Contributed by Mingliang Liu 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
6b9915fcbd HDFS-11774. Ozone: KSM: add deleteVolume. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
236c410881 HDFS-11776. Ozone: KSM: add SetBucketProperty. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
72b228a9e6 HDFS-11853. Ozone: KSM: Add getKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
59d273b175 HDFS-11655. Ozone: CLI: Guarantees user runs SCM commands has appropriate permission. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
b71efcf1b0 HDFS-11865. Ozone: Do not initialize Ratis cluster during datanode startup. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
e3c8f6a24d HDFS-11780. Ozone: KSM: Add putKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ff5dbeec07 HDFS-11846. Ozone: Fix Http connection leaks in ozone clients. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
84294de9a2 HDFS-11778. Ozone: KSM: add getBucketInfo. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
47c4867765 HDFS-11859. Ozone: SCM: Separate BlockLocationProtocol from ContainerLocationProtocol. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
0753e094d7 HDFS-11727. Block Storage: Retry Blocks should be requeued when cblock is restarted. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
37642c12df HDFS-11770. Ozone: KSM: Add setVolumeProperty. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
56966a0535 HDFS-11860. Ozone: SCM: SCMContainerPlacementCapacity#chooseNode sometimes does not remove chosen node from healthy list. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
a0f51ee3d7 HDFS-11850. Ozone: Stack Overflow in XceiverClientManager because of race condition in accessing openClient. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Weiwei Yang
ae5242accb HDFS-11830. Ozone: Datanode needs to re-register to SCM if SCM is restarted. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
877e751c84 HDFS-11843. Ozone: XceiverClientRatis should implement XceiverClientSpi.connect(). 2018-04-26 05:34:18 -07:00
Chen Liang
4fb9064523 HDFS-11835. Block Storage: Overwrite of blocks fails. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ca70300eea HDFS-11775. Ozone: KSM: add createBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Weiwei Yang
e9d09c209e HDFS-11824. Ozone: Fix javac warnings. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
7f0be64284 HDFS-11831. Ozone: Improve the way of getting test file path in unit tests for Windows. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
c9e6d4378d HDFS-11769. Ozone: KSM: Add createVolume API. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
9c9be9f7f7 HDFS-11693. Ozone:Add archive support to containers. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Chen Liang
162dac96ce HDFS-11650. Ozone: fix the consistently timeout test testUpgradeFromRel22Image. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Chen Liang
dfb0dab966 HDFS-11694. Block Storage: Add Support for 2 BlockIDBuffers and also for periodic flush of BlockIDBuffer. Contributed by Mukul Kumar Singh 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
821740e4d3 HDFS-11815. CBlockManager#main should join() after start() service. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
a6013e2b5b HDFS-11805. Ensure LevelDB DBIterator is closed. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
b592df6413 HDFS-11811. Ozone: SCM: Support Delete Block. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
9bc494b909 HDFS-11802. Ozone : add DEBUG CLI support for open container db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
edff6c6a24 HDFS-11768. Ozone: KSM: Create Key Space manager service. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Chen Liang
169713f3f3 HDFS-11788. Ozone : add DEBUG CLI support for nodepool db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
070960ed8a HDFS-11798. Ozone: Correct description for ozone.handler.type in ozone-default.xml. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
744a052dc2 HDFS-11783. Ozone: Fix spotbugs warnings. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
4be9710712 HDFS-11680. Ozone: SCM CLI: Implement info container command. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Chen Liang
9fcaeceb54 HDFS-11756. Ozone : add DEBUG CLI support of blockDB file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Weiwei Yang
c18229f0df HDFS-11716. Ozone: SCM: CLI: Revisit delete container API. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
67c81a8b18 HDFS-11761. Ozone: Get container report should only report closed containers. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
fc9130f6c5 HDFS-11759. Ozone : SCMNodeManager#close() should also close node pool manager object. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
d9be9a8c30 HDFS-11698. Ozone: SCM: CLI: Add Debug command. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
a28557ac03 HDFS-11712. Ozone: Reuse ObjectMapper instance to improve the performance. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
ee8708ca31 HDFS-11747. Ozone : need to fix OZONE_SCM_DEFAULT_PORT. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
3158f479d3 HDFS-11725. Ozone: Revise create container CLI specification and implementation. Contributed by Weiwei Yang 2018-04-26 05:34:18 -07:00
Weiwei Yang
b581cde542 HDFS-11728. Ozone: add the DB names to OzoneConsts. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
6b3da448eb HDFS-11675. Ozone: SCM CLI: Implement delete container command. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Chen Liang
e9588c6422 HDFS-11627. Block Storage: Cblock cache should register with flusher to upload blocks to containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Chen Liang
668b056984 HDFS-11667. Block Storage:Handling flushing of incomplete block id buffers during shutdown. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
1a245accb5 HDFS-11668. Ozone: misc improvements for SCM CLI. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Yiqun Lin
5984d40b86 HDFS-11666. Ozone: Fix compile error due to inconsistent package name. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
a9ba8b5ded HDFS-11504. Ozone: SCM: Add Block APIs. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Weiwei Yang
720e5c40f6 HDFS-11649. Ozone: SCM: CLI: Add shell code placeholder classes. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
6c4bd1647c HDFS-11581. Ozone: Support force delete a container. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Anu Engineer
d3f93d55d9 HDFS-11631. Block Storage : allow cblock server to be started from hdfs commandline. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
05b12c48cf HDFS-11657. Ozone: Add unit test for storage container metrics. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
783b7c27b1 HDFS-11582. Block Storage : add SCSI target access daemon. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
7d4a2d4104 HDFS-11569. Ozone: Implement listKey function for KeyManager. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Chen Liang
cbb7cc26a5 HDFS-11642. Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
5b19a03f63 HDFS-11635. Block Storage: Add metrics for Container Flushes. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
2129c6b276 HDFS-11062. Ozone:SCM: Remove null command. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Chen Liang
b7463924af HDFS-11618. Block Storage: Add Support for Direct I/O. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
57b2a9b17d HDFS-11006. Ozone: support setting chunk size in streaming API. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
cd361fa490 HDFS-11625. Ozone: Fix UT failures that caused by hard coded datanode data dirs. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00