big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
18,817 Commits 2 Branches 2 Tags 264 MiB
11473b8204
Commit Graph

7539 Commits

Author SHA1 Message Date
Weiwei Yang
63ce883746 HDFS-12035. Ozone: listKey doesn't work from ozone commandline. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
e07662cb04 HDFS-12048. Ozone:TestOzoneContainerRatis & TestRatisManager are failing consistently. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d1ac8611c6 HDFS-12023. Ozone: Test if all the configuration keys are documented in ozone-defaults.xml. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c398ab49d3 HDFS-12047. Ozone: Add REST API documentation. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
5379ffa8de HDFS-12030. Ozone: CLI: support infoKey command. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
4b63e3e504 HDFS-12002. Ozone : SCM cli misc fixes/improvements. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c567e6766b HDFS-11955. Ozone: Set proper parameter default values for listBuckets http request. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
b7b8511bae HDFS-11773. Ozone: KSM : add listVolumes. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang
b49c165d28 HDFS-12063. Ozone: Ozone shell: Multiple RPC calls for put/get key. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Chen Liang
0a8c903d3f HDFS-12050. Ozone: StorageHandler: Implementation of close for releasing resources during shutdown. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
8ef1163eca HDFS-12031. Ozone: Rename OzoneClient to OzoneRestClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7e834c839d HDFS-12018. Ozone: Misc: Add CBlocks defaults to ozone-defaults.xml. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fea1ab6192 HDFS-12007. Ozone: Enable HttpServer2 for SCM and KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
723da57644 HDFS-12016. Ozone: SCM: Container metadata are not loaded properly after datanode restart. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
77a4b248e4 HDFS-11990. Ozone: Add all configurable entries into ozone-default.xml. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
77e3bb783b HDFS-11970. Ozone: TestXceiverClientManager.testFreeByEviction fails occasionally. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Yiqun Lin
52aa60ed2f HDFS-11963. Ozone: Documentation: Add getting started page (addendum). Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
3d2339de38 HDFS-11983. Ozone: Add documentation for metrics in KSMMetrics to OzoneMetrics.md. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
3b0996e2a1 HDFS-11585. Ozone: Support force update a container. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
8d289df627 HDFS-11963. Ozone: Documentation: Add getting started page. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
e73d285567 HDFS-11782. Ozone: KSM: Add listKey. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
721e7a6418 HDFS-11979. Ozone: TestContainerPersistence never uses MiniOzoneCluster. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
03606dd1a8 HDFS-11962. Ozone: Add stop-ozone.sh script. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fc48e18151 HDFS-11961. Ozone: Add start-ozone.sh to quickly start ozone. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
18ddec6144 HDFS-11977. Ozone: cannot enable test debug/trace log. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
8916c70d90 HDFS-11969. Block Storage: Convert unnecessary info log levels to debug. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
15cc4007b0 HDFS-11946. Ozone: Containers in different datanodes are mapped to the same location. Contributed by Nandakumar 2018-04-26 05:36:04 -07:00
Anu Engineer
938728744c HDFS-11845. Ozone: Output error when DN handshakes with SCM. Contributed by Weiwei Yang 2018-04-26 05:36:04 -07:00
Anu Engineer
863e4cfb56 HDFS-11958. Ozone: Ensure KSM is initiated using ProtobufRpcEngine. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fa26dea89d HDFS-11185. Ozone: remove disabled tests. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
e3591aeec0 HDFS-11953. Remove Guava v21 usage from HDFS-7240. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Lei Xu
08460f1559 HDFS-11659. TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Contributed by Lei (Eddy) Xu. 2018-04-26 05:36:04 -07:00
Rakesh Radhakrishnan
665f7c8b69 HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen. 2018-04-26 05:36:04 -07:00
Arpit Agarwal
d18909d338 HDFS-11866. JournalNode Sync should be off by default in hdfs-default.xml. Contributed by Hanisha Koneru. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
ddc88eaf88 HDFS-11952. Ozone: Fix TestContainerSQLCli#testConvertContainerDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0c37e05b10 HDFS-11779. Ozone: KSM: add listBuckets. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
bacd1188f1 HDFS-11880. Ozone: KSM: Remove protobuf formats from KSM wrappers. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2007e85d5b HDFS-11777. Ozone: KSM: add deleteBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Anu Engineer
3c0ed97df9 HDFS-11926. Ozone: Implement a common helper to return a range of KVs in levelDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
f617b2dddb HDFS-11568. Ozone: Create metadata path automatically after null checking. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d6dd557b24 HDFS-11887. Shared XceiverClient should be closed if there is no open clients to avoid resource leak. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
f5d17b8f7c HDFS-11919. Ozone: SCM: TestNodeManager takes too long to execute. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Chen Liang
d67542c115 HDFS-11822. Block Storage: Fix TestCBlockCLI, failing because of " Address already in use". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
bf8c168a80 HDFS-11781. Ozone: KSM: Add deleteKey. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
42eaecae55 HDFS-11771. Ozone: KSM: Add checkVolumeAccess. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b2d0b4fec HDFS-11796. Ozone: MiniOzoneCluster should set "ozone.handler.type" key correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
aa5722dc44 HDFS-11913. Ozone: TestKeySpaceManager#testDeleteVolume fails. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4893692972 HDFS-11597. Ozone: Add Ratis management API. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
a8c0976fb4 HDFS-11894. Ozone: Cleanup imports. Contributed by Weiwei Yang. 2018-04-26 05:35:59 -07:00
Mingliang Liu
677dcf529c HDFS-11903. Ozone: Cleaning up local storage when closing MiniOzoneCluster. Contributed by Mingliang Liu 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
6b9915fcbd HDFS-11774. Ozone: KSM: add deleteVolume. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
236c410881 HDFS-11776. Ozone: KSM: add SetBucketProperty. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
72b228a9e6 HDFS-11853. Ozone: KSM: Add getKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
59d273b175 HDFS-11655. Ozone: CLI: Guarantees user runs SCM commands has appropriate permission. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
b71efcf1b0 HDFS-11865. Ozone: Do not initialize Ratis cluster during datanode startup. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
e3c8f6a24d HDFS-11780. Ozone: KSM: Add putKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ff5dbeec07 HDFS-11846. Ozone: Fix Http connection leaks in ozone clients. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
84294de9a2 HDFS-11778. Ozone: KSM: add getBucketInfo. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
47c4867765 HDFS-11859. Ozone: SCM: Separate BlockLocationProtocol from ContainerLocationProtocol. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
0753e094d7 HDFS-11727. Block Storage: Retry Blocks should be requeued when cblock is restarted. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
37642c12df HDFS-11770. Ozone: KSM: Add setVolumeProperty. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
56966a0535 HDFS-11860. Ozone: SCM: SCMContainerPlacementCapacity#chooseNode sometimes does not remove chosen node from healthy list. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
a0f51ee3d7 HDFS-11850. Ozone: Stack Overflow in XceiverClientManager because of race condition in accessing openClient. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Weiwei Yang
ae5242accb HDFS-11830. Ozone: Datanode needs to re-register to SCM if SCM is restarted. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
877e751c84 HDFS-11843. Ozone: XceiverClientRatis should implement XceiverClientSpi.connect(). 2018-04-26 05:34:18 -07:00
Chen Liang
4fb9064523 HDFS-11835. Block Storage: Overwrite of blocks fails. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ca70300eea HDFS-11775. Ozone: KSM: add createBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Weiwei Yang
e9d09c209e HDFS-11824. Ozone: Fix javac warnings. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
7f0be64284 HDFS-11831. Ozone: Improve the way of getting test file path in unit tests for Windows. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
c9e6d4378d HDFS-11769. Ozone: KSM: Add createVolume API. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
9c9be9f7f7 HDFS-11693. Ozone:Add archive support to containers. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Chen Liang
162dac96ce HDFS-11650. Ozone: fix the consistently timeout test testUpgradeFromRel22Image. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Chen Liang
dfb0dab966 HDFS-11694. Block Storage: Add Support for 2 BlockIDBuffers and also for periodic flush of BlockIDBuffer. Contributed by Mukul Kumar Singh 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
821740e4d3 HDFS-11815. CBlockManager#main should join() after start() service. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
a6013e2b5b HDFS-11805. Ensure LevelDB DBIterator is closed. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
b592df6413 HDFS-11811. Ozone: SCM: Support Delete Block. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
9bc494b909 HDFS-11802. Ozone : add DEBUG CLI support for open container db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
edff6c6a24 HDFS-11768. Ozone: KSM: Create Key Space manager service. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Chen Liang
169713f3f3 HDFS-11788. Ozone : add DEBUG CLI support for nodepool db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
070960ed8a HDFS-11798. Ozone: Correct description for ozone.handler.type in ozone-default.xml. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
744a052dc2 HDFS-11783. Ozone: Fix spotbugs warnings. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
4be9710712 HDFS-11680. Ozone: SCM CLI: Implement info container command. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Chen Liang
9fcaeceb54 HDFS-11756. Ozone : add DEBUG CLI support of blockDB file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Weiwei Yang
c18229f0df HDFS-11716. Ozone: SCM: CLI: Revisit delete container API. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
67c81a8b18 HDFS-11761. Ozone: Get container report should only report closed containers. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
fc9130f6c5 HDFS-11759. Ozone : SCMNodeManager#close() should also close node pool manager object. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
d9be9a8c30 HDFS-11698. Ozone: SCM: CLI: Add Debug command. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
a28557ac03 HDFS-11712. Ozone: Reuse ObjectMapper instance to improve the performance. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
ee8708ca31 HDFS-11747. Ozone : need to fix OZONE_SCM_DEFAULT_PORT. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
3158f479d3 HDFS-11725. Ozone: Revise create container CLI specification and implementation. Contributed by Weiwei Yang 2018-04-26 05:34:18 -07:00
Weiwei Yang
b581cde542 HDFS-11728. Ozone: add the DB names to OzoneConsts. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
6b3da448eb HDFS-11675. Ozone: SCM CLI: Implement delete container command. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Chen Liang
e9588c6422 HDFS-11627. Block Storage: Cblock cache should register with flusher to upload blocks to containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Chen Liang
668b056984 HDFS-11667. Block Storage:Handling flushing of incomplete block id buffers during shutdown. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
1a245accb5 HDFS-11668. Ozone: misc improvements for SCM CLI. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Yiqun Lin
5984d40b86 HDFS-11666. Ozone: Fix compile error due to inconsistent package name. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
a9ba8b5ded HDFS-11504. Ozone: SCM: Add Block APIs. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Weiwei Yang
720e5c40f6 HDFS-11649. Ozone: SCM: CLI: Add shell code placeholder classes. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
6c4bd1647c HDFS-11581. Ozone: Support force delete a container. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Anu Engineer
d3f93d55d9 HDFS-11631. Block Storage : allow cblock server to be started from hdfs commandline. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
05b12c48cf HDFS-11657. Ozone: Add unit test for storage container metrics. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
783b7c27b1 HDFS-11582. Block Storage : add SCSI target access daemon. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
7d4a2d4104 HDFS-11569. Ozone: Implement listKey function for KeyManager. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Chen Liang
cbb7cc26a5 HDFS-11642. Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
5b19a03f63 HDFS-11635. Block Storage: Add metrics for Container Flushes. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
2129c6b276 HDFS-11062. Ozone:SCM: Remove null command. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Chen Liang
b7463924af HDFS-11618. Block Storage: Add Support for Direct I/O. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
57b2a9b17d HDFS-11006. Ozone: support setting chunk size in streaming API. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
cd361fa490 HDFS-11625. Ozone: Fix UT failures that caused by hard coded datanode data dirs. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
7573a0bc61 HDFS-11620. Ozone: Allocate container for MiniOzone cluster fails because of insufficient space error. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
fadfe37019 HDFS-11619. Ozone: Unit tests running failed in Windows. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
aa16380d71 HDFS-11566. Ozone: Document missing metrics for container operations. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
82d814fa79 HDFS-11564. Ozone: SCM: Add Comparable Metric Support. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
68da45a789 HDFS-11519. Ozone: Implement XceiverServerSpi and XceiverClientSpi using Ratis. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:34:18 -07:00
Chen Liang
3966f42895 HDFS-11591. Block Storage: fix TestCBlockServer test cleanup. Contributed by Xiaoyu Yao 2018-04-26 05:34:18 -07:00
Anu Engineer
7c59add3da HDFS-11594. Ozone: close container should call compactDB. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
bc9c313907 HDFS-11567. Ozone: SCM: Support update container. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
17a6e62629 HDFS-11537. Block Storage : add cache layer. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
3598d4b8cf HDFS-11548. Ozone: SCM: Add node pool management API. Contributed by Xiaoyu Yao 2018-04-26 05:34:18 -07:00
Anu Engineer
dc8e73a7d5 HDFS-11355. Block Storage : Merge configuration into ozone configuration classes. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
860a4e0044 HDFS-11550. Ozone: Add a check to prevent removing a container that has keys in it. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
23044c1db8 HDFS-11361. Block Storage: add cache interface. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
d251a55ff9 HDFS-11559. Ozone: MiniOzoneCluster prints too many log messages by default. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:34:18 -07:00
Anu Engineer
1e892f5748 HDFS-11162. Block Storage: add command line tool. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
85c2312e7d HDFS-11492. Ozone: Add the ability to handle sendContainerReport Command. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
98d3360e3a HDFS-11444. Ozone: Fix datanode ID handling in MiniOzoneCluster. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
88371ff07f HDFS-11491. Ozone: SCM: Add close container RPC. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
932423211f HDFS-11469. Ozone: SCM: Container allocation based on node report. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
fc7d678d3d HDFS-11071. Ozone: SCM: Move SCM config keys to ScmConfig. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
7b7d186b82 HDFS-11513. Ozone: Separate XceiverServer and XceiverClient into interfaces and implementations. 2018-04-26 05:34:18 -07:00
Anu Engineer
1058aa2523 HDFS-10928. Ozone:SCM: Support MXBean for SCM and NodeManager. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
386dbc11da HDFS-11196. Ozone: Improve logging and error handling in the container layer. Contributed by Anu Engineer. 2018-04-25 15:55:06 -07:00
Anu Engineer
32cc2b8f1a HDFS-11509. Ozone: Fix TestEndpoint test regression. Contributed by Anu Engineer. 2018-04-25 15:54:56 -07:00
Anu Engineer
9c57a61f68 HDFS-11424. Block Storage: add container representation to meta data. Contributed Chen Liang. 2018-04-25 15:54:56 -07:00
Anu Engineer
3e1317de02 HDFS-11451. Ozone: Add protobuf definitions for container reports. Contributed by Anu Engineer. 2018-04-25 15:54:56 -07:00
Anu Engineer
02fb9aed32 HDFS-11158. Ozone: TestDatanodeStateMachine is failing intermittently. Contributed by Weiwei Yang. 2018-04-25 15:54:56 -07:00
Anu Engineer
753979865b HDFS-11137. Ozone: SCM: Add negative tests cases for datanodeStatemachine. Contributed by Weiwei Yang. 2018-04-25 15:54:42 -07:00
Anu Engineer
d55564143d HDFS-11484. Ozone: Fix flaky TestNodeManager#testScmNodeReportUpdate. Contributed by Xiaoyu Yao. 2018-04-25 15:54:42 -07:00
Anu Engineer
a2328d36ac HDFS-11480. Ozone: TestEndpoint task failure. Contributed by Xiaoyu Yao. 2018-04-25 15:54:42 -07:00
Anu Engineer
2cba0daddc HDFS-11425. Ozone: add client-facing container APIs and container references. Contributed by Chen Liang. 2018-04-25 15:54:42 -07:00
Anu Engineer
56f011fd02 HDFS-11447. Ozone: SCM: Send node report to SCM with heartbeat.Contributed by Xiaoyu Yao. 2018-04-25 15:54:41 -07:00
Anu Engineer
f4529316bd HDFS-11474. Ozone: TestContainerMapping needs to cleanup levelDB files.Contributed by Xiaoyu Yao. 2018-04-25 15:54:36 -07:00
Anu Engineer
b29ae6e5f7 HDFS-11473. Ozone : exclude cblock protobuf files from findbugs check. Contributed by Xiaoyu Yao. 2018-04-25 15:54:36 -07:00
Anu Engineer
b8b72241aa HDFS-11036. Ozone: reuse Xceiver connection. Contributed by Chen Liang. 2018-04-25 15:54:36 -07:00
Arpit Agarwal
754ed7d651 HDFS-11414. Ozone : move StorageContainerLocation protocol to hdfs-client. Contributed by Chen Liang. 2018-04-25 15:54:34 -07:00
Anu Engineer
bb410de10c HDFS-11184. Ozone: SCM: Make SCM use container protocol. Contributed by Anu Engineer. 2018-04-25 15:54:26 -07:00
Anu Engineer
c169dd1306 HDFS-11463. Ozone: Add metrics for container operations and export over JMX. Contributed by Mukul Kumar Singh. 2018-04-25 15:54:03 -07:00
Anu Engineer
a7f82cba71 HDFS-11154. Block Storage: store server state to persistent storage. Contributed by Chen Liang 2018-04-25 15:53:58 -07:00
Anu Engineer
9997d36eb6 HDFS-11138. Block Storage: add block storage server. Contributed by Chen Liang 2018-04-25 15:53:58 -07:00
Anu Engineer
48db56adea HDFS-11133. Ozone: Add allocateContainer RPC. Contributed by Anu Engineer 2018-04-25 15:53:58 -07:00
Anu Engineer
ca12aac5a4 HDFS-11126. Ozone: Add small file support RPC. Contributed by Anu Engineer. 2018-04-25 15:53:58 -07:00
Anu Engineer
05b44e1ad8 HDFS-11108. Ozone: use containers with the state machine. Contributed by Anu Engineer 2018-04-25 15:53:56 -07:00
Anu Engineer
8bd85268e6 HDFS-11081. Ozone:SCM: Add support for registerNode in datanode. Contributed by Anu Engineer. 2018-04-25 15:53:50 -07:00
Anu Engineer
2b8b3754f6 HDFS-11130. Block Storage : Add storage client to server protocol. Contributed by Chen Lian 2018-04-25 15:53:49 -07:00
Anu Engineer
3d83ee25c5 HDFS-11127. Block Storage : add block storage service protocol. Contributed by Chen Liang 2018-04-25 15:53:43 -07:00
Anu Engineer
8274ff356a HDFS-11103. Ozone: Cleanup some dependencies. Contributed by Anu Engineer. 2018-04-25 15:53:39 -07:00
Anu Engineer
e49e305f25 HDFS-11001. Ozone:SCM: Add support for registerNode in SCM. Contributed by Anu Engineer. 2018-04-25 15:53:34 -07:00
Anu Engineer
964daed853 fix a build break due to merge 2018-04-25 15:53:34 -07:00
Anu Engineer
b186fc81f7 HDFS-10938. Ozone:SCM: Add datanode protocol. Contributed by Anu Engineer. 2018-04-25 15:53:32 -07:00
Anu Engineer
ee119ff60a HDFS-11004. Ozone : move Chunk IO and container protocol calls to hdfs-client. Contributed by Chen Liang. 2018-04-25 15:53:27 -07:00
Anu Engineer
ed84388fca HDFS-10912. Ozone:SCM: Add chill mode support to NodeManager. Contributed by Anu Engineer. 2018-04-25 15:53:27 -07:00
Anu Engineer
5520f73dee Fixing merge conflicts due to HDFS-10637 2018-04-25 15:53:27 -07:00
Anu Engineer
56b0fddfbd HDFS-10995. Ozone: Move ozone XceiverClient to hdfs-client. Contributed by Chen Liang. 2018-04-25 15:53:25 -07:00
Anu Engineer
d0422a74ea HDFS-10932. Ozone: fix XceiverClient slow shutdown. Contributed by Chen Liang. 2018-04-25 15:53:21 -07:00
Anu Engineer
4c95c3d6ab HDFS-10897. Ozone: SCM: Add NodeManager. Contributed by Anu Engineer. 2018-04-25 15:53:21 -07:00
Arpit Agarwal
e3aa2b687c HDFS-10420. Fix Ozone unit tests to use MiniOzoneCluster. Contributed by Arpit Agarwal. 2018-04-25 15:53:21 -07:00
Anu Engineer
3beee0b3d1 HDFS-10363. Ozone: Introduce new config keys for SCM service endpoints. Contributed by Arpit Agarwal. 2018-04-25 15:53:19 -07:00
Arpit Agarwal
6c61902389 HDFS-10361. Support starting StorageContainerManager as a daemon. (Contributed by Arpit Agarwal) 2018-04-25 15:53:12 -07:00
Anu Engineer
aa3ca951f9 HDFS-10351. Ozone: Optimize key writes to chunks by providing a bulk write implementation in ChunkOutputStream. Contributed by Chris Nauroth. 2018-04-25 15:53:07 -07:00
Chris Nauroth
acf0b220ce HDFS-10349. Ozone: StorageContainerManager fails to compile after merge of HDFS-10312 maxDataLength enforcement. Contributed by Chris Nauroth. 2018-04-25 15:53:07 -07:00
Anu Engineer
7d661a289a HDFS-10232. Ozone: Make config key naming consistent. Contributed by Anu Engineer. 2018-04-25 15:53:05 -07:00
Anu Engineer
0ecbc13128 HDFS-10278. Ozone: Add paging support to list Volumes. Contributed by Anu Engineer. 2018-04-25 15:52:58 -07:00
Anu Engineer
7bb2ab7d7c HDFS-10251. Ozone: Shutdown cleanly. Contributed by Anu Engineer 2018-04-25 15:52:58 -07:00
Anu Engineer
e11e824c9b HDFS-10268. Ozone: end-to-end integration for create/get volumes, buckets and keys. Contributed by Chris Nauroth. 2018-04-25 15:52:56 -07:00
Chris Nauroth
0addb1033e HDFS-10250. Ozone: Add key Persistence. Contributed by Anu Engineer. 2018-04-25 15:52:48 -07:00
Chris Nauroth
c6fd5ea3f0 HDFS-10238. Ozone : Add chunk persistance. Contributed by Anu Engineer. 2018-04-25 15:52:48 -07:00
Anu Engineer
0744d0a947 HDFS-8210. Ozone: Implement storage container manager. Contributed by Chris Nauroth. 2018-04-25 15:52:46 -07:00
Chris Nauroth
e758f90f93 HDFS-10195. Ozone: Add container persistence. Contributed by Anu Engineer. 2018-04-25 15:52:33 -07:00
Chris Nauroth
c18c6801ef HDFS-10202. ozone : Add key commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:25 -07:00
Chris Nauroth
f69ce5f3c6 HDFS-10196. Ozone : Enable better error reporting for failed commands in ozone shell. Contributed by Anu Engineer. 2018-04-25 15:52:09 -07:00
Chris Nauroth
fd2fe47b32 HDFS-10179. Ozone: Adding logging support. Contributed by Anu Engineer. 2018-04-25 15:52:07 -07:00
Chris Nauroth
7c55d7feb5 HDFS-9960. OzoneHandler : Add localstorage support for keys. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
4009b66090 HDFS-10180. Ozone: Refactor container Namespace. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
044cc47145 HDFS-9961. Ozone: Add buckets commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
bb47d03906 HDFS-9944. Ozone : Add container dispatcher. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
b9d53ed836 HDFS-9926. ozone : Add volume commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
60fbef08ec HDFS-9925. Ozone: Add Ozone Client lib for bucket handling. Contributed by Anu Engineer. 
Fix build break
 2018-04-25 15:52:00 -07:00
Chris Nauroth
a9879798a6 HDFS-9916. OzoneHandler : Add Key handler. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
93201330b8 HDFS-9916. OzoneHandler : Add Key handler. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Anu Engineer
2b387d622c HDFS-9920. Stop tracking CHANGES.txt in the HDFS-7240 feature branch. Contributed by Chris Nauroth. 2018-04-25 15:52:00 -07:00
Chris Nauroth
085c62545d HDFS-9891. Ozone: Add container transport client. Contributed by Anu Engineer. 2018-04-25 15:51:58 -07:00
Chris Nauroth
9b158f3b57 HDFS-9873. Ozone: Add container transport server. Contributed by Anu Engineer. 2018-04-25 15:51:53 -07:00
Anu Engineer
3c39c18a6d HDFS-9907. Exclude Ozone protobuf-generated classes from Findbugs analysis. Contributed by Chris Nauroth 2018-04-25 15:51:53 -07:00
Chris Nauroth
0a43e272f1 HDFS-9848. Ozone: Add Ozone Client lib for volume handling. Contributed by Anu Engineer. 2018-04-25 15:51:53 -07:00
Anu Engineer
3481897bde HDFS-9853. Ozone: Add container definitions. Contributed by Anu Engineer. 2018-04-25 15:51:51 -07:00
cnauroth
11a6cc2a4d HDFS-9845. OzoneHandler : Support List and Info Volumes. Contributed by Anu Engineer. 2018-04-25 15:51:39 -07:00
cnauroth
f6ef841f74 HDFS-9834. OzoneHandler : Enable MiniDFSCluster based testing for Ozone. Contributed by Anu Engineer. 2018-04-25 15:51:39 -07:00
Anu Engineer
f6b9c0f8a1 HDFS-8757 : OzoneHandler : Add localStorageHandler support for Buckets. Contributed by Anu Engineer 2018-04-25 15:51:39 -07:00
Anu Engineer
ae109d1de7 HDFS-8527. OzoneHandler: Integration of REST interface and container data pipeline back-end. Contributed by Chris Nauroth 2018-04-25 15:51:36 -07:00
Anu Engineer
a301827c6a HDFS-8695. OzoneHandler : Add Bucket REST Interface. (aengineer) 2018-04-25 15:51:00 -07:00
Anu Engineer
764ff0d505 HDFS-8753. Ozone: Unify StorageContainerConfiguration with ozone-default.xml & ozone-site.xml. Contributed by kanaka kumar avvaru 2018-04-25 15:51:00 -07:00
Arpit Agarwal
507e7274aa HDFS-8717. OzoneHandler : Add common bucket objects. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
07213f4eb5 HDFS-8680. OzoneHandler : Add Local StorageHandler support for volumes. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
835f872a0a HDFS-8654. OzoneHandler : Add ACL support. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
b1fa06aacb HDFS-8644. OzoneHandler : Add volume handler. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
2793b4c96f HDFS-8634. OzoneHandler: Add userAuth Interface and Simple userAuth handler. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
d65b373ec4 HDFS-8637. OzoneHandler : Add Error Table. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
cnauroth
2ea29ee062 HDFS-8448. Create REST Interface for Volumes. Contributed by Anu Engineer. 2018-04-25 15:51:00 -07:00
Arpit Agarwal
0a90e6d45c HDFS-8614. OzoneHandler : Add Quota Support. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
d41245ca56 HDFS-8456. Introduce STORAGE_CONTAINER_SERVICE as a new NodeType. (Contributed by Arpit Agarwal) 2018-04-25 15:50:52 -07:00
Yiqun Lin
b06601acce HDFS-13490. RBF: Fix setSafeMode in the Router. Contributed by Inigo Goiri. 2018-04-24 11:25:23 +08:00
Inigo Goiri
df92a17e02 HDFS-13336. Test cases of TestWriteToReplica failed in windows. Contributed by Xiao Liang. 2018-04-23 19:12:16 -07:00
Bharat Viswanadham
9b5375e0c1 HDFS-13356. Balancer:Set default value of minBlockSize to 10mb. Contributed by Bharat Viswanadham 2018-04-23 16:09:01 -07:00
Lei Xu
42e82f0281 Update CHANGES, RELEASENOTES, and jdiff for 3.0.2 release. 
(cherry picked from commit f6ecb76d0b919b9836600fe28ec9e637b223cd54)

Conflicts:
	hadoop-project-dist/pom.xml
 2018-04-23 14:14:05 -07:00
Chris Douglas
f411de6a79 HDFS-13408. MiniDFSCluster to support being built on randomized base directory. Contributed by Xiao Liang 2018-04-23 11:13:18 -07:00
Arpit Agarwal
c533c77047 HDFS-13433. webhdfs requests can be routed incorrectly in federated cluster. Contributed by Arpit Agarwal. 2018-04-23 10:08:53 -07:00
Inigo Goiri
63803e7051 HDFS-13388. RequestHedgingProxyProvider calls multiple configured NNs all the time. Contributed by Jinglun. 2018-04-22 19:49:35 -07:00
Arpit Agarwal
1c1ce63cda HDFS-13055. Aggregate usage statistics from datanodes. Contributed by Ajay Kumar. 2018-04-22 08:07:19 -07:00
Yiqun Lin
b6dae26f44 HDFS-13478. RBF: Disabled Nameservice store API. Contributed by Inigo Goiri. 2018-04-21 13:19:09 +08:00
Sangjin Lee
f40969a141 HDFS-10183. Prevent race condition during class initialization. Contributed by Pavel Avgustinov. 2018-04-20 20:33:10 -07:00
Inigo Goiri
1134af9ad1 HDFS-13453. RBF: getMountPointDates should fetch latest subdir time/date when parent dir is not present but /parent/child dirs are present in mount table. Contributed by Dibyendu Karmakar. 2018-04-19 14:56:36 -07:00
Shashikant Banerjee
e4c39f3247
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-19 08:35:38 +09:00
Bharat Viswanadham
bf7694d020 HDFS-13464. Fix javadoc in FsVolumeList#handleVolumeFailures. Contributed by Shashikant Banerjee 2018-04-18 08:23:45 -07:00
Mukul Kumar Singh
bf2f493f81 HDFS-13463. Fix javadoc in FsDatasetImpl#checkAndUpdate. Contributed by Shashikant Banerjee. 2018-04-18 16:11:37 +05:30
Erik Krogen
2d9e791a90
HDFS-12828. OIV ReverseXML Processor fails with escaped characters 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-18 14:38:23 +09:00
Wei Yan
e4313e7e47 HDFS-13466. RBF: Add more router-related information to the UI. 2018-04-17 15:01:07 -07:00
Inigo Goiri
c9b33514b8 HDFS-13462. Add BIND_HOST configuration for JournalNode's HTTP and RPC Servers. Contributed by Lukas Majercak. 2018-04-17 14:19:23 -07:00
Bharat Viswanadham
a68b04370f HDFS-13129. Add a test for DfsAdmin refreshSuperUserGroupsConfiguration. Contributed by Mukul Kumar Singh 2018-04-17 12:37:30 -07:00
Yiqun Lin
c4d3636c21 HDFS-13435. RBF: Improve the error loggings for printing the stack trace. 2018-04-17 11:23:22 +08:00
Arpit Agarwal
d907fdc3cd HDFS-13439. Add test case for read block operation when it is moved. Contributed by Ajay Kumar. 2018-04-16 14:16:59 -07:00
Wei-Chiu Chuang
e66e287efe HDFS-13330. ShortCircuitCache#fetchOrCreate never retries. Contributed by Gabor Bota. 2018-04-13 09:17:34 -07:00
Xiao Chen
6503593711 HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445. 2018-04-13 09:05:17 -07:00
Bharat Viswanadham
fa8b88ab2b HDFS-13438. Fix javadoc in FsVolumeList#removeVolume. Contributed by Shashikant Banerjee. 2018-04-13 08:56:02 -07:00
Yiqun Lin
0725953efe HDFS-13418. NetworkTopology should be configurable when enable DFSNetworkTopology. Contributed by Tao Jie. 2018-04-13 17:55:45 +08:00
Akira Ajisaka
9b0b9f2af2
HDFS-13436. Fix javadoc of package-info.java 2018-04-13 13:23:51 +09:00
Inigo Goiri
044341b4e4 HDFS-13386. RBF: Wrong date information in list file(-ls) result. Contributed by Dibyendu Karmakar. 2018-04-12 09:30:11 -07:00
Inigo Goiri
b5353c75d9 Revert "HDFS-13388. RequestHedgingProxyProvider calls multiple configured NNs all the time. Contributed by Jinglun." 
This reverts commit ac32b3576d.
 2018-04-12 09:28:23 -07:00
Inigo Goiri
d272056fcb Revert "HDFS-13386. RBF: Wrong date information in list file(-ls) result. Contributed by Dibyendu Karmakar." 
This reverts commit 18de6f2042.
 2018-04-12 08:42:19 -07:00
Mukul Kumar Singh
7ed8511ad8 HDFS-13426. Fix javadoc in FsDatasetAsyncDiskService#removeVolume. Contributed by Shashikant Banerjee. 2018-04-12 20:12:31 +05:30
Akira Ajisaka
113af12cfb
HDFS-7101. Potential null dereference in DFSck#doWork(). Contributed by skrho and Ted Yu. 2018-04-12 17:47:37 +09:00
Akira Ajisaka
c7cd362afd
HDFS-13427. Fix the section titles of transparent encryption document. 2018-04-12 10:44:56 +09:00
Inigo Goiri
18de6f2042 HDFS-13386. RBF: Wrong date information in list file(-ls) result. Contributed by Dibyendu Karmakar. 2018-04-11 12:03:14 -07:00
Inigo Goiri
f7d5bace43 HDFS-13428. RBF: Remove LinkedList From StateStoreFileImpl.java. Contributed by BELUGA BEHR. 2018-04-11 10:43:43 -07:00
Wei Yan
0c93d43f3d HDFS-13045. RBF: Improve error message returned from subcluster. Contributed by Inigo Goiri. 2018-04-11 08:37:43 -07:00
James Clampffer
eefe2a147c HDFS-13403: libhdfs++ Use hdfs::IoService object rather than asio::io_service. Contributed by James Clampffer. 2018-04-11 10:27:23 -04:00
Xiao Chen
7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Xiao Chen
e76c2aeb28 HDFS-13363. Record file path when FSDirAclOp throws AclException. Contributed by Gabor Bota. 2018-04-10 11:19:48 -07:00
Rakesh Radhakrishnan
f89594f0b8 HDFS-13328. Abstract ReencryptionHandler recursive logic in separate class. Contributed by Surendra Singh Lilhore. 2018-04-10 23:35:00 +05:30
Weiwei Yang
6729047a8b HDFS-13420. License header is displayed in ArchivalStorage/MemoryStorage html pages. Contributed by Akira Ajisaka. 2018-04-10 22:10:44 +08:00
Yiqun Lin
e87be8a2a4 HDFS-13384. RBF: Improve timeout RPC call mechanism. Contributed by Inigo Goiri. 2018-04-10 15:34:42 +08:00