big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
19,434 Commits 2 Branches 2 Tags 264 MiB
ea9b608237
Commit Graph

7351 Commits

Author SHA1 Message Date
Nanda kumar
11473b8204 HDFS-12898. Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
207523915d HDFS-12588. Use GenericOptionsParser for scm and ksm daemon. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
7bba3943f0 HDFS-12876. Ozone: moving NodeType from OzoneConsts to Ozone.proto. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
85ec48a646 HDFS-12867. Ozone: TestOzoneConfigurationFields fails consistently. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
538e17824d HDFS-12850. Ozone: TestContainerPersistence#testListContainer fails frequently due to timed out. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
e32e187448 HDFS-12864. Ozone: Turn off full synced write in RocksDB MetadataStore. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
8023ba50b5 HDFS-12807. Ozone: Expose RockDB stats via JMX for Ozone metadata stores. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
86802a461c HDFS-12851. Ozone: Upgrade to latest ratis build. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nanda kumar
ea86945754 HDFS-12842. Ozone: SCM: During startup SCM should check if the cluster is INITIALIZED. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
73ca8608da HDFS-12786. Ozone: add port/service names to the ksm/scm web ui. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
f9c11d952c HDFS-12740. SCM should support a RPC to share the cluster Id with KSM and DataNodes. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c8d8270f72 HDFS-12787. Ozone: SCM: Aggregate the metrics from all the container reports. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
cbe4f314b3 HDFS-12731. Ozone: Fix inconsistent default value in ozone settings. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Anu Engineer
4deb845d7c HDFS-12759. Ozone: web: integrate configuration reader page to the SCM/KSM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
8e8ff85b16 HDFS-12830. Ozone: TestOzoneRpcClient#testPutKeyRatisThreeNodes fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
214b52dfce HDFS-12500. Ozone: add logger for oz shell commands and move error stack traces to DEBUG level. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
419e86d580 HDFS-12805. Ozone: Redundant characters printed in exception log. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
93f1b04949 HDFS-12798. Ozone: scm web: fix the node status table. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
4460ac9c57 HDFS-12756. Ozone: Add datanodeID to heartbeat responses and container protocol. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
446e84357d HDFS-12796. SCM should not start if Cluster Version file does not exist. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
740a06cdd7 HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Yiqun Lin
761926db2c HDFS-12793. Ozone : TestSCMCli is failing consistently. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Chen Liang
e395244e95 HDFS-12758. Ozone: Correcting assertEquals argument order in test cases. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
cec96b296f HDFS-12549. Ozone: OzoneClient: Support for REST protocol. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
9734f505ea HDFS-12739. Add Support for SCM --init command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
709d56fdc9 HDFS-12474. Ozone: SCM: Handling container report with key count and container usage. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2ec24a7542 HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Yiqun Lin
132f30c8ea HDFS-12443. Ozone: Improve SCM block deletion throttling algorithm. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
7ebe79e879 HDFS-12720. Ozone: Ratis options are not passed from KSM Client protobuf helper correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0559265822 HDFS-12750. Ozone: Fix TestStorageContainerManager#testBlockDeletionTransactions. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
0760418bec HDFS-12697. Ozone services must stay disabled in secure setup for alpha. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
b687902590 HDFS-12718. Ozone: fix thread number calculation in CBlockManager. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
76d34bca62 HDFS-12521. Ozone: SCM should read all Container info into memory when booting up. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
6291ca1e32 HDFS-12663. Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
80357c0eee HDFS-12678. Ozone: Corona: Add statistical information to json output. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b56a73a4f HDFS-12680. Ozone: SCM: Lease support for container creation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
338c909ac6 HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
333dae888c HDFS-12690. Ozone: generate swagger descriptor for the Ozone REST Api. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
0582d8b1b1 Revert "HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton." 
This reverts commit 18c0cdcec9ff5676b10ba711039143d07dd4a510.
 2018-04-26 05:36:04 -07:00
Anu Engineer
402a1c1ba3 HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
731a69faa1 HDFS-12693. Ozone: Enable XFrame options for KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6783dad766 HDFS-11468. Ozone: SCM: Add Node Metrics for SCM. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
393a02d8e1 HDFS-12692. Ozone: fix javadoc/unit test for calling MetadataStore.getRangeKVs with non existing key. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang
6d9f914b55 HDFS-12684. Ozone: SCMMXBean NodeCount is overlapping with NodeManagerMXBean. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Nanda kumar
cc86719451 HDFS-12675. Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0f0417a12c HDFS-12673. Ozone: Log elapsed time for block deletion tasks. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
06c6582694 HDFS-12585. Ozone: KSM UI: Add description for configs in UI. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
763484e596 HDFS-12661. Ozone: Support optional documentation link in KSM/SCM webui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
a467017068 HDFS-12411. Ozone: Add container usage information to DN container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
fb545e4291 HDFS-12415. Ozone: TestXceiverClientManager and TestAllocateContainer occasionally fails. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
895a4cc60a HDFS-12632. Ozone: OzoneFileSystem: Add contract tests to OzoneFileSystem. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
9bb516118d HDFS-12490. Ozone: OzoneClient: Add creation/modification time information in OzoneVolume/OzoneBucket/OzoneKey. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nandakumar
8881309002 HDFS-12519. Ozone: Lease Manager framework. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
ebc4d4ebc3 HDFS-12475. Ozone: add documentation about Datanode http address. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
c85d3b2d25 HDFS-12593. Ozone: update Ratis to the latest snapshot. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Nandakumar
5ed663b32f HDFS-12631. Ozone: ContainerStorageLocation#scmUsage should count only SCM usage. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Yiqun Lin
d19b4c8763 HDFS-12583. Ozone: Fix swallow exceptions which makes hard to debug failures. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
9445a9267f HDFS-12572. Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
ae2b77a0e5 HDFS-12546. Ozone: DB listing operation performance improvement. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang
b05ad0b315 HDFS-12611. Ozone: SCM Cli: Use the hostname of SCM server instead of the bind address. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
yuanbo
e76e10fd7a HDFS-12609. Ozone: SCM: Refactoring of chill mode logic in NodeManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
e3b51d9074 HDFS-12598. Ozone: Fix 3 node ratis replication in Ozone. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nandakumar
3504af9411 HDFS-12537. Ozone: Reduce key creation overhead in Corona. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Nandakumar
9630621be7 HDFS-12610. Ozone: OzoneClient: RpcClient list calls throw NPE when iterating over empty list. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nandakumar
c3ef381011 HDFS-12543. Ozone : allow create key without specifying size. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c97ce55d06 HDFS-12602. Add Ozone config UI to KSM and SCM index page. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
303ecae5d5 HDFS-12557. Ozone: Improve the formatting of the RPC stats on web UI. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
c17521b1bd HDFS-12387. Ozone: Support Ratis as a first class replication mechanism. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer
0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
c52019c291 HDFS-12539. Ozone: refactor some functions in KSMMetadataManagerImpl to be more readable and reusable. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
ceec14bff5 HDFS-12467. Ozone: SCM: NodeManager should log when it comes out of chill mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
d7a94a21ae HDFS-12471. Ozone: Reduce some KSM/SCM deletion log messages from INFO to DEBUG. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6a754caca0 HDFS-12568. Ozone: Cleanup the ozone-default.xml. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
05246e2b32 HDFS-12060. Ozone: OzoneClient: Add list calls in RpcClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
7846636a5d HDFS-12551. Ozone: Documentation: Add Ozone overview documentation. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Yiqun Lin
cb1fb3436e HDFS-12563. Ozone: Creation time wrongly showed in OzoneRest.md. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
aaec2689f4 HDFS-12511. Ozone: Add tags to config. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7363c8fc09 HDFS-12540. Ozone: node status text reported by SCM is a bit confusing. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
13fdb58490 HDFS-12554. Ozone: Fix TestDatanodeStateMachine#testDatanodeStateMachineWithInvalidConfiguration. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Chen Liang
9530153f33 HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
94090acf3a HDFS-12469. Ozone: Create docker-compose definition to easily test real clusters. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
a44f3a777d HDFS-12501. Ozone: Cleanup javac issues. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6b10723ec3 HDFS-12525. Ozone: OzoneClient: Verify bucket/volume name in create calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5e4a6b686c HDFS-12425. Ozone: OzoneFileSystem: read/write/create/open/getFileInfo APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
a1a3ba6529 HDFS-12454. Ozone : the sample ozone-site.xml in OzoneGettingStarted does not work. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2804435386 HDFS-12017. Ozone: Container: Move IPC port to 98xx range. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
b2447f8dd9 HDFS-11614. Ozone: Cleanup javadoc issues. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
9ff136bb02 HDFS-12210. Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
dddded0016 Revert "Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh." to fix commit message. 
This reverts commit 087c69ba2434813044089f733581607d172efbd1.
 2018-04-26 05:36:04 -07:00
Weiwei Yang
d303b7f40f HDFS-12524. Ozone: Record number of keys scanned and hinted for getRangeKVs call. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Yiqun Lin
712bd70ac7 HDFS-12205. Ozone: List Key on an empty ozone bucket fails with command failed error. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
8052374e79 Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
fd1564b87e HDFS-12506. Ozone: ListBucket is too slow. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
46b65ec4fb HDFS-12503. Ozone: some UX improvements to oz_debug. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1c354e51ac HDFS-12477. Ozone: Some minor text improvement in SCM web UI. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nandakumar
0313bdf886 HDFS-11897. Ozone: KSM: Changing log level for client calls in KSM. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Weiwei Yang
3b290fe3bf HDFS-12466. Ozone: KSM: Make ozone.ksm.address as mandatory property for client. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
8ddf75da17 HDFS-12489. Ozone: OzoneRestClientException swallows exceptions which makes client hard to debug failures. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c329d3b4b4 HDFS-12329. Ozone: Ratis: Readonly calls in XceiverClientRatis should use sendReadOnly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
607d1fcd31 HDFS-12256. Ozone : handle inactive containers on DataNode. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
77d4b18700 HDFS-12385. Ozone: OzoneClient: Refactoring OzoneClient API. Contributed by Nadakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
b984e903f3 HDFS-12481. Ozone: Corona: Support for variable key length in offline mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
d9c3d9a5ef HDFS-11873. Ozone: Object store handler supports reusing http client for multiple requests. Contributed by Xiaoyu Yao and Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fe4139bdc3 HDFS-11612. Ozone: Cleanup Checkstyle issues. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Chen Liang
b127ecd1ba Revert "HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin." as it uses class not part of JRE. 
This reverts commit c29aff44cfc78a2839f15e8cde3b80f0a31c80dd.
 2018-04-26 05:36:04 -07:00
Weiwei Yang
eaf3732217 HDFS-12465. Ozone: fix a KeySpaceManager startup message typo. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c551025ed6 HDFS-12468. Ozone: fix hard coded version in the Ozone GettingStarted guide. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
cf841286b8 HDFS-12426. Ozone: BlockManager MBean unregister failure upon shutdown. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
f66a522695 HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
efa25373d7 HDFS-12019. Ozone: Misc : Make sure that ozone-site.xml is in etc/hadoop in tarball created from mvn package. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
80c50a1eac HDFS-12423. Ozone: TopN container choosing policy should ignore containers that has no pending deletion blocks. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1087ce4cd2 HDFS-11676. Ozone: SCM CLI: Implement close container command. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0c17776646 HDFS-12389. Ozone: oz commandline list calls should return valid JSON format output. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
fec04f1158 HDFS-12370. Ozone: Implement TopN container choosing policy for BlockDeletionService. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
743be0d7c0 HDFS-12235. Ozone: DeleteKey-3: KSM SCM block deletion message and ACK interactions. Contributed by Weiwei Yang and Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Weiwei Yang
3aa846412d HDFS-12397. Ozone: KSM: multiple delete methods in KSMMetadataManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang
e06a9a5bb1 HDFS-12391. Ozone: TestKSMSQLCli is not working as expected. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
1a21136322 HDFS-12382. Ozone: SCM: BlockManager creates a new container for each allocateBlock call. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
2e2e30373c HDFS-12333. Ozone: Extend Datanode web interface with SCM information. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
ca72c2342c HDFS-12366. Ozone: Refactor KSM metadata class names to avoid confusion. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7b3e7eb565 HDFS-12354. Ozone: Shuffle container list for datanode BlockDeletingService. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
eef437d5e2 HDFS-12282. Ozone: DeleteKey-4: Block delete via HB between SCM and DN. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2f4dfbc8fb HDFS-12275. Ozone: Corona: Support for random validation of writes. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
2dfd0a2f4f HDFS-12341. Ozone: maven dist compilation fails with "Duplicate classes found" error. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
532ced18fc HDFS-12280. Ozone: TestOzoneContainer#testCreateOzoneContainer fails. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Weiwei Yang
4d3b8d332d HDFS-12283. Ozone: DeleteKey-5: Implement SCM DeletedBlockLog. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
f51189e2bb HDFS-12324. Ozone: TestQueryNode#testStaleNodesCount is failing. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
02f8c2224f HDFS-11888. Ozone: SCM: use state machine for open containers allocated for key/blocks. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
f352b104f7 HDFS-12337. Ozone: Concurrent RocksDB open calls fail because of "No locks available". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
9e733215a8 HDFS-12327. Ozone: support setting timeout in background service. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
38bc1d1097 HDFS-12216. Ozone: TestKeys is failing consistently. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
bd33038bf4 HDFS-12159. Ozone: SCM: Add create replication pipeline RPC. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer
3a661b7f82 HDFS-12238. Ozone: Add valid trace ID check in sendCommandAsync. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Chen Liang
f9bce29dcc HDFS-12290. Block Storage: Change dfs.cblock.jscsi.server.address default bind address to 0.0.0.0. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
6a16d7c7ab HDFS-12255. Block Storage: Cblock should generated unique trace ID for the ops. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
e61530f5d9 HDFS-12298. Ozone: Block deletion service floods the log when deleting large number of block files. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
25586607a1 HDFS-12230. Ozone: KSM: Add creation time field in bucket info. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
dcc21a4f93 HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
c9ea431267 HDFS-12281. Ozone: Ozone-default.xml has 3 properties that do not match the default Config value. Contributed by Ajay Yadav. 2018-04-26 05:36:04 -07:00
Weiwei Yang
639b4fb8a9 HDFS-12196. Ozone: DeleteKey-2: Implement block deleting service to delete stale blocks at background. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
285d61cd8d HDFS-12274. Ozone: Corona: move corona from test to tools package. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
bb9a4793d3 HDFS-12259. Ozone: OzoneClient: Refactor move ozone client from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
b4aecc1009 HDFS-12262. Ozone: KSM: Reduce default handler thread count from 200. Contributed by Ajay Yadav. 2018-04-26 05:36:04 -07:00
Anu Engineer
bb5f1c4228 HDFS-12231. Ozone: KSM: Add creation time field in volume info. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
6f83e9c51e HDFS-12163. Ozone: Reduce MiniOzoneCluster handler thread count. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
89d8d20c73 HDFS-12246. Ozone: potential thread leaks. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
23dee0f123 HDFS-11935. Ozone: TestStorageContainerManager#testRpcPermission fails when ipv6 address used. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
ccc1607424 HDFS-12178. Ozone: OzoneClient: Handling SCM container creationFlag at client side. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
37a0ab5ca3 HDFS-12247. Ozone: KeySpaceManager should unregister KSMMetrics upon stop. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
9cf40547ce HDFS-11580. Ozone: Support asynchronus client API for SCM and containers. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1ad95cf2a9 HDFS-11984. Ozone: Ensures listKey lists all required key fields. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
acfae2716a HDFS-12034. Ozone: Web interface for KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Chen Liang
070f7e2e59 HDFS-11920. Ozone : add key partition. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
bc413a69cc HDFS-12195. Ozone: DeleteKey-1: KSM replies delete key request asynchronously. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
86b42907cf HDFS-12183. Ozone: KSM : Use proper defaults for block client address. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
7907fe1c80 HDFS-12059. Ozone: OzoneClient: OzoneClientImpl Add setBucketProperty and delete calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
fcd4537e63 HDFS-12180. Ozone: Corona: Add stats and progress bar to corona. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
a4b3160efb HDFS-12145. Ozone: OzoneFileSystem: Ozone & KSM should support / delimited key names. Contributed by Mukul Kumar Singh. 
(cherry picked from commit fe0a957e92d506330ba005abba18f665eb5ac1cc)
 2018-04-26 05:36:04 -07:00
Yiqun Lin
8cf0c864c4 HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
349a87ac9c HDFS-12192. Ozone: Fix the remaining failure tests for Windows caused by incorrect path generated. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d50a743015 HDFS-12155. Ozone : add RocksDB support to DEBUG CLI. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
0cc166c053 HDFS-12115. Ozone: SCM: Add queryNode RPC Call. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Weiwei Yang
539842ed8b HDFS-12187. Ozone : add support to DEBUG CLI for ksm.db. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
b00830142b HDFS-12167. Ozone: Intermittent failure TestContainerPersistence#testListKey. Contributed by Surendra Singh Lilhore. 2018-04-26 05:36:04 -07:00
Anu Engineer
487aa2f7bf HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
d02a4e9dd1 HDFS-12175. Ozone: Fix Leaking in TestStorageContainerManager#testRpcPermission. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Chen Liang
d05b44f417 Revert "HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao" 
This reverts commit f801454ad320e0d9c544c4249456bcaf50b4f30b.
 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
eda559ffd2 HDFS-12181. Ozone: Fix TestContainerReplicationManager by setting proper log level for LogCapturer. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
ca096501bf HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao 2018-04-26 05:36:04 -07:00
Weiwei Yang
f1e2592bf8 HDFS-12127. Ozone: Ozone shell: Add more testing for key shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
a652a72286 HDFS-11936. Ozone: TestNodeManager times out before it is able to find all nodes. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
bbbfc885e7 HDFS-12149. Ozone: RocksDB implementation of ozone metadata store. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6225622dee HDFS-12071. Ozone: Corona: Implementation of Corona. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d2ef3e40bc HDFS-12126. Ozone: Ozone shell: Add more testing for bucket shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
5604c3788f HDFS-11996. Ozone : add an UT to test partial read of chunks. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4d333f8db1 HDFS-12058. Ozone: OzoneClient: OzoneClientImpl add getDetails calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang
27262d8889 HDFS-12118. Ozone: Ozone shell: Add more testing for volume shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
db859e8172 HDFS-12148. Ozone: TestOzoneConfigurationFields is failing because ozone-default.xml has some missing properties. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
9522356b46 HDFS-12069. Ozone: Create a general abstraction for metadata store. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
b94b5021b8 HDFS-11989. Ozone: add TestKeysRatis, TestBucketsRatis and TestVolumeRatis. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
ab3d510c1d HDFS-12123. Ozone: OzoneClient: Abstraction of OzoneClient and default implementation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
3d430a6076 HDFS-11493. Ozone: SCM: Add the ability to handle container reports. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Yiqun Lin
58e850f262 HDFS-12083. Ozone: KSM: previous key has to be excluded from result in listVolumes, listBuckets and listKeys. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
a4cbd33dd5 HDFS-12129. Ozone: SCM http server is not stopped with SCM#stop(). Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
ea5e24c925 HDFS-12105. Ozone: listVolumes doesn't work from ozone commandline. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7afddca9d5 HDFS-12037. Ozone: Improvement rest API output format for better looking. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
25c1b296e6 HDFS-11679. Ozone: SCM CLI: Implement list container command. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Weiwei Yang
eb18b86e00 HDFS-12080. Ozone: Fix UT failure in TestOzoneConfigurationFields. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
eab407beac HDFS-12057. Ozone: OzoneClient: Implementation of OzoneClient. Contributed by Nandakumar 2018-04-26 05:36:04 -07:00
Anu Engineer
a353094ba4 HDFS-12076. Ozone: Review all cases where we are returning FAILED_INTERNAL_ERROR. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
8caa5d1ac0 HDFS-12086. Ozone: Add the unit test for KSMMetrics. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
5fe9143a0e HDFS-12081. Ozone: Add infoKey REST API document. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4c27dba08a HDFS-12053. Ozone: ozone server should create missing metadata directory if it has permission to. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
63ce883746 HDFS-12035. Ozone: listKey doesn't work from ozone commandline. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
e07662cb04 HDFS-12048. Ozone:TestOzoneContainerRatis & TestRatisManager are failing consistently. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d1ac8611c6 HDFS-12023. Ozone: Test if all the configuration keys are documented in ozone-defaults.xml. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c398ab49d3 HDFS-12047. Ozone: Add REST API documentation. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
5379ffa8de HDFS-12030. Ozone: CLI: support infoKey command. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
4b63e3e504 HDFS-12002. Ozone : SCM cli misc fixes/improvements. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c567e6766b HDFS-11955. Ozone: Set proper parameter default values for listBuckets http request. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00