big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
19,729 Commits 2 Branches 2 Tags 264 MiB
3159b39cf8
Commit Graph

4456 Commits

Author SHA1 Message Date
Xiaoyu Yao
18ddec6144 HDFS-11977. Ozone: cannot enable test debug/trace log. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
15cc4007b0 HDFS-11946. Ozone: Containers in different datanodes are mapped to the same location. Contributed by Nandakumar 2018-04-26 05:36:04 -07:00
Anu Engineer
fa26dea89d HDFS-11185. Ozone: remove disabled tests. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Lei Xu
08460f1559 HDFS-11659. TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Contributed by Lei (Eddy) Xu. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0c37e05b10 HDFS-11779. Ozone: KSM: add listBuckets. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
bacd1188f1 HDFS-11880. Ozone: KSM: Remove protobuf formats from KSM wrappers. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2007e85d5b HDFS-11777. Ozone: KSM: add deleteBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Anu Engineer
3c0ed97df9 HDFS-11926. Ozone: Implement a common helper to return a range of KVs in levelDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d6dd557b24 HDFS-11887. Shared XceiverClient should be closed if there is no open clients to avoid resource leak. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
f5d17b8f7c HDFS-11919. Ozone: SCM: TestNodeManager takes too long to execute. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Chen Liang
d67542c115 HDFS-11822. Block Storage: Fix TestCBlockCLI, failing because of " Address already in use". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
bf8c168a80 HDFS-11781. Ozone: KSM: Add deleteKey. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
42eaecae55 HDFS-11771. Ozone: KSM: Add checkVolumeAccess. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b2d0b4fec HDFS-11796. Ozone: MiniOzoneCluster should set "ozone.handler.type" key correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
aa5722dc44 HDFS-11913. Ozone: TestKeySpaceManager#testDeleteVolume fails. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4893692972 HDFS-11597. Ozone: Add Ratis management API. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Mingliang Liu
677dcf529c HDFS-11903. Ozone: Cleaning up local storage when closing MiniOzoneCluster. Contributed by Mingliang Liu 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
6b9915fcbd HDFS-11774. Ozone: KSM: add deleteVolume. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
236c410881 HDFS-11776. Ozone: KSM: add SetBucketProperty. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
72b228a9e6 HDFS-11853. Ozone: KSM: Add getKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
59d273b175 HDFS-11655. Ozone: CLI: Guarantees user runs SCM commands has appropriate permission. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
b71efcf1b0 HDFS-11865. Ozone: Do not initialize Ratis cluster during datanode startup. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
e3c8f6a24d HDFS-11780. Ozone: KSM: Add putKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ff5dbeec07 HDFS-11846. Ozone: Fix Http connection leaks in ozone clients. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
84294de9a2 HDFS-11778. Ozone: KSM: add getBucketInfo. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
47c4867765 HDFS-11859. Ozone: SCM: Separate BlockLocationProtocol from ContainerLocationProtocol. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
0753e094d7 HDFS-11727. Block Storage: Retry Blocks should be requeued when cblock is restarted. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
37642c12df HDFS-11770. Ozone: KSM: Add setVolumeProperty. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Weiwei Yang
ae5242accb HDFS-11830. Ozone: Datanode needs to re-register to SCM if SCM is restarted. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
877e751c84 HDFS-11843. Ozone: XceiverClientRatis should implement XceiverClientSpi.connect(). 2018-04-26 05:34:18 -07:00
Chen Liang
4fb9064523 HDFS-11835. Block Storage: Overwrite of blocks fails. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ca70300eea HDFS-11775. Ozone: KSM: add createBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Weiwei Yang
e9d09c209e HDFS-11824. Ozone: Fix javac warnings. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
7f0be64284 HDFS-11831. Ozone: Improve the way of getting test file path in unit tests for Windows. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
c9e6d4378d HDFS-11769. Ozone: KSM: Add createVolume API. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Chen Liang
162dac96ce HDFS-11650. Ozone: fix the consistently timeout test testUpgradeFromRel22Image. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Chen Liang
dfb0dab966 HDFS-11694. Block Storage: Add Support for 2 BlockIDBuffers and also for periodic flush of BlockIDBuffer. Contributed by Mukul Kumar Singh 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
b592df6413 HDFS-11811. Ozone: SCM: Support Delete Block. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
9bc494b909 HDFS-11802. Ozone : add DEBUG CLI support for open container db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
edff6c6a24 HDFS-11768. Ozone: KSM: Create Key Space manager service. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Chen Liang
169713f3f3 HDFS-11788. Ozone : add DEBUG CLI support for nodepool db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
070960ed8a HDFS-11798. Ozone: Correct description for ozone.handler.type in ozone-default.xml. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Weiwei Yang
4be9710712 HDFS-11680. Ozone: SCM CLI: Implement info container command. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Chen Liang
9fcaeceb54 HDFS-11756. Ozone : add DEBUG CLI support of blockDB file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Weiwei Yang
c18229f0df HDFS-11716. Ozone: SCM: CLI: Revisit delete container API. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
67c81a8b18 HDFS-11761. Ozone: Get container report should only report closed containers. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
d9be9a8c30 HDFS-11698. Ozone: SCM: CLI: Add Debug command. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
3158f479d3 HDFS-11725. Ozone: Revise create container CLI specification and implementation. Contributed by Weiwei Yang 2018-04-26 05:34:18 -07:00
Anu Engineer
6b3da448eb HDFS-11675. Ozone: SCM CLI: Implement delete container command. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Chen Liang
e9588c6422 HDFS-11627. Block Storage: Cblock cache should register with flusher to upload blocks to containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Chen Liang
668b056984 HDFS-11667. Block Storage:Handling flushing of incomplete block id buffers during shutdown. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
1a245accb5 HDFS-11668. Ozone: misc improvements for SCM CLI. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Yiqun Lin
5984d40b86 HDFS-11666. Ozone: Fix compile error due to inconsistent package name. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
a9ba8b5ded HDFS-11504. Ozone: SCM: Add Block APIs. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Weiwei Yang
720e5c40f6 HDFS-11649. Ozone: SCM: CLI: Add shell code placeholder classes. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
6c4bd1647c HDFS-11581. Ozone: Support force delete a container. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Anu Engineer
05b12c48cf HDFS-11657. Ozone: Add unit test for storage container metrics. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
7d4a2d4104 HDFS-11569. Ozone: Implement listKey function for KeyManager. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Chen Liang
cbb7cc26a5 HDFS-11642. Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
5b19a03f63 HDFS-11635. Block Storage: Add metrics for Container Flushes. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
2129c6b276 HDFS-11062. Ozone:SCM: Remove null command. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Chen Liang
b7463924af HDFS-11618. Block Storage: Add Support for Direct I/O. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
cd361fa490 HDFS-11625. Ozone: Fix UT failures that caused by hard coded datanode data dirs. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
7573a0bc61 HDFS-11620. Ozone: Allocate container for MiniOzone cluster fails because of insufficient space error. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
fadfe37019 HDFS-11619. Ozone: Unit tests running failed in Windows. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
82d814fa79 HDFS-11564. Ozone: SCM: Add Comparable Metric Support. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
68da45a789 HDFS-11519. Ozone: Implement XceiverServerSpi and XceiverClientSpi using Ratis. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:34:18 -07:00
Chen Liang
3966f42895 HDFS-11591. Block Storage: fix TestCBlockServer test cleanup. Contributed by Xiaoyu Yao 2018-04-26 05:34:18 -07:00
Anu Engineer
7c59add3da HDFS-11594. Ozone: close container should call compactDB. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
bc9c313907 HDFS-11567. Ozone: SCM: Support update container. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
17a6e62629 HDFS-11537. Block Storage : add cache layer. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
3598d4b8cf HDFS-11548. Ozone: SCM: Add node pool management API. Contributed by Xiaoyu Yao 2018-04-26 05:34:18 -07:00
Anu Engineer
860a4e0044 HDFS-11550. Ozone: Add a check to prevent removing a container that has keys in it. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
23044c1db8 HDFS-11361. Block Storage: add cache interface. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
d251a55ff9 HDFS-11559. Ozone: MiniOzoneCluster prints too many log messages by default. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:34:18 -07:00
Anu Engineer
1e892f5748 HDFS-11162. Block Storage: add command line tool. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
85c2312e7d HDFS-11492. Ozone: Add the ability to handle sendContainerReport Command. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
98d3360e3a HDFS-11444. Ozone: Fix datanode ID handling in MiniOzoneCluster. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
88371ff07f HDFS-11491. Ozone: SCM: Add close container RPC. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
932423211f HDFS-11469. Ozone: SCM: Container allocation based on node report. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
fc7d678d3d HDFS-11071. Ozone: SCM: Move SCM config keys to ScmConfig. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
7b7d186b82 HDFS-11513. Ozone: Separate XceiverServer and XceiverClient into interfaces and implementations. 2018-04-26 05:34:18 -07:00
Anu Engineer
1058aa2523 HDFS-10928. Ozone:SCM: Support MXBean for SCM and NodeManager. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
386dbc11da HDFS-11196. Ozone: Improve logging and error handling in the container layer. Contributed by Anu Engineer. 2018-04-25 15:55:06 -07:00
Anu Engineer
32cc2b8f1a HDFS-11509. Ozone: Fix TestEndpoint test regression. Contributed by Anu Engineer. 2018-04-25 15:54:56 -07:00
Anu Engineer
9c57a61f68 HDFS-11424. Block Storage: add container representation to meta data. Contributed Chen Liang. 2018-04-25 15:54:56 -07:00
Anu Engineer
3e1317de02 HDFS-11451. Ozone: Add protobuf definitions for container reports. Contributed by Anu Engineer. 2018-04-25 15:54:56 -07:00
Anu Engineer
02fb9aed32 HDFS-11158. Ozone: TestDatanodeStateMachine is failing intermittently. Contributed by Weiwei Yang. 2018-04-25 15:54:56 -07:00
Anu Engineer
753979865b HDFS-11137. Ozone: SCM: Add negative tests cases for datanodeStatemachine. Contributed by Weiwei Yang. 2018-04-25 15:54:42 -07:00
Anu Engineer
d55564143d HDFS-11484. Ozone: Fix flaky TestNodeManager#testScmNodeReportUpdate. Contributed by Xiaoyu Yao. 2018-04-25 15:54:42 -07:00
Anu Engineer
a2328d36ac HDFS-11480. Ozone: TestEndpoint task failure. Contributed by Xiaoyu Yao. 2018-04-25 15:54:42 -07:00
Anu Engineer
2cba0daddc HDFS-11425. Ozone: add client-facing container APIs and container references. Contributed by Chen Liang. 2018-04-25 15:54:42 -07:00
Anu Engineer
56f011fd02 HDFS-11447. Ozone: SCM: Send node report to SCM with heartbeat.Contributed by Xiaoyu Yao. 2018-04-25 15:54:41 -07:00
Anu Engineer
f4529316bd HDFS-11474. Ozone: TestContainerMapping needs to cleanup levelDB files.Contributed by Xiaoyu Yao. 2018-04-25 15:54:36 -07:00
Arpit Agarwal
754ed7d651 HDFS-11414. Ozone : move StorageContainerLocation protocol to hdfs-client. Contributed by Chen Liang. 2018-04-25 15:54:34 -07:00
Anu Engineer
bb410de10c HDFS-11184. Ozone: SCM: Make SCM use container protocol. Contributed by Anu Engineer. 2018-04-25 15:54:26 -07:00
Anu Engineer
c169dd1306 HDFS-11463. Ozone: Add metrics for container operations and export over JMX. Contributed by Mukul Kumar Singh. 2018-04-25 15:54:03 -07:00
Anu Engineer
a7f82cba71 HDFS-11154. Block Storage: store server state to persistent storage. Contributed by Chen Liang 2018-04-25 15:53:58 -07:00
Anu Engineer
9997d36eb6 HDFS-11138. Block Storage: add block storage server. Contributed by Chen Liang 2018-04-25 15:53:58 -07:00
Anu Engineer
48db56adea HDFS-11133. Ozone: Add allocateContainer RPC. Contributed by Anu Engineer 2018-04-25 15:53:58 -07:00
Anu Engineer
ca12aac5a4 HDFS-11126. Ozone: Add small file support RPC. Contributed by Anu Engineer. 2018-04-25 15:53:58 -07:00
Anu Engineer
05b44e1ad8 HDFS-11108. Ozone: use containers with the state machine. Contributed by Anu Engineer 2018-04-25 15:53:56 -07:00
Anu Engineer
8bd85268e6 HDFS-11081. Ozone:SCM: Add support for registerNode in datanode. Contributed by Anu Engineer. 2018-04-25 15:53:50 -07:00
Anu Engineer
8274ff356a HDFS-11103. Ozone: Cleanup some dependencies. Contributed by Anu Engineer. 2018-04-25 15:53:39 -07:00
Anu Engineer
e49e305f25 HDFS-11001. Ozone:SCM: Add support for registerNode in SCM. Contributed by Anu Engineer. 2018-04-25 15:53:34 -07:00
Anu Engineer
ed84388fca HDFS-10912. Ozone:SCM: Add chill mode support to NodeManager. Contributed by Anu Engineer. 2018-04-25 15:53:27 -07:00
Anu Engineer
56b0fddfbd HDFS-10995. Ozone: Move ozone XceiverClient to hdfs-client. Contributed by Chen Liang. 2018-04-25 15:53:25 -07:00
Anu Engineer
4c95c3d6ab HDFS-10897. Ozone: SCM: Add NodeManager. Contributed by Anu Engineer. 2018-04-25 15:53:21 -07:00
Arpit Agarwal
e3aa2b687c HDFS-10420. Fix Ozone unit tests to use MiniOzoneCluster. Contributed by Arpit Agarwal. 2018-04-25 15:53:21 -07:00
Anu Engineer
3beee0b3d1 HDFS-10363. Ozone: Introduce new config keys for SCM service endpoints. Contributed by Arpit Agarwal. 2018-04-25 15:53:19 -07:00
Anu Engineer
7d661a289a HDFS-10232. Ozone: Make config key naming consistent. Contributed by Anu Engineer. 2018-04-25 15:53:05 -07:00
Anu Engineer
0ecbc13128 HDFS-10278. Ozone: Add paging support to list Volumes. Contributed by Anu Engineer. 2018-04-25 15:52:58 -07:00
Anu Engineer
e11e824c9b HDFS-10268. Ozone: end-to-end integration for create/get volumes, buckets and keys. Contributed by Chris Nauroth. 2018-04-25 15:52:56 -07:00
Chris Nauroth
0addb1033e HDFS-10250. Ozone: Add key Persistence. Contributed by Anu Engineer. 2018-04-25 15:52:48 -07:00
Chris Nauroth
c6fd5ea3f0 HDFS-10238. Ozone : Add chunk persistance. Contributed by Anu Engineer. 2018-04-25 15:52:48 -07:00
Anu Engineer
0744d0a947 HDFS-8210. Ozone: Implement storage container manager. Contributed by Chris Nauroth. 2018-04-25 15:52:46 -07:00
Chris Nauroth
e758f90f93 HDFS-10195. Ozone: Add container persistence. Contributed by Anu Engineer. 2018-04-25 15:52:33 -07:00
Chris Nauroth
fd2fe47b32 HDFS-10179. Ozone: Adding logging support. Contributed by Anu Engineer. 2018-04-25 15:52:07 -07:00
Chris Nauroth
7c55d7feb5 HDFS-9960. OzoneHandler : Add localstorage support for keys. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
4009b66090 HDFS-10180. Ozone: Refactor container Namespace. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
bb47d03906 HDFS-9944. Ozone : Add container dispatcher. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
60fbef08ec HDFS-9925. Ozone: Add Ozone Client lib for bucket handling. Contributed by Anu Engineer. 
Fix build break
 2018-04-25 15:52:00 -07:00
Chris Nauroth
085c62545d HDFS-9891. Ozone: Add container transport client. Contributed by Anu Engineer. 2018-04-25 15:51:58 -07:00
Chris Nauroth
9b158f3b57 HDFS-9873. Ozone: Add container transport server. Contributed by Anu Engineer. 2018-04-25 15:51:53 -07:00
Chris Nauroth
0a43e272f1 HDFS-9848. Ozone: Add Ozone Client lib for volume handling. Contributed by Anu Engineer. 2018-04-25 15:51:53 -07:00
cnauroth
11a6cc2a4d HDFS-9845. OzoneHandler : Support List and Info Volumes. Contributed by Anu Engineer. 2018-04-25 15:51:39 -07:00
cnauroth
f6ef841f74 HDFS-9834. OzoneHandler : Enable MiniDFSCluster based testing for Ozone. Contributed by Anu Engineer. 2018-04-25 15:51:39 -07:00
Anu Engineer
ae109d1de7 HDFS-8527. OzoneHandler: Integration of REST interface and container data pipeline back-end. Contributed by Chris Nauroth 2018-04-25 15:51:36 -07:00
Anu Engineer
764ff0d505 HDFS-8753. Ozone: Unify StorageContainerConfiguration with ozone-default.xml & ozone-site.xml. Contributed by kanaka kumar avvaru 2018-04-25 15:51:00 -07:00
Arpit Agarwal
507e7274aa HDFS-8717. OzoneHandler : Add common bucket objects. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
07213f4eb5 HDFS-8680. OzoneHandler : Add Local StorageHandler support for volumes. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
835f872a0a HDFS-8654. OzoneHandler : Add ACL support. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
b1fa06aacb HDFS-8644. OzoneHandler : Add volume handler. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
0a90e6d45c HDFS-8614. OzoneHandler : Add Quota Support. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Inigo Goiri
df92a17e02 HDFS-13336. Test cases of TestWriteToReplica failed in windows. Contributed by Xiao Liang. 2018-04-23 19:12:16 -07:00
Chris Douglas
f411de6a79 HDFS-13408. MiniDFSCluster to support being built on randomized base directory. Contributed by Xiao Liang 2018-04-23 11:13:18 -07:00
Arpit Agarwal
c533c77047 HDFS-13433. webhdfs requests can be routed incorrectly in federated cluster. Contributed by Arpit Agarwal. 2018-04-23 10:08:53 -07:00
Arpit Agarwal
1c1ce63cda HDFS-13055. Aggregate usage statistics from datanodes. Contributed by Ajay Kumar. 2018-04-22 08:07:19 -07:00
Erik Krogen
2d9e791a90
HDFS-12828. OIV ReverseXML Processor fails with escaped characters 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-18 14:38:23 +09:00
Inigo Goiri
c9b33514b8 HDFS-13462. Add BIND_HOST configuration for JournalNode's HTTP and RPC Servers. Contributed by Lukas Majercak. 2018-04-17 14:19:23 -07:00
Bharat Viswanadham
a68b04370f HDFS-13129. Add a test for DfsAdmin refreshSuperUserGroupsConfiguration. Contributed by Mukul Kumar Singh 2018-04-17 12:37:30 -07:00
Arpit Agarwal
d907fdc3cd HDFS-13439. Add test case for read block operation when it is moved. Contributed by Ajay Kumar. 2018-04-16 14:16:59 -07:00
Wei-Chiu Chuang
e66e287efe HDFS-13330. ShortCircuitCache#fetchOrCreate never retries. Contributed by Gabor Bota. 2018-04-13 09:17:34 -07:00
Xiao Chen
6503593711 HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445. 2018-04-13 09:05:17 -07:00
Yiqun Lin
0725953efe HDFS-13418. NetworkTopology should be configurable when enable DFSNetworkTopology. Contributed by Tao Jie. 2018-04-13 17:55:45 +08:00
Xiao Chen
7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Rakesh Radhakrishnan
f89594f0b8 HDFS-13328. Abstract ReencryptionHandler recursive logic in separate class. Contributed by Surendra Singh Lilhore. 2018-04-10 23:35:00 +05:30
Surendra Singh Lilhore
70590cd8d9 HDFS-13292. Crypto command should give proper exception when trying to set key on existing EZ directory. Contributed by Ranith Sardar. 2018-04-07 11:23:49 +05:30
Lei Xu
d737bf99d4 HDFS-13350. Negative legacy block ID will confuse Erasure Coding to be considered as striped block. (Contributed by Lei (Eddy) Xu). 2018-04-05 09:59:10 -07:00
Xiao Chen
57374c4737 HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 2018-04-02 22:48:26 -07:00
Konstantin V Shvachko
acfd764fcc HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 2018-03-30 18:55:35 -07:00
Xiao Chen
2c6cfad5a3 HDFS-13087. Snapshotted encryption zone information should be immutable. Contributed by LiXin Ge. 2018-03-29 15:46:34 -07:00
Xiaoyu Yao
389bc6d3da HDFS-11043. TestWebHdfsTimeouts fails. Contributed by Xiaoyu Yao and Chao Sun. 2018-03-21 13:53:35 -07:00
Yongjun Zhang
6c63cc7d30 HDFS-13315. Add a test for the issue reported in HDFS-11481 which is fixed by HDFS-10997. Contributed by Yongjun Zhang. 2018-03-20 23:04:05 -07:00
weiy
6e2b5fa493 HDFS-13215. RBF: Move Router to its own module. Contributed by Wei Yan 2018-03-19 22:13:40 -07:00
Inigo Goiri
08ff1586d5 HDFS-12886. Ignore minReplication for block recovery. Contributed by Lukas Majercak. 2018-03-16 10:29:19 -07:00
Xiaoyu Yao
78b05fde6c HDFS-13280. WebHDFS: Fix NPE in get snasphottable directory list call. Contributed by Lokesh Jain. 2018-03-15 11:02:27 -07:00
Inigo Goiri
e71bc00a47 HDFS-13224. RBF: Resolvers to support mount points across multiple subclusters. Contributed by Inigo Goiri. 2018-03-15 10:32:30 -07:00
Xiaoyu Yao
da777a5498 HDFS-13251. Avoid using hard coded datanode data dirs in unit tests(addendum). Contributed by Ajay Kumar. 2018-03-15 10:18:44 -07:00
Inigo Goiri
6de135169e HDFS-12723. TestReadStripedFileWithMissingBlocks#testReadFileWithMissingBlocks failing consistently. Contributed by Ajay Kumar. 2018-03-15 10:14:35 -07:00
Inigo Goiri
41fc7f80be HDFS-13198. RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router. Contributed by Wei Yan. 2018-03-14 13:14:36 -07:00
Xiaoyu Yao
f83716b7f2 HDFS-13251. Avoid using hard coded datanode data dirs in unit tests. Contributed by Ajay Kumar. 2018-03-14 11:26:58 -07:00
Arpit Agarwal
4c57fb0cd9 HDFS-13257. Code cleanup: INode never throws QuotaExceededException. Contributed by Tsz Wo Nicholas Sze. 2018-03-14 09:42:48 -07:00
Sammi Chen
ad1b988a82 HDFS-11600. Refactor TestDFSStripedOutputStreamWithFailure test classes. Contributed by Sammi Chen. 2018-03-14 14:39:53 +08:00
Yiqun Lin
76be6cbf6c HDFS-12773. RBF: Improve State Store FS implementation. Contributed by Inigo Goiri. 2018-03-14 11:20:59 +08:00
Arpit Agarwal
9714fc1dd4 HDFS-336. dfsadmin -report should report number of blocks from datanode. Contributed by Bharat Viswanadham. 2018-03-13 16:39:17 -07:00
Xiao Chen
8211a3d469 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 2018-03-13 10:48:45 -07:00
Xiao Chen
f82d38dcb3 Revert "HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma." 
This reverts commit 84c1095586.
 2018-03-13 10:36:16 -07:00
Xiao Chen
3b8dbc2cb7 HDFS-12587. Use Parameterized tests in TestBlockInfoStriped and TestLowRedundancyBlockQueues to test all EC policies. Contributed by Takanobu Asanuma. 2018-03-13 10:14:05 -07:00
Xiao Chen
d6931c30c5 HDFS-13239. Fix non-empty dir warning message when setting default EC policy. Contributed by Bharat Viswanadham. 2018-03-13 10:07:26 -07:00
Xiao Chen
84c1095586 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 2018-03-13 09:58:03 -07:00
Xiaoyu Yao
0355ec20eb HDFS-13141. WebHDFS: Add support for getting snasphottable directory list. Contributed by Lokesh Jain. 2018-03-12 20:41:37 -07:00
Yiqun Lin
19292bc264 HDFS-13226. RBF: Throw the exception if mount table entry validated failed. Contributed by maobaolong. 2018-03-13 11:03:31 +08:00
Yiqun Lin
7fab787de7 HDFS-13253. RBF: Quota management incorrect parent-child relationship judgement. Contributed by Yiqun Lin. 2018-03-13 10:30:20 +08:00
Akira Ajisaka
319defafc1
HDFS-12156. TestFSImage fails without -Pnative 2018-03-13 11:26:56 +09:00
Yiqun Lin
4afd50b106 HDFS-10803. TestBalancerWithMultipleNameNodes#testBalancing2OutOf3Blockpools fails intermittently due to no free space available. Contributed by Yiqun Lin. 2018-03-13 10:15:51 +08:00
Inigo Goiri
91c82c90f0 HDFS-13241. RBF: TestRouterSafemode failed if the port 8888 is in use. Contributed by maobaolong. 2018-03-12 17:28:15 -07:00
Chris Douglas
39a5fbae47 HDFS-12677. Extend TestReconstructStripedFile with a random EC policy. Contributed by Takanobu Asanuma 2018-03-12 14:29:44 -07:00
Anu Engineer
cceb68ffea HDFS-10618. TestPendingReconstruction#testPendingAndInvalidate is flaky due to race condition. Contributed by Eric Badger. 2018-03-12 12:07:22 -07:00
Inigo Goiri
ac627f561f HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin. 2018-03-12 09:58:56 -07:00
Inigo Goiri
8133cd5305 HDFS-13232. RBF: ConnectionPool should return first usable connection. Contributed by Ekanth S. 2018-03-09 18:25:05 -08:00
Inigo Goiri
afe1a3ccd5 HDFS-13212. RBF: Fix router location cache issue. Contributed by Weiwei Wu. 2018-03-09 17:18:51 -08:00
Tsz-Wo Nicholas Sze
ba0da2785d HDFS-13252. Code refactoring: Remove Diff.ListType. 2018-03-09 15:50:26 -08:00
Hanisha Koneru
9a082fbe6e HDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus. 2018-03-09 13:20:32 -08:00
Yiqun Lin
122805b43a HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 2018-03-09 15:42:57 +08:00
Inigo Goiri
0c2b969e01 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 2018-03-08 09:32:05 -08:00
Sean Mackrory
46d29e3d7e HDFS-13176. WebHdfs file path gets truncated when having semicolon (;) inside. Contributed by Zsolt Venczel. 2018-03-07 13:33:41 -07:00
Tsz-Wo Nicholas Sze
88fba00caa HDFS-13222. Update getBlocks method to take minBlockSize in RPC calls. Contributed by Bharat Viswanadham 2018-03-07 11:27:53 -08:00
Yiqun Lin
58ea2d7a65 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 2018-03-07 15:26:38 +08:00
Xiaoyu Yao
edf9445708 HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 2018-03-06 16:44:20 -08:00
Tsz-Wo Nicholas Sze
871d0d39fa HDFS-13223. Reduce DiffListBySkipList memory usage. Contributed by Shashikant Banerjee 2018-03-06 12:23:03 -08:00
Inigo Goiri
7060725662 HDFS-13188. Disk Balancer: Support multiple block pools during block move. Contributed by Bharat Viswanadham. 2018-03-06 09:09:32 -08:00
Arpit Agarwal
a3c304c5dd HDFS-13178. Disk Balancer: Add a force option to DiskBalancer Execute command. Contributed by Bharat Viswanadham. 2018-03-03 15:42:16 -08:00
Tsz-Wo Nicholas Sze
ba82e5c488 HDFS-13173. Replace ArrayList with DirectoryDiffList(SnapshotSkipList) to store DirectoryDiffs. Contributed by Shashikant Banerjee 2018-03-02 17:47:48 -08:00
Xiao Chen
c75105f07b HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 2018-03-02 16:46:04 -08:00
Tsz-Wo Nicholas Sze
60080fbfcf HDFS-13171. Handle Deletion of nodes in SnasphotSkipList. Contributed by Shashikant Banerjee 2018-03-02 10:51:05 -08:00
Tsz-Wo Nicholas Sze
850b2f2567 HDFS-1686. Federation: Add more Balancer tests with federation setting. Contributed by Bharat Viswanadham 2018-03-02 10:42:39 -08:00
Yiqun Lin
55669515f6 HDFS-13210. Fix the typo in MiniDFSCluster class. Contributed by fang zhenyi. 2018-03-02 11:49:12 +08:00
Tsz-Wo Nicholas Sze
96e8f260ab HDFS-13211. Fix a bug in DirectoryDiffList.getMinListForRange. Contributed by Shashikant Banerjee 2018-03-01 14:12:15 -08:00
Tsz-Wo Nicholas Sze
81d9446a92 HDFS-13102. Implement SnapshotSkipList class to store Multi level DirectoryDiffs. Contributed by Shashikant Banerjee 2018-02-28 13:42:58 -08:00
Xiaoyu Yao
f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Yiqun Lin
a9c14b1119 HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang. 2018-02-28 10:17:43 +08:00
Tsz-Wo Nicholas Sze
55c77bf722 HDFS-13143. SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDiff calculation happens between a snapshot and the current tree. Contributed by Shashikant Banerjee 2018-02-27 15:28:41 -08:00
Arpit Agarwal
1cc9a58dda HDFS-13181. DiskBalancer: Add an configuration for valid plan hours . Contributed by Bharat Viswanadham. 2018-02-27 10:18:07 -08:00
Inigo Goiri
cd4a97bf80 HDFS-13192. Change the code order in getFileEncryptionInfo to avoid unnecessary call of assignment. Contributed by LiXin Ge. 2018-02-27 08:53:00 -08:00
Yiqun Lin
1e85a995d1 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 2018-02-27 10:48:52 +08:00
Chao Sun
ae290a4bb4 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 2018-02-26 16:15:00 -08:00
Kihwal Lee
451265a83d HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 2018-02-26 10:28:04 -06:00
Arpit Agarwal
2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Xiaoyu Yao
1e84e46f16 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 2018-02-23 19:35:12 -08:00
Xiao Chen
51088d3233 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 2018-02-23 13:49:09 -08:00
Xiaoyu Yao
84a1321f6a HDFS-13136. Avoid taking FSN lock while doing group member lookup for FSD permission check. Contributed by Xiaoyu Yao. 2018-02-22 11:32:32 -08:00
Arpit Agarwal
9028ccaf83 HDFS-13159. TestTruncateQuotaUpdate fails in trunk. Contributed by Nanda kumar. 2018-02-20 11:40:20 -08:00
Yiqun Lin
8896d20b91 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 2018-02-20 09:37:08 +08:00
Tsz-Wo Nicholas Sze
83e2bb98ee HDFS-12998. SnapshotDiff - Provide an iterator-based listing API for calculating snapshotDiff. Contributed by Shashikant Banerjee 2018-02-19 11:42:10 +08:00
Arpit Agarwal
4c2119f04e HDFS-13058. Fix dfs.namenode.shared.edits.dir in TestJournalNode. Contributed by Bharat Viswanadham. 2018-02-16 10:29:22 -08:00
Kihwal Lee
47473952e5 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 2018-02-15 15:32:42 -06:00
Kihwal Lee
da59acd8ca xattr api cleanup 2018-02-15 11:11:55 -06:00
Tsz-Wo Nicholas Sze
6ea7d78ccb HDFS-13142. Define and Implement a DiifList Interface to store and manage SnapshotDiffs. Contributed by Shashikant Banerjee 2018-02-15 19:33:44 +08:00
Steve Loughran
f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-14 16:20:14 +00:00
Yiqun Lin
543f3abbee HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 2018-02-09 13:57:42 +08:00
Xiaoyu Yao
8faf0b50d4 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 2018-02-08 08:59:48 -08:00
Jitendra Pandey
b061215ecf HDFS-11701. NPE from Unresolved Host causes permanent DFSInputStream failures. Contributed by Lokesh Jain. 2018-02-07 11:22:36 -08:00
Brahma Reddy Battula
01bd6ab18f HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-07 23:10:33 +05:30
Tsz-Wo Nicholas Sze
4fadcf1631 HDFS-13107. Add Mover Cli Unit Tests for Federated cluster. Contributed by Bharat Viswanadham 2018-02-06 16:59:26 -08:00
Sean Mackrory
b14db95c35 HDFS-13106. Need to exercise all HDFS APIs for EC. Contributed by Haibo Yan. 2018-02-06 16:39:00 -07:00
Tsz-Wo Nicholas Sze
4d1bf33d0f HDFS-12947. Limit the number of Snapshots allowed to be created for a Snapshottable Directory. Contributed by Shashikant Banerjee 2018-02-06 14:46:44 -08:00
Anu Engineer
4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Wei-Chiu Chuang
2021f4bdce HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Wei-Chiu Chuang. 2018-02-02 17:18:42 -08:00
Akira Ajisaka
4aef8bd2ef
HDFS-13048. LowRedundancyReplicatedBlocks metric can be negative 2018-02-02 14:34:07 +09:00
Yiqun Lin
b0627c891b HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 2018-02-02 11:25:41 +08:00
Anu Engineer
aa45faf0b2 HDFS-12942. Synchronization issue in FSDataSetImpl#moveBlock. Contributed by Ajay Kumar. 2018-02-01 18:03:01 -08:00
Hanisha Koneru
dd50f53997 HDFS-13062. Provide support for JN to use separate journal disk per namespace. Contributed by Bharat Viswanadham. 2018-02-01 12:28:17 -08:00
Xiaoyu Yao
b3ae11d597 HDFS-12997. Move logging to slf4j in BlockPoolSliceStorage and Storage. Contributed by Ajay Kumar. 2018-02-01 10:45:34 -08:00
Inigo Goiri
6ca7204ceb HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 2018-02-01 10:37:14 -08:00
Xiao Chen
ae2177d296 HDFS-12897. getErasureCodingPolicy should handle .snapshot dir better. Contributed by LiXin Ge. 2018-01-31 22:50:25 -08:00
Xiaoyu Yao
af015c0b23 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 2018-01-31 22:39:51 -08:00
Xiaoyu Yao
37b7536568 HDFS-13061. SaslDataTransferClient#checkTrustAndSend should not trust a partially trusted channel. Contributed by Ajay Kumar. 2018-01-31 10:47:02 -08:00
Weiwei Yang
2e7331ca26 HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 2018-01-31 07:25:03 +08:00
Yiqun Lin
dbb9dded33 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 2018-01-30 12:12:08 +08:00
Kihwal Lee
fde95d463c HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah 2018-01-29 17:23:29 -06:00
Xiao Chen
b63dcd583f HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 2018-01-28 22:19:49 -08:00
Xiao Chen
6bc2f7f4b4 HDFS-13065. TestErasureCodingMultipleRacks#testSkewedRack3 is failing. Contributed by Gabor Bota. 2018-01-28 22:12:05 -08:00
Arpit Agarwal
e990904dd5 HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 13:09:13 -08:00
Yiqun Lin
7721fff744 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 2018-01-25 15:51:26 +08:00
Inigo Goiri
95743c672e HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 2018-01-23 19:15:44 -08:00
Inigo Goiri
130f8bce00 HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 2018-01-19 09:47:40 -08:00
Lei Xu
3bd9ea63df HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei) 2018-01-16 15:15:11 -08:00
Brahma Reddy Battula
880b9d24ff HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 2018-01-16 16:21:02 +05:30
Yiqun Lin
9afb8025d6 HDFS-12972. RBF: Display mount table quota info in Web UI and admin command. Contributed by Yiqun Lin. 2018-01-13 13:16:43 +08:00
Inigo Goiri
d5d6a0353b HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-12 16:25:39 -08:00
Yiqun Lin
d98a2e6e23 HDFS-12934. RBF: Federation supports global quota. Contributed by Yiqun Lin. 2018-01-10 13:59:11 +08:00
Inigo Goiri
d9006d8a4e HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 2018-01-09 18:53:25 -08:00
Lei Xu
47563d86fe HDFS-12994. TestReconstructStripedFile.testNNSendsErasureCodingTasks fails due to socket timeout. (Contributed by Lei (Eddy) Xu) 2018-01-09 11:53:49 -08:00
Inigo Goiri
b3290c41a9 HDFS-12808. Add LOG.isDebugEnabled() guard for LOG.debug(...). Contributed by Bharat Viswanadham. 2018-01-08 17:51:09 -08:00
Manoj Govindassamy
73ff09b79a HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 2018-01-08 15:34:00 -08:00
Yiqun Lin
bf5c948995 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-06 14:31:08 +08:00
Inigo Goiri
83b513ac6d HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 2018-01-05 09:11:38 -08:00
Lei Xu
dc735b286b HDFS-12860. StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB. (Contributed by Lei (Eddy) Xu) 2018-01-04 10:54:56 -08:00
Sean Mackrory
45a47198b4 HDFS-12913. TestDNFencingWithReplication sometimes fails because of NameNode in standby state. Contributed by Zsolt Venczel. 2018-01-04 08:37:21 -07:00
Yiqun Lin
2a48b3594c HDFS-12948. DiskBalancer report command top option should only take positive numeric values. Contributed by Shashikant Banerjee. 2018-01-04 10:48:44 +08:00
Xiaoyu Yao
3ba985997d HDFS-12931. Handle InvalidEncryptionKeyException during DistributedFileSystem#getFileChecksum. Contributed by Mukul Kumar Singh. 2018-01-03 14:54:20 -08:00
Manoj Govindassamy
42a1c98597 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-02 14:59:36 -08:00
Chris Douglas
7fe6f83c8f HADOOP-15117. open(PathHandle) contract test should be exhaustive for default options 2017-12-30 17:58:35 -08:00
Manoj Govindassamy
76e664e931 HDFS-12959. Fix TestOpenFilesWithSnapshot redundant configurations. 2017-12-21 15:47:15 -08:00
Lei Xu
b318bed01a HDFS-12938. TestErasureCodigCLI testAll failing consistently. (Contributed by Ajay Kumar) 2017-12-21 10:28:24 -08:00
Yiqun Lin
e040c97b77 HDFS-12937. RBF: Add more unit tests for router admin commands. Contributed by Yiqun Lin. 2017-12-19 15:31:34 +08:00
Tsz-Wo Nicholas Sze
c7499f2d24 HDFS-12347. TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently. Contributed by Bharat Viswanadham 2017-12-19 10:02:30 +08:00
Erik Krogen
94576b17fb HDFS-12818. Support multiple storages in DataNodeCluster / SimulatedFSDataset. Contributed by Erik Krogen. 2017-12-18 11:36:22 -08:00
Virajith Jalaparti
8239e3afb3 HDFS-12712. [9806] Code style cleanup 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
80c3fec3a1 HDFS-12912. [READ] Fix configuration and implementation of LevelDB-based alias maps 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
4531588a94 HDFS-11640. [READ] Datanodes should use a unique identifier when reading from external stores 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
9c35be86e1 HDFS-12713. [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata. Contributed by Ewan Higgs 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
b634053c4d HDFS-12591. [READ] Implement LevelDBFileRegionFormat. Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
352f994b64 HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
cc933cba77 HDFS-12685. [READ] FsVolumeImpl exception when scanning Provided storage volume 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
3b1d30301b HDFS-12775. [READ] Fix reporting of Provided volumes 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
e1a28f95b8 HDFS-12777. [READ] Reduce memory and CPU footprint for PROVIDED volumes. 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
98f5ed5aa3 HDFS-11902. [READ] Merge BlockFormatProvider and FileRegionProvider. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
aca023b72c HDFS-12289. [READ] HDFS-12091 breaks the tests for provided block reads 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
2407c9b93a HDFS-12093. [READ] Share remoteFS between ProvidedReplica instances. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
663b3c08b1 HDFS-12091. [READ] Check that the replicas served from a ProvidedVolumeImpl belong to the correct external storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
546b95f484 HDFS-11673. [READ] Handle failures of Datanode with PROVIDED storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
55ade54b8e HDFS-11792. [READ] Test cases for ProvidedVolumeDF and ProviderBlockIteratorImpl 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
89b9faf529 HDFS-11703. [READ] Tests for ProvidedStorageMap 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
1108cb7691 HDFS-11653. [READ] ProvidedReplica should return an InputStream that is bounded by its length 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
d65df0f273 HDFS-11190. [READ] Namenode support for data stored in external stores. 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Chen Liang
aa503a29d0 HDFS-12917. Fix description errors in testErasureCodingConf.xml. Contributed by chencan. 2017-12-15 13:45:37 -08:00
Lei Xu
1c15b1751c HDFS-12819. Setting/Unsetting EC policy shows warning if the directory is not empty. (Contributed by Lei (Eddy) Xu) 2017-12-15 10:04:43 -08:00
Yiqun Lin
ee028bfdf1 HDFS-12895. RBF: Add ACL support for mount table. Contributed by Yiqun Lin. 2017-12-15 14:09:24 +08:00
Xiao Chen
e1cb278cd0 HDFS-12910. Secure Datanode Starter should log the port when it fails to bind. Contributed by Stephen O'Donnell and Nanda kumar. 2017-12-14 19:22:51 -08:00
Kihwal Lee
f5a72424c0 HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 2017-12-14 14:58:47 -06:00
Jason Lowe
80db744ee5 HDFS-12881. Output streams closed with IOUtils suppressing write errors. Contributed by Ajay Kumar 2017-12-14 09:45:44 -06:00
Chris Douglas
693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Inigo Goiri
5cd1056ad7 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 2017-12-11 15:14:57 -08:00
Manoj Govindassamy
ef7d334d36 HDFS-12825. Fsck report shows config key name for min replication issues (Contributed by Gabor Bota). 2017-12-08 11:46:30 -08:00
Xiao Chen
19e0894209 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 2017-12-07 21:05:55 -08:00
Lei Xu
67662e2ac9 HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 2017-12-07 11:15:40 -08:00
Xiao Chen
56b1ff80dd HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 2017-12-05 20:48:16 -08:00
Arpit Agarwal
e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00
Xiao Chen
404eab4dc0 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:11 -08:00
Chris Douglas
42307e3c3a HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 22:34:30 -08:00
Wei-Chiu Chuang
0faf506245 HDFS-12836. startTxId could be greater than endTxId when tailing in-progress edit log. Contributed by Chao Sun. 2017-12-01 12:01:21 -08:00
Chris Douglas
53bbef3802 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 5304698dc8.
 2017-12-01 11:19:38 -08:00
Chris Douglas
5304698dc8 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 10:29:30 -08:00
Konstantin V Shvachko
60fd0d7fd7 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 2017-11-30 18:18:28 -08:00
Tsz-Wo Nicholas Sze
b1c7654ee4 HDFS-12594. snapshotDiff fails if the report exceeds the RPC response limit. Contributed by Shashikant Banerjee 2017-11-30 12:18:29 -08:00
Chris Douglas
0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Kihwal Lee
53509f295b HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 2017-11-29 15:11:14 -06:00
Surendra Singh Lilhore
30941d99c9 HDFS-12857. StoragePolicyAdmin should support schema based path. Contributed by Surendra Singh Lilhore. 2017-11-28 23:57:03 +05:30
Kihwal Lee
738d1a206a HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 2017-11-22 13:01:55 -06:00
Lei Xu
785732c13e HDFS-12847. Regenerate editsStored and editsStored.xml in HDFS tests. Contributed by Lei (Eddy) Xu. 2017-11-22 10:22:32 -08:00
Chen Liang
60fc2a1388 HDFS-12804. Use slf4j instead of log4j in FSEditLog. Contributed by Mukul Kumar Singh. 2017-11-20 12:49:53 -08:00
Manoj Govindassamy
9fb4effd2c HDFS-12730. Verify open files captured in the snapshots across config disable and enable. 2017-11-20 10:28:12 -08:00
Chris Douglas
675e9a8f57 Revert "HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus." 
This reverts commit b85603e3f8.
 2017-11-15 19:20:37 -08:00
Arpit Agarwal
4908a8970e HDFS-12705. WebHdfsFileSystem exceptions should retain the caused by exception. Contributed by Hanisha Koneru. 2017-11-13 11:30:39 -08:00
Arpit Agarwal
6d201f77c7 HDFS-12498. Journal Syncer is not started in Federated + HA cluster. Contributed by Bharat Viswanadham. 2017-11-10 16:30:38 -08:00
Arpit Agarwal
1d6f8bebe9 HADOOP-14929. Cleanup usage of decodecomponent and use QueryStringDecoder from netty. Contributed by Bharat Viswanadham. 2017-11-10 16:28:12 -08:00
Chris Douglas
b85603e3f8 HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus. 2017-11-03 14:30:57 -07:00
Kihwal Lee
4d2dce40bb HDFS-12771. Add genstamp and block size to metasave Corrupt blocks list. Contributed by Kuhu Shukla. 2017-11-03 15:16:46 -05:00
Xiao Chen
299d38295d HDFS-11467. Support ErasureCoding section in OIV XML/ReverseXML. Contributed by Huafeng Wang. 2017-11-03 12:05:45 -07:00
Xiao Chen
b00f828d84 HDFS-12725. BlockPlacementPolicyRackFaultTolerant fails with very uneven racks. 2017-11-02 21:53:13 -07:00
Xiao Chen
e565b5277d HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 2017-11-02 21:27:35 -07:00
Lei Xu
9367c25dbd HDFS-12482. Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery. (lei) 2017-10-31 21:58:14 -07:00
Andrew Wang
5f681fa821 Revert "HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham." 
This reverts commit b922ba7393.
 2017-10-31 10:46:10 -07:00
Arpit Agarwal
b922ba7393 HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham. 2017-10-31 10:23:00 -07:00
Inigo Goiri
982bd2a5bf HDFS-12699. TestMountTable fails with Java 7. Contributed by Inigo Goiri. 2017-10-31 10:21:42 -07:00
Chris Douglas
d015e0bbd5 HDFS-7878. API - expose a unique file identifier. 2017-10-31 09:44:01 -07:00
Sean Mackrory
81225430d0 HDFS-206. Support for head in FSShell. Contributed by Gabor Bota. 2017-10-31 08:34:01 -06:00
Chris Douglas
d55a84951a HDFS-12582. Replace HdfsFileStatus constructor with a builder pattern. Contributed by Bharat Viswanadham 2017-10-27 15:36:03 -07:00
Manoj Govindassamy
075dd45a24 HDFS-12544. SnapshotDiff - support diff generation on any snapshot root descendant directory. 2017-10-25 10:54:40 -07:00
Zhe Zhang
17cd8d0c17 Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit 60bfee270e.
 2017-10-24 15:06:50 -07:00
Wei-Chiu Chuang
0f2a69127a HDFS-12249. dfsadmin -metaSave to output maintenance mode blocks. Contributed by Wellington Chevreuil. 2017-10-23 15:24:34 -07:00
Arpit Agarwal
96be795656 HDFS-12650. Use slf4j instead of log4j in LeaseManager. Contributed by Ajay Kumar. 2017-10-23 13:20:46 -07:00
Xiao Chen
248d9b6fff HDFS-12518. Re-encryption should handle task cancellation and progress better. 2017-10-20 16:25:04 -07:00
Andrew Wang
0477eff8be HDFS-12497. Re-enable TestDFSStripedOutputStreamWithFailure tests. Contributed by Huafeng Wang. 2017-10-20 13:27:21 -07:00
Kai Zheng
ce7cf66e5e HDFS-12448. Make sure user defined erasure coding policy ID will not overflow. Contributed by Huafeng Wang 2017-10-20 09:42:04 +08:00
Inigo Goiri
0f1c037618 HDFS-12620. Backporting HDFS-10467 to branch-2. Contributed by Inigo Goiri. 2017-10-19 18:08:45 -07:00
Zhe Zhang
60bfee270e HDFS-12502. nntop should support a category based on FilesInGetListingOps. 2017-10-18 23:51:24 -07:00
Lei Xu
f27a4ad032 HDFS-12612. DFSStripedOutputStream.close will throw if called a second time with a failed streamer. (Lei (Eddy) Xu) 2017-10-17 15:52:09 -07:00
Lei Xu
31ebccc962 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 2017-10-16 19:44:30 -07:00
Manoj Govindassamy
b406d8e375 HDFS-12614. FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured. 2017-10-16 17:42:41 -07:00
Wei-Chiu Chuang
8dbc8909c9 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 2017-10-16 12:57:48 -07:00
Xiao Chen
7bd700941d HDFS-12637. Extend TestDistributedFileSystemWithECFile with a random EC policy. Contributed by Takanobu Asanuma. 2017-10-16 09:55:22 -07:00
Xiao Chen
035c6ee587 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 2017-10-16 09:51:10 -07:00
Xiao Chen
20575ececb HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 2017-10-15 19:28:22 -07:00
Manoj Govindassamy
133d7ca76e HADOOP-13055. Implement linkMergeSlash and linkFallback for ViewFileSystem 2017-10-13 17:43:21 -07:00
Arpit Agarwal
8dd1eeb94f HDFS-12553. Add nameServiceId to QJournalProtocol. Contributed by Bharat Viswanadham 2017-10-13 14:22:21 -07:00
Arpit Agarwal
bb0a742aac HDFS-12627. Fix typo in DFSAdmin command output. Contributed by Ajay Kumar. 2017-10-11 12:26:14 -07:00