big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,506 Commits 2 Branches 2 Tags 264 MiB
b8a98e4f82
Commit Graph

8359 Commits

Author SHA1 Message Date
Konstantin V Shvachko
ebe5853a45 HDFS-12978. Fine-grained locking while consuming journal stream. Contributed by Konstantin Shvachko. 2018-05-31 14:56:32 -07:00
Inigo Goiri
1f324e9661 HDFS-13631. TestDFSAdmin#testCheckNumOfBlocksInReportCommand should use a separate MiniDFSCluster path. Contributed by Anbang Hu. 2018-05-31 10:12:38 -07:00
Sean Mackrory
1361030e59 HDFS-13582. Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside). Contributed by Zsolt Venczel. 2018-05-31 07:59:21 -06:00
Inigo Goiri
47c31ff16b HDFS-13629. Some tests in TestDiskBalancerCommand fail on Windows due to MiniDFSCluster path conflict and improper path usage. Contributed by Anbang Hu. 2018-05-30 10:22:04 -07:00
Inigo Goiri
8197b9b560 HDFS-13632. Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA. Contributed by Anbang Hu. 2018-05-30 10:13:52 -07:00
Yiqun Lin
b24098bc8f HDFS-13626. Fix incorrect username when deny the setOwner operation. Contributed by Zsolt Venczel. 2018-05-30 16:52:21 +08:00
Inigo Goiri
9dbf4f0166 HDFS-13591. TestDFSShell#testSetrepLow fails on Windows. Contributed by Anbang Hu. 2018-05-28 16:34:02 -07:00
Inigo Goiri
91d7c74e6a HDFS-13627. TestErasureCodingExerciseAPIs fails on Windows. Contributed by Anbang Hu. 2018-05-28 10:26:47 -07:00
Takanobu Asanuma
04757e5864
HDFS-13628. Update Archival Storage doc for Provided Storage 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-28 19:06:34 +09:00
Inigo Goiri
8605a38514 HDFS-13620. Randomize the test directory path for TestHDFSFileSystemContract. Contributed by Anbang Hu. 2018-05-25 19:43:33 -07:00
Inigo Goiri
13d2528907 HDFS-13619. TestAuditLoggerWithCommands fails on Windows. Contributed by Anbang Hu. 2018-05-25 13:32:34 -07:00
Inigo Goiri
1e0d4b1c28 HDFS-13618. Fix TestDataNodeFaultInjector test failures on Windows. Contributed by Xiao Liang. 2018-05-25 09:14:28 -07:00
Yiqun Lin
7a87add4ea HDFS-13598. Reduce unnecessary byte-to-string transform operation in INodesInPath#toString. Contributed by Gabor Bota. 2018-05-24 10:57:35 +08:00
Inigo Goiri
cddbbe5f69 HDFS-13493. Reduce the HttpServer2 thread count on DataNodes. Contributed by Erik Krogen. 2018-05-23 12:12:08 -07:00
Inigo Goiri
c13dea87d9 HDFS-13587. TestQuorumJournalManager fails on Windows. Contributed by Anbang Hu. 2018-05-23 11:36:03 -07:00
Bharat Viswanadham
bc6d9d4c79 HDFS-13589: Add dfsAdmin command to query if upgrade is finalized. Contributed by Hanisha Koneru 2018-05-23 10:15:40 -07:00
Inigo Goiri
c0c9b7a8ef HDFS-13588. Fix TestFsDatasetImpl test failures on Windows. Contributed by Xiao Liang. 2018-05-23 09:46:35 -07:00
Sammi Chen
34e8b9f9a8 HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen. 2018-05-23 19:10:09 +08:00
Andrew Wang
1d2640b613 HDFS-13601. Optimize ByteString conversions in PBHelper. 2018-05-22 23:55:20 -07:00
Inigo Goiri
9775ecb235 HDFS-13593. TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows. Contributed by Anbang Hu. 2018-05-18 09:46:02 -07:00
Inigo Goiri
57b893de3d HDFS-13592. TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly. Contributed by Anbang Hu. 2018-05-18 09:36:58 -07:00
Yiqun Lin
f749517cc7 HDFS-13573. Javadoc for BlockPlacementPolicyDefault is inaccurate. Contributed by Zsolt Venczel. 2018-05-18 10:43:53 +08:00
Inigo Goiri
53b807a6a8 HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 2018-05-17 17:03:23 -07:00
Inigo Goiri
a97a2042f2 HDFS-13556. TestNestedEncryptionZones does not shut down cluster. Contributed by Anbang Hu. 2018-05-17 16:53:23 -07:00
Inigo Goiri
65476458fa HDFS-13554. TestDatanodeRegistration#testForcedRegistration does not shut down cluster. Contributed by Anbang Hu. 2018-05-17 14:48:04 -07:00
Inigo Goiri
26f1e22fc9 HDFS-13558. TestDatanodeHttpXFrame does not shut down cluster. Contributed by Anbang Hu. 2018-05-17 13:35:09 -07:00
Inigo Goiri
c0ec061e28 HDFS-13570. TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows. Contributed by Anbang Hu. 2018-05-17 09:21:57 -07:00
Kihwal Lee
0ce6290de6 HDFS-13581. DN UI logs link is broken when https is enabled. Contributed by Shashikant Banerjee. 2018-05-17 09:23:24 -05:00
Inigo Goiri
454de3b543 HDFS-13559. TestBlockScanner does not close TestContext properly. Contributed by Anbang Hu. 2018-05-16 19:23:02 -07:00
Arpit Agarwal
0fc988e6a3 HDFS-13512. WebHdfs getFileStatus doesn't return ecPolicy. Contributed by Ajay Kumar. 2018-05-16 11:28:39 -07:00
Inigo Goiri
55d5549087 HDFS-13550.TestDebugAdmin#testComputeMetaCommand fails on Windows. Contributed by Anbang Hu. 2018-05-16 10:08:49 -07:00
Inigo Goiri
6b67161ca3 HDFS-13557. TestDFSAdmin#testListOpenFiles fails on Windows. Contributed by Anbang Hu. 2018-05-16 09:31:46 -07:00
Inigo Goiri
63480976a0 HDFS-13567. TestNameNodeMetrics#testGenerateEDEKTime,TestNameNodeMetrics#testResourceCheck should use a different cluster basedir. Contributed by Anbang Hu. 2018-05-15 15:34:54 -07:00
Inigo Goiri
2d6195c0af HDFS-13548. TestResolveHdfsSymlink#testFcResolveAfs fails on Windows. Contributed by Anbang Hu. 2018-05-15 11:20:32 -07:00
Inigo Goiri
eaca7980ad HDFS-11700. TestHDFSServerPorts#testBackupNodePorts doesn't pass on Windows. Contributed by Anbang Hu. 2018-05-15 10:27:36 -07:00
Inigo Goiri
92ebd466c7 HDFS-13551. TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster. Contributed by Anbang Hu. 2018-05-15 10:21:42 -07:00
Hanisha Koneru
6beb25ab7e HDFS-13544. Improve logging for JournalNode in federated cluster. 2018-05-14 10:12:08 -07:00
Xiao Chen
960940e0e0 HDFS-13539. DFSStripedInputStream NPE when reportCheckSumFailure. 2018-05-14 09:28:39 -07:00
Sammi Chen
f3f544b004 Add 2.9.1 release notes and changes documents 2018-05-14 15:14:02 +08:00
sammichen
66c9905b57 Revert "Add 2.9.1 release notes and changes documents" 
This reverts commit e4dc346d65.
 2018-05-14 14:38:40 +08:00
littlezhou
e4dc346d65 Add 2.9.1 release notes and changes documents 2018-05-14 14:24:01 +08:00
Anu Engineer
4b4f24ad5f HDDS-40. Separating packaging of Ozone/HDDS from the main Hadoop. 
Contributed by Elek, Marton.
 2018-05-11 14:24:36 -07:00
Inigo Goiri
d50c4d71dc HDFS-13542. TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows. Contributed by Anbang Hu. 2018-05-11 09:47:57 -07:00
Xiao Chen
696a4be0da HDFS-12981. renameSnapshot a Non-Existent snapshot to itself should throw error. Contributed by Kitti Nanasi. 2018-05-07 15:36:29 -07:00
Xiao Chen
118bd75805 Revert "HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445." 
This reverts commit 6503593711.
 2018-05-07 13:32:27 -07:00
Aaron Fabbri
a732acd873
HDFS-5926 Documentation should clarify dfs.datanode.du.reserved impact from reserved disk capacity (Gabor Bota) 2018-05-04 16:38:30 -07:00
Inigo Goiri
9e2cfb2d3f HDFS-13503. Fix TestFsck test failures on Windows. Contributed by Xiao Liang. 2018-05-01 08:12:46 -07:00
Inigo Goiri
fc074a359c HDFS-13283. Percentage based Reserved Space Calculation for DataNode. Contributed by Lukas Majercak. 2018-04-30 13:28:33 -07:00
Inigo Goiri
eb7fe1d588 HDFS-13509. Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows. Contributed by Xiao Liang. 2018-04-28 09:05:30 -07:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Inigo Goiri
09c9cf1d43 HDFS-13326. RBF: Improve the interfaces to modify and view mount tables. Contributed by Gang Li. 2018-04-26 13:54:40 -07:00
Lei Xu
a8e428b2dc HDFS-13468. Add erasure coding metrics into ReadStatistics. (Contributed by Lei (Eddy) Xu) 2018-04-26 13:54:40 -07:00
Yiqun Lin
48269c370c HDFS-13484. RBF: Disable Nameservices from the federation. Contributed by Inigo Goiri. 2018-04-26 13:54:40 -07:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Anu Engineer
3d18ca4926 HDFS-13459. Ozone: Clean-up of ozone related change from MiniDFSCluste. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
979bbb4019 HDFS-13423. Ozone: Clean-up of ozone related change from hadoop-hdfs-project. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
25f2398bbd HDFS-13395. Ozone: Plugins support in HDSL Datanode Service. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Anu Engineer
5e7164c614 HDFS-13137. Ozone: Ozonefs read fails because ChunkGroupInputStream#read does not iterate through all the blocks in the key. 
Contributed by Mukul Kumar Singh.
 2018-04-26 05:36:04 -07:00
Anu Engineer
61651dcf5c HDFS-13108. Ozone: OzoneFileSystem: Simplified url schema for Ozone File System. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
9a914126a7 HDFS-13008. Ozone: Add DN container open/close state to container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Nanda kumar
f4466402ae HDFS-12965. Ozone: Documentation : Add ksm -createObjectStore command documentation. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Nanda kumar
1094af072c HDFS-11699. Ozone:SCM: Add support for close containers in SCM. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
b4a3cf1476 HDFS-13070. Ozone: SCM: Support for container replica reconciliation - 1. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
4c10a849e8 HDFS-13078. Ozone: Update Ratis on Ozone to 0.1.1-alpha-8fd74ed-SNAPSHOT. 
To fix large chunk reads (>4M) from Datanodes. Contributed by Mukul Kumar Singh.
 2018-04-26 05:36:04 -07:00
Anu Engineer
136b419594 HDFS-13149. Ozone: Rename Corona to Freon. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
9272e1021d HDFS-12636. Ozone: OzoneFileSystem: Implement seek functionality for rpc client. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
377b31ffa1 HDFS-13022. Block Storage: Kubernetes dynamic persistent volume provisioner. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
ee5495456e HDFS-13116. Ozone: Refactor Pipeline to have transport and container specific information. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
3cf07b43bc HDFS-13127. Fix TestContainerStateManager and TestOzoneConfigurationFields. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
443425a5d9 HDFS-12522. Ozone: Remove the Priority Queues used in the Container State Manager. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
32245c78e2 HDFS-13080. Ozone: Make finalhash in ContainerInfo of StorageContainerDatanodeProtocol.proto optional. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
150c4e1d0b HDFS-12879. Ozone : add scm init command to document. Contributed by Rahul Pathak. 2018-04-26 05:36:04 -07:00
Nanda kumar
7b3179f551 HDFS-13072. Ozone: DatanodeStateMachine: Handling Uncaught Exception in command handler thread. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
6b03fca37d HDFS-12868. Ozone: Service Discovery API. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
fb09d75379 HDFS-13018. Block Storage: make the iscsi target addres configurable for discovery. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
4bb9ad8e8f HDFS-13017. Block Storage: implement simple iscsi discovery in jscsi server. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
ba4d5a52a8 HDFS-12986. Ozone: Update ozone to latest ratis snapshot build (0.1.1-alpha-0f7169d-SNAPSHOT). Contributed by Lokesh Jain 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
94c0346f35 HDFS-13024. Ozone: ContainerStateMachine should synchronize operations between createContainer and writeChunk. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
4a051ba494 HDFS-12940. Ozone: KSM: TestKeySpaceManager#testExpiredOpenKey fails occasionally. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
1fb1ce107f HDFS-13012. TestOzoneConfigurationFields fails due to missing configs in ozone-default.xml. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
916f31f7d1 HDFS-13026. Ozone: TestContainerPersistence is failing becaue of container data mismatch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
28b87af51b HDFS-13013. Fix closeContainer API with the right container state change. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
c4b88454a7 HDFS-12843. Ozone: Client: TestOzoneRpcClient#testPutKeyRatisThreeNodes is failing. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
98d62e55c4 Revert "HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee." 
This reverts commit 6ce5ec676164b84a9e2f8dc65b5f2199a141506d.
 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
98eeabbdb3 HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
0c48cb2cf2 HFDS-13007. Ozone: Fix TestKeySpaceManager#testKSMInitializationFailure. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
98973cd0c4 HDFS-12966. Ozone: owner name should be set properly when the container allocation happens. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5e31b920f0 HDFS-12871. Ozone: Service Discovery: Adding REST server details in ServiceList. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
3965f1ec99 HDFS-12853. Ozone: Optimize chunk writes for Ratis by avoiding double writes. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5cc5149bd8 HDFS-12961. Ozone: Use relative resource urls in KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
5766792cad HDFS-12870. Ozone: Service Discovery: REST endpoint in KSM for getServiceList. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
5f10c2d8ce HDFS-12980. Ozone: SCM: Restructuring container state transition and event. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
87b7774488 HDFS-12958. Ozone: remove setAllocatedBytes method in ContainerInfo. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Nanda kumar
ce19f09bd0 HDFS-12908. Ozone: write chunk call fails because of Metrics registry exception. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
4b9f66a19e HDFS-12869. Ozone: Service Discovery: RPC endpoint in KSM for getServiceList. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
79432ee6cf HDFS-12863. Ozone: Avoid configuration.get() call for each container report. Contributed by Akira Ajisaka. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
0380dbb928 HDFS-12941. Ozone: ConfServlet does not trim values during the description parsing. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
4dae68eebf HDFS-12741. Ozone: KSM: ADD support for KSM --createObjectStore command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Chen Liang
caeeb78ca4 HDFS-12799. Ozone: SCM: Close containers: extend SCMCommandResponseProto with SCMCloseContainerCmdResponseProto. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
966853894f HDFS-12795. Ozone: SCM: Support for Container LifeCycleState PENDING_CLOSE and LifeCycleEvent FULL_CONTAINER. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
26e270b908 HDFS-12751. Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fd09c2ce5b HDFS-12890 . Ozone: XceiverClient should have upper bound on async requests. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
2f3068bc72 HDFS-12698. Ozone: Use time units in the Ozone configuration values. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
8ff98e2af3 HDFS-12000. Ozone: Container : Add key versioning support-1. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c0c87dea9b HDFS-12626. Ozone : delete open key entries that will no longer be closed. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6c630d0b09 HDFS-12899. Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
74362cb503 HDFS-12901. Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
11473b8204 HDFS-12898. Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
207523915d HDFS-12588. Use GenericOptionsParser for scm and ksm daemon. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
7bba3943f0 HDFS-12876. Ozone: moving NodeType from OzoneConsts to Ozone.proto. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
85ec48a646 HDFS-12867. Ozone: TestOzoneConfigurationFields fails consistently. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
538e17824d HDFS-12850. Ozone: TestContainerPersistence#testListContainer fails frequently due to timed out. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
e32e187448 HDFS-12864. Ozone: Turn off full synced write in RocksDB MetadataStore. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
8023ba50b5 HDFS-12807. Ozone: Expose RockDB stats via JMX for Ozone metadata stores. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
86802a461c HDFS-12851. Ozone: Upgrade to latest ratis build. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nanda kumar
ea86945754 HDFS-12842. Ozone: SCM: During startup SCM should check if the cluster is INITIALIZED. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
73ca8608da HDFS-12786. Ozone: add port/service names to the ksm/scm web ui. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nanda kumar
f9c11d952c HDFS-12740. SCM should support a RPC to share the cluster Id with KSM and DataNodes. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
c8d8270f72 HDFS-12787. Ozone: SCM: Aggregate the metrics from all the container reports. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
cbe4f314b3 HDFS-12731. Ozone: Fix inconsistent default value in ozone settings. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Anu Engineer
4deb845d7c HDFS-12759. Ozone: web: integrate configuration reader page to the SCM/KSM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
8e8ff85b16 HDFS-12830. Ozone: TestOzoneRpcClient#testPutKeyRatisThreeNodes fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
214b52dfce HDFS-12500. Ozone: add logger for oz shell commands and move error stack traces to DEBUG level. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
419e86d580 HDFS-12805. Ozone: Redundant characters printed in exception log. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
93f1b04949 HDFS-12798. Ozone: scm web: fix the node status table. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
4460ac9c57 HDFS-12756. Ozone: Add datanodeID to heartbeat responses and container protocol. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Nanda kumar
446e84357d HDFS-12796. SCM should not start if Cluster Version file does not exist. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
740a06cdd7 HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Yiqun Lin
761926db2c HDFS-12793. Ozone : TestSCMCli is failing consistently. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Chen Liang
e395244e95 HDFS-12758. Ozone: Correcting assertEquals argument order in test cases. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
cec96b296f HDFS-12549. Ozone: OzoneClient: Support for REST protocol. Contributed by Nanda Kumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
9734f505ea HDFS-12739. Add Support for SCM --init command. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
709d56fdc9 HDFS-12474. Ozone: SCM: Handling container report with key count and container usage. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2ec24a7542 HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar. 2018-04-26 05:36:04 -07:00
Yiqun Lin
132f30c8ea HDFS-12443. Ozone: Improve SCM block deletion throttling algorithm. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
7ebe79e879 HDFS-12720. Ozone: Ratis options are not passed from KSM Client protobuf helper correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0559265822 HDFS-12750. Ozone: Fix TestStorageContainerManager#testBlockDeletionTransactions. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
0760418bec HDFS-12697. Ozone services must stay disabled in secure setup for alpha. Contributed by Bharat Viswanadham. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
b687902590 HDFS-12718. Ozone: fix thread number calculation in CBlockManager. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
76d34bca62 HDFS-12521. Ozone: SCM should read all Container info into memory when booting up. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
6291ca1e32 HDFS-12663. Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nanda kumar
80357c0eee HDFS-12678. Ozone: Corona: Add statistical information to json output. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b56a73a4f HDFS-12680. Ozone: SCM: Lease support for container creation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
338c909ac6 HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
333dae888c HDFS-12690. Ozone: generate swagger descriptor for the Ozone REST Api. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
0582d8b1b1 Revert "HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton." 
This reverts commit 18c0cdcec9ff5676b10ba711039143d07dd4a510.
 2018-04-26 05:36:04 -07:00
Anu Engineer
402a1c1ba3 HDFS-12558. Ozone: Clarify the meaning of rpc.metrics.percentiles.intervals on KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
731a69faa1 HDFS-12693. Ozone: Enable XFrame options for KSM/SCM web ui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6783dad766 HDFS-11468. Ozone: SCM: Add Node Metrics for SCM. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
393a02d8e1 HDFS-12692. Ozone: fix javadoc/unit test for calling MetadataStore.getRangeKVs with non existing key. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang
6d9f914b55 HDFS-12684. Ozone: SCMMXBean NodeCount is overlapping with NodeManagerMXBean. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Nanda kumar
cc86719451 HDFS-12675. Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0f0417a12c HDFS-12673. Ozone: Log elapsed time for block deletion tasks. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
06c6582694 HDFS-12585. Ozone: KSM UI: Add description for configs in UI. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
763484e596 HDFS-12661. Ozone: Support optional documentation link in KSM/SCM webui. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
a467017068 HDFS-12411. Ozone: Add container usage information to DN container report. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
fb545e4291 HDFS-12415. Ozone: TestXceiverClientManager and TestAllocateContainer occasionally fails. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
895a4cc60a HDFS-12632. Ozone: OzoneFileSystem: Add contract tests to OzoneFileSystem. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nanda kumar
9bb516118d HDFS-12490. Ozone: OzoneClient: Add creation/modification time information in OzoneVolume/OzoneBucket/OzoneKey. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Nandakumar
8881309002 HDFS-12519. Ozone: Lease Manager framework. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
ebc4d4ebc3 HDFS-12475. Ozone: add documentation about Datanode http address. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
c85d3b2d25 HDFS-12593. Ozone: update Ratis to the latest snapshot. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Nandakumar
5ed663b32f HDFS-12631. Ozone: ContainerStorageLocation#scmUsage should count only SCM usage. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Yiqun Lin
d19b4c8763 HDFS-12583. Ozone: Fix swallow exceptions which makes hard to debug failures. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
9445a9267f HDFS-12572. Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
ae2b77a0e5 HDFS-12546. Ozone: DB listing operation performance improvement. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang
b05ad0b315 HDFS-12611. Ozone: SCM Cli: Use the hostname of SCM server instead of the bind address. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
yuanbo
e76e10fd7a HDFS-12609. Ozone: SCM: Refactoring of chill mode logic in NodeManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
e3b51d9074 HDFS-12598. Ozone: Fix 3 node ratis replication in Ozone. Contributed by Mukul Kumar Singh 2018-04-26 05:36:04 -07:00
Nandakumar
3504af9411 HDFS-12537. Ozone: Reduce key creation overhead in Corona. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Nandakumar
9630621be7 HDFS-12610. Ozone: OzoneClient: RpcClient list calls throw NPE when iterating over empty list. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Nandakumar
c3ef381011 HDFS-12543. Ozone : allow create key without specifying size. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c97ce55d06 HDFS-12602. Add Ozone config UI to KSM and SCM index page. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
303ecae5d5 HDFS-12557. Ozone: Improve the formatting of the RPC stats on web UI. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
c17521b1bd HDFS-12387. Ozone: Support Ratis as a first class replication mechanism. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer
0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
c52019c291 HDFS-12539. Ozone: refactor some functions in KSMMetadataManagerImpl to be more readable and reusable. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
ceec14bff5 HDFS-12467. Ozone: SCM: NodeManager should log when it comes out of chill mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
d7a94a21ae HDFS-12471. Ozone: Reduce some KSM/SCM deletion log messages from INFO to DEBUG. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6a754caca0 HDFS-12568. Ozone: Cleanup the ozone-default.xml. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
05246e2b32 HDFS-12060. Ozone: OzoneClient: Add list calls in RpcClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
7846636a5d HDFS-12551. Ozone: Documentation: Add Ozone overview documentation. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Yiqun Lin
cb1fb3436e HDFS-12563. Ozone: Creation time wrongly showed in OzoneRest.md. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
aaec2689f4 HDFS-12511. Ozone: Add tags to config. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7363c8fc09 HDFS-12540. Ozone: node status text reported by SCM is a bit confusing. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
13fdb58490 HDFS-12554. Ozone: Fix TestDatanodeStateMachine#testDatanodeStateMachineWithInvalidConfiguration. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Chen Liang
9530153f33 HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
94090acf3a HDFS-12469. Ozone: Create docker-compose definition to easily test real clusters. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
a44f3a777d HDFS-12501. Ozone: Cleanup javac issues. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
6b10723ec3 HDFS-12525. Ozone: OzoneClient: Verify bucket/volume name in create calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
5e4a6b686c HDFS-12425. Ozone: OzoneFileSystem: read/write/create/open/getFileInfo APIs. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
a1a3ba6529 HDFS-12454. Ozone : the sample ozone-site.xml in OzoneGettingStarted does not work. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2804435386 HDFS-12017. Ozone: Container: Move IPC port to 98xx range. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
b2447f8dd9 HDFS-11614. Ozone: Cleanup javadoc issues. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
9ff136bb02 HDFS-12210. Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
dddded0016 Revert "Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh." to fix commit message. 
This reverts commit 087c69ba2434813044089f733581607d172efbd1.
 2018-04-26 05:36:04 -07:00
Weiwei Yang
d303b7f40f HDFS-12524. Ozone: Record number of keys scanned and hinted for getRangeKVs call. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Yiqun Lin
712bd70ac7 HDFS-12205. Ozone: List Key on an empty ozone bucket fails with command failed error. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
8052374e79 Block Storage: volume creation times out while creating 3TB volume because of too many containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
fd1564b87e HDFS-12506. Ozone: ListBucket is too slow. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
46b65ec4fb HDFS-12503. Ozone: some UX improvements to oz_debug. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1c354e51ac HDFS-12477. Ozone: Some minor text improvement in SCM web UI. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Nandakumar
0313bdf886 HDFS-11897. Ozone: KSM: Changing log level for client calls in KSM. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Weiwei Yang
3b290fe3bf HDFS-12466. Ozone: KSM: Make ozone.ksm.address as mandatory property for client. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
8ddf75da17 HDFS-12489. Ozone: OzoneRestClientException swallows exceptions which makes client hard to debug failures. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c329d3b4b4 HDFS-12329. Ozone: Ratis: Readonly calls in XceiverClientRatis should use sendReadOnly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
607d1fcd31 HDFS-12256. Ozone : handle inactive containers on DataNode. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
77d4b18700 HDFS-12385. Ozone: OzoneClient: Refactoring OzoneClient API. Contributed by Nadakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
b984e903f3 HDFS-12481. Ozone: Corona: Support for variable key length in offline mode. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
d9c3d9a5ef HDFS-11873. Ozone: Object store handler supports reusing http client for multiple requests. Contributed by Xiaoyu Yao and Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fe4139bdc3 HDFS-11612. Ozone: Cleanup Checkstyle issues. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Chen Liang
b127ecd1ba Revert "HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin." as it uses class not part of JRE. 
This reverts commit c29aff44cfc78a2839f15e8cde3b80f0a31c80dd.
 2018-04-26 05:36:04 -07:00
Weiwei Yang
eaf3732217 HDFS-12465. Ozone: fix a KeySpaceManager startup message typo. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c551025ed6 HDFS-12468. Ozone: fix hard coded version in the Ozone GettingStarted guide. Contributed by Elek Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
cf841286b8 HDFS-12426. Ozone: BlockManager MBean unregister failure upon shutdown. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
f66a522695 HDFS-12268. Ozone: Add metrics for pending storage container requests. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
efa25373d7 HDFS-12019. Ozone: Misc : Make sure that ozone-site.xml is in etc/hadoop in tarball created from mvn package. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
80c50a1eac HDFS-12423. Ozone: TopN container choosing policy should ignore containers that has no pending deletion blocks. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1087ce4cd2 HDFS-11676. Ozone: SCM CLI: Implement close container command. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0c17776646 HDFS-12389. Ozone: oz commandline list calls should return valid JSON format output. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
fec04f1158 HDFS-12370. Ozone: Implement TopN container choosing policy for BlockDeletionService. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
743be0d7c0 HDFS-12235. Ozone: DeleteKey-3: KSM SCM block deletion message and ACK interactions. Contributed by Weiwei Yang and Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Weiwei Yang
3aa846412d HDFS-12397. Ozone: KSM: multiple delete methods in KSMMetadataManager. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang
e06a9a5bb1 HDFS-12391. Ozone: TestKSMSQLCli is not working as expected. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
1a21136322 HDFS-12382. Ozone: SCM: BlockManager creates a new container for each allocateBlock call. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
2e2e30373c HDFS-12333. Ozone: Extend Datanode web interface with SCM information. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
ca72c2342c HDFS-12366. Ozone: Refactor KSM metadata class names to avoid confusion. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7b3e7eb565 HDFS-12354. Ozone: Shuffle container list for datanode BlockDeletingService. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
eef437d5e2 HDFS-12282. Ozone: DeleteKey-4: Block delete via HB between SCM and DN. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
2f4dfbc8fb HDFS-12275. Ozone: Corona: Support for random validation of writes. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
2dfd0a2f4f HDFS-12341. Ozone: maven dist compilation fails with "Duplicate classes found" error. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
532ced18fc HDFS-12280. Ozone: TestOzoneContainer#testCreateOzoneContainer fails. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Weiwei Yang
4d3b8d332d HDFS-12283. Ozone: DeleteKey-5: Implement SCM DeletedBlockLog. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
f51189e2bb HDFS-12324. Ozone: TestQueryNode#testStaleNodesCount is failing. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
02f8c2224f HDFS-11888. Ozone: SCM: use state machine for open containers allocated for key/blocks. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
f352b104f7 HDFS-12337. Ozone: Concurrent RocksDB open calls fail because of "No locks available". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
9e733215a8 HDFS-12327. Ozone: support setting timeout in background service. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
38bc1d1097 HDFS-12216. Ozone: TestKeys is failing consistently. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
bd33038bf4 HDFS-12159. Ozone: SCM: Add create replication pipeline RPC. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Anu Engineer
3a661b7f82 HDFS-12238. Ozone: Add valid trace ID check in sendCommandAsync. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Chen Liang
f9bce29dcc HDFS-12290. Block Storage: Change dfs.cblock.jscsi.server.address default bind address to 0.0.0.0. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
6a16d7c7ab HDFS-12255. Block Storage: Cblock should generated unique trace ID for the ops. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
e61530f5d9 HDFS-12298. Ozone: Block deletion service floods the log when deleting large number of block files. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
25586607a1 HDFS-12230. Ozone: KSM: Add creation time field in bucket info. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
dcc21a4f93 HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
c9ea431267 HDFS-12281. Ozone: Ozone-default.xml has 3 properties that do not match the default Config value. Contributed by Ajay Yadav. 2018-04-26 05:36:04 -07:00
Weiwei Yang
639b4fb8a9 HDFS-12196. Ozone: DeleteKey-2: Implement block deleting service to delete stale blocks at background. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
285d61cd8d HDFS-12274. Ozone: Corona: move corona from test to tools package. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
bb9a4793d3 HDFS-12259. Ozone: OzoneClient: Refactor move ozone client from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
b4aecc1009 HDFS-12262. Ozone: KSM: Reduce default handler thread count from 200. Contributed by Ajay Yadav. 2018-04-26 05:36:04 -07:00
Anu Engineer
bb5f1c4228 HDFS-12231. Ozone: KSM: Add creation time field in volume info. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
6f83e9c51e HDFS-12163. Ozone: Reduce MiniOzoneCluster handler thread count. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
89d8d20c73 HDFS-12246. Ozone: potential thread leaks. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
23dee0f123 HDFS-11935. Ozone: TestStorageContainerManager#testRpcPermission fails when ipv6 address used. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
ccc1607424 HDFS-12178. Ozone: OzoneClient: Handling SCM container creationFlag at client side. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
37a0ab5ca3 HDFS-12247. Ozone: KeySpaceManager should unregister KSMMetrics upon stop. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
9cf40547ce HDFS-11580. Ozone: Support asynchronus client API for SCM and containers. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Weiwei Yang
1ad95cf2a9 HDFS-11984. Ozone: Ensures listKey lists all required key fields. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
acfae2716a HDFS-12034. Ozone: Web interface for KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Chen Liang
070f7e2e59 HDFS-11920. Ozone : add key partition. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
bc413a69cc HDFS-12195. Ozone: DeleteKey-1: KSM replies delete key request asynchronously. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
86b42907cf HDFS-12183. Ozone: KSM : Use proper defaults for block client address. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Chen Liang
7907fe1c80 HDFS-12059. Ozone: OzoneClient: OzoneClientImpl Add setBucketProperty and delete calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
fcd4537e63 HDFS-12180. Ozone: Corona: Add stats and progress bar to corona. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
a4b3160efb HDFS-12145. Ozone: OzoneFileSystem: Ozone & KSM should support / delimited key names. Contributed by Mukul Kumar Singh. 
(cherry picked from commit fe0a957e92d506330ba005abba18f665eb5ac1cc)
 2018-04-26 05:36:04 -07:00
Yiqun Lin
8cf0c864c4 HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
349a87ac9c HDFS-12192. Ozone: Fix the remaining failure tests for Windows caused by incorrect path generated. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d50a743015 HDFS-12155. Ozone : add RocksDB support to DEBUG CLI. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
0cc166c053 HDFS-12115. Ozone: SCM: Add queryNode RPC Call. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Weiwei Yang
539842ed8b HDFS-12187. Ozone : add support to DEBUG CLI for ksm.db. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
b00830142b HDFS-12167. Ozone: Intermittent failure TestContainerPersistence#testListKey. Contributed by Surendra Singh Lilhore. 2018-04-26 05:36:04 -07:00
Anu Engineer
487aa2f7bf HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
d02a4e9dd1 HDFS-12175. Ozone: Fix Leaking in TestStorageContainerManager#testRpcPermission. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Chen Liang
d05b44f417 Revert "HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao" 
This reverts commit f801454ad320e0d9c544c4249456bcaf50b4f30b.
 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
eda559ffd2 HDFS-12181. Ozone: Fix TestContainerReplicationManager by setting proper log level for LogCapturer. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Chen Liang
ca096501bf HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao 2018-04-26 05:36:04 -07:00
Weiwei Yang
f1e2592bf8 HDFS-12127. Ozone: Ozone shell: Add more testing for key shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
a652a72286 HDFS-11936. Ozone: TestNodeManager times out before it is able to find all nodes. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
bbbfc885e7 HDFS-12149. Ozone: RocksDB implementation of ozone metadata store. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
6225622dee HDFS-12071. Ozone: Corona: Implementation of Corona. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d2ef3e40bc HDFS-12126. Ozone: Ozone shell: Add more testing for bucket shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
5604c3788f HDFS-11996. Ozone : add an UT to test partial read of chunks. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4d333f8db1 HDFS-12058. Ozone: OzoneClient: OzoneClientImpl add getDetails calls. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Chen Liang
27262d8889 HDFS-12118. Ozone: Ozone shell: Add more testing for volume shell commands. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
db859e8172 HDFS-12148. Ozone: TestOzoneConfigurationFields is failing because ozone-default.xml has some missing properties. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
9522356b46 HDFS-12069. Ozone: Create a general abstraction for metadata store. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
b94b5021b8 HDFS-11989. Ozone: add TestKeysRatis, TestBucketsRatis and TestVolumeRatis. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
ab3d510c1d HDFS-12123. Ozone: OzoneClient: Abstraction of OzoneClient and default implementation. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
3d430a6076 HDFS-11493. Ozone: SCM: Add the ability to handle container reports. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Yiqun Lin
58e850f262 HDFS-12083. Ozone: KSM: previous key has to be excluded from result in listVolumes, listBuckets and listKeys. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
a4cbd33dd5 HDFS-12129. Ozone: SCM http server is not stopped with SCM#stop(). Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
ea5e24c925 HDFS-12105. Ozone: listVolumes doesn't work from ozone commandline. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7afddca9d5 HDFS-12037. Ozone: Improvement rest API output format for better looking. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
25c1b296e6 HDFS-11679. Ozone: SCM CLI: Implement list container command. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Weiwei Yang
eb18b86e00 HDFS-12080. Ozone: Fix UT failure in TestOzoneConfigurationFields. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
eab407beac HDFS-12057. Ozone: OzoneClient: Implementation of OzoneClient. Contributed by Nandakumar 2018-04-26 05:36:04 -07:00
Anu Engineer
a353094ba4 HDFS-12076. Ozone: Review all cases where we are returning FAILED_INTERNAL_ERROR. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
8caa5d1ac0 HDFS-12086. Ozone: Add the unit test for KSMMetrics. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
5fe9143a0e HDFS-12081. Ozone: Add infoKey REST API document. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4c27dba08a HDFS-12053. Ozone: ozone server should create missing metadata directory if it has permission to. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
63ce883746 HDFS-12035. Ozone: listKey doesn't work from ozone commandline. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
e07662cb04 HDFS-12048. Ozone:TestOzoneContainerRatis & TestRatisManager are failing consistently. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d1ac8611c6 HDFS-12023. Ozone: Test if all the configuration keys are documented in ozone-defaults.xml. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Weiwei Yang
c398ab49d3 HDFS-12047. Ozone: Add REST API documentation. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
5379ffa8de HDFS-12030. Ozone: CLI: support infoKey command. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Weiwei Yang
4b63e3e504 HDFS-12002. Ozone : SCM cli misc fixes/improvements. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
c567e6766b HDFS-11955. Ozone: Set proper parameter default values for listBuckets http request. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
b7b8511bae HDFS-11773. Ozone: KSM : add listVolumes. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Chen Liang
b49c165d28 HDFS-12063. Ozone: Ozone shell: Multiple RPC calls for put/get key. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Chen Liang
0a8c903d3f HDFS-12050. Ozone: StorageHandler: Implementation of close for releasing resources during shutdown. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
8ef1163eca HDFS-12031. Ozone: Rename OzoneClient to OzoneRestClient. Contributed by Nandakumar. 2018-04-26 05:36:04 -07:00
Weiwei Yang
7e834c839d HDFS-12018. Ozone: Misc: Add CBlocks defaults to ozone-defaults.xml. Contributed by Chen Liang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fea1ab6192 HDFS-12007. Ozone: Enable HttpServer2 for SCM and KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
723da57644 HDFS-12016. Ozone: SCM: Container metadata are not loaded properly after datanode restart. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Anu Engineer
77a4b248e4 HDFS-11990. Ozone: Add all configurable entries into ozone-default.xml. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
77e3bb783b HDFS-11970. Ozone: TestXceiverClientManager.testFreeByEviction fails occasionally. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Yiqun Lin
52aa60ed2f HDFS-11963. Ozone: Documentation: Add getting started page (addendum). Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Yiqun Lin
3d2339de38 HDFS-11983. Ozone: Add documentation for metrics in KSMMetrics to OzoneMetrics.md. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
3b0996e2a1 HDFS-11585. Ozone: Support force update a container. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
8d289df627 HDFS-11963. Ozone: Documentation: Add getting started page. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
e73d285567 HDFS-11782. Ozone: KSM: Add listKey. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Anu Engineer
721e7a6418 HDFS-11979. Ozone: TestContainerPersistence never uses MiniOzoneCluster. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
03606dd1a8 HDFS-11962. Ozone: Add stop-ozone.sh script. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fc48e18151 HDFS-11961. Ozone: Add start-ozone.sh to quickly start ozone. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
18ddec6144 HDFS-11977. Ozone: cannot enable test debug/trace log. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
8916c70d90 HDFS-11969. Block Storage: Convert unnecessary info log levels to debug. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Tsz-Wo Nicholas Sze
15cc4007b0 HDFS-11946. Ozone: Containers in different datanodes are mapped to the same location. Contributed by Nandakumar 2018-04-26 05:36:04 -07:00
Anu Engineer
863e4cfb56 HDFS-11958. Ozone: Ensure KSM is initiated using ProtobufRpcEngine. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
fa26dea89d HDFS-11185. Ozone: remove disabled tests. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
e3591aeec0 HDFS-11953. Remove Guava v21 usage from HDFS-7240. Contributed by Xiaoyu Yao. 2018-04-26 05:36:04 -07:00
Lei Xu
08460f1559 HDFS-11659. TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Contributed by Lei (Eddy) Xu. 2018-04-26 05:36:04 -07:00
Rakesh Radhakrishnan
665f7c8b69 HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen. 2018-04-26 05:36:04 -07:00
Arpit Agarwal
d18909d338 HDFS-11866. JournalNode Sync should be off by default in hdfs-default.xml. Contributed by Hanisha Koneru. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
ddc88eaf88 HDFS-11952. Ozone: Fix TestContainerSQLCli#testConvertContainerDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Weiwei Yang
0c37e05b10 HDFS-11779. Ozone: KSM: add listBuckets. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
bacd1188f1 HDFS-11880. Ozone: KSM: Remove protobuf formats from KSM wrappers. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Xiaoyu Yao
2007e85d5b HDFS-11777. Ozone: KSM: add deleteBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:36:04 -07:00
Anu Engineer
3c0ed97df9 HDFS-11926. Ozone: Implement a common helper to return a range of KVs in levelDB. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
f617b2dddb HDFS-11568. Ozone: Create metadata path automatically after null checking. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
d6dd557b24 HDFS-11887. Shared XceiverClient should be closed if there is no open clients to avoid resource leak. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
f5d17b8f7c HDFS-11919. Ozone: SCM: TestNodeManager takes too long to execute. Contributed by Yiqun Lin. 2018-04-26 05:36:04 -07:00
Chen Liang
d67542c115 HDFS-11822. Block Storage: Fix TestCBlockCLI, failing because of " Address already in use". Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Weiwei Yang
bf8c168a80 HDFS-11781. Ozone: KSM: Add deleteKey. Contributed by Yuanbo Liu. 2018-04-26 05:36:04 -07:00
Anu Engineer
42eaecae55 HDFS-11771. Ozone: KSM: Add checkVolumeAccess. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
1b2d0b4fec HDFS-11796. Ozone: MiniOzoneCluster should set "ozone.handler.type" key correctly. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
aa5722dc44 HDFS-11913. Ozone: TestKeySpaceManager#testDeleteVolume fails. Contributed by Weiwei Yang. 2018-04-26 05:36:04 -07:00
Anu Engineer
4893692972 HDFS-11597. Ozone: Add Ratis management API. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:36:04 -07:00
Anu Engineer
a8c0976fb4 HDFS-11894. Ozone: Cleanup imports. Contributed by Weiwei Yang. 2018-04-26 05:35:59 -07:00
Mingliang Liu
677dcf529c HDFS-11903. Ozone: Cleaning up local storage when closing MiniOzoneCluster. Contributed by Mingliang Liu 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
6b9915fcbd HDFS-11774. Ozone: KSM: add deleteVolume. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
236c410881 HDFS-11776. Ozone: KSM: add SetBucketProperty. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
72b228a9e6 HDFS-11853. Ozone: KSM: Add getKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
59d273b175 HDFS-11655. Ozone: CLI: Guarantees user runs SCM commands has appropriate permission. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
b71efcf1b0 HDFS-11865. Ozone: Do not initialize Ratis cluster during datanode startup. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
e3c8f6a24d HDFS-11780. Ozone: KSM: Add putKey. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ff5dbeec07 HDFS-11846. Ozone: Fix Http connection leaks in ozone clients. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
84294de9a2 HDFS-11778. Ozone: KSM: add getBucketInfo. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
47c4867765 HDFS-11859. Ozone: SCM: Separate BlockLocationProtocol from ContainerLocationProtocol. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
0753e094d7 HDFS-11727. Block Storage: Retry Blocks should be requeued when cblock is restarted. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
37642c12df HDFS-11770. Ozone: KSM: Add setVolumeProperty. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
56966a0535 HDFS-11860. Ozone: SCM: SCMContainerPlacementCapacity#chooseNode sometimes does not remove chosen node from healthy list. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Weiwei Yang
ae5242accb HDFS-11830. Ozone: Datanode needs to re-register to SCM if SCM is restarted. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
877e751c84 HDFS-11843. Ozone: XceiverClientRatis should implement XceiverClientSpi.connect(). 2018-04-26 05:34:18 -07:00
Chen Liang
4fb9064523 HDFS-11835. Block Storage: Overwrite of blocks fails. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
ca70300eea HDFS-11775. Ozone: KSM: add createBucket. Contributed by Nandakumar Vadivelu. 2018-04-26 05:34:18 -07:00
Weiwei Yang
e9d09c209e HDFS-11824. Ozone: Fix javac warnings. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
7f0be64284 HDFS-11831. Ozone: Improve the way of getting test file path in unit tests for Windows. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
c9e6d4378d HDFS-11769. Ozone: KSM: Add createVolume API. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Chen Liang
162dac96ce HDFS-11650. Ozone: fix the consistently timeout test testUpgradeFromRel22Image. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Chen Liang
dfb0dab966 HDFS-11694. Block Storage: Add Support for 2 BlockIDBuffers and also for periodic flush of BlockIDBuffer. Contributed by Mukul Kumar Singh 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
821740e4d3 HDFS-11815. CBlockManager#main should join() after start() service. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
a6013e2b5b HDFS-11805. Ensure LevelDB DBIterator is closed. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
b592df6413 HDFS-11811. Ozone: SCM: Support Delete Block. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
9bc494b909 HDFS-11802. Ozone : add DEBUG CLI support for open container db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
edff6c6a24 HDFS-11768. Ozone: KSM: Create Key Space manager service. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Chen Liang
169713f3f3 HDFS-11788. Ozone : add DEBUG CLI support for nodepool db file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Anu Engineer
070960ed8a HDFS-11798. Ozone: Correct description for ozone.handler.type in ozone-default.xml. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
744a052dc2 HDFS-11783. Ozone: Fix spotbugs warnings. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
4be9710712 HDFS-11680. Ozone: SCM CLI: Implement info container command. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Chen Liang
9fcaeceb54 HDFS-11756. Ozone : add DEBUG CLI support of blockDB file. Contributed by Chen Liang 2018-04-26 05:34:18 -07:00
Weiwei Yang
c18229f0df HDFS-11716. Ozone: SCM: CLI: Revisit delete container API. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
67c81a8b18 HDFS-11761. Ozone: Get container report should only report closed containers. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Weiwei Yang
fc9130f6c5 HDFS-11759. Ozone : SCMNodeManager#close() should also close node pool manager object. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
d9be9a8c30 HDFS-11698. Ozone: SCM: CLI: Add Debug command. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
a28557ac03 HDFS-11712. Ozone: Reuse ObjectMapper instance to improve the performance. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Weiwei Yang
3158f479d3 HDFS-11725. Ozone: Revise create container CLI specification and implementation. Contributed by Weiwei Yang 2018-04-26 05:34:18 -07:00
Weiwei Yang
b581cde542 HDFS-11728. Ozone: add the DB names to OzoneConsts. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
6b3da448eb HDFS-11675. Ozone: SCM CLI: Implement delete container command. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Chen Liang
e9588c6422 HDFS-11627. Block Storage: Cblock cache should register with flusher to upload blocks to containers. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Chen Liang
668b056984 HDFS-11667. Block Storage:Handling flushing of incomplete block id buffers during shutdown. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
1a245accb5 HDFS-11668. Ozone: misc improvements for SCM CLI. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Yiqun Lin
5984d40b86 HDFS-11666. Ozone: Fix compile error due to inconsistent package name. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
a9ba8b5ded HDFS-11504. Ozone: SCM: Add Block APIs. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Weiwei Yang
720e5c40f6 HDFS-11649. Ozone: SCM: CLI: Add shell code placeholder classes. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
6c4bd1647c HDFS-11581. Ozone: Support force delete a container. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Anu Engineer
d3f93d55d9 HDFS-11631. Block Storage : allow cblock server to be started from hdfs commandline. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
05b12c48cf HDFS-11657. Ozone: Add unit test for storage container metrics. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
783b7c27b1 HDFS-11582. Block Storage : add SCSI target access daemon. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
7d4a2d4104 HDFS-11569. Ozone: Implement listKey function for KeyManager. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Chen Liang
cbb7cc26a5 HDFS-11642. Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Chen Liang
5b19a03f63 HDFS-11635. Block Storage: Add metrics for Container Flushes. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
2129c6b276 HDFS-11062. Ozone:SCM: Remove null command. Contributed by Yuanbo Liu. 2018-04-26 05:34:18 -07:00
Chen Liang
b7463924af HDFS-11618. Block Storage: Add Support for Direct I/O. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
57b2a9b17d HDFS-11006. Ozone: support setting chunk size in streaming API. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
cd361fa490 HDFS-11625. Ozone: Fix UT failures that caused by hard coded datanode data dirs. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
7573a0bc61 HDFS-11620. Ozone: Allocate container for MiniOzone cluster fails because of insufficient space error. Contributed by Mukul Kumar Singh. 2018-04-26 05:34:18 -07:00
Anu Engineer
fadfe37019 HDFS-11619. Ozone: Unit tests running failed in Windows. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
aa16380d71 HDFS-11566. Ozone: Document missing metrics for container operations. Contributed by Yiqun Lin. 2018-04-26 05:34:18 -07:00
Anu Engineer
82d814fa79 HDFS-11564. Ozone: SCM: Add Comparable Metric Support. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
68da45a789 HDFS-11519. Ozone: Implement XceiverServerSpi and XceiverClientSpi using Ratis. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:34:18 -07:00
Chen Liang
3966f42895 HDFS-11591. Block Storage: fix TestCBlockServer test cleanup. Contributed by Xiaoyu Yao 2018-04-26 05:34:18 -07:00
Anu Engineer
7c59add3da HDFS-11594. Ozone: close container should call compactDB. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
bc9c313907 HDFS-11567. Ozone: SCM: Support update container. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
17a6e62629 HDFS-11537. Block Storage : add cache layer. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
3598d4b8cf HDFS-11548. Ozone: SCM: Add node pool management API. Contributed by Xiaoyu Yao 2018-04-26 05:34:18 -07:00
Anu Engineer
dc8e73a7d5 HDFS-11355. Block Storage : Merge configuration into ozone configuration classes. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
860a4e0044 HDFS-11550. Ozone: Add a check to prevent removing a container that has keys in it. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
23044c1db8 HDFS-11361. Block Storage: add cache interface. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
d251a55ff9 HDFS-11559. Ozone: MiniOzoneCluster prints too many log messages by default. Contributed by Tsz Wo Nicholas Sze. 2018-04-26 05:34:18 -07:00
Anu Engineer
1e892f5748 HDFS-11162. Block Storage: add command line tool. Contributed by Chen Liang. 2018-04-26 05:34:18 -07:00
Anu Engineer
85c2312e7d HDFS-11492. Ozone: Add the ability to handle sendContainerReport Command. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
98d3360e3a HDFS-11444. Ozone: Fix datanode ID handling in MiniOzoneCluster. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
88371ff07f HDFS-11491. Ozone: SCM: Add close container RPC. Contributed by Anu Engineer. 2018-04-26 05:34:18 -07:00
Anu Engineer
932423211f HDFS-11469. Ozone: SCM: Container allocation based on node report. Contributed by Xiaoyu Yao. 2018-04-26 05:34:18 -07:00
Xiaoyu Yao
fc7d678d3d HDFS-11071. Ozone: SCM: Move SCM config keys to ScmConfig. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Tsz-Wo Nicholas Sze
7b7d186b82 HDFS-11513. Ozone: Separate XceiverServer and XceiverClient into interfaces and implementations. 2018-04-26 05:34:18 -07:00
Anu Engineer
1058aa2523 HDFS-10928. Ozone:SCM: Support MXBean for SCM and NodeManager. Contributed by Weiwei Yang. 2018-04-26 05:34:18 -07:00
Anu Engineer
386dbc11da HDFS-11196. Ozone: Improve logging and error handling in the container layer. Contributed by Anu Engineer. 2018-04-25 15:55:06 -07:00
Anu Engineer
32cc2b8f1a HDFS-11509. Ozone: Fix TestEndpoint test regression. Contributed by Anu Engineer. 2018-04-25 15:54:56 -07:00
Anu Engineer
9c57a61f68 HDFS-11424. Block Storage: add container representation to meta data. Contributed Chen Liang. 2018-04-25 15:54:56 -07:00
Anu Engineer
3e1317de02 HDFS-11451. Ozone: Add protobuf definitions for container reports. Contributed by Anu Engineer. 2018-04-25 15:54:56 -07:00
Anu Engineer
02fb9aed32 HDFS-11158. Ozone: TestDatanodeStateMachine is failing intermittently. Contributed by Weiwei Yang. 2018-04-25 15:54:56 -07:00
Anu Engineer
753979865b HDFS-11137. Ozone: SCM: Add negative tests cases for datanodeStatemachine. Contributed by Weiwei Yang. 2018-04-25 15:54:42 -07:00
Anu Engineer
d55564143d HDFS-11484. Ozone: Fix flaky TestNodeManager#testScmNodeReportUpdate. Contributed by Xiaoyu Yao. 2018-04-25 15:54:42 -07:00
Anu Engineer
a2328d36ac HDFS-11480. Ozone: TestEndpoint task failure. Contributed by Xiaoyu Yao. 2018-04-25 15:54:42 -07:00
Anu Engineer
2cba0daddc HDFS-11425. Ozone: add client-facing container APIs and container references. Contributed by Chen Liang. 2018-04-25 15:54:42 -07:00
Anu Engineer
56f011fd02 HDFS-11447. Ozone: SCM: Send node report to SCM with heartbeat.Contributed by Xiaoyu Yao. 2018-04-25 15:54:41 -07:00
Anu Engineer
f4529316bd HDFS-11474. Ozone: TestContainerMapping needs to cleanup levelDB files.Contributed by Xiaoyu Yao. 2018-04-25 15:54:36 -07:00
Anu Engineer
b29ae6e5f7 HDFS-11473. Ozone : exclude cblock protobuf files from findbugs check. Contributed by Xiaoyu Yao. 2018-04-25 15:54:36 -07:00
Arpit Agarwal
754ed7d651 HDFS-11414. Ozone : move StorageContainerLocation protocol to hdfs-client. Contributed by Chen Liang. 2018-04-25 15:54:34 -07:00
Anu Engineer
bb410de10c HDFS-11184. Ozone: SCM: Make SCM use container protocol. Contributed by Anu Engineer. 2018-04-25 15:54:26 -07:00
Anu Engineer
c169dd1306 HDFS-11463. Ozone: Add metrics for container operations and export over JMX. Contributed by Mukul Kumar Singh. 2018-04-25 15:54:03 -07:00
Anu Engineer
a7f82cba71 HDFS-11154. Block Storage: store server state to persistent storage. Contributed by Chen Liang 2018-04-25 15:53:58 -07:00
Anu Engineer
9997d36eb6 HDFS-11138. Block Storage: add block storage server. Contributed by Chen Liang 2018-04-25 15:53:58 -07:00
Anu Engineer
48db56adea HDFS-11133. Ozone: Add allocateContainer RPC. Contributed by Anu Engineer 2018-04-25 15:53:58 -07:00
Anu Engineer
ca12aac5a4 HDFS-11126. Ozone: Add small file support RPC. Contributed by Anu Engineer. 2018-04-25 15:53:58 -07:00
Anu Engineer
05b44e1ad8 HDFS-11108. Ozone: use containers with the state machine. Contributed by Anu Engineer 2018-04-25 15:53:56 -07:00
Anu Engineer
8bd85268e6 HDFS-11081. Ozone:SCM: Add support for registerNode in datanode. Contributed by Anu Engineer. 2018-04-25 15:53:50 -07:00
Anu Engineer
2b8b3754f6 HDFS-11130. Block Storage : Add storage client to server protocol. Contributed by Chen Lian 2018-04-25 15:53:49 -07:00
Anu Engineer
3d83ee25c5 HDFS-11127. Block Storage : add block storage service protocol. Contributed by Chen Liang 2018-04-25 15:53:43 -07:00
Anu Engineer
8274ff356a HDFS-11103. Ozone: Cleanup some dependencies. Contributed by Anu Engineer. 2018-04-25 15:53:39 -07:00
Anu Engineer
e49e305f25 HDFS-11001. Ozone:SCM: Add support for registerNode in SCM. Contributed by Anu Engineer. 2018-04-25 15:53:34 -07:00
Anu Engineer
964daed853 fix a build break due to merge 2018-04-25 15:53:34 -07:00
Anu Engineer
b186fc81f7 HDFS-10938. Ozone:SCM: Add datanode protocol. Contributed by Anu Engineer. 2018-04-25 15:53:32 -07:00
Anu Engineer
ee119ff60a HDFS-11004. Ozone : move Chunk IO and container protocol calls to hdfs-client. Contributed by Chen Liang. 2018-04-25 15:53:27 -07:00
Anu Engineer
ed84388fca HDFS-10912. Ozone:SCM: Add chill mode support to NodeManager. Contributed by Anu Engineer. 2018-04-25 15:53:27 -07:00
Anu Engineer
5520f73dee Fixing merge conflicts due to HDFS-10637 2018-04-25 15:53:27 -07:00
Anu Engineer
56b0fddfbd HDFS-10995. Ozone: Move ozone XceiverClient to hdfs-client. Contributed by Chen Liang. 2018-04-25 15:53:25 -07:00
Anu Engineer
d0422a74ea HDFS-10932. Ozone: fix XceiverClient slow shutdown. Contributed by Chen Liang. 2018-04-25 15:53:21 -07:00
Anu Engineer
4c95c3d6ab HDFS-10897. Ozone: SCM: Add NodeManager. Contributed by Anu Engineer. 2018-04-25 15:53:21 -07:00
Arpit Agarwal
e3aa2b687c HDFS-10420. Fix Ozone unit tests to use MiniOzoneCluster. Contributed by Arpit Agarwal. 2018-04-25 15:53:21 -07:00
Anu Engineer
3beee0b3d1 HDFS-10363. Ozone: Introduce new config keys for SCM service endpoints. Contributed by Arpit Agarwal. 2018-04-25 15:53:19 -07:00
Arpit Agarwal
6c61902389 HDFS-10361. Support starting StorageContainerManager as a daemon. (Contributed by Arpit Agarwal) 2018-04-25 15:53:12 -07:00
Anu Engineer
aa3ca951f9 HDFS-10351. Ozone: Optimize key writes to chunks by providing a bulk write implementation in ChunkOutputStream. Contributed by Chris Nauroth. 2018-04-25 15:53:07 -07:00
Chris Nauroth
acf0b220ce HDFS-10349. Ozone: StorageContainerManager fails to compile after merge of HDFS-10312 maxDataLength enforcement. Contributed by Chris Nauroth. 2018-04-25 15:53:07 -07:00
Anu Engineer
7d661a289a HDFS-10232. Ozone: Make config key naming consistent. Contributed by Anu Engineer. 2018-04-25 15:53:05 -07:00
Anu Engineer
0ecbc13128 HDFS-10278. Ozone: Add paging support to list Volumes. Contributed by Anu Engineer. 2018-04-25 15:52:58 -07:00
Anu Engineer
7bb2ab7d7c HDFS-10251. Ozone: Shutdown cleanly. Contributed by Anu Engineer 2018-04-25 15:52:58 -07:00
Anu Engineer
e11e824c9b HDFS-10268. Ozone: end-to-end integration for create/get volumes, buckets and keys. Contributed by Chris Nauroth. 2018-04-25 15:52:56 -07:00
Chris Nauroth
0addb1033e HDFS-10250. Ozone: Add key Persistence. Contributed by Anu Engineer. 2018-04-25 15:52:48 -07:00
Chris Nauroth
c6fd5ea3f0 HDFS-10238. Ozone : Add chunk persistance. Contributed by Anu Engineer. 2018-04-25 15:52:48 -07:00
Anu Engineer
0744d0a947 HDFS-8210. Ozone: Implement storage container manager. Contributed by Chris Nauroth. 2018-04-25 15:52:46 -07:00
Chris Nauroth
e758f90f93 HDFS-10195. Ozone: Add container persistence. Contributed by Anu Engineer. 2018-04-25 15:52:33 -07:00
Chris Nauroth
c18c6801ef HDFS-10202. ozone : Add key commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:25 -07:00
Chris Nauroth
f69ce5f3c6 HDFS-10196. Ozone : Enable better error reporting for failed commands in ozone shell. Contributed by Anu Engineer. 2018-04-25 15:52:09 -07:00
Chris Nauroth
fd2fe47b32 HDFS-10179. Ozone: Adding logging support. Contributed by Anu Engineer. 2018-04-25 15:52:07 -07:00
Chris Nauroth
7c55d7feb5 HDFS-9960. OzoneHandler : Add localstorage support for keys. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
4009b66090 HDFS-10180. Ozone: Refactor container Namespace. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
044cc47145 HDFS-9961. Ozone: Add buckets commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
bb47d03906 HDFS-9944. Ozone : Add container dispatcher. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
b9d53ed836 HDFS-9926. ozone : Add volume commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
60fbef08ec HDFS-9925. Ozone: Add Ozone Client lib for bucket handling. Contributed by Anu Engineer. 
Fix build break
 2018-04-25 15:52:00 -07:00
Chris Nauroth
a9879798a6 HDFS-9916. OzoneHandler : Add Key handler. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
93201330b8 HDFS-9916. OzoneHandler : Add Key handler. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Anu Engineer
2b387d622c HDFS-9920. Stop tracking CHANGES.txt in the HDFS-7240 feature branch. Contributed by Chris Nauroth. 2018-04-25 15:52:00 -07:00
Chris Nauroth
085c62545d HDFS-9891. Ozone: Add container transport client. Contributed by Anu Engineer. 2018-04-25 15:51:58 -07:00
Chris Nauroth
9b158f3b57 HDFS-9873. Ozone: Add container transport server. Contributed by Anu Engineer. 2018-04-25 15:51:53 -07:00
Anu Engineer
3c39c18a6d HDFS-9907. Exclude Ozone protobuf-generated classes from Findbugs analysis. Contributed by Chris Nauroth 2018-04-25 15:51:53 -07:00
Chris Nauroth
0a43e272f1 HDFS-9848. Ozone: Add Ozone Client lib for volume handling. Contributed by Anu Engineer. 2018-04-25 15:51:53 -07:00
Anu Engineer
3481897bde HDFS-9853. Ozone: Add container definitions. Contributed by Anu Engineer. 2018-04-25 15:51:51 -07:00
cnauroth
11a6cc2a4d HDFS-9845. OzoneHandler : Support List and Info Volumes. Contributed by Anu Engineer. 2018-04-25 15:51:39 -07:00
cnauroth
f6ef841f74 HDFS-9834. OzoneHandler : Enable MiniDFSCluster based testing for Ozone. Contributed by Anu Engineer. 2018-04-25 15:51:39 -07:00
Anu Engineer
f6b9c0f8a1 HDFS-8757 : OzoneHandler : Add localStorageHandler support for Buckets. Contributed by Anu Engineer 2018-04-25 15:51:39 -07:00
Anu Engineer
ae109d1de7 HDFS-8527. OzoneHandler: Integration of REST interface and container data pipeline back-end. Contributed by Chris Nauroth 2018-04-25 15:51:36 -07:00
Anu Engineer
a301827c6a HDFS-8695. OzoneHandler : Add Bucket REST Interface. (aengineer) 2018-04-25 15:51:00 -07:00
Anu Engineer
764ff0d505 HDFS-8753. Ozone: Unify StorageContainerConfiguration with ozone-default.xml & ozone-site.xml. Contributed by kanaka kumar avvaru 2018-04-25 15:51:00 -07:00
Arpit Agarwal
507e7274aa HDFS-8717. OzoneHandler : Add common bucket objects. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
07213f4eb5 HDFS-8680. OzoneHandler : Add Local StorageHandler support for volumes. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
835f872a0a HDFS-8654. OzoneHandler : Add ACL support. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
b1fa06aacb HDFS-8644. OzoneHandler : Add volume handler. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
2793b4c96f HDFS-8634. OzoneHandler: Add userAuth Interface and Simple userAuth handler. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
d65b373ec4 HDFS-8637. OzoneHandler : Add Error Table. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
cnauroth
2ea29ee062 HDFS-8448. Create REST Interface for Volumes. Contributed by Anu Engineer. 2018-04-25 15:51:00 -07:00
Arpit Agarwal
0a90e6d45c HDFS-8614. OzoneHandler : Add Quota Support. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
d41245ca56 HDFS-8456. Introduce STORAGE_CONTAINER_SERVICE as a new NodeType. (Contributed by Arpit Agarwal) 2018-04-25 15:50:52 -07:00
Inigo Goiri
df92a17e02 HDFS-13336. Test cases of TestWriteToReplica failed in windows. Contributed by Xiao Liang. 2018-04-23 19:12:16 -07:00
Bharat Viswanadham
9b5375e0c1 HDFS-13356. Balancer:Set default value of minBlockSize to 10mb. Contributed by Bharat Viswanadham 2018-04-23 16:09:01 -07:00
Lei Xu
42e82f0281 Update CHANGES, RELEASENOTES, and jdiff for 3.0.2 release. 
(cherry picked from commit f6ecb76d0b919b9836600fe28ec9e637b223cd54)

Conflicts:
	hadoop-project-dist/pom.xml
 2018-04-23 14:14:05 -07:00
Chris Douglas
f411de6a79 HDFS-13408. MiniDFSCluster to support being built on randomized base directory. Contributed by Xiao Liang 2018-04-23 11:13:18 -07:00
Arpit Agarwal
c533c77047 HDFS-13433. webhdfs requests can be routed incorrectly in federated cluster. Contributed by Arpit Agarwal. 2018-04-23 10:08:53 -07:00
Arpit Agarwal
1c1ce63cda HDFS-13055. Aggregate usage statistics from datanodes. Contributed by Ajay Kumar. 2018-04-22 08:07:19 -07:00
Sangjin Lee
f40969a141 HDFS-10183. Prevent race condition during class initialization. Contributed by Pavel Avgustinov. 2018-04-20 20:33:10 -07:00
Bharat Viswanadham
bf7694d020 HDFS-13464. Fix javadoc in FsVolumeList#handleVolumeFailures. Contributed by Shashikant Banerjee 2018-04-18 08:23:45 -07:00
Mukul Kumar Singh
bf2f493f81 HDFS-13463. Fix javadoc in FsDatasetImpl#checkAndUpdate. Contributed by Shashikant Banerjee. 2018-04-18 16:11:37 +05:30
Erik Krogen
2d9e791a90
HDFS-12828. OIV ReverseXML Processor fails with escaped characters 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-18 14:38:23 +09:00
Inigo Goiri
c9b33514b8 HDFS-13462. Add BIND_HOST configuration for JournalNode's HTTP and RPC Servers. Contributed by Lukas Majercak. 2018-04-17 14:19:23 -07:00
Bharat Viswanadham
a68b04370f HDFS-13129. Add a test for DfsAdmin refreshSuperUserGroupsConfiguration. Contributed by Mukul Kumar Singh 2018-04-17 12:37:30 -07:00
Arpit Agarwal
d907fdc3cd HDFS-13439. Add test case for read block operation when it is moved. Contributed by Ajay Kumar. 2018-04-16 14:16:59 -07:00
Wei-Chiu Chuang
e66e287efe HDFS-13330. ShortCircuitCache#fetchOrCreate never retries. Contributed by Gabor Bota. 2018-04-13 09:17:34 -07:00
Xiao Chen
6503593711 HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445. 2018-04-13 09:05:17 -07:00
Bharat Viswanadham
fa8b88ab2b HDFS-13438. Fix javadoc in FsVolumeList#removeVolume. Contributed by Shashikant Banerjee. 2018-04-13 08:56:02 -07:00
Yiqun Lin
0725953efe HDFS-13418. NetworkTopology should be configurable when enable DFSNetworkTopology. Contributed by Tao Jie. 2018-04-13 17:55:45 +08:00
Akira Ajisaka
9b0b9f2af2
HDFS-13436. Fix javadoc of package-info.java 2018-04-13 13:23:51 +09:00
Mukul Kumar Singh
7ed8511ad8 HDFS-13426. Fix javadoc in FsDatasetAsyncDiskService#removeVolume. Contributed by Shashikant Banerjee. 2018-04-12 20:12:31 +05:30
Akira Ajisaka
113af12cfb
HDFS-7101. Potential null dereference in DFSck#doWork(). Contributed by skrho and Ted Yu. 2018-04-12 17:47:37 +09:00
Akira Ajisaka
c7cd362afd
HDFS-13427. Fix the section titles of transparent encryption document. 2018-04-12 10:44:56 +09:00
Xiao Chen
7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Xiao Chen
e76c2aeb28 HDFS-13363. Record file path when FSDirAclOp throws AclException. Contributed by Gabor Bota. 2018-04-10 11:19:48 -07:00
Rakesh Radhakrishnan
f89594f0b8 HDFS-13328. Abstract ReencryptionHandler recursive logic in separate class. Contributed by Surendra Singh Lilhore. 2018-04-10 23:35:00 +05:30
Weiwei Yang
6729047a8b HDFS-13420. License header is displayed in ArchivalStorage/MemoryStorage html pages. Contributed by Akira Ajisaka. 2018-04-10 22:10:44 +08:00
Surendra Singh Lilhore
70590cd8d9 HDFS-13292. Crypto command should give proper exception when trying to set key on existing EZ directory. Contributed by Ranith Sardar. 2018-04-07 11:23:49 +05:30
Wangda Tan
6cf023f9b7 Added CHANGES/RELEASES/Jdiff for 3.1.0 release 
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
 2018-04-05 15:52:39 -07:00
Lei Xu
d737bf99d4 HDFS-13350. Negative legacy block ID will confuse Erasure Coding to be considered as striped block. (Contributed by Lei (Eddy) Xu). 2018-04-05 09:59:10 -07:00
Xiao Chen
57374c4737 HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 2018-04-02 22:48:26 -07:00
Konstantin V Shvachko
acfd764fcc HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 2018-03-30 18:55:35 -07:00
Xiao Chen
2c6cfad5a3 HDFS-13087. Snapshotted encryption zone information should be immutable. Contributed by LiXin Ge. 2018-03-29 15:46:34 -07:00
Wei-Chiu Chuang
ad10cbd91b HDFS-13357. Improve AclException message "Invalid ACL: only directories may have a default ACL.". Contributed by Gabor Bota. 2018-03-29 07:08:22 -07:00
Arpit Agarwal
a991e899fb HDFS-13314. NameNode should optionally exit if it detects FsImage corruption. Contributed by Arpit Agarwal. 2018-03-28 12:49:17 -07:00
Lei Xu
7616683db5 Update 3.0.1 jdiff file and jdiff stable api version 2018-03-23 11:48:36 -07:00
weiy
6e31a09084 HDFS-12512. RBF: Add WebHDFS. 2018-03-23 08:32:25 -07:00
Kihwal Lee
66461ed98b HDFS-13195. DataNode conf page cannot display the current value after reconfig. Contributed by maobaolong. 2018-03-22 12:21:10 -05:00
Konstantin V Shvachko
8d898ab25f HDFS-12884. BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo. Contributed by chencan. 2018-03-21 16:46:03 -07:00
Xiaoyu Yao
389bc6d3da HDFS-11043. TestWebHdfsTimeouts fails. Contributed by Xiaoyu Yao and Chao Sun. 2018-03-21 13:53:35 -07:00
Yongjun Zhang
6c63cc7d30 HDFS-13315. Add a test for the issue reported in HDFS-11481 which is fixed by HDFS-10997. Contributed by Yongjun Zhang. 2018-03-20 23:04:05 -07:00
weiy
6e2b5fa493 HDFS-13215. RBF: Move Router to its own module. Contributed by Wei Yan 2018-03-19 22:13:40 -07:00
Inigo Goiri
08ff1586d5 HDFS-12886. Ignore minReplication for block recovery. Contributed by Lukas Majercak. 2018-03-16 10:29:19 -07:00
Xiaoyu Yao
78b05fde6c HDFS-13280. WebHDFS: Fix NPE in get snasphottable directory list call. Contributed by Lokesh Jain. 2018-03-15 11:02:27 -07:00
Inigo Goiri
e71bc00a47 HDFS-13224. RBF: Resolvers to support mount points across multiple subclusters. Contributed by Inigo Goiri. 2018-03-15 10:32:30 -07:00
Xiaoyu Yao
da777a5498 HDFS-13251. Avoid using hard coded datanode data dirs in unit tests(addendum). Contributed by Ajay Kumar. 2018-03-15 10:18:44 -07:00
Inigo Goiri
6de135169e HDFS-12723. TestReadStripedFileWithMissingBlocks#testReadFileWithMissingBlocks failing consistently. Contributed by Ajay Kumar. 2018-03-15 10:14:35 -07:00
fang zhenyi
5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-15 16:12:07 +09:00
Xiao Chen
5ff22d4c3a HDFS-13246. FileInputStream redundant closes in readReplicasFromCache. Contributed by liaoyuxiangqin. 2018-03-14 21:59:51 -07:00
Yiqun Lin
3a0f4bc0d5 HDFS-13261. Fix incorrect null value check. Contributed by Jianfei Jiang. 2018-03-15 11:06:04 +08:00
Inigo Goiri
41fc7f80be HDFS-13198. RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router. Contributed by Wei Yan. 2018-03-14 13:14:36 -07:00
Xiaoyu Yao
f83716b7f2 HDFS-13251. Avoid using hard coded datanode data dirs in unit tests. Contributed by Ajay Kumar. 2018-03-14 11:26:58 -07:00
Xiaoyu Yao
0be0f1ce1d HDFS-13275. Adding log for BlockPoolManager#refreshNamenodes failures. Contributed by Ajay Kumar. 2018-03-14 10:38:00 -07:00
Arpit Agarwal
4c57fb0cd9 HDFS-13257. Code cleanup: INode never throws QuotaExceededException. Contributed by Tsz Wo Nicholas Sze. 2018-03-14 09:42:48 -07:00
Xiaoyu Yao
a34d83d2fd HDFS-13249. Document webhdfs support for getting snapshottable directory list. Contributed by Lokesh Jain. 2018-03-14 09:14:26 -07:00
Sammi Chen
ad1b988a82 HDFS-11600. Refactor TestDFSStripedOutputStreamWithFailure test classes. Contributed by Sammi Chen. 2018-03-14 14:39:53 +08:00
Yiqun Lin
76be6cbf6c HDFS-12773. RBF: Improve State Store FS implementation. Contributed by Inigo Goiri. 2018-03-14 11:20:59 +08:00
Arpit Agarwal
9714fc1dd4 HDFS-336. dfsadmin -report should report number of blocks from datanode. Contributed by Bharat Viswanadham. 2018-03-13 16:39:17 -07:00
Arpit Agarwal
39537b7c84 HDFS-13235. DiskBalancer: Update Documentation to add newly added options. Contributed by Bharat Viswanadham. 2018-03-13 16:35:51 -07:00
Xiao Chen
8211a3d469 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 2018-03-13 10:48:45 -07:00
Xiao Chen
f82d38dcb3 Revert "HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma." 
This reverts commit 84c1095586.
 2018-03-13 10:36:16 -07:00
Xiao Chen
3b8dbc2cb7 HDFS-12587. Use Parameterized tests in TestBlockInfoStriped and TestLowRedundancyBlockQueues to test all EC policies. Contributed by Takanobu Asanuma. 2018-03-13 10:14:05 -07:00
Xiao Chen
d6931c30c5 HDFS-13239. Fix non-empty dir warning message when setting default EC policy. Contributed by Bharat Viswanadham. 2018-03-13 10:07:26 -07:00
Xiao Chen
84c1095586 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 2018-03-13 09:58:03 -07:00
Xiaoyu Yao
0355ec20eb HDFS-13141. WebHDFS: Add support for getting snasphottable directory list. Contributed by Lokesh Jain. 2018-03-12 20:41:37 -07:00
Chris Douglas
45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Yiqun Lin
19292bc264 HDFS-13226. RBF: Throw the exception if mount table entry validated failed. Contributed by maobaolong. 2018-03-13 11:03:31 +08:00
Yiqun Lin
7fab787de7 HDFS-13253. RBF: Quota management incorrect parent-child relationship judgement. Contributed by Yiqun Lin. 2018-03-13 10:30:20 +08:00
Akira Ajisaka
319defafc1
HDFS-12156. TestFSImage fails without -Pnative 2018-03-13 11:26:56 +09:00
Yiqun Lin
4afd50b106 HDFS-10803. TestBalancerWithMultipleNameNodes#testBalancing2OutOf3Blockpools fails intermittently due to no free space available. Contributed by Yiqun Lin. 2018-03-13 10:15:51 +08:00
Inigo Goiri
91c82c90f0 HDFS-13241. RBF: TestRouterSafemode failed if the port 8888 is in use. Contributed by maobaolong. 2018-03-12 17:28:15 -07:00
Chris Douglas
39a5fbae47 HDFS-12677. Extend TestReconstructStripedFile with a random EC policy. Contributed by Takanobu Asanuma 2018-03-12 14:29:44 -07:00
Chris Douglas
ddb67ca707 HADOOP-14742. Document multi-URI replication Inode for ViewFS. Contributed by Gera Shegalov 2018-03-12 13:43:27 -07:00
Anu Engineer
cceb68ffea HDFS-10618. TestPendingReconstruction#testPendingAndInvalidate is flaky due to race condition. Contributed by Eric Badger. 2018-03-12 12:07:22 -07:00
Inigo Goiri
ac627f561f HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin. 2018-03-12 09:58:56 -07:00
Yiqun Lin
4743d4a2c7 HDFS-13240. RBF: Update some inaccurate document descriptions. Contributed by Yiqun Lin. 2018-03-10 11:28:55 +08:00
Inigo Goiri
8133cd5305 HDFS-13232. RBF: ConnectionPool should return first usable connection. Contributed by Ekanth S. 2018-03-09 18:25:05 -08:00
Inigo Goiri
afe1a3ccd5 HDFS-13212. RBF: Fix router location cache issue. Contributed by Weiwei Wu. 2018-03-09 17:18:51 -08:00
Tsz-Wo Nicholas Sze
ba0da2785d HDFS-13252. Code refactoring: Remove Diff.ListType. 2018-03-09 15:50:26 -08:00
Hanisha Koneru
4eeff62f69 HDFS-13244. Add stack, conf, metrics links to utilities dropdown in NN webUI. Contributed by Bharat Viswanadham. 2018-03-09 15:27:17 -08:00
Lokesh Jain
7b0dc31020
HDFS-13190. Document WebHDFS support for snapshot diff 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
 2018-03-09 15:06:15 -08:00
Hanisha Koneru
9a082fbe6e HDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus. 2018-03-09 13:20:32 -08:00
Yiqun Lin
122805b43a HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 2018-03-09 15:42:57 +08:00
Inigo Goiri
0c2b969e01 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 2018-03-08 09:32:05 -08:00
Sean Mackrory
46d29e3d7e HDFS-13176. WebHdfs file path gets truncated when having semicolon (;) inside. Contributed by Zsolt Venczel. 2018-03-07 13:33:41 -07:00
Tsz-Wo Nicholas Sze
88fba00caa HDFS-13222. Update getBlocks method to take minBlockSize in RPC calls. Contributed by Bharat Viswanadham 2018-03-07 11:27:53 -08:00
Yiqun Lin
58ea2d7a65 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 2018-03-07 15:26:38 +08:00
Xiaoyu Yao
edf9445708 HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 2018-03-06 16:44:20 -08:00
Tsz-Wo Nicholas Sze
346caa2095 HDFS-13227. Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList. Contributed by Shashikant Banerjee 2018-03-06 13:35:03 -08:00
Tsz-Wo Nicholas Sze
871d0d39fa HDFS-13223. Reduce DiffListBySkipList memory usage. Contributed by Shashikant Banerjee 2018-03-06 12:23:03 -08:00
Inigo Goiri
7060725662 HDFS-13188. Disk Balancer: Support multiple block pools during block move. Contributed by Bharat Viswanadham. 2018-03-06 09:09:32 -08:00
Arpit Agarwal
a3c304c5dd HDFS-13178. Disk Balancer: Add a force option to DiskBalancer Execute command. Contributed by Bharat Viswanadham. 2018-03-03 15:42:16 -08:00
Tsz-Wo Nicholas Sze
ba82e5c488 HDFS-13173. Replace ArrayList with DirectoryDiffList(SnapshotSkipList) to store DirectoryDiffs. Contributed by Shashikant Banerjee 2018-03-02 17:47:48 -08:00
Xiao Chen
c75105f07b HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 2018-03-02 16:46:04 -08:00
Tsz-Wo Nicholas Sze
60080fbfcf HDFS-13171. Handle Deletion of nodes in SnasphotSkipList. Contributed by Shashikant Banerjee 2018-03-02 10:51:05 -08:00
Tsz-Wo Nicholas Sze
850b2f2567 HDFS-1686. Federation: Add more Balancer tests with federation setting. Contributed by Bharat Viswanadham 2018-03-02 10:42:39 -08:00
Yiqun Lin
55669515f6 HDFS-13210. Fix the typo in MiniDFSCluster class. Contributed by fang zhenyi. 2018-03-02 11:49:12 +08:00
Tsz-Wo Nicholas Sze
96e8f260ab HDFS-13211. Fix a bug in DirectoryDiffList.getMinListForRange. Contributed by Shashikant Banerjee 2018-03-01 14:12:15 -08:00
Yiqun Lin
ececeea8f2 HDFS-13202. Fix the outdated javadocs in HAUtil. Contributed by Chao Sun. 2018-03-01 17:51:43 +08:00
Tsz-Wo Nicholas Sze
81d9446a92 HDFS-13102. Implement SnapshotSkipList class to store Multi level DirectoryDiffs. Contributed by Shashikant Banerjee 2018-02-28 13:42:58 -08:00
Xiaoyu Yao
f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Xiaoyu Yao
2574375bf5 HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru. 2018-02-28 08:37:01 -08:00
Yiqun Lin
a9c14b1119 HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang. 2018-02-28 10:17:43 +08:00
Inigo Goiri
d86f301d46 HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 2018-02-27 17:39:02 -08:00
Tsz-Wo Nicholas Sze
55c77bf722 HDFS-13143. SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDiff calculation happens between a snapshot and the current tree. Contributed by Shashikant Banerjee 2018-02-27 15:28:41 -08:00
Arpit Agarwal
1cc9a58dda HDFS-13181. DiskBalancer: Add an configuration for valid plan hours . Contributed by Bharat Viswanadham. 2018-02-27 10:18:07 -08:00
Inigo Goiri
cd4a97bf80 HDFS-13192. Change the code order in getFileEncryptionInfo to avoid unnecessary call of assignment. Contributed by LiXin Ge. 2018-02-27 08:53:00 -08:00
Inigo Goiri
225d877092 HDFS-13193. Various Improvements for BlockTokenSecretManager. Contributed by BELUGA BEHR. 2018-02-27 08:48:03 -08:00
Yiqun Lin
1e85a995d1 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 2018-02-27 10:48:52 +08:00
Chao Sun
ae290a4bb4 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 2018-02-26 16:15:00 -08:00
Wei Yan
6ce9f79cc9 HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 2018-02-26 15:13:41 -08:00
Arpit Agarwal
46c93453e9 HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-26 13:56:34 -08:00
Kihwal Lee
451265a83d HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 2018-02-26 10:28:04 -06:00
Arpit Agarwal
2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Xiaoyu Yao
1e84e46f16 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 2018-02-23 19:35:12 -08:00
Xiao Chen
51088d3233 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 2018-02-23 13:49:09 -08:00
Xiaoyu Yao
84a1321f6a HDFS-13136. Avoid taking FSN lock while doing group member lookup for FSD permission check. Contributed by Xiaoyu Yao. 2018-02-22 11:32:32 -08:00
Anu Engineer
121e1e1280 HDFS-13175. Add more information for checking argument in DiskBalancerVolume. 
Contributed by  Lei (Eddy) Xu.
 2018-02-20 19:16:30 -08:00
Inigo Goiri
6f81cc0bee HDFS-13167. DatanodeAdminManager Improvements. Contributed by BELUGA BEHR. 2018-02-20 15:18:27 -08:00
Inigo Goiri
17c592e6cf HDFS-13168. XmlImageVisitor - Prefer Array over LinkedList. Contributed by BELUGA BEHR. 2018-02-20 15:16:01 -08:00
Arpit Agarwal
9028ccaf83 HDFS-13159. TestTruncateQuotaUpdate fails in trunk. Contributed by Nanda kumar. 2018-02-20 11:40:20 -08:00
Yiqun Lin
8896d20b91 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 2018-02-20 09:37:08 +08:00
Tsz-Wo Nicholas Sze
83e2bb98ee HDFS-12998. SnapshotDiff - Provide an iterator-based listing API for calculating snapshotDiff. Contributed by Shashikant Banerjee 2018-02-19 11:42:10 +08:00
Chen Liang
c008d10a11 HDFS-13161. Update comment in start-dfs.sh to mention correct variable for secure datanode user. Contributed by Ajay Kumar. 2018-02-16 16:04:50 -08:00
Arpit Agarwal
4c2119f04e HDFS-13058. Fix dfs.namenode.shared.edits.dir in TestJournalNode. Contributed by Bharat Viswanadham. 2018-02-16 10:29:22 -08:00
Arpit Agarwal
8d5ea7470a HDFS-13153. Enable HDFS diskbalancer by default. Contributed by Ajay Kumar. 2018-02-16 09:38:34 -08:00
Bharat Viswanadham
a1e05e0292
HDFS-13151. Fix the javadoc error in ReplicaInfo 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-16 16:22:24 +09:00
Kihwal Lee
47473952e5 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 2018-02-15 15:32:42 -06:00
Kihwal Lee
da59acd8ca xattr api cleanup 2018-02-15 11:11:55 -06:00
Tsz-Wo Nicholas Sze
6ea7d78ccb HDFS-13142. Define and Implement a DiifList Interface to store and manage SnapshotDiffs. Contributed by Shashikant Banerjee 2018-02-15 19:33:44 +08:00
Steve Loughran
f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-14 16:20:14 +00:00
Brahma Reddy Battula
35c17351ca HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 2018-02-12 22:14:34 +05:30
Arpit Agarwal
96bb6a51ec HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:17:40 -08:00
Yiqun Lin
25fbec67d1 HDFS-13130. Log object instance get incorrectly in SlowDiskTracker. Contributed by Jianfei Jiang. 2018-02-11 12:02:10 +08:00
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Yiqun Lin
543f3abbee HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 2018-02-09 13:57:42 +08:00
Steve Loughran
996796f104 HADOOP-15214. Make Hadoop compatible with Guava 21.0. 
Contributed by Igor Dvorzhak
 2018-02-08 10:55:54 -08:00
Xiaoyu Yao
8faf0b50d4 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 2018-02-08 08:59:48 -08:00
Yongjun Zhang
f491f717e9 HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 2018-02-07 12:58:09 -08:00
Jitendra Pandey
b061215ecf HDFS-11701. NPE from Unresolved Host causes permanent DFSInputStream failures. Contributed by Lokesh Jain. 2018-02-07 11:22:36 -08:00
Brahma Reddy Battula
01bd6ab18f HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-07 23:10:33 +05:30
Tsz-Wo Nicholas Sze
4fadcf1631 HDFS-13107. Add Mover Cli Unit Tests for Federated cluster. Contributed by Bharat Viswanadham 2018-02-06 16:59:26 -08:00
Sean Mackrory
b14db95c35 HDFS-13106. Need to exercise all HDFS APIs for EC. Contributed by Haibo Yan. 2018-02-06 16:39:00 -07:00
Tsz-Wo Nicholas Sze
4d1bf33d0f HDFS-12947. Limit the number of Snapshots allowed to be created for a Snapshottable Directory. Contributed by Shashikant Banerjee 2018-02-06 14:46:44 -08:00
Anu Engineer
4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Wei-Chiu Chuang
2021f4bdce HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Wei-Chiu Chuang. 2018-02-02 17:18:42 -08:00
Hanisha Koneru
0ef639235b HDFS-13073. Cleanup code in InterQJournalProtocol.proto. Contributed by Bharat Viswanadham. 2018-02-02 16:28:03 -08:00
Akira Ajisaka
4aef8bd2ef
HDFS-13048. LowRedundancyReplicatedBlocks metric can be negative 2018-02-02 14:34:07 +09:00
Yiqun Lin
b0627c891b HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 2018-02-02 11:25:41 +08:00
Anu Engineer
aa45faf0b2 HDFS-12942. Synchronization issue in FSDataSetImpl#moveBlock. Contributed by Ajay Kumar. 2018-02-01 18:03:01 -08:00
Hanisha Koneru
dd50f53997 HDFS-13062. Provide support for JN to use separate journal disk per namespace. Contributed by Bharat Viswanadham. 2018-02-01 12:28:17 -08:00
Xiaoyu Yao
b3ae11d597 HDFS-12997. Move logging to slf4j in BlockPoolSliceStorage and Storage. Contributed by Ajay Kumar. 2018-02-01 10:45:34 -08:00
Inigo Goiri
6ca7204ceb HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 2018-02-01 10:37:14 -08:00
Xiao Chen
ae2177d296 HDFS-12897. getErasureCodingPolicy should handle .snapshot dir better. Contributed by LiXin Ge. 2018-01-31 22:50:25 -08:00
Xiaoyu Yao
af015c0b23 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 2018-01-31 22:39:51 -08:00
Hanisha Koneru
3ce2190b58 HDFS-13092. Reduce verbosity for ThrottledAsyncChecker#schedule. Contributed by Mukul Kumar Singh. 2018-01-31 11:05:17 -08:00
Xiaoyu Yao
37b7536568 HDFS-13061. SaslDataTransferClient#checkTrustAndSend should not trust a partially trusted channel. Contributed by Ajay Kumar. 2018-01-31 10:47:02 -08:00
Inigo Goiri
5206b2c7ca HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 2018-01-30 18:27:18 -08:00
Weiwei Yang
2e7331ca26 HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 2018-01-31 07:25:03 +08:00
Yiqun Lin
dbb9dded33 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 2018-01-30 12:12:08 +08:00
Kihwal Lee
fde95d463c HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah 2018-01-29 17:23:29 -06:00
Xiao Chen
b63dcd583f HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 2018-01-28 22:19:49 -08:00
Xiao Chen
6bc2f7f4b4 HDFS-13065. TestErasureCodingMultipleRacks#testSkewedRack3 is failing. Contributed by Gabor Bota. 2018-01-28 22:12:05 -08:00
Arpit Agarwal
e990904dd5 HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 13:09:13 -08:00
Yiqun Lin
7721fff744 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 2018-01-25 15:51:26 +08:00
Jianfei Jiang
eb2dd08688
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-25 15:30:42 +09:00
Inigo Goiri
95743c672e HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 2018-01-23 19:15:44 -08:00
Yiqun Lin
d95c13774e HDFS-12963. Error log level in ShortCircuitRegistry#removeShm. Contributed by hu xiaodong. 2018-01-24 10:43:36 +08:00
Hanisha Koneru
6347b2253d HDFS-13023. Journal Sync does not work on a secure cluster. Contributed by Bharat Viswanadham. 2018-01-22 16:02:32 -08:00
Chen Liang
62c9e7fa99 HDFS-13036. Reusing the volume storage ID obtained by replicaInfo. Contributed by liaoyuxiangqin. 2018-01-19 17:51:53 -08:00
Lei Xu
2ed9d61aad HDFS-13039. StripedBlockReader.createBlockReader leaks socket on IOException. (Lei (Eddy) Xu) 2018-01-19 16:40:58 -08:00
Inigo Goiri
130f8bce00 HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 2018-01-19 09:47:40 -08:00
Yiqun Lin
9e4f52d323 HDFS-12973. RBF: Document global quota supporting in federation. Contributed by Yiqun Lin. 2018-01-19 14:18:19 +08:00
Vinayakumar B
09efdfe9e1 HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 2018-01-17 14:16:48 +05:30
Lei Xu
3bd9ea63df HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei) 2018-01-16 15:15:11 -08:00
Yufei Gu
b202935353 HADOOP-15157. Zookeeper authentication related properties to support CredentialProviders. (Contributed by Gergo Repas) 2018-01-16 10:15:22 -08:00
Brahma Reddy Battula
880b9d24ff HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 2018-01-16 16:21:02 +05:30
Yiqun Lin
9afb8025d6 HDFS-12972. RBF: Display mount table quota info in Web UI and admin command. Contributed by Yiqun Lin. 2018-01-13 13:16:43 +08:00
Inigo Goiri
d5d6a0353b HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-12 16:25:39 -08:00
Arpit Agarwal
b278f7b293 HDFS-12984. BlockPoolSlice can leak in a mini dfs cluster. Contributed by Ajay Kumar. 2018-01-12 08:04:00 -08:00
Yiqun Lin
d98a2e6e23 HDFS-12934. RBF: Federation supports global quota. Contributed by Yiqun Lin. 2018-01-10 13:59:11 +08:00
Inigo Goiri
d9006d8a4e HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 2018-01-09 18:53:25 -08:00
Lei Xu
47563d86fe HDFS-12994. TestReconstructStripedFile.testNNSendsErasureCodingTasks fails due to socket timeout. (Contributed by Lei (Eddy) Xu) 2018-01-09 11:53:49 -08:00
Inigo Goiri
b3290c41a9 HDFS-12808. Add LOG.isDebugEnabled() guard for LOG.debug(...). Contributed by Bharat Viswanadham. 2018-01-08 17:51:09 -08:00
Manoj Govindassamy
73ff09b79a HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 2018-01-08 15:34:00 -08:00
Wei Yan
2ee0d64ace HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 2018-01-08 14:25:39 -08:00
Yiqun Lin
bf5c948995 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-06 14:31:08 +08:00
Inigo Goiri
83b513ac6d HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 2018-01-05 09:11:38 -08:00
Lei Xu
dc735b286b HDFS-12860. StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB. (Contributed by Lei (Eddy) Xu) 2018-01-04 10:54:56 -08:00
Arpit Agarwal
739d3c394d HDFS-12987. Document - Disabling the Lazy persist file scrubber.. Contributed by Karthik Palanisamy. 2018-01-04 10:32:00 -08:00
Sean Mackrory
45a47198b4 HDFS-12913. TestDNFencingWithReplication sometimes fails because of NameNode in standby state. Contributed by Zsolt Venczel. 2018-01-04 08:37:21 -07:00
Yiqun Lin
2a48b3594c HDFS-12948. DiskBalancer report command top option should only take positive numeric values. Contributed by Shashikant Banerjee. 2018-01-04 10:48:44 +08:00
Xiaoyu Yao
3ba985997d HDFS-12931. Handle InvalidEncryptionKeyException during DistributedFileSystem#getFileChecksum. Contributed by Mukul Kumar Singh. 2018-01-03 14:54:20 -08:00
Manoj Govindassamy
5c28804b95 HDFS-12629. NameNode UI should report total blocks count by type - replicated and erasure coded. 2018-01-02 15:05:27 -08:00
Manoj Govindassamy
42a1c98597 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-02 14:59:36 -08:00
Arpit Agarwal
b4d11337c9 HDFS-12351. Explicitly describe the minimal number of DataNodes required to support an EC policy in EC document.. Contributed by Hanisha Koneru. 2018-01-02 12:54:40 -08:00
Chris Douglas
7fe6f83c8f HADOOP-15117. open(PathHandle) contract test should be exhaustive for default options 2017-12-30 17:58:35 -08:00
Lei Xu
6e3e1b8cde HDFS-12915. Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy. (Contributed by Chris Douglas) 2017-12-29 12:21:57 -08:00
Xiao Chen
5bf7e594d7 HDFS-9023. When NN is not able to identify DN for replication, reason behind it can be logged. 2017-12-28 11:54:08 -08:00
Manoj Govindassamy
76e664e931 HDFS-12959. Fix TestOpenFilesWithSnapshot redundant configurations. 2017-12-21 15:47:15 -08:00
Chen Liang
826507c41b HDFS-12951. Incorrect javadoc in SaslDataTransferServer.java#receive. Contributed by Mukul Kumar Singh. 2017-12-21 11:20:30 -08:00
Lei Xu
b318bed01a HDFS-12938. TestErasureCodigCLI testAll failing consistently. (Contributed by Ajay Kumar) 2017-12-21 10:28:24 -08:00
Wei Yan
a78db99190 HDFS-12932. Fix confusing LOG message for block replication. Contributed by Chao Sun. 2017-12-20 08:55:46 -08:00
Yiqun Lin
e040c97b77 HDFS-12937. RBF: Add more unit tests for router admin commands. Contributed by Yiqun Lin. 2017-12-19 15:31:34 +08:00
Junping Du
a7f8caf58e Add 2.8.3 release jdiff files. 
(cherry picked from commit c89f99aade575ab1f6a9836df719cda272293d90)
 2017-12-18 22:39:01 -08:00
Yiqun Lin
25a36b7452 HDFS-12930. Remove the extra space in HdfsImageViewer.md. Contributed by Rahul Pathak. 2017-12-19 11:23:16 +08:00
Tsz-Wo Nicholas Sze
c7499f2d24 HDFS-12347. TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently. Contributed by Bharat Viswanadham 2017-12-19 10:02:30 +08:00
Erik Krogen
94576b17fb HDFS-12818. Support multiple storages in DataNodeCluster / SimulatedFSDataset. Contributed by Erik Krogen. 2017-12-18 11:36:22 -08:00
Virajith Jalaparti
8239e3afb3 HDFS-12712. [9806] Code style cleanup 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
80c3fec3a1 HDFS-12912. [READ] Fix configuration and implementation of LevelDB-based alias maps 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
c89b29bd42 HDFS-12893. [READ] Support replication of Provided blocks with non-default topologies. 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
0f6aa9564c HDFS-12905. [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage. 2017-12-15 17:51:41 -08:00
Chris Douglas
2298f2d76b HDFS-12874. Documentation for provided storage. Contributed by Virajith Jalaparti 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
71ec170107 HDFS-12887. [READ] Allow Datanodes with Provided volumes to start when blocks with the same id exist locally 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
4531588a94 HDFS-11640. [READ] Datanodes should use a unique identifier when reading from external stores 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
fb996a32a9 HDFS-12894. [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
9c35be86e1 HDFS-12713. [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata. Contributed by Ewan Higgs 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
a027055dd2 HDFS-12885. Add visibility/stability annotations. Contributed by Chris Douglas 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
b634053c4d HDFS-12591. [READ] Implement LevelDBFileRegionFormat. Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
352f994b64 HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
cc933cba77 HDFS-12685. [READ] FsVolumeImpl exception when scanning Provided storage volume 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
4d59dabb7f HDFS-12809. [READ] Fix the randomized selection of locations in {{ProvidedBlocksBuilder}}. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
3d3be87e30 HDFS-12778. [READ] Report multiple locations for PROVIDED blocks 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
3b1d30301b HDFS-12775. [READ] Fix reporting of Provided volumes 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
e1a28f95b8 HDFS-12777. [READ] Reduce memory and CPU footprint for PROVIDED volumes. 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
6cd80b2521 HDFS-12779. [READ] Allow cluster id to be specified to the Image generation tool 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
90d1b47a2a HDFS-12776. [READ] Increasing replication for PROVIDED files should create local replicas 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
71d0a82571 HDFS-12607. [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
98f5ed5aa3 HDFS-11902. [READ] Merge BlockFormatProvider and FileRegionProvider. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
d6a9a89973 HDFS-12605. [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
aca023b72c HDFS-12289. [READ] HDFS-12091 breaks the tests for provided block reads 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
2407c9b93a HDFS-12093. [READ] Share remoteFS between ProvidedReplica instances. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
663b3c08b1 HDFS-12091. [READ] Check that the replicas served from a ProvidedVolumeImpl belong to the correct external storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
546b95f484 HDFS-11673. [READ] Handle failures of Datanode with PROVIDED storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
55ade54b8e HDFS-11792. [READ] Test cases for ProvidedVolumeDF and ProviderBlockIteratorImpl 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
89b9faf529 HDFS-11703. [READ] Tests for ProvidedStorageMap 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
aa5ec85f7f HDFS-11663. [READ] Fix NullPointerException in ProvidedBlocksBuilder 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
1108cb7691 HDFS-11653. [READ] ProvidedReplica should return an InputStream that is bounded by its length 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
d65df0f273 HDFS-11190. [READ] Namenode support for data stored in external stores. 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Chen Liang
aa503a29d0 HDFS-12917. Fix description errors in testErasureCodingConf.xml. Contributed by chencan. 2017-12-15 13:45:37 -08:00
Lei Xu
949be14b08 HDFS-12927. Update erasure coding doc to address unsupported APIs. (Contributed by Lei (Eddy) Xu) 2017-12-15 10:20:29 -08:00
Lei Xu
1c15b1751c HDFS-12819. Setting/Unsetting EC policy shows warning if the directory is not empty. (Contributed by Lei (Eddy) Xu) 2017-12-15 10:04:43 -08:00
Yiqun Lin
ee028bfdf1 HDFS-12895. RBF: Add ACL support for mount table. Contributed by Yiqun Lin. 2017-12-15 14:09:24 +08:00
Xiao Chen
e1cb278cd0 HDFS-12910. Secure Datanode Starter should log the port when it fails to bind. Contributed by Stephen O'Donnell and Nanda kumar. 2017-12-14 19:22:51 -08:00
Kihwal Lee
f5a72424c0 HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 2017-12-14 14:58:47 -06:00
Jason Lowe
80db744ee5 HDFS-12881. Output streams closed with IOUtils suppressing write errors. Contributed by Ajay Kumar 2017-12-14 09:45:44 -06:00
Yiqun Lin
91c96bdf8e HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 2017-12-14 16:00:53 +08:00
Chen Liang
46e18c8da7 HADOOP-14914. Change to a safely casting long to int. Contributed by Ajay Kumar. 2017-12-13 14:56:14 -08:00
Andrew Wang
d447152d49 Update CHANGES, RELEASENOTES, jdiff for 3.0.0 release. 2017-12-13 13:05:32 -08:00
Chris Douglas
693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Wei-Chiu Chuang
55fc2d6485 HDFS-12891. Do not invalidate blocks if toInvalidate is empty. Contributed by Zsolt Venczel. 2017-12-11 16:43:03 -08:00
Inigo Goiri
5cd1056ad7 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 2017-12-11 15:14:57 -08:00
Manoj Govindassamy
ef7d334d36 HDFS-12825. Fsck report shows config key name for min replication issues (Contributed by Gabor Bota). 2017-12-08 11:46:30 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
f19638333b HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 2017-12-08 08:00:21 -08:00
Xiao Chen
19e0894209 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 2017-12-07 21:05:55 -08:00
Wei-Chiu Chuang
d6c31a3e6b HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 2017-12-07 14:50:30 -08:00
Lei Xu
67662e2ac9 HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 2017-12-07 11:15:40 -08:00
Akira Ajisaka
6cca5b3bcb
HDFS-12889. Addendum patch to add missing file. 2017-12-07 13:23:37 +09:00
Xiao Chen
56b1ff80dd HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 2017-12-05 20:48:16 -08:00
Akira Ajisaka
44b06d34a5
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 2017-12-06 11:40:33 +09:00
Brahma Reddy Battula
6d16a99ae8 HDFS-11751. DFSZKFailoverController daemon exits with wrong status code. Contributed by Bharat Viswanadham 2017-12-05 19:53:46 +05:30
Arpit Agarwal
e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00
Xiao Chen
404eab4dc0 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:11 -08:00
Chris Douglas
42307e3c3a HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 22:34:30 -08:00
Wei-Chiu Chuang
0faf506245 HDFS-12836. startTxId could be greater than endTxId when tailing in-progress edit log. Contributed by Chao Sun. 2017-12-01 12:01:21 -08:00
Chris Douglas
53bbef3802 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 5304698dc8.
 2017-12-01 11:19:38 -08:00
Chris Douglas
5304698dc8 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 10:29:30 -08:00
Konstantin V Shvachko
60fd0d7fd7 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 2017-11-30 18:18:28 -08:00
Tsz-Wo Nicholas Sze
b1c7654ee4 HDFS-12594. snapshotDiff fails if the report exceeds the RPC response limit. Contributed by Shashikant Banerjee 2017-11-30 12:18:29 -08:00
Chris Douglas
0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Kihwal Lee
53509f295b HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 2017-11-29 15:11:14 -06:00
Wei Yan
301641811d HDFS-12835. Fix the javadoc errors in Router-based federation. 2017-11-29 09:43:03 -08:00
Konstantin V Shvachko
d331762f24 HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 2017-11-28 17:14:23 -08:00
Surendra Singh Lilhore
30941d99c9 HDFS-12857. StoragePolicyAdmin should support schema based path. Contributed by Surendra Singh Lilhore. 2017-11-28 23:57:03 +05:30
Yiqun Lin
94bed50471 HDFS-12858. Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 2017-11-28 11:48:55 +08:00
Kihwal Lee
738d1a206a HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 2017-11-22 13:01:55 -06:00
Lei Xu
785732c13e HDFS-12847. Regenerate editsStored and editsStored.xml in HDFS tests. Contributed by Lei (Eddy) Xu. 2017-11-22 10:22:32 -08:00
Chen Liang
60fc2a1388 HDFS-12804. Use slf4j instead of log4j in FSEditLog. Contributed by Mukul Kumar Singh. 2017-11-20 12:49:53 -08:00
Manoj Govindassamy
9fb4effd2c HDFS-12730. Verify open files captured in the snapshots across config disable and enable. 2017-11-20 10:28:12 -08:00
Inigo Goiri
e182e77794 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 2017-11-16 16:58:47 -08:00
Weiwei Yang
462e25a3b2 HDFS-12814. Add blockId when warning slow mirror/disk in BlockReceiver. Contributed by Jiandan Yang. 2017-11-16 16:19:53 +08:00
Chris Douglas
675e9a8f57 Revert "HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus." 
This reverts commit b85603e3f8.
 2017-11-15 19:20:37 -08:00
Arpit Agarwal
4908a8970e HDFS-12705. WebHdfsFileSystem exceptions should retain the caused by exception. Contributed by Hanisha Koneru. 2017-11-13 11:30:39 -08:00
Arpit Agarwal
6d201f77c7 HDFS-12498. Journal Syncer is not started in Federated + HA cluster. Contributed by Bharat Viswanadham. 2017-11-10 16:30:38 -08:00
Arpit Agarwal
1d6f8bebe9 HADOOP-14929. Cleanup usage of decodecomponent and use QueryStringDecoder from netty. Contributed by Bharat Viswanadham. 2017-11-10 16:28:12 -08:00
Chen Liang
10a1f557e7 HDFS-12791. NameNode Fsck http Connection can timeout for directories with multiple levels. Contributed by Mukul Kumar Singh. 2017-11-09 18:47:34 -08:00
Akira Ajisaka
3a3566e1d1
HDFS-12732. Correct spellings of ramdomly to randomly in log. Contributed by hu xiaodong. 2017-11-09 15:14:46 +09:00
Brahma Reddy Battula
410d0319cf HDFS-12788. Reset the upload button when file upload fails. Contributed by Brahma Reddy Battula 2017-11-08 14:41:16 +05:30
Weiwei Yang
bb8a6eea52 HDFS-7060. Avoid taking locks when sending heartbeats from the DataNode. Contributed by Jiandan Yang. 2017-11-08 10:22:13 +08:00
Xiaoyu Yao
169cdaa38e HDFS-10528. Add logging to successful standby checkpointing. Contributed by Xiaoyu Yao. 2017-11-03 20:34:40 -07:00
Chris Douglas
b85603e3f8 HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus. 2017-11-03 14:30:57 -07:00
Kihwal Lee
4d2dce40bb HDFS-12771. Add genstamp and block size to metasave Corrupt blocks list. Contributed by Kuhu Shukla. 2017-11-03 15:16:46 -05:00
Xiao Chen
299d38295d HDFS-11467. Support ErasureCoding section in OIV XML/ReverseXML. Contributed by Huafeng Wang. 2017-11-03 12:05:45 -07:00
Xiao Chen
b00f828d84 HDFS-12725. BlockPlacementPolicyRackFaultTolerant fails with very uneven racks. 2017-11-02 21:53:13 -07:00
Xiao Chen
e565b5277d HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 2017-11-02 21:27:35 -07:00
Weiwei Yang
56b88b0670 HDFS-12744. More logs when short-circuit read is failed and disabled. Contributed by Weiwei Yang. 2017-11-01 16:41:45 +08:00
Yiqun Lin
20304b91cc HDFS-12219. Javadoc for FSNamesystem#getMaxObjects is incorrect. Contributed by Erik Krogen. 2017-11-01 14:37:08 +08:00
Lei Xu
9367c25dbd HDFS-12482. Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery. (lei) 2017-10-31 21:58:14 -07:00
Andrew Wang
5f681fa821 Revert "HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham." 
This reverts commit b922ba7393.
 2017-10-31 10:46:10 -07:00
Arpit Agarwal
b922ba7393 HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham. 2017-10-31 10:23:00 -07:00
Inigo Goiri
982bd2a5bf HDFS-12699. TestMountTable fails with Java 7. Contributed by Inigo Goiri. 2017-10-31 10:21:42 -07:00
Chris Douglas
d015e0bbd5 HDFS-7878. API - expose a unique file identifier. 2017-10-31 09:44:01 -07:00
Sean Mackrory
81225430d0 HDFS-206. Support for head in FSShell. Contributed by Gabor Bota. 2017-10-31 08:34:01 -06:00
Chris Douglas
d55a84951a HDFS-12582. Replace HdfsFileStatus constructor with a builder pattern. Contributed by Bharat Viswanadham 2017-10-27 15:36:03 -07:00
Junping Du
a25b5aa0cf Add several jdiff xml files for 2.8.2 release. 2017-10-26 19:30:48 -07:00
Arpit Agarwal
7da15eac7a HDFS-12579. JournalNodeSyncer should use fromUrl field of EditLogManifestResponse to construct servlet Url. Contributed by Hanisha Koneru. 2017-10-25 14:24:22 -07:00
Xiao Chen
f66ad1fab9 HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 2017-10-25 14:11:30 -07:00
Manoj Govindassamy
075dd45a24 HDFS-12544. SnapshotDiff - support diff generation on any snapshot root descendant directory. 2017-10-25 10:54:40 -07:00
Zhe Zhang
17cd8d0c17 Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit 60bfee270e.
 2017-10-24 15:06:50 -07:00
Wei-Chiu Chuang
0f2a69127a HDFS-12249. dfsadmin -metaSave to output maintenance mode blocks. Contributed by Wellington Chevreuil. 2017-10-23 15:24:34 -07:00
Arpit Agarwal
96be795656 HDFS-12650. Use slf4j instead of log4j in LeaseManager. Contributed by Ajay Kumar. 2017-10-23 13:20:46 -07:00
Arpit Agarwal
4b00c9a47f HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:16 -07:00
Xiao Chen
248d9b6fff HDFS-12518. Re-encryption should handle task cancellation and progress better. 2017-10-20 16:25:04 -07:00
Andrew Wang
0477eff8be HDFS-12497. Re-enable TestDFSStripedOutputStreamWithFailure tests. Contributed by Huafeng Wang. 2017-10-20 13:27:21 -07:00
Kai Zheng
ce7cf66e5e HDFS-12448. Make sure user defined erasure coding policy ID will not overflow. Contributed by Huafeng Wang 2017-10-20 09:42:04 +08:00
Inigo Goiri
0f1c037618 HDFS-12620. Backporting HDFS-10467 to branch-2. Contributed by Inigo Goiri. 2017-10-19 18:08:45 -07:00
Wei-Chiu Chuang
4ab0c8f96a HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 2017-10-19 06:17:59 -07:00
Zhe Zhang
60bfee270e HDFS-12502. nntop should support a category based on FilesInGetListingOps. 2017-10-18 23:51:24 -07:00
Lei Xu
f27a4ad032 HDFS-12612. DFSStripedOutputStream.close will throw if called a second time with a failed streamer. (Lei (Eddy) Xu) 2017-10-17 15:52:09 -07:00
Lei Xu
31ebccc962 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 2017-10-16 19:44:30 -07:00
Manoj Govindassamy
b406d8e375 HDFS-12614. FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured. 2017-10-16 17:42:41 -07:00
Wei-Chiu Chuang
8dbc8909c9 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 2017-10-16 12:57:48 -07:00
Xiao Chen
21bc855587 HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 2017-10-16 10:34:06 -07:00
Xiao Chen
7bd700941d HDFS-12637. Extend TestDistributedFileSystemWithECFile with a random EC policy. Contributed by Takanobu Asanuma. 2017-10-16 09:55:22 -07:00
Xiao Chen
035c6ee587 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 2017-10-16 09:51:10 -07:00
Xiao Chen
20575ececb HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 2017-10-15 19:28:22 -07:00
Manoj Govindassamy
133d7ca76e HADOOP-13055. Implement linkMergeSlash and linkFallback for ViewFileSystem 2017-10-13 17:43:21 -07:00
Arpit Agarwal
8dd1eeb94f HDFS-12553. Add nameServiceId to QJournalProtocol. Contributed by Bharat Viswanadham 2017-10-13 14:22:21 -07:00
Arpit Agarwal
8acdf5c274 HDFS-12542. Update javadoc and documentation for listStatus. Contributed by Ajay Kumar. 2017-10-11 12:29:35 -07:00
Arpit Agarwal
bb0a742aac HDFS-12627. Fix typo in DFSAdmin command output. Contributed by Ajay Kumar. 2017-10-11 12:26:14 -07:00
Akira Ajisaka
dc63a6a52b
HDFS-12622. Fix enumerate in HDFSErasureCoding.md. Contributed by Yiqun Lin. 2017-10-11 18:09:53 +09:00
Andrew Wang
a297fb0886 HDFS-12547. Extend TestQuotaWithStripedBlocks with a random EC policy. Contributed by Takanobu Asanuma. 2017-10-10 17:35:49 -07:00
Manoj Govindassamy
78af6cdc53 HDFS-12573. Divide the total blocks metrics into replicated and erasure coded. Contributed by Takanobu Asanuma. 2017-10-10 14:23:29 -07:00
Jitendra Pandey
f29e55bf5e HADOOP-14910. Upgrade netty-all jar to latest 4.0.x.Final. Contributed by Vinayakumar B. 2017-10-10 11:00:07 -07:00
Andrew Wang
82cd85d868 Revert "HDFS-12603. Enable async edit logging by default." 
This reverts commit afb42aeabf.
 2017-10-09 17:53:56 -07:00
Andrew Wang
7938208233 HDFS-12541. Extend TestSafeModeWithStripedFile with a random EC policy. Contributed by Takanobu Asanuma. 2017-10-09 13:03:32 -07:00
Andrew Wang
afb42aeabf HDFS-12603. Enable async edit logging by default. 2017-10-09 11:20:00 -07:00
Arpit Agarwal
09ad848b64 HDFS-12599. Remove Mockito dependency from DataNodeTestUtils. Contributed by Ted Yu. 2017-10-09 10:29:01 -07:00
Lei Xu
46644319e1 HDFS-12606. When using native decoder, DFSStripedStream.close crashes JVM after being called multiple times. (Lei (Eddy) Xu) 2017-10-09 10:08:30 -07:00
Inigo Goiri
5d63a388d1 HDFS-12577. Rename Router tooling. Contributed by Inigo Goiri. 
(cherry picked from commit 53e8d0d030525e4c7f3875e23807c6dbe778890f)
 2017-10-06 18:50:52 -07:00
Inigo Goiri
81601dac8e HDFS-12273. Federation UI. Contributed by Inigo Goiri. 
(cherry picked from commit adbb2e00c7b85524fd43bd68895d49814c16680a)
 2017-10-06 18:50:52 -07:00
Inigo Goiri
6c69e23dcd HDFS-12580. Rebasing HDFS-10467 after HDFS-12447. Contributed by Inigo Goiri. 
(cherry picked from commit b12440d3479f19138bc66ea59baf41eb89061906)
 2017-10-06 18:50:51 -07:00
Inigo Goiri
ad41c81559 HDFS-12381. [Documentation] Adding configuration keys for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit c2d6aa79055ef72406fa598e1c743b0c994b5da8)
 2017-10-06 18:50:51 -07:00
Inigo Goiri
67785fe006 HDFS-12450. Fixing TestNamenodeHeartbeat and support non-HA. Contributed by Inigo Goiri. 
(cherry picked from commit 928d1e87f9dbe64f89b858ccc1780723f3af58e7)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
1f06b81ecb HDFS-12430. Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218. Contributed by Inigo Goiri. 
(cherry picked from commit 3302e792d469b7e8f3bfa10151e4e1c546589734)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
bc9e588a19 HDFS-12335. Federation Metrics. Contributed by Inigo Goiri. 
(cherry picked from commit 3b19e77752afce87936f5c0d1e6d272fba798d7b)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
cc58e7a983 HDFS-12384. Fixing compilation issue with BanDuplicateClasses. Contributed by Inigo Goiri. 
(cherry picked from commit fabe02c8fafa807198054da0c02b2ebaafda76aa)
 2017-10-06 18:50:49 -07:00
Inigo Goiri
7cb6bdf09e HDFS-10631. Federation State Store ZooKeeper implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 23c4ddee11ab1300325a6361124ee8ad6f68d7a4)
 2017-10-06 18:50:49 -07:00
Inigo Goiri
346c9fce43 HDFS-12312. Rebasing HDFS-10467 (2). Contributed by Inigo Goiri. 
(cherry picked from commit 90ba6843fb3ac5dc7576535e66a75a5e3433247b)
 2017-10-06 18:50:48 -07:00
Inigo Goiri
67d10087af HDFS-11554. [Documentation] Router-based federation documentation. Contributed by Inigo Goiri. 
(cherry picked from commit ee3260211d94aed223dd6f2386a166eb2c7d67af)
 2017-10-06 18:50:48 -07:00
Inigo Goiri
b3e6bd22e3 HDFS-10646. Federation admin tool. Contributed by Inigo Goiri. 
(cherry picked from commit ae27e31fbcf546481db0b0345772db2e9132372e)
 2017-10-06 18:50:48 -07:00
Inigo Goiri
6f0de27318 HDFS-10880. Federation Mount Table State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 58b97df661441150d35abd44b3a8606206b46441)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
d8c8107332 HDFS-11826. Federation Namenode Heartbeat. Contributed by Inigo Goiri. 
(cherry picked from commit 928f8dab52191e733984d37f47b69719ccf11313)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
55da7fd7eb HDFS-10687. Federation Membership State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 95cae08849d23cf12c5d280c29f90908e6a90d40)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
0ec82b8cdf HDFS-12223. Rebasing HDFS-10467. Contributed by Inigo Goiri. 
(cherry picked from commit 47db6e9d8e2c264671c89fdd6cb11a7c762d2cce)
 2017-10-06 18:50:46 -07:00
Inigo Goiri
ca4f209b49 HDFS-11546. Federation Router RPC server. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 8a9cdebebf26841a0f1e99fb08135f4597f2eba2)
 2017-10-06 18:50:46 -07:00
Inigo Goiri
4bf877b03f HDFS-10630. Federation State Store FS Implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit c6e0bd640cdaf83a660fa050809cad6f1d4c6f4d)
 2017-10-06 18:50:45 -07:00
Inigo
2c740a684a HDFS-10882. Federation State Store Interface API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6d94c90ece1c1d23d4c97e72c54e9991f5dbc481)
 2017-10-06 18:50:45 -07:00
Inigo
533b986633 HDFS-10881. Federation State Store Driver API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 0f88e049156dce173afc0dbda864e29190dd2210)
 2017-10-06 18:50:45 -07:00
Inigo
2761bbc91a HDFS-10629. Federation Roter. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6821e801724ac38e9737538b2164c9ae88792282)
 2017-10-06 18:50:44 -07:00
Xiao Chen
25f31d9fc4 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-10-05 19:46:47 -07:00
Andrew Wang
644c2f6924 HDFS-12567. BlockPlacementPolicyRackFaultTolerant fails with racks with very few nodes. 2017-10-05 16:58:43 -07:00
Arpit Agarwal
b6942cbe9b HDFS-12420. Add an option to disallow 'namenode format -force'. Contributed by Ajay Kumar. 2017-10-05 15:26:52 -07:00
Xiaoyu Yao
107c177782 HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 2017-10-03 13:02:12 -07:00
Arpit Agarwal
b91305119b HDFS-11968. ViewFS: StoragePolicies commands fail with HDFS federation. Contributed by Mukul Kumar Singh. 2017-10-03 11:23:40 -07:00
Anu Engineer
015abcd8ce HDFS-12576. JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured. Contributed by Bharat Viswanadham. 2017-10-02 15:40:00 -07:00
Lei Xu
27ffd43b64 HDFS-12569. Unset EC policy logs empty payload in edit log. (Lei (Eddy) Xu) 2017-10-02 15:35:49 -07:00
Chen Liang
2c62ff7fd4 HDFS-12271. Incorrect statement in Downgrade section of HDFS Rolling Upgrade document. Contributed by Nandakumar. 2017-10-02 13:18:00 -07:00
Chen Liang
5a68c2cef3 HDFS-12552. Use slf4j instead of log4j in FSNamesystem. Contributed by Ajay Kumar. 2017-10-02 11:26:53 -07:00
Lei Xu
f40dbc170e HDFS-12453. TestDataNodeHotSwapVolumes fails in trunk Jenkins runs. (Lei (Eddy) Xu) 2017-09-29 10:46:17 -07:00
Chen Liang
8e1bd114ee HDFS-12560. Remove the extra word it in HdfsUserGuide.md. Contributed by fang zhenyi. 2017-09-28 10:22:27 -07:00
Wei-Chiu Chuang
7c34ceaf4f HDFS-12458. TestReencryptionWithKMS fails regularly. Contributed by Xiao Chen. 2017-09-28 05:10:26 -07:00
Daryn Sharp
0da29cbeea [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:15:24 -05:00
Lei Xu
1267ff22ce HDFS-12523. Thread pools in ErasureCodingWorker do not shutdown. (Huafeng Wang via Lei) 2017-09-26 10:08:16 -07:00
Brahma Reddy Battula
9d3e4cccf9 HDFS-5040.Audit log for admin commands/ logging output of all DFS admin commands. Contributed by Kuhu Shukla. 2017-09-26 21:59:22 +05:30
Yiqun Lin
e9b790db9d HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 2017-09-26 17:21:42 +08:00
Akira Ajisaka
02e2a9b115
HDFS-12304. Remove unused parameter from FsDatasetImpl#addVolume. Contributed by Chen Liang. 2017-09-26 01:25:07 +09:00
Anu Engineer
d0b2c5850b HDFS-12516. Suppress the fsnamesystem lock warning on nn startup. Contributed by Ajay Kumar. 2017-09-24 21:03:06 -07:00
Anu Engineer
415e5a157a HDFS-12375. Fail to start/stop journalnodes using start-dfs.sh/stop-dfs.sh. Contributed by Bharat Viswanadham. 2017-09-23 15:05:25 -07:00
Anu Engineer
40ef9fa171 HADOOP-14904. Fix javadocs issues in Hadoop HDFS. Contributed by Mukul Kumar Singh. 2017-09-23 12:16:48 -07:00
Anu Engineer
2a81af447f HDFS-12536. Add documentation for getconf command with -journalnodes option. Contributed by Bharat Viswanadham. 2017-09-23 10:53:59 -07:00
Anu Engineer
423ebcaf51 HDFS-12320. Add quantiles for transactions batched in Journal sync. Contributed by Hanisha Koneru. 2017-09-23 10:34:13 -07:00
Anu Engineer
36fb90c939 HDFS-12064. Reuse object mapper in HDFS. Contributed by Hanisha Koneru. 2017-09-23 10:11:00 -07:00
Jitendra Pandey
cda3378659 HDFS-12486. GetConf to get journalnodeslist. Contributed by Bharat Viswanadham. 2017-09-22 16:52:47 -07:00
Arpit Agarwal
08fca508e6 HDFS-12530. Processor argument in Offline Image Viewer should be case insensitive. Contributed by Hanisha Koneru. 2017-09-22 09:47:57 -07:00
Arpit Agarwal
b9e423fa8d HDFS-12496. Make QuorumJournalManager timeout properties configurable. Contributed by Ajay Kumar. 2017-09-21 08:44:43 -07:00
Kihwal Lee
6bf921a5c3 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 2017-09-21 08:42:50 -05:00
Akira Ajisaka
50849ec9eb
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 2017-09-21 19:27:48 +09:00
Andrew Wang
a12f09ba3c HDFS-12447. Rename AddECPolicyResponse to AddErasureCodingPolicyResponse. Contributed by SammiChen. 2017-09-20 11:51:17 -07:00
Ming Ma
ce943eb17a HDFS-11035. Better documentation for maintenace mode and upgrade domain. 2017-09-20 09:36:33 -07:00
Ming Ma
230b85d586 HDFS-12473. Change hosts JSON file format. 2017-09-20 09:03:59 -07:00
Andrew Wang
12d9d7bc50 HDFS-12437. Fix test setup in TestLeaseRecoveryStriped. 2017-09-19 16:42:20 -07:00
Andrew Wang
51edaacd09 HDFS-12445. Correct spellings of choosen to chosen. Contributed by hu xiaodong. 2017-09-19 13:48:23 -07:00
Andrew Wang
59830ca772 HDFS-12444. Reduce runtime of TestWriteReadStripedFile. Contributed by Huafeng Wang and Andrew Wang. 2017-09-19 13:44:42 -07:00
Lei Xu
7bbeacb75e HDFS-12449. TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s. (SammiChen via lei) 2017-09-19 11:50:01 -07:00
Brahma Reddy Battula
fda1221c55 HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 2017-09-19 11:25:45 +05:30
Brahma Reddy Battula
31b58406ac HDFS-12480. TestNameNodeMetrics#testTransactionAndCheckpointMetrics Fails in trunk. Contributed by Hanisha Koneru 2017-09-19 10:37:07 +05:30
Arpit Agarwal
a2dcba1853 HDFS-12470. DiskBalancer: Some tests create plan files under system directory. Contributed by Hanisha Koneru. 2017-09-18 09:53:24 -07:00
Kai Zheng
0f9af246e8 HDFS-12460. Make addErasureCodingPolicy an idempotent operation. Contributed by Sammi Chen 2017-09-18 18:07:12 +08:00
Arpit Agarwal
8d7cc22ac2 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:09:27 -07:00
Andrew Wang
ef8cd5dc56 HDFS-10701. TestDFSStripedOutputStreamWithFailure#testBlockTokenExpired occasionally fails. Contributed by SammiChen. 2017-09-15 16:20:36 -07:00
Erik Krogen
90894c7262 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 2017-09-15 13:56:27 -07:00
Lei Xu
fbe06b5880 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (Lei (Eddy) Xu) 2017-09-15 12:12:42 -07:00
Nathan Roberts
de197fc156 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 2017-09-15 13:08:41 -05:00
Kihwal Lee
78bdf10ae4 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 2017-09-15 09:28:31 -05:00
Arpit Agarwal
50764ef3e4 HDFS-12456. TestNamenodeMetrics.testSyncAndBlockReportMetric fails. Contributed by Bharat Viswanadham. 2017-09-14 22:23:36 -07:00
Kai Zheng
08d996d3e9 HDFS-12395. Support erasure coding policy operations in namenode edit log. Contributed by Sammi Chen 2017-09-15 09:43:39 +08:00
Kai Zheng
ae8f55b932 HDFS-7859. Erasure Coding: Persist erasure coding policies in NameNode. Contributed by Sammi Chen 2017-09-15 09:08:18 +08:00
Lei Xu
61cee3a0b9 HDFS-12378. TestClientProtocolForPipelineRecovery#testZeroByteBlockRecovery fails on trunk. (Lei (Eddy) Xu) 2017-09-14 17:03:25 -07:00
Andrew Wang
390c2b5df0 Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 7fcc73fc0d.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
 2017-09-14 15:12:26 -07:00
John Zhuge
7ee02d1065 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 2017-09-14 11:26:49 -07:00
Arpit Agarwal
65a941008d Revert "HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak." 
This reverts commit aa4b6fbe75.
 2017-09-14 11:17:08 -07:00
Kai Zheng
e12f3e85bd HDFS-12438. Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.threads. Contributed by Andrew Wang 2017-09-14 15:39:10 +08:00
ChenSammi
e0b3c644e1 HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen 2017-09-14 09:15:29 +08:00
Lei Xu
73aed34dff HDFS-12409. Add metrics of execution time of different stages in EC recovery task. (Lei (Eddy) Xu) 2017-09-13 17:14:13 -07:00
Arpit Agarwal
f153e60576 HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:54:02 -07:00
Xiao Chen
fa6cc43edd Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 0a6d5c0cf1.
 2017-09-12 19:53:07 -07:00
Lei Xu
123342cd07 HDFS-12412. Change ErasureCodingWorker.stripedReadPool to cached thread pool. (Lei (Eddy) Xu) 2017-09-12 18:12:07 -07:00
Andrew Wang
f4b6267465 HDFS-12222. Document and test BlockLocation for erasure-coded files. Contributed by Huafeng Wang. 2017-09-12 17:35:30 -07:00
Arpit Agarwal
68282c8eac HDFS-12407. Journal node fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start. Contributed by Ajay Kumar. 2017-09-12 16:18:41 -07:00
Andrew Wang
80ee89b287 HDFS-12417. Disable flaky TestDFSStripedOutputStreamWithFailure. 2017-09-12 11:20:56 -07:00
Chen Liang
738c2a9682 HDFS-12406. dfsadmin command prints "Exception encountered" even if there is no exception, when debug is enabled. Contributed by Nandakumar. 2017-09-11 14:48:07 -07:00
Lei Xu
de9994bd89 Revert "HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (lei)" 
This reverts commit 3e6d0ca2b2.
 2017-09-11 10:06:05 -07:00
Arpit Agarwal
aa4b6fbe75 HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak. 2017-09-09 08:40:53 -07:00
Xiao Chen
b3a4d7d2a0 HDFS-12400. Provide a way for NN to drain the local key cache before re-encryption. 2017-09-07 20:51:02 -07:00
Lei Xu
3e6d0ca2b2 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (lei) 2017-09-07 18:01:37 -07:00
Manoj Govindassamy
3b3be355b3 HDFS-12404. Rename hdfs config authorization.provider.bypass.users to attributes.provider.bypass.users. 2017-09-07 17:20:42 -07:00
Andrew Wang
4e50dc976a HDFS-12218. Addendum. Rename split EC / replicated block metrics in BlockManager. 2017-09-07 16:57:19 -07:00
Andrew Wang
40c2f31f8d HDFS-12218. Rename split EC / replicated block metrics in BlockManager. 2017-09-07 16:56:35 -07:00
Xiao Chen
52b894db33 HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 2017-09-07 16:30:12 -07:00
Yongjun Zhang
d77ed238a9 HDFS-12357. Let NameNode to bypass external attribute provider for configured users. Contributed by Yongjun Zhang, Arun Suresh. 2017-09-07 09:50:36 -07:00
Arpit Agarwal
5ff74167dd HDFS-12376. Enable JournalNode Sync by default. Contributed by Hanisha Koneru. 2017-09-07 08:18:28 -07:00
Kai Zheng
2adf8bed71 HDFS-12402. Refactor ErasureCodingPolicyManager and related codes. Contributed by Sammi Chen 2017-09-07 20:38:23 +08:00
Chris Douglas
1f3bc63e67 HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov 2017-09-05 23:51:51 -07:00
Kai Zheng
d7f27043ce HDFS-12392. Writing striped file failed due to different cell size. Contributed by Sammi Chen 2017-09-06 13:29:52 +08:00
Andrew Wang
d4035d42f0 HDFS-12377. Refactor TestReadStripedFileWithDecoding to avoid test timeouts. 2017-09-05 16:33:29 -07:00
Andrew Wang
ccd2ac60ec HDFS-11882. Precisely calculate acked length of striped block groups in updatePipeline. 2017-09-05 14:16:03 -07:00
Xiao Chen
0ba8ff4b77 HDFS-12359. Re-encryption should operate with minimum KMS ACL requirements. 2017-09-05 10:08:27 -07:00
Xiao Chen
633c1ea455 HDFS-12383. Re-encryption updater should handle canceled tasks better. 2017-09-03 19:50:41 -07:00
Andrew Wang
0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Xiao Chen
1b3b9938cf HDFS-12300. Audit-log delegation token related operations. 2017-08-31 23:20:01 -07:00
Mingliang Liu
1fbb662c70 HDFS-12363. Possible NPE in BlockManager$StorageInfoDefragmenter#scanAndCompactStorages. Contributed by Xiao Chen 2017-08-31 22:36:56 -07:00
Arpit Agarwal
fd66a243bf HDFS-12356. Unit test for JournalNode sync during Rolling Upgrade. Contributed by Hanisha Koneru. 2017-08-30 10:29:42 -07:00
Rakesh Radhakrishnan
200b11368d HDFS-12258. ec -listPolicies should list all policies in system, no matter it's enabled or disabled. Contributed by Wei Zhou. 2017-08-30 12:58:56 +05:30
Xiao Chen
f3661fd08e HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 2017-08-29 21:30:02 -07:00
Yongjun Zhang
cf93d60d3f HDFS-12191. Provide option to not capture the accessTime change of a file to snapshot if no other modification has been done to this file. Contributed by Yongjun Zhang. 2017-08-29 16:57:03 -07:00
Manoj Govindassamy
63fc1b0b6d HDFS-11912. Snapshot functionality test along with randomized file IO operations. (George Huang via Manoj Govindassamy) 2017-08-29 11:27:47 -07:00
Arpit Agarwal
a1e3f84afe HDFS-12293. DataNode should log file name on disk error. Contributed by Ajay Kumar. 2017-08-28 15:49:59 -07:00
Arpit Agarwal
077a5eed9f HDFS-12358. Handle IOException when transferring edit log to Journal current dir through JN sync. Contributed by Hanisha Koneru. 2017-08-26 22:47:55 -07:00
Brahma Reddy Battula
bb6a3c8330 HDFS-12248. SNN will not upload fsimage on IOE and Interrupted exceptions. (Brahma Reddy Battula) 2017-08-26 13:47:12 +08:00
Manoj Govindassamy
b89ffcff36 HDFS-11986. Dfsadmin should report erasure coding related information separately. 2017-08-25 17:21:56 -07:00
Lei Xu
36bada3032 HDFS-12215. DataNode.transferBlock does not create its daemon in the xceiver thread group. (Lei Xu) 2017-08-25 16:01:14 -07:00
Andrew Wang
f29a0fc288 HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou. 2017-08-25 14:14:23 -07:00
Kihwal Lee
8455d70756 HDFS-12299. Race Between update pipeline and DN Re-Registration 2017-08-25 14:49:29 -05:00
Arpit Agarwal
4b2c442d4e HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 10:41:52 -07:00
John Zhuge
3123895db0 HDFS-12318. Fix IOException condition for openInfo in DFSInputStream. Contributed by legend. 2017-08-24 00:26:37 -07:00
Xiao Chen
1000a2af04 HDFS-10899. Add functionality to re-encrypt EDEKs. 2017-08-23 17:06:16 -07:00
John Zhuge
b6bfb2fcb2 HDFS-11738. Hedged pread takes more time when block moved from initial locations. Contributed by Vinayakumar B. 2017-08-21 13:45:30 -07:00
Manoj Govindassamy
913760cb4f HDFS-11988. Verify HDFS Snapshots with open files captured are consistent across truncates and appends to current version file. 2017-08-21 11:08:38 -07:00
Manoj Govindassamy
4230872dd6 HDFS-12316. Verify HDFS snapshot deletion doesn't crash the ongoing file writes. 2017-08-17 16:23:48 -07:00
Andrew Wang
b298948897 HDFS-12072. Provide fairness between EC and non-EC recovery tasks. Contributed by Eddy Xu. 2017-08-17 15:26:11 -07:00
Andrew Wang
dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Andrew Wang
96b3a6b972 HDFS-11082. Provide replicated EC policy to replicate files. Contributed by SammiChen. 2017-08-16 22:17:06 -07:00
Akira Ajisaka
08aaa4b36f
HDFS-12269. Better to return a Map rather than HashMap in getErasureCodingCodecs. Contributed by Huafeng Wang. 2017-08-17 13:20:27 +09:00
Ravi Prakash
f34646d652 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 2017-08-15 15:44:59 -07:00
Wei-Chiu Chuang
e3ae3e2644 HDFS-12066. When Namenode is in safemode,may not allowed to remove an user's erasure coding policy. Contributed by lufei. 2017-08-15 07:41:43 -07:00
Wei-Chiu Chuang
1040bae6fc HDFS-12054. FSNamesystem#addErasureCodingPolicies should call checkNameNodeSafeMode() to ensure Namenode is not in safemode. Contributed by lufei. 2017-08-15 07:38:43 -07:00
Yiqun Lin
2e43c28e01 HDFS-11696. Fix warnings from Spotbugs in hadoop-hdfs. Contributed by Yiqun Lin. 2017-08-15 16:48:49 +08:00
Chris Douglas
645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
Anu Engineer
d72124a442 HDFS-12162. Update listStatus document to describe the behavior when the argument is a file. Contributed by Ajay Kumar. 2017-08-14 11:32:49 -07:00
Lei Xu
ce797a1706 HDFS-12221. Replace xcerces in XmlEditsVisitor. (Ajay Kumar via lei) 2017-08-14 10:27:47 -07:00
John Zhuge
8b242f09a6 HDFS-11303. Hedged read might hang infinitely if read data from all DN failed . Contributed by Chen Zhang, Wei-chiu Chuang, and John Zhuge. 2017-08-11 19:42:07 -07:00
Yiqun Lin
f13ca94954 HDFS-12287. Remove a no-longer applicable TODO comment in DatanodeManager. Contributed by Chen Liang. 2017-08-11 14:13:45 +08:00
John Zhuge
312e57b954 HDFS-11957. Enable POSIX ACL inheritance by default. Contributed by John Zhuge. 2017-08-10 10:30:47 -07:00
Kihwal Lee
b5c02f95b5 HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 2017-08-09 16:46:05 -05:00
Kihwal Lee
69afa26f19 HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by inayakumar B. 2017-08-09 09:03:51 -05:00
Wei-Chiu Chuang
9a3c2379ef HDFS-12182. BlockManager.metaSave does not distinguish between "under replicated" and "missing" blocks. Contributed by Wellington Chevreuil. 2017-08-08 23:44:18 -07:00
Kai Zheng
a53b8b6fdc HDFS-11975. Provide a system-default EC policy. Contributed by Huichun Lu 2017-08-09 10:12:58 +08:00
Haohui Mai
1db4788b7d HADOOP-14598. Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory. Contributed by Steve Loughran. 2017-08-08 16:33:18 -07:00
Haohui Mai
71b8dda4f6 HDFS-10326. Disable setting tcp socket send/receive buffers for write pipelines. Contributed by Daryn Sharp. 2017-08-08 14:58:16 -07:00
Kai Zheng
0b67436068 HDFS-12306. Add audit log for some erasure coding operations. Contributed by Huafeng Wang 2017-08-07 19:30:10 +08:00
Arpit Agarwal
bbc6d254c8 HDFS-12224. Add tests to TestJournalNodeSync for sync after JN downtime. Contributed by Hanisha Koneru. 2017-08-04 12:51:33 -07:00
Lei Xu
fe3341786a HDFS-12251. Add document for StreamCapabilities. (Lei (Eddy) Xu) 2017-08-04 11:21:58 -07:00
Andrew Wang
f4c6b00a9f HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:44:51 -07:00
Manoj Govindassamy
79df1e750e HDFS-9388. Decommission related code to support Maintenance State for datanodes. 2017-08-02 14:22:46 -07:00
Chris Douglas
12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Sean Mackrory
1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Haohui Mai
6814324c33 HDFS-12107. FsDatasetImpl#removeVolumes floods the logs when removing the volume. Contributed by Kelvin Chu. 2017-08-01 18:34:07 -07:00
Manoj Govindassamy
52d7bafcf4 HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted. 2017-08-01 16:28:20 -07:00
Sean Mackrory
7774759830 HDFS-12151. Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes. 2017-08-01 14:34:25 -06:00
Akira Ajisaka
b38a1eea8e
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor. 2017-08-01 15:15:43 +09:00
Weiwei Yang
ea568123fa HDFS-12154. Incorrect javadoc description in StorageLocationChecker#check. Contributed by Nandakumar. 2017-07-31 17:02:44 -07:00
Arpit Agarwal
3e23415a92 HDFS-12082. BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured. Contributed by Weiwei Yang. 2017-07-31 11:33:55 -07:00
Andrew Wang
480c8db40c HDFS-12206. Rename the split EC / replicated block metrics. 2017-07-28 11:24:44 -07:00
Lei Xu
77791e4c36 HDFS-12044. Mismatch between BlockManager.maxReplicationStreams and ErasureCodingWorker.stripedReconstructionPool pool size causes slow and bursty recovery. (Contributed by Lei (Eddy) Xu) 2017-07-28 10:50:49 -07:00
Wei-Chiu Chuang
369f731264 HDFS-12062. removeErasureCodingPolicy needs super user permission. Contributed by Wei-Chiu Chuang. 2017-07-28 00:51:03 -07:00
Yongjun Zhang
c6330f22a5 HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 2017-07-27 16:48:24 -07:00
Jitendra Pandey
e3c7300225 HDFS-2319. Add test cases for FSshell -stat. Contributed by XieXianshan and Bharat Viswanadham. 2017-07-27 13:23:15 -07:00
Brahma Reddy Battula
c4a85c694f HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 2017-07-27 12:02:57 -07:00
Kihwal Lee
a68b5b31cf HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 2017-07-25 11:03:49 -05:00
Brahma Reddy Battula
6d983cca52 HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-25 23:51:53 +08:00
Kihwal Lee
1a79dcfc45 HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 2017-07-25 10:28:57 -05:00
Akira Ajisaka
770cc46228
HDFS-12176. dfsadmin shows DFS Used%: NaN% if the cluster has zero block. Contributed by Weiwei Yang. 2017-07-24 16:24:23 +09:00
Kihwal Lee
8e3a992ecc HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 2017-07-21 09:14:19 -05:00
Konstantin V Shvachko
8c2c812832 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 2017-07-20 14:47:25 -07:00
Anu Engineer
413b23eb04 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 2017-07-19 10:29:06 -07:00
Brahma Reddy Battula
04ff412dab HDFS-12133. Correct ContentSummaryComputationContext Logger class name.. Contributed by Surendra Singh Lilhore. 2017-07-19 23:43:10 +08:00
Brahma Reddy Battula
f8cd55fe33 HDFS-12067. Correct dfsadmin commands usage message to reflects IPC port. Contributed by steven-wugang. 2017-07-19 23:21:43 +08:00
Tsz-Wo Nicholas Sze
ed27f2b2cc HDFS-12138. Remove redundant 'public' modifiers from BlockCollection. Contributed by Chen Liang 2017-07-17 13:54:16 -07:00
Brahma Reddy Battula
b778887af5 HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 2017-07-15 10:38:31 +08:00
Xiao Chen
0a6d5c0cf1 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-07-14 22:22:53 -07:00
Tsz-Wo Nicholas Sze
f413ee33df HDFS-12130. Optimizing permission check for getContentSummary. Contributed by Chen Liang 2017-07-14 14:35:51 -07:00
Tsz-Wo Nicholas Sze
a1f12bb543 Revert "HDFS-12130. Optimizing permission check for getContentSummary." to fix commit message. 
This reverts commit a29fe100b3.
 2017-07-14 14:34:01 -07:00
Kihwal Lee
e7d187a1b6 HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 2017-07-14 16:07:17 -05:00
Kihwal Lee
8d86a93915 HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 2017-07-14 15:41:43 -05:00
Tsz-Wo Nicholas Sze
a29fe100b3 HDFS-12130. Optimizing permission check for getContentSummary. 2017-07-14 13:36:27 -07:00
Xiaoyu Yao
e15e2713e1 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu. 2017-07-12 16:26:19 -07:00
Akira Ajisaka
9144fd9e9b
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 2017-07-12 11:35:50 +09:00
Yiqun Lin
3a7f02b815 HDFS-12085. Reconfigure namenode heartbeat interval fails if the interval was set with time unit. Contributed by Weiwei Yang. 2017-07-11 20:40:11 +08:00
Andrew Wang
f10864a820 Add release notes, changes, jdiff for 3.0.0-alpha4 2017-07-07 11:01:59 -07:00
Akira Ajisaka
8fc5dcc2a1
Revert "HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He." 
This reverts commit 82cb2a6497.
 2017-07-08 02:53:18 +09:00
Akira Ajisaka
82cb2a6497
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He. 2017-07-07 15:00:47 +09:00
Akira Ajisaka
186650d21d
HDFS-12078. Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml. Contributed by Weiwei Yang. 2017-07-04 14:51:52 +09:00
Akira Ajisaka
b0560e0624
HDFS-12079. Description of dfs.block.invalidate.limit is incorrect in hdfs-default.xml. Contributed by Weiwei Yang. 2017-07-04 14:02:14 +09:00
Wei-Chiu Chuang
bcba844d11 HDFS-12042. Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption. Contributed by Misha Dmitriev. 2017-06-30 10:28:01 -07:00
Arpit Agarwal
6a9dc5f44b HDFS-12043. Add counters for block re-replication. Contributed by Chen Liang. 2017-06-30 10:20:12 -07:00
Arpit Agarwal
a2f0cbd92f Revert "HDFS-12043. Add counters for block re-replication. Contributed by Chen Liang." 
Accidentally committed the wrong patch version, reverting to fix that.

This reverts commit 900221f95e.
 2017-06-30 10:19:27 -07:00
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Arpit Agarwal
900221f95e HDFS-12043. Add counters for block re-replication. Contributed by Chen Liang. 2017-06-29 17:15:13 -07:00
Andrew Wang
441378e7e4 Revert "HDFS-11696. Fix warnings from Spotbugs in hadoop-hdfs. Contributed by Yiqun Lin." 
This reverts commit 89a8edc014.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
 2017-06-29 13:54:16 -07:00
Wei-Chiu Chuang
16c8dbde57 HDFS-11881. NameNode consumes a lot of memory for snapshot diff report generation. Contributed by Manoj Govindassamy. 2017-06-29 06:38:41 -07:00
Wei-Chiu Chuang
e9d8bdfdf5 HADOOP-14297. Update the documentation about the new ec codecs config keys. Contributed by Kai Sasaki. 2017-06-28 13:53:54 -07:00
Lei Xu
d6df0fdbbd Add -E option in 'ls' to list erasure coding policy of each file and directory if applicable. Contributed by luhuichun via lei. 2017-06-28 13:47:23 -07:00
Wei-Chiu Chuang
f99b6d19de HDFS-11870. Add CLI cmd to enable/disable an erasure code policy. Contributed by lufei. 2017-06-28 10:54:25 -07:00
Akira Ajisaka
07defa4c09
HDFS-12040. TestFsDatasetImpl.testCleanShutdownOfVolume fails. Contributed by hu xiaodong. 2017-06-27 20:49:26 +09:00
Anu Engineer
8641a2c08b HDFS-12045. Add log when Diskbalancer volume is transient storage type. Contributed by steven-wugang. 2017-06-27 00:39:47 -07:00
Lei Xu
144753e87f HDFS-12033. DatanodeManager picking EC recovery tasks should also consider the number of regular replication tasks. Contributed by Lei (Eddy) Xu. 2017-06-26 15:43:50 -07:00
Andrew Wang
2c367b464c HDFS-11956. Do not require a storage ID or target storage IDs when writing a block. Contributed by Ewan Higgs. 2017-06-26 11:20:07 -07:00
Andrew Wang
06c8ca3bb3 HDFS-12032. Inaccurate comment on DatanodeDescriptor#getNumberOfBlocksToBeErasureCoded. 2017-06-26 10:54:01 -07:00
Brahma Reddy Battula
abdea26280 HDFS-12024. Fix typo's in FsDatasetImpl.java. Contributed by Yasen liu. 2017-06-23 22:56:28 +08:00
Xiao Chen
68dc7c2405 HDFS-12009. Accept human-friendly units in dfsadmin -setBalancerBandwidth and -setQuota. Contributed by Andrew Wang. 2017-06-22 19:57:54 -07:00
Arpit Agarwal
6d116ffad2 HDFS-11789. Maintain Short-Circuit Read Statistics. Contributed by Hanisha Koneru. 2017-06-22 13:35:56 -07:00
Vinayakumar B
8dbd53ef9f HDFS-11067. DFS#listStatusIterator(..) should throw FileNotFoundException if the directory deleted before fetching next batch of entries. Contributed by Vinayakumar B. 2017-06-22 17:37:08 +05:30
Arpit Agarwal
9ae9467f92 HDFS-11998. Enable DFSNetworkTopology as default. Contributed by Chen Liang. 2017-06-21 22:01:41 -07:00
Zhe Zhang
c22cf00442 Confusion/name conflict between NameNodeActivity#BlockReportNumOps and RpcDetailedActivity#BlockReportNumOps. Contributed by Erik Krogen. 2017-06-21 16:34:19 -07:00
Kai Zheng
5db3f98468 HDFS-11606. Add CLI cmd to remove an erasure code policy. Contributed by Tim Yao 2017-06-21 14:01:55 +08:00
Akira Ajisaka
1a598479a9
HDFS-11992. Replace commons-logging APIs with slf4j in FsDatasetImpl. Contributed by hu xiaodong. 2017-06-21 11:19:48 +09:00
Lei Xu
45ff4d38e6 HDFS-11647. Add -E option in hdfs "count" command to show erasure policy summarization. Contributed by luhuichun. 2017-06-20 11:55:09 -07:00
Wei-Chiu Chuang
2b654a493c HDFS-11978. Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs' in doc. Contributed by wenxin he. 2017-06-20 09:36:55 -07:00
Kihwal Lee
8c0769dee4 HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee. 2017-06-20 09:11:23 -05:00
Brahma Reddy Battula
2e9daa2e27 HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 2017-06-20 20:44:31 +08:00
Akira Ajisaka
2c769167db
HDFS-11345. Document the configuration key for FSNamesystem lock fairness. Contributed by Erik Krogen. 2017-06-20 16:35:54 +09:00
Brahma Reddy Battula
099cbb427a HDFS-11890. Handle NPE in BlockRecoveryWorker when DN is getting shoutdown. Contributed by Surendra Singh Lilhore. 2017-06-20 11:03:56 +08:00
Masatake Iwasaki
d954a64730 HDFS-11995. HDFS Architecture documentation incorrectly describes writing to a local temporary file. Contributed by Nandakumar. 2017-06-20 08:07:42 +09:00
Lei Xu
73fb75017e HDFS-11916. Extend TestErasureCodingPolicies/TestErasureCodingPolicyWithSnapshot with a random EC policy. Contributed by Takanobu Asanuma. 2017-06-19 10:25:20 -07:00
Lei Xu
6460df21a0 HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 2017-06-16 17:24:00 -07:00
Andrew Wang
fb68980959 HDFS-10480. Add an admin command to list currently open files. Contributed by Manoj Govindassamy. 2017-06-15 14:46:55 -07:00
Lei Xu
3f5108723c HDFS-11682. TestBalancer.testBalancerWithStripedFile is flaky. (lei) 2017-06-15 11:04:50 -07:00
Lei Xu
5e7cfdca7b HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 2017-06-15 10:59:24 -07:00
Akira Ajisaka
315f07700d
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 2017-06-15 13:46:20 +09:00
Lei Xu
999c8fcbef HDFS-10999. Introduce separate stats for Replicated and Erasure Coded Blocks apart from the current Aggregated stats. (Manoj Govindassamy via lei) 2017-06-14 10:44:19 -07:00
Arpit Agarwal
8633ef8e10 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-12 20:43:43 -07:00
Tsz-Wo Nicholas Sze
bec79ca249 HDFS-11947. When constructing a thread name, BPOfferService may print a bogus warning message. Contributed by Weiwei Yang 2017-06-13 09:45:10 +08:00
Arpit Agarwal
b3d3ede91a HDFS-11967. TestJMXGet fails occasionally. Contributed by Arpit Agarwal. 2017-06-12 16:03:47 -07:00
Arpit Agarwal
3f0a727f75 HDFS-11907. Add metric for time taken by NameNode resource check. Contributed by Chen Liang. 2017-06-12 15:18:38 -07:00
Mingliang Liu
1a33c9d589 HDFS-11945. Internal lease recovery may not be retried for a long time. Contributed by Kihwal Lee 2017-06-08 14:36:47 -07:00
Yiqun Lin
5ec71639cf HDFS-11929. Document missing processor of hdfs oiv_legacy command. Contributed by LiXin Ge. 2017-06-07 13:42:13 +08:00
Vinayakumar B
70fc6746b3 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:55:53 +05:30
Brahma Reddy Battula
1869e1771c HDFS-11711. DN should not delete the block On "Too many open files" Exception. Contributed by Brahma Reddy Battula. 2017-06-07 13:05:33 +08:00
Andrew Wang
c31cb879a3 HDFS-11840. Log HDFS Mover exception message of exit to its own log. Contributed by LiXin Ge. 2017-06-06 14:57:48 -07:00
Arpit Agarwal
867903db2b HDFS-11932. BPServiceActor thread name is not correctly set. Contributed by Chen Liang. 2017-06-06 13:51:02 -07:00
Brahma Reddy Battula
19ef3a81f8 HADOOP-14485. Redundant 'final' modifier in try-with-resources statement. Contributed by wenxin he. 2017-06-06 23:11:47 +08:00
Yongjun Zhang
6a28a2b914 HDFS-11914. Add more diagnosis info for fsimage transfer failure. Contributed by Yongjun Zhang. 2017-06-05 16:31:03 -07:00
Arpit Agarwal
835560983e HDFS-11928. Segment overflow in FileDistributionCalculator. Contributed by LiXin Ge. 2017-06-05 13:21:22 -07:00
Kihwal Lee
e4e203e080 HDFS-10816. TestComputeInvalidateWork#testDatanodeReRegistration fails due to race between test and replication monitor. Contributed by Eric Badger. 2017-06-05 15:18:27 -05:00
Yiqun Lin
367da9b7b8 HDFS-11899. ASF License warnings generated intermittently in trunk. Contributed by Yiqun Lin. 2017-06-03 22:07:24 +08:00
Brahma Reddy Battula
056cc72885 HADOOP-14436. Remove the redundant colon in ViewFs.md. Contributed by maobaolong. 2017-06-02 22:39:10 +08:00
Yiqun Lin
8d9084eb62 HDFS-11359. DFSAdmin report command supports displaying maintenance state datanodes. Contributed by Yiqun Lin. 2017-06-02 12:48:30 +08:00
Xiao Chen
219f4c199e HDFS-11904. Reuse iip in unprotectedRemoveXAttrs calls. 2017-06-01 14:13:57 -07:00
Xiao Chen
6a3fc685a9 HDFS-11741. Long running balancer may fail due to expired DataEncryptionKey. Contributed by Wei-Chiu Chuang and Xiao Chen. 2017-06-01 14:05:37 -07:00
Brahma Reddy Battula
ff3103556b HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 00:28:33 +08:00
Brahma Reddy Battula
0dcf843c00 HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula. 2017-06-01 22:29:29 +08:00
Kihwal Lee
1543d0f5be HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-05-31 10:55:03 -05:00
Brahma Reddy Battula
13de636b40 HDFS-11901. Modifier 'static' is redundant for inner enums. Contributed by ZhangBing Lin. 2017-05-31 23:09:08 +08:00
Lei Xu
91d6fe151f HDFS-11659. TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Contributed by Lei (Eddy) Xu. 2017-05-30 11:10:12 -07:00
Akira Ajisaka
a7f085d6bf
HDFS-11832. Switch leftover logs to slf4j format in BlockManager.java. Contributed by Hui Xu and Chen Liang. 2017-05-29 17:30:23 +09:00
Yiqun Lin
31058b243e HDFS-11446. TestMaintenanceState#testWithNNAndDNRestart fails intermittently. Contributed by Yiqun Lin. 2017-05-28 11:23:32 +08:00
Andrew Wang
2cd612ba8e Update 3.0.0-alpha3 changes, release notes, jdiff. 2017-05-26 14:14:38 -07:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Inigo Goiri
303c8dc9b6 Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 2017-05-26 13:16:01 -07:00
Kihwal Lee
2b5ad48762 HDFS-11817. A faulty node can cause a lease leak and NPE on accessing data. Contributed by Kihwal Lee. 2017-05-25 17:17:38 -05:00
Arpit Agarwal
1161563136 HDFS-11879. Fix JN sync interval in case of exception. Contributed by Hanisha Koneru. 2017-05-25 14:01:53 -07:00
Kihwal Lee
29b7df960f HDFS-11856. Ability to re-add Upgrading Nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-25 13:05:23 -05:00
Arpit Agarwal
4fb41b31db HDFS-11878. Fix journal missing log httpServerUrl address in JournalNodeSyncer. Contributed by Hanisha Koneru. 2017-05-25 10:42:24 -07:00
Brahma Reddy Battula
2e41f8803d HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 2017-05-25 22:35:10 +08:00