Commit Graph

3272 Commits

Author SHA1 Message Date
Brahma Reddy Battula
55d5993a8e HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 2016-07-27 10:26:42 +05:30
Wei-Chiu Chuang
d84ab8a578 HDFS-10598. DiskBalancer does not execute multi-steps plan. Contributed by Lei (Eddy) Xu. 2016-07-26 16:01:50 -07:00
Arpit Agarwal
d2cf8b54c5 HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu) 2016-07-26 12:27:46 -07:00
Steve Loughran
2aa18e1925 Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu."
This reverts commit 7cac7655fd.
2016-07-26 14:30:02 +01:00
Brahma Reddy Battula
7cac7655fd HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 2016-07-26 10:54:24 +05:30
Vinitha Reddy Gankidi
85a20508bd HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi. 2016-07-25 18:50:59 -07:00
Kai Zheng
b79ba4fe96 HDFS-10651. Clean up some configuration related codes about legacy block reader. Contributed by Youwei Wang 2016-07-23 17:45:33 +08:00
Arpit Agarwal
c6e3a0020f HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R) 2016-07-22 12:33:50 -07:00
Xiao Chen
132deb4cac HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu. 2016-07-21 16:41:02 -07:00
Akira Ajisaka
fcde6940e0 HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor. 2016-07-21 11:26:08 -07:00
Jing Zhao
bd3dcf46e2 HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp. 2016-07-21 11:14:39 -07:00
Akira Ajisaka
38128baff4 HDFS-10425. Clean up NNStorage and TestSaveNamespace. Contributed by Andras Bokor. 2016-07-20 10:44:11 -07:00
Akira Ajisaka
fc570b55b9 HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin. 2016-07-19 13:49:24 -07:00
Akira Ajisaka
5537c6b234 HADOOP-13351. TestDFSClientSocketSize buffer size tests are flaky. Contributed by Aaron Fabbri and Mingliang Liu. 2016-07-14 14:40:58 -07:00
Uma Maheswara Rao G
438b7c5935 HDFS-10590: Fix TestReconstructStripedBlocks.testCountLiveReplicas test failures. Contributed by Rakesh R 2016-07-13 00:07:11 -07:00
Zhe Zhang
087290e6b1 HDFS-10544. Balancer doesn't work with IPFailoverProxyProvider. 2016-07-12 23:18:37 -07:00
Jitendra Pandey
c447efebdb HDFS-10579. HDFS web interfaces lack configs for X-FRAME-OPTIONS protection. Contributed by Anu Engineer. 2016-07-11 14:55:33 -07:00
Yongjun Zhang
da6f1b88dd HDFS-10512. VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks. Contributed by Wei-Chiu Chuang and Yiqun Lin. 2016-07-08 19:40:44 -07:00
Akira Ajisaka
932aed64d7 HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin. 2016-07-08 14:39:11 -07:00
Jitendra Pandey
6e597600f7 HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu. 2016-07-07 14:58:19 -07:00
Uma Maheswara Rao G
ab092c56c2 HDFS-10592:Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning. Contributed by Rakesh R 2016-07-07 13:52:35 -07:00
Kai Zheng
8b281bce85 HDFS-10548. Remove the long deprecated BlockReaderRemote. Contributed by Kai Zheng 2016-07-03 11:56:23 +08:00
Colin Patrick Mccabe
e4a2545620 HDFS-9805. Add server-side configuration for enabling TCP_NODELAY for DataTransferProtocol and default it to true (Gary Helmling via cmccabe) 2016-06-29 12:41:42 -07:00
Andrew Wang
c0829f4493 HDFS-6434. Default permission for creating file should be 644 for WebHdfs/HttpFS. Contributed by Wellington Chevreuil. 2016-06-28 13:55:26 -07:00
Vinayakumar B
2a0082c51d HDFS-10440. Improve DataNode web UI (Contributed by Weiwei Yang) 2016-06-28 16:49:39 +05:30
Vinayakumar B
73615a789d HDFS-10536. Standby NN can not trigger log roll after EditLogTailer thread failed 3 times in EditLogTailer.triggerActiveLogRoll method (Contributed by XingFeng Shen 2016-06-26 15:45:32 +05:30
Zhe Zhang
22be604fa3 Revert "HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki."
This reverts commit 0424056a77.
2016-06-24 09:36:03 -07:00
Zhe Zhang
0424056a77 HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki. 2016-06-24 08:58:29 -07:00
Anu Engineer
b2584bee45 HDFS-10571. TestDiskBalancerCommand#testPlanNode failed with IllegalArgumentException. Contributed by Xiaobing Zhou. 2016-06-23 18:27:02 -07:00
Anu Engineer
8a6e354122 HDFS-10552. DiskBalancer "-query" results in NPE if no plan for the node. Contributed by Anu Engineer. 2016-06-23 18:27:02 -07:00
Anu Engineer
e8de28181a HDFS-10551. o.a.h.h.s.diskbalancer.command.Command does not actually verify options as expected. Contributed by Anu Engineer. 2016-06-23 18:27:02 -07:00
Arpit Agarwal
66fa34c839 HDFS-10557. Fix handling of the -fs Generic option. (Arpit Agarwal) 2016-06-23 18:27:01 -07:00
Anu Engineer
7b23ad1ef7 HDFS-10550. DiskBalancer: fix issue of order dependency in iteration in ReportCommand test. Contributed by Xiaobing Zhou. 2016-06-23 18:27:01 -07:00
Anu Engineer
3225c24e0e HDFS-10520. DiskBalancer: Fix Checkstyle issues in test code. Contributed by Anu Engineer. 2016-06-23 18:27:00 -07:00
Anu Engineer
7e2be5c4a0 HDFS-10518. DiskBalancer: Pretty-print json in Query command. Contributed by Anu Engineer. 2016-06-23 18:21:08 -07:00
Anu Engineer
b502102bb1 HDFS-9461. DiskBalancer: Add Report Command. Contributed by Xiaobing Zhou. 2016-06-23 18:21:08 -07:00
Anu Engineer
64ccb232cc HDFS-10478. DiskBalancer: resolve volume path names. Contributed by Anu Engineer. 2016-06-23 18:21:08 -07:00
Anu Engineer
5df2d2b8fd HDFS-10399. DiskBalancer: Add JMX for DiskBalancer. Contributed by Anu Engineer. 2016-06-23 18:21:08 -07:00
Anu Engineer
75882ec0b0 HDFS-9545: DiskBalancer: Add Plan Command. Contributed by Anu Engineer. 2016-06-23 18:21:08 -07:00
Anu Engineer
1594b472bb HDFS-9543. DiskBalancer: Add Data mover. Contributed by Anu Engineer. 2016-06-23 18:20:24 -07:00
Anu Engineer
7820737cfa HDFS-9735. DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer. Contributed by Anu Engineer. 2016-06-23 18:20:24 -07:00
Anu Engineer
050677077b HDFS-9720. DiskBalancer : Add configuration parameters. Contributed by Anu Engineer. 2016-06-23 18:18:48 -07:00
Anu Engineer
6c606bf5c8 HDFS-9709. DiskBalancer : Add tests for disk balancer using a Mock Mover class. Contributed by Anu Engineer. 2016-06-23 18:18:48 -07:00
Arpit Agarwal
75a711a2d5 HDFS-9703. DiskBalancer: getBandwidth implementation. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
918722bdd2 HDFS-9702. DiskBalancer: getVolumeMap implementation. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
9847640603 HDFS-9683. DiskBalancer: Add cancelPlan implementation. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
e646c2eb50 HDFS-9681. DiskBalancer: Add QueryPlan implementation. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
2b1b2faf76 HDFS-9671. DiskBalancer: SubmitPlan implementation. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
66f0bb646d HDFS-9647. DiskBalancer: Add getRuntimeSettings. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
96fe685b7a HDFS-9645. DiskBalancer: Add Query RPC. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
0501d430e2 HDFS-9595. DiskBalancer: Add cancelPlan RPC. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
7100c0da35 HDFS-9588. DiskBalancer: Add submitDiskbalancer RPC. (Anu Engineer via Arpit Agarwal) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
5724a10316 HDFS-9469. DiskBalancer: Add Planner. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Tsz-Wo Nicholas Sze
30c6ebd699 HDFS-9449. DiskBalancer: Add connectors. Contributed by Anu Engineer 2016-06-23 18:18:48 -07:00
Tsz-Wo Nicholas Sze
91a5c48143 HDFS-9420. Add DataModels for DiskBalancer. Contributed by Anu Engineer 2016-06-23 18:18:48 -07:00
Xiaoyu Yao
dca298d79e HDFS-10469. Add number of active xceivers to datanode metrics. Contributed by Hanisha Koneru. 2016-06-23 14:13:39 -07:00
Kai Zheng
e6cb07520f HDFS-10460. Recompute block checksum for a particular range less than file size on the fly by reconstructing missed block. Contributed by Rakesh R 2016-06-24 17:39:32 +08:00
Uma Maheswara Rao G
17eae9ebb3 HDFS-10473: Allow only suitable storage policies to be set on striped files. Contributed by Uma Maheswara Rao G 2016-06-22 11:17:43 -07:00
Akira Ajisaka
d433b16ce6 HDFS-7597. DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping. Contributed by Daryn Sharp, Bob Hansen, and Xiao Chen. 2016-06-22 11:00:16 +09:00
Brahma Reddy Battula
f2ac132d6a HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula) 2016-06-21 15:42:28 +05:30
Jitendra Pandey
7adc4d1769 HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey. 2016-06-20 17:22:55 -07:00
Colin Patrick Mccabe
5f6b4157a4 HDFS-10328. Add per-cache-pool default replication num configuration (xupeng via cmccabe) 2016-06-20 10:42:50 -07:00
Arpit Agarwal
0319d73c3b HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (Arpit Agarwal) 2016-06-18 16:16:54 -07:00
Akira Ajisaka
09e82acaf9 HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang. 2016-06-17 16:20:38 +09:00
Vinayakumar B
cc6c265171 HDFS-10256. Use GenericTestUtils.getTestDir method in tests for temporary directories (Contributed by Vinayakumar B) 2016-06-16 16:47:06 +05:30
Colin Patrick Mccabe
2449db507d HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe) 2016-06-15 22:35:19 -07:00
Ming Ma
b48f27e794 HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma) 2016-06-15 22:00:52 -07:00
Ming Ma
7d521a29ee HDFS-9016. Display upgrade domain information in fsck. (mingma) 2016-06-14 20:05:50 -07:00
Andrew Wang
709a814fe0 HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen. 2016-06-13 11:50:19 -07:00
Ravi Prakash
ae047655f4 HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak) 2016-06-08 13:44:22 -07:00
Andrew Wang
1ee9ea0026 Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou. 2016-06-08 12:58:56 -07:00
Jing Zhao
be34e85e68 HDFS-10468. HDFS read ends up ignoring an interrupt. Contributed by Jing Zhao 2016-06-07 10:48:21 -07:00
Tsz-Wo Nicholas Sze
35f255b03b Revert "Revert "HADOOP-13226 Support async call retry and failover.""
This reverts commit 5360da8bd9.
2016-06-06 16:31:43 +08:00
Tsz-Wo Nicholas Sze
7e7b1ae037 Revert "Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.""
This reverts commit 8cf47d8589.
2016-06-06 16:31:38 +08:00
Tsz-Wo Nicholas Sze
db41e6d285 Revert "Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou""
This reverts commit 5ee5912ebd.
2016-06-06 16:31:34 +08:00
Tsz-Wo Nicholas Sze
b3d81f38da Revert "Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou""
This reverts commit b82c74b910.
2016-06-06 16:31:30 +08:00
Tsz-Wo Nicholas Sze
cba9a01889 Revert "Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou""
This reverts commit f23d5dfc60.
2016-06-06 16:29:38 +08:00
Tsz-Wo Nicholas Sze
aa20fa150d Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou""
This reverts commit 4d36b221a2.
2016-06-06 16:28:47 +08:00
Tsz-Wo Nicholas Sze
eded3d109e Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou""
This reverts commit 106234d873.
2016-06-06 16:28:21 +08:00
Andrew Wang
106234d873 Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"
This reverts commit fc94810d3f.
2016-06-03 18:09:55 -07:00
Andrew Wang
4d36b221a2 Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"
This reverts commit 1b9f18623a.
2016-06-03 18:09:18 -07:00
Andrew Wang
f23d5dfc60 Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou"
This reverts commit 7251bb922b.
2016-06-03 18:09:17 -07:00
Andrew Wang
b82c74b910 Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou"
This reverts commit 02d4e478a3.
2016-06-03 18:09:14 -07:00
Andrew Wang
5ee5912ebd Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou"
This reverts commit f4b9bcd87c.
2016-06-03 18:09:13 -07:00
Andrew Wang
8cf47d8589 Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou."
This reverts commit 21890c4239.
2016-06-03 18:09:12 -07:00
Andrew Wang
5360da8bd9 Revert "HADOOP-13226 Support async call retry and failover."
This reverts commit 83f2f78c11.
2016-06-03 18:09:09 -07:00
Arpit Agarwal
97e2449477 HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (Contributed by Akira Ajisaka) 2016-06-02 13:14:45 -07:00
Akira Ajisaka
1df6f5735c HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin. 2016-06-03 04:10:32 +09:00
Masatake Iwasaki
aadb77e412 HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula. 2016-06-02 20:55:04 +09:00
Akira Ajisaka
69555fca06 HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki. 2016-06-02 18:52:47 +09:00
Kai Zheng
d749cf65e1 HDFS-9833. Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data. Contributed by Rakesh R. 2016-06-02 12:56:21 +08:00
Colin Patrick Mccabe
8ceb06e239 HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe) 2016-05-31 17:54:34 -07:00
Colin Patrick Mccabe
c7921c9bdd HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe) 2016-05-31 17:37:52 -07:00
Colin Patrick Mccabe
29d6cadc52 HDFS-10415. TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called (Mingliang Liu via cmccabe) 2016-05-31 16:54:52 -07:00
Tsz-Wo Nicholas Sze
83f2f78c11 HADOOP-13226 Support async call retry and failover. 2016-05-31 16:30:11 -07:00
Kai Zheng
93d8a7f2a2 Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng"
This reverts commit a6c79f92d5.
2016-06-01 08:41:15 +08:00
Chris Nauroth
21890c4239 HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou. 2016-05-27 14:20:18 -07:00
Yongjun Zhang
5ea6fd85c7 HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang) 2016-05-27 10:02:02 -07:00
Kai Zheng
916140604f HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li 2016-05-28 14:23:39 +08:00
Karthik Kambatla
55c3e2de3d HDFS-10463. TestRollingFileSystemSinkWithHdfs needs some cleanup. (Daniel Templeton via kasha) 2016-05-26 14:24:31 -07:00