Colin Patrick Mccabe
46f1602e89
HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe)
2016-06-20 18:25:09 -07:00
Jitendra Pandey
7adc4d1769
HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey.
2016-06-20 17:22:55 -07:00
Colin Patrick Mccabe
5f6b4157a4
HDFS-10328. Add per-cache-pool default replication num configuration (xupeng via cmccabe)
2016-06-20 10:42:50 -07:00
Arpit Agarwal
0319d73c3b
HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (Arpit Agarwal)
2016-06-18 16:16:54 -07:00
Kihwal Lee
280069510b
HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang.
2016-06-17 11:05:44 -05:00
Akira Ajisaka
09e82acaf9
HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang.
2016-06-17 16:20:38 +09:00
Arpit Agarwal
e983eafb44
HDFS-10532. Typo in RollingUpgrade docs. (Contributed by Yiqun Lin)
2016-06-16 10:10:33 -07:00
Vinayakumar B
cc6c265171
HDFS-10256. Use GenericTestUtils.getTestDir method in tests for temporary directories (Contributed by Vinayakumar B)
2016-06-16 16:47:06 +05:30
Colin Patrick Mccabe
2ca73445f5
HDFS-10525. Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap (Xiao Chen via cmccabe)
2016-06-15 22:47:01 -07:00
Colin Patrick Mccabe
2449db507d
HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe)
2016-06-15 22:35:19 -07:00
Ming Ma
b48f27e794
HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma)
2016-06-15 22:00:52 -07:00
Ming Ma
7d521a29ee
HDFS-9016. Display upgrade domain information in fsck. (mingma)
2016-06-14 20:05:50 -07:00
Andrew Wang
709a814fe0
HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen.
2016-06-13 11:50:19 -07:00
Allen Wittenauer
d44f4745b4
HDFS-7987. Allow files / directories to be moved (Ravi Prakash via aw)
2016-06-10 09:02:28 -07:00
Ravi Prakash
ae047655f4
HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak)
2016-06-08 13:44:22 -07:00
Andrew Wang
1ee9ea0026
Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou.
2016-06-08 12:58:56 -07:00
Jing Zhao
be34e85e68
HDFS-10468. HDFS read ends up ignoring an interrupt. Contributed by Jing Zhao
2016-06-07 10:48:21 -07:00
Akira Ajisaka
e620530301
HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka)
2016-06-07 17:52:55 +09:00
Zhe Zhang
6de9213df1
HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster.
2016-06-06 15:52:39 -07:00
Tsz-Wo Nicholas Sze
35f255b03b
Revert "Revert "HADOOP-13226 Support async call retry and failover.""
...
This reverts commit 5360da8bd9
2016-06-06 16:31:43 +08:00
Tsz-Wo Nicholas Sze
7e7b1ae037
Revert "Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.""
...
This reverts commit 8cf47d8589
2016-06-06 16:31:38 +08:00
Tsz-Wo Nicholas Sze
db41e6d285
Revert "Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou""
...
This reverts commit 5ee5912ebd
2016-06-06 16:31:34 +08:00
Tsz-Wo Nicholas Sze
b3d81f38da
Revert "Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit b82c74b910
2016-06-06 16:31:30 +08:00
Tsz-Wo Nicholas Sze
cba9a01889
Revert "Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit f23d5dfc60
2016-06-06 16:29:38 +08:00
Tsz-Wo Nicholas Sze
aa20fa150d
Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou""
...
This reverts commit 4d36b221a2
2016-06-06 16:28:47 +08:00
Tsz-Wo Nicholas Sze
eded3d109e
Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit 106234d873
2016-06-06 16:28:21 +08:00
Andrew Wang
106234d873
Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit fc94810d3f
2016-06-03 18:09:55 -07:00
Andrew Wang
4d36b221a2
Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"
...
This reverts commit 1b9f18623a
2016-06-03 18:09:18 -07:00
Andrew Wang
f23d5dfc60
Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 7251bb922b
2016-06-03 18:09:17 -07:00
Andrew Wang
b82c74b910
Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 02d4e478a3
2016-06-03 18:09:14 -07:00
Andrew Wang
5ee5912ebd
Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou"
...
This reverts commit f4b9bcd87c
2016-06-03 18:09:13 -07:00
Andrew Wang
8cf47d8589
Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou."
...
This reverts commit 21890c4239
2016-06-03 18:09:12 -07:00
Andrew Wang
5360da8bd9
Revert "HADOOP-13226 Support async call retry and failover."
...
This reverts commit 83f2f78c11
2016-06-03 18:09:09 -07:00
Allen Wittenauer
99a771cd7a
HDFS-7767. Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI (Ravi Prakash via aw)
2016-06-03 17:07:39 -07:00
Allen Wittenauer
15f018434c
HDFS-9877. HDFS Namenode UI: Fix browsing directories that need to be encoded (Ravi Prakash via aw)
2016-06-03 17:06:29 -07:00
Arpit Agarwal
97e2449477
HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (Contributed by Akira Ajisaka)
2016-06-02 13:14:45 -07:00
Akira Ajisaka
1df6f5735c
HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin.
2016-06-03 04:10:32 +09:00
Masatake Iwasaki
aadb77e412
HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula.
2016-06-02 20:55:04 +09:00
Akira Ajisaka
69555fca06
HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki.
2016-06-02 18:52:47 +09:00
Kai Zheng
d749cf65e1
HDFS-9833. Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data. Contributed by Rakesh R.
2016-06-02 12:56:21 +08:00
Colin Patrick Mccabe
8ceb06e239
HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:54:34 -07:00
Colin Patrick Mccabe
c7921c9bdd
HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:37:52 -07:00
Colin Patrick Mccabe
29d6cadc52
HDFS-10415. TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called (Mingliang Liu via cmccabe)
2016-05-31 16:54:52 -07:00
Tsz-Wo Nicholas Sze
83f2f78c11
HADOOP-13226 Support async call retry and failover.
2016-05-31 16:30:11 -07:00
Kai Zheng
93d8a7f2a2
Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng"
...
This reverts commit a6c79f92d5
2016-06-01 08:41:15 +08:00
Chris Nauroth
21890c4239
HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.
2016-05-27 14:20:18 -07:00
Yongjun Zhang
5ea6fd85c7
HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang)
2016-05-27 10:02:02 -07:00
Kai Zheng
916140604f
HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
2016-05-28 14:23:39 +08:00
Zhe Zhang
8c84a2a93c
HDFS-10236. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-3]. Contributed by Rakesh R.
2016-05-26 16:50:55 -07:00
Karthik Kambatla
55c3e2de3d
HDFS-10463. TestRollingFileSystemSinkWithHdfs needs some cleanup. (Daniel Templeton via kasha)
2016-05-26 14:24:31 -07:00
Tsz-Wo Nicholas Sze
f4b9bcd87c
HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou
2016-05-26 12:15:17 -07:00
Kai Zheng
77202fa103
HADOOP-13010. Refactor raw erasure coders. Contributed by Kai Zheng
2016-05-27 13:23:34 +08:00
Kai Zheng
f69f5ab3b6
HDFS-10434. Fix intermittent test failure of TestDataNodeErasureCodingMetrics. Contributed by Rakesh R
2016-05-27 04:42:53 +08:00
Kai Zheng
a6c79f92d5
HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng
2016-05-27 04:29:42 +08:00
Vinayakumar B
2dab64da2f
HDFS-10217. show 'blockScheduled' tooltip in datanodes table.
2016-05-25 12:34:52 +05:30
Tsz-Wo Nicholas Sze
02d4e478a3
HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-24 14:51:27 -07:00
Tsz-Wo Nicholas Sze
15ed080e36
HDFS-9365. Balaner does not work with the HDFS-6376 HA setup.
2016-05-24 12:49:48 -07:00
Karthik Kambatla
57c31a3fef
HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha)
2016-05-24 10:37:00 -07:00
Allen Wittenauer
4b0f55b6ea
HDFS-7766. Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect (Ravi Prakash via aw)
2016-05-23 15:52:01 -07:00
Masatake Iwasaki
12eb5a7b11
HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge.
2016-05-22 16:53:30 +09:00
Chris Nauroth
500e946729
HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth.
2016-05-20 13:01:43 -07:00
Chris Nauroth
bcde1562d2
HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth.
2016-05-20 12:47:41 -07:00
Kihwal Lee
204de430bf
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-19 15:52:32 -05:00
Andrew Wang
010e6ac328
HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor.
2016-05-18 14:13:41 -07:00
Lei Xu
cf552aa87b
HDFS-10360. DataNode may format directory and lose blocks if current/VERSION is missing. (Wei-Chiu Chuang via lei)
2016-05-18 09:38:37 -07:00
Arpit Agarwal
f4d8fde822
HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser)
2016-05-18 09:20:57 -07:00
Xiaoyu Yao
dd99f5fc79
HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu.
2016-05-17 16:12:00 -07:00
Junping Du
f6ef876fe1
MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
2016-05-17 14:41:47 -07:00
Yongjun Zhang
e24fe2641b
HDFS-9732, Improve DelegationTokenIdentifier.toString() for better logging. Contributed by Yongjun Zhang.
2016-05-17 14:03:08 -07:00
Kihwal Lee
1356cbe994
Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu."
...
This reverts commit 0942954e8a
2016-05-17 15:20:03 -05:00
Kihwal Lee
0942954e8a
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-17 15:06:25 -05:00
Colin Patrick Mccabe
7cd5ae62f6
HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe)
2016-05-17 11:10:48 -07:00
Kihwal Lee
4a5819dae2
HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. Contributed by Surendra Singh Lilhore.
2016-05-17 08:53:31 -05:00
Akira Ajisaka
9fe5828f05
HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi.
2016-05-17 15:53:09 +09:00
Sangjin Lee
61f46be071
HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee)
2016-05-16 18:49:47 -07:00
Allen Wittenauer
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
Lei Xu
6a6e74acf5
HDFS-10410. RedundantEditLogInputStream.LOG is set to wrong class. (John Zhuge via lei)
2016-05-16 17:05:46 -07:00
Kai Zheng
ad9441122f
HDFS-8449. Add tasks count metrics to datanode for ECWorker. Contributed by Bo Li.
2016-05-15 23:39:09 -07:00
Andrew Wang
45788204ae
HDFS-10333. Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk. Contributed by Yiqun Lin.
2016-05-15 22:15:05 -07:00
Andrew Wang
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
Lei Xu
a2774debf7
HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei)
2016-05-12 15:42:54 -07:00
Andrew Wang
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
Tsz-Wo Nicholas Sze
7251bb922b
HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-11 17:53:40 -07:00
Colin Patrick Mccabe
687233f20d
HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe)
2016-05-11 13:45:39 -07:00
Zhe Zhang
307db61aac
HDFS-10172. hdfs erasurecode command should remove the redundant -usage option. Contributed by Yuanbo Liu.
2016-05-10 15:20:17 -07:00
Colin Patrick Mccabe
f0252ad252
HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe)
2016-05-10 11:15:05 -07:00
Kihwal Lee
b9e5a32fa1
HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah.
2016-05-09 15:50:09 -05:00
Akira Ajisaka
411fb4b2b7
HDFS-10337. OfflineEditsViewer stats option should print 0 instead of null for the count of operations. Contributed by Yiqun Lin.
2016-05-08 15:49:13 -07:00
Masatake Iwasaki
6957e45699
HDFS-10374. typo in HDFSHighAvailabilityWithQJM doc. Contributed by Takashi Ohnishi.
2016-05-07 11:26:42 +09:00
Masatake Iwasaki
2835f146b5
HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun.
2016-05-06 15:36:47 +09:00
Xiaoyu Yao
dacd1f50fe
HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang.
2016-05-05 18:08:47 -07:00
Ravi Prakash
d8faf47f32
HADOOP-13051. Test for special characters in path being respected during globPaths. Contributed by Harsh J.
2016-05-05 14:18:39 -07:00
Ming Ma
1268cf5fbe
HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma)
2016-05-04 17:02:26 -07:00
Arpit Agarwal
6d77d6eab7
HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula)
2016-05-03 16:52:43 -07:00
Xiaoyu Yao
45a753ccf7
HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao.
2016-05-02 19:30:47 -07:00
Tsz-Wo Nicholas Sze
1b9f18623a
HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou
2016-05-02 11:16:30 -07:00
Ravi Prakash
2753185a01
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-29 22:32:27 -07:00
Kihwal Lee
7da540d03e
HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah.
2016-04-29 15:50:43 -05:00
Tsz-Wo Nicholas Sze
4da6f69ca1
HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu
2016-04-29 11:22:33 -07:00
Kihwal Lee
af9b000535
HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah.
2016-04-29 09:58:44 -05:00
Kihwal Lee
6243eabb48
HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla
2016-04-28 12:44:53 -05:00
Lei Xu
0a152103f1
HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu)
2016-04-27 14:22:51 -07:00
Kihwal Lee
919a1d824a
HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla.
2016-04-27 08:19:48 -05:00
Tsz-Wo Nicholas Sze
fc94810d3f
HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-04-26 17:10:13 -07:00
Andrew Wang
6be22ddbf1
HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge.
2016-04-26 13:04:18 -07:00
Kihwal Lee
c6a2430b47
HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun.
2016-04-26 11:45:29 -05:00
Zhe Zhang
5865fe2bf0
HDFS-9869. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-2]. Contributed by Rakesh R.
2016-04-25 22:01:54 -07:00
Masatake Iwasaki
8eadd7145e
HDFS-10325. TestDFSShell#testMoveWithTargetPortEmpty fails. Contributed by Takanobu Asanuma.
2016-04-26 07:11:13 +09:00
Tsz-Wo Nicholas Sze
f308561f1d
HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma
2016-04-25 12:01:48 -07:00
Kihwal Lee
10f0f7851a
HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor.
2016-04-25 11:38:14 -05:00
Masatake Iwasaki
6fcde2e38d
HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims)
2016-04-23 23:37:56 +09:00
Jing Zhao
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
Ravi Prakash
d6402faded
Revert "HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano"
...
This reverts commit 4838b735f0
2016-04-22 10:17:20 -07:00
Kihwal Lee
5d4255a801
HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang.
2016-04-22 11:08:44 -05:00
Tsz-Wo Nicholas Sze
14ab7a81e2
HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand
2016-04-21 15:41:50 -07:00
Tsz-Wo Nicholas Sze
bbce1d525e
HDFS-10319. Balancer should not try to pair storages with different types.
2016-04-21 15:20:12 -07:00
Ravi Prakash
4838b735f0
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-21 11:32:39 -07:00
Xiaoyu Yao
b4be288c5d
HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou.
2016-04-21 10:18:48 -07:00
Xiaoyu Yao
5bd7b592e5
HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao.
2016-04-20 23:10:52 -07:00
Chris Nauroth
63ac2db59a
HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth.
2016-04-20 13:39:44 -07:00
Colin Patrick Mccabe
ad36fa6f42
HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe)
2016-04-20 11:33:11 -07:00
Arpit Agarwal
af9bdbe447
HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou)
2016-04-19 11:26:30 -07:00
Walter Su
90d1af1416
HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun.
2016-04-19 18:54:11 +08:00
Walter Su
4b4df2cd48
HDFS-10284. o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently. Contributed by Mingliang Liu.
2016-04-19 18:07:46 +08:00
Steve Loughran
438efc3634
HDFS-10291 TestShortCircuitLocalRead failing (stevel)
...
(cherry picked from commit dd64769699ea4c85abdf9c590b9db680837e8b27)
2016-04-19 17:48:20 +08:00
Kai Zheng
b5d4c7dc76
HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li.
2016-04-20 15:47:01 +08:00
Jing Zhao
be0bce1b71
HDFS-10306. SafeModeMonitor should not leave safe mode if name system is starting active service. Contributed by Mingliang Liu.
2016-04-18 18:05:49 -07:00
Colin Patrick Mccabe
cb3ca460ef
HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe)
2016-04-18 13:47:56 -07:00
Kihwal Lee
d8b729e16f
HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun.
2016-04-18 07:58:55 -05:00
Walter Su
ab903029a9
HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun.
2016-04-18 20:29:29 +08:00
Walter Su
67523ffcf4
HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi.
2016-04-18 09:28:02 +08:00
Jing Zhao
55e19b7f0c
HDFS-10293. StripedFileTestUtil#readAll flaky. Contributed by Mingliang Liu.
2016-04-15 10:53:40 -07:00
Jing Zhao
89a838769f
HDFS-10283. o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently. Contributed by Mingliang Liu.
2016-04-15 10:49:21 -07:00
Kihwal Lee
b9c9d03591
HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu.
2016-04-14 15:24:39 -05:00
Xiaoyu Yao
809226752d
HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. Contributed by Xiaobing Zhou.
2016-04-14 10:57:02 -07:00
Kihwal Lee
c970f1d005
HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang.
2016-04-14 12:45:47 -05:00
Kihwal Lee
0d1c1152f1
HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe.
2016-04-14 07:58:24 -05:00
Xiaoyu Yao
5566177c9a
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou.
2016-04-13 16:51:13 -07:00
Xiaoyu Yao
4895c73dd4
Revert "HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode"
...
This reverts commit 192112d5a2
2016-04-13 16:45:18 -07:00
Andrew Wang
314aa21a89
HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun.
2016-04-13 16:39:50 -07:00
Xiaobing Zhou
192112d5a2
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode
2016-04-13 15:34:41 -07:00
Kihwal Lee
d2f3bbc290
HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák
2016-04-13 11:22:36 -05:00
Walter Su
903428bf94
HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun.
2016-04-13 17:49:45 +08:00
Colin Patrick Mccabe
35f0770555
HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe)
2016-04-12 16:28:42 -07:00
Zhe Zhang
6ef42873a0
HDFS-9918. Erasure Coding: Sort located striped blocks based on decommissioned states. Contributed by Rakesh R.
2016-04-12 13:38:58 -07:00
Colin Patrick Mccabe
600d129bb8
HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe)
2016-04-12 10:52:39 -07:00
Akira Ajisaka
209303be3a
HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang.
2016-04-12 15:26:50 +09:00
Vinayakumar B
a9a607f8fc
HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula)
2016-04-12 11:44:34 +05:30
Akira Ajisaka
9c32f8785e
HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.
2016-04-08 15:32:13 +09:00
Uma Maheswara Rao G
3c18a53cbd
HDFS-9719. Refactoring ErasureCodingWorker into smaller reusable constructs. Contributed by Kai Zheng.
2016-04-06 22:50:24 -07:00
Tsz-Wo Nicholas Sze
654cd1d0c0
HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R
2016-04-07 12:38:42 +08:00
Colin Patrick Mccabe
4bd7cbc29d
HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose
2016-04-06 21:07:31 -07:00
Eric Payne
aede8c10ec
HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee
2016-04-06 20:20:14 +00:00
Colin Patrick Mccabe
188f65287d
HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe)
2016-04-06 11:28:34 -07:00
Jing Zhao
221b3a8722
HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula.
2016-04-06 10:42:59 -07:00
Kihwal Lee
9ba1e5af06
HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger.
2016-04-05 16:26:18 -05:00
Ravi Prakash
0cd320a846
HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula.
2016-04-05 13:41:19 -07:00
Kihwal Lee
917464505c
HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla.
2016-04-05 09:07:24 -05:00
Vinayakumar B
818d6b799e
HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula)
2016-04-05 09:49:39 +08:00
Colin Patrick Mccabe
f6b1a81812
HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe)
2016-04-04 18:02:15 -07:00
Kihwal Lee
a7d1fb0cd2
HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee.
2016-04-04 16:40:00 -05:00
Masatake Iwasaki
154d2532cf
HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims)
2016-04-05 03:19:48 +09:00
Vinayakumar B
54b2e78fd2
HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao)
2016-04-03 13:27:49 +05:30
Allen Wittenauer
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
Colin Patrick Mccabe
37e23ce45c
HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe)
2016-03-30 13:37:37 -07:00
Vinayakumar B
09d63d5a19
HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B)
2016-03-30 14:22:00 +08:00
Arpit Agarwal
ddfe6774c2
HDFS-9439. Support reconfiguring fs.protected.directories without NN restart. (Contributed by Xiaobing Zhou)
2016-03-29 13:55:00 -07:00
Arpit Agarwal
c9307e48b7
HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA)
2016-03-29 13:50:08 -07:00
Andrew Wang
f2aec4eb82
HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun.
2016-03-29 11:54:42 -07:00
Vinayakumar B
1f004b3367
HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla)
2016-03-29 11:25:41 +08:00
Uma Maheswara Rao G
3a4ff7776e
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 19:58:09 -07:00
Arpit Agarwal
a337ceb74e
Revert "HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng"
...
This reverts commit e5ff0ea7ba
2016-03-26 09:20:01 -07:00
Uma Maheswara Rao G
e5ff0ea7ba
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 00:52:50 -07:00
Lei Xu
fde8ac5d85
Add missing files from HDFS-9005. (lei)
2016-03-25 17:11:35 -07:00
Lei Xu
4fcfea71bf
HDFS-9005. Provide support for upgrade domain script. (Ming Ma via Lei Xu)
2016-03-25 17:09:12 -07:00
Allen Wittenauer
738155063e
HADOOP-12857. rework hadoop-tools (aw)
2016-03-23 13:46:38 -07:00
Andrew Wang
8f85e5d212
HDFS-10200. Docs for WebHDFS still describe GETDELEGATIONTOKENS operation. Contributed by Wellington Chevreuil.
2016-03-23 12:55:15 -07:00
Colin Patrick Mccabe
a7d8f2b396
HDFS-10189. PacketResponder#toString should include the downstreams for PacketResponderType.HAS_DOWNSTREAM_IN_PIPELINE (Joe Pallas via cmccabe)
2016-03-22 14:56:02 -07:00
Colin Patrick Mccabe
680716f31e
HDFS-9951. Use string constants for XML tags in OfflineImageReconstructor (Lin Yiqun via cmccabe)
2016-03-21 11:40:37 -07:00
Andrew Wang
e3bb38d625
HDFS-9405. Warmup NameNode EDEK caches in background thread. Contributed by Xiao Chen.
2016-03-21 11:39:05 -07:00
Haohui Mai
8a3f0cb255
HDFS-7166. SbNN Web UI shows #Under replicated blocks and #pending deletion blocks. Contributed by Wei-Chiu Chuang.
2016-03-20 17:21:25 -07:00
Sangjin Lee
cd8b6889a7
HDFS-9579. Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level (Ming Ma via sjlee)
2016-03-19 14:02:04 -07:00
Akira Ajisaka
9b623fbaf7
HDFS-3677. dfs.namenode.edits.dir.required is missing from hdfs-default.xml. Contributed by Mark Yang.
2016-03-19 01:11:15 +09:00
Kihwal Lee
63c966a3fb
HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah.
2016-03-18 10:24:59 -05:00
Colin Patrick Mccabe
dc951e606f
HDFS-9949. Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared (Harsh J via cmccabe)
2016-03-17 10:37:42 -07:00
Andrew Wang
02a250db9f
HDFS-10173. Typo in DataXceiverServer. Contributed by Michael Han.
2016-03-16 18:49:57 -07:00
Zhe Zhang
32d043d9c5
HDFS-9857. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-1]. Contributed by Rakesh R.
2016-03-16 16:53:58 -07:00
Kihwal Lee
d457401784
HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun.
2016-03-15 10:52:47 -05:00
Masatake Iwasaki
5de848cd5d
HDFS-9928. Make HDFS commands guide up to date (Wei-Chiu Chuang via iwasakims)
2016-03-15 20:42:05 +09:00
Ravi Prakash
bd5556ba12
Download File from UI broken after pagination. Contributed by Brahma Reddy Battula
2016-03-14 14:30:13 -07:00
Chris Nauroth
5644137ada
HDFS-9941. Do not log StandbyException on NN, other minor logging fixes. Contributed by Arpit Agarwal.
2016-03-14 09:54:54 -07:00
Arpit Agarwal
e01c6ea688
HDFS-1477. Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart. (Contributed by Xiaobing Zhou)
2016-03-10 19:03:55 -08:00
Colin Patrick Mccabe
79961ecea8
HDFS-9933. ReverseXML should be capitalized in oiv usage message (cmccabe)
2016-03-10 13:43:27 -08:00
Colin Patrick Mccabe
bd49354c6d
HDFS-9934. ReverseXML oiv processor should bail out if the XML file's layoutVersion doesn't match oiv's (cmccabe)
2016-03-10 13:41:06 -08:00
Colin Patrick Mccabe
500875dfcc
HDFS-9927. Document the new OIV ReverseXML processor (Wei-Chiu Chuang via cmccabe)
2016-03-10 13:38:00 -08:00
Zhe Zhang
7600e3c48f
HDFS-7866. Erasure coding: NameNode manages multiple erasure coding policies. Contributed by Rui Li.
2016-03-08 22:30:30 -08:00
Andrew Wang
a14a6f08ee
HDFS-9919. TestStandbyCheckpoints#testNonPrimarySBNUploadFSImage waitForCheckpoint incorrectly. Contributed by Lin Yiqun.
2016-03-08 11:54:50 -08:00
Jing Zhao
743a99f2db
HDFS-8786. Erasure coding: use simple replication for internal blocks on decommissioning datanodes. Contributed by Rakesh R.
2016-03-08 10:24:57 -08:00
Arpit Agarwal
c2140d05ef
HDFS-9882. Add heartbeatsTotal in Datanode metrics. (Contributed by Hua Liu)
2016-03-07 21:10:24 -08:00
Arpit Agarwal
724d2299cd
HDFS-9906. Remove spammy log spew when a datanode is restarted. (Contributed by Brahma Reddy Battula)
2016-03-07 12:19:53 -08:00
Masatake Iwasaki
d718fc1ee5
HDFS-9865. TestBlockReplacement fails intermittently in trunk (Lin Yiqun via iwasakims)
2016-03-08 02:07:18 +09:00
Harsh J
fd1c09be3e
HDFS-9521. TransferFsImage.receiveFile should account and log separate times for image download and fsync to disk. Contributed by Wellington Chevreuil
2016-03-07 17:14:51 +05:30
Chris Nauroth
2759689d7d
HDFS-9239. DataNode Lifeline Protocol: an alternative protocol for reporting DataNode liveness. Contributed by Chris Nauroth.
2016-03-04 15:29:50 -08:00
Yongjun Zhang
8e08861a14
HDFS-9889. Update balancer/mover document about HDFS-6133 feature. Contributed by Yongjun Zhang.
2016-03-04 13:25:29 -08:00
Zhe Zhang
3e8099a45a
HDFS-9888. Allow reseting KerberosName in unit tests. Contributed by Xiao Chen.
2016-03-04 09:48:05 -08:00
Masatake Iwasaki
cbd31328a6
HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)
2016-03-04 14:11:36 +09:00
Masatake Iwasaki
f70e317ccc
HADOOP-12793. Write a new group mapping service guide (Wei-Chiu Chuang via iwasakims)
2016-03-03 17:14:32 +09:00
Arpit Agarwal
27941a1811
HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou)
2016-03-02 18:35:28 -08:00
Colin Patrick Mccabe
700b0e4019
HDFS-9835. OIV: add ReverseXML processor which reconstructs an fsimage from an XML file (cmccabe)
2016-03-02 17:56:11 -08:00
Xiaoyu Yao
5abf051249
HDFS-9887. WebHdfs socket timeouts should be configurable. Contributed by Austin Donnelly and Chris Douglas.
2016-03-02 09:23:17 -08:00
Akira Ajisaka
27e0681f28
HDFS-9851. NameNode throws NPE when setPermission is called on a path that does not exist. Contributed by Brahma Reddy Battula.
2016-03-02 14:43:03 +09:00
Akira Ajisaka
e2ddf82469
HDFS-9766. TestDataNodeMetrics#testDataNodeTimeSpend fails intermittently. Contributed by Xiao Chen.
2016-03-02 14:17:35 +09:00
Jing Zhao
f2ba7da4f0
HDFS-9876. shouldProcessOverReplicated should not count number of pending replicas. Contributed by Jing Zhao.
2016-03-01 18:41:57 -08:00
Andrew Wang
4abb2fa687
HDFS-9881. DistributedFileSystem#getTrashRoot returns incorrect path for encryption zones.
2016-03-01 16:35:39 -08:00
Kihwal Lee
e76b13c415
HDFS-9880. TestDatanodeRegistration fails occasionally. Contributed by Kihwal Lee.
2016-03-01 17:00:18 -06:00
Kihwal Lee
2c8496ebf3
HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo.
2016-03-01 15:04:59 -06:00
Chris Nauroth
2137e8feeb
HDFS-9870. Remove unused imports from DFSUtil. Contributed by Brahma Reddy Battula.
2016-03-01 11:15:13 -08:00
Uma Maheswara Rao G
307ec80aca
HDFS-9733. Refactor DFSClient#getFileChecksum and DataXceiver#blockChecksum. Contributed by Kai Zheng
2016-02-29 21:52:20 -08:00
Jing Zhao
2151716832
HDFS-7964. Add support for async edit logging. Contributed by Daryn Sharp.
2016-02-29 15:34:43 -08:00
Akira Ajisaka
8bc023b3b1
HDFS-9864. Correct reference for RENEWDELEGATIONTOKEN and CANCELDELEGATIONTOKEN in webhdfs doc. Contributed by Brahma Reddy Battula.
2016-02-29 18:42:21 +09:00
Jing Zhao
408f2c807b
HDFS-9866. BlockManager#chooseExcessReplicasStriped may weaken rack fault tolerance. Contributed by Jing Zhao.
2016-02-28 14:54:49 -08:00
Tsz-Wo Nicholas Sze
d1d4e16690
HDFS-9710. DN can be configured to send block receipt IBRs in batches.
2016-02-26 15:32:25 -08:00
Xiaoyu Yao
eab52dfb35
HDFS-9831. Document webhdfs retry configuration keys introduced by HDFS-5219/HDFS-5122. Contributed by Xiaobing Zhou.
2016-02-26 14:14:12 -08:00
Karthik Kambatla
c2460dad64
HDFS-9858. RollingFileSystemSink can throw an NPE on non-secure clusters. (Daniel Templeton via kasha)
2016-02-25 16:31:09 -08:00
Zhe Zhang
8808779db3
HDFS-9734. Refactoring of checksum failure report related codes. Contributed by Kai Zheng.
...
Change-Id: Ie69a77e3498a360959f8e213c51fb2b17c28b64a
2016-02-25 09:55:50 -08:00
Zhe Zhang
ccff6035f5
HDFS-9804. Allow long-running Balancer to login with keytab. Contributed by Xiao Chen.
...
Change-Id: I54b82e0b11929082057b4d21bfcf285222c94cdb
2016-02-25 09:24:32 -08:00
Tsz-Wo Nicholas Sze
6979cbfc1f
HDFS-9838. Refactor the excessReplicateMap to a class.
2016-02-24 19:42:47 -08:00
Chris Nauroth
dbbfc58c33
HDFS-9843. Document distcp options required for copying between encrypted locations. Contributed by Xiaoyu Yao.
2016-02-24 15:16:05 -08:00
Jing Zhao
47b92f2b6f
HDFS-9837. BlockManager#countNodes should be able to detect duplicated internal blocks. Contributed by Jing Zhao.
2016-02-24 15:13:56 -08:00
cnauroth
d1dd248b75
HDFS-9854. Log cipher suite negotiation more verbosely. Contributed by Wei-Chiu Chuang.
2016-02-24 12:11:35 -08:00
Kihwal Lee
d27d7fc72e
HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla.
2016-02-24 12:04:01 -06:00
Zhe Zhang
def754ec06
HDFS-9816. Erasure Coding: allow to use multiple EC policies in striping related tests [Part 3]. Contributed by Rui Li.
...
Change-Id: I64b57bab4722cdc6e1e3148c3a3a401370249afe
2016-02-23 14:05:17 -08:00
Zhe Zhang
647a35e996
HDFS-9844. Correct path creation in getTrashRoot to handle root dir.
...
Change-Id: I5afe6c7176063892e3e632698c849bd9f4b92906
2016-02-23 13:29:39 -08:00
Colin Patrick Mccabe
211c78c090
HDFS-9549. TestCacheDirectives#testExceedsCapacity is flaky (Xiao Chen via cmccabe)
2016-02-23 12:01:20 -08:00
Tsuyoshi Ozawa
48c61cd6aa
HDFS-9084. Pagination, sorting and filtering of files/directories in the HDFS Web UI. (Contributed by Ravi Prakash)
2016-02-23 19:30:58 +09:00
Tsz-Wo Nicholas Sze
66289a3bf4
HDFS-8578. On upgrade, Datanode should process all storage/data dirs in parallel. Contributed by vinayakumarb and szetszwo
2016-02-22 15:01:15 -08:00
Zhe Zhang
a2fdfff02d
HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh.
...
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
2016-02-22 14:13:54 -08:00
Arpit Agarwal
a63301a864
HDFS-9842. dfs.datanode.balance.bandwidthPerSec should accept friendly size units. (Contributed by Lin Yiqun)
2016-02-22 11:34:09 -08:00
Vinayakumar B
342c9572bf
HDFS-9425. Expose number of blocks per volume as a metric (Contributed by Brahma Reddy Battula)
2016-02-22 09:29:23 +05:30
Vinayakumar B
f313516731
HDFS-7452. skip StandbyException log for getCorruptFiles() (Contributed by Bramha Reddy Battula)
2016-02-22 09:21:46 +05:30
Arpit Agarwal
d5abd293a8
HDFS-9839. Reduce verbosity of processReport logging. (Contributed by Arpit Agarwal)
...
This closes #78
2016-02-20 23:19:09 -08:00
Jing Zhao
e54cc29312
HDFS-9818. Correctly handle EC reconstruction work caused by not enough racks. Contributed by Jing Zhao.
2016-02-19 19:02:23 -08:00
Jing Zhao
6546d9e7ff
HDFS-9829. Erasure Coding: Improve few exception handling logic of ErasureCodingWorker. Contributed by Rakesh R.
2016-02-19 10:40:28 -08:00
Tsz-Wo Nicholas Sze
46f7790236
HDFS-8923. Add -source flag to balancer usage message. Contributed by Chris Trezzo
2016-02-18 18:18:12 -08:00
Akira Ajisaka
c7ac25e52c
HDFS-9830. Remove references to hftp in ViewFs documentation in trunk. Contributed by Wei-Chiu Chuang.
2016-02-19 05:02:57 +09:00
Tsz-Wo Nicholas Sze
bbfaf3c271
HADOOP-10865. Add a Crc32 chunked verification benchmark for both directly and non-directly buffer cases.
2016-02-18 11:47:33 -08:00
Kihwal Lee
ba6d5ed955
HDFS-4946. Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable. Contributed by James Kinley and Nathan Roberts.
2016-02-18 13:21:51 -06:00
cnauroth
5d1889a66d
HDFS-9711. Integrate CSRF prevention filter in WebHDFS. Contributed by Chris Nauroth.
2016-02-18 10:07:28 -08:00
Akira Ajisaka
e78d8e66f7
HDFS-6832. Fix the usage of 'hdfs namenode' command. Contributed by Manjunath Ballur.
2016-02-19 01:35:06 +09:00
Andrew Wang
3a23dc683c
HDFS-9608. Disk IO imbalance in HDFS with heterogeneous storages. Contributed by Wei Zhou.
2016-02-17 11:29:58 -08:00
Haohui Mai
77f7ca3e94
HDFS-9815. Move o.a.h.fs.Hdfs to hadoop-hdfs-client. Contributed by Vinayakumar B.
2016-02-17 10:56:11 -08:00
Vinayakumar B
2536ece7b6
HDFS-9787. SNNs stop uploading FSImage to ANN once isPrimaryCheckPointer changed to false. (Contributed by Guocui Mi)
2016-02-17 10:41:24 +05:30
Junping Du
77ba5add0d
HDFS-9456. BlockPlacementPolicyWithNodeGroup should override verifyBlockPlacement(). Contributed by Xiaobing Zhou.
2016-02-16 18:55:55 -08:00
Akira Ajisaka
8ed07bd3e6
HDFS-9765. TestBlockScanner#testVolumeIteratorWithCaching fails intermittently. (aajisaka)
2016-02-15 19:08:03 +09:00
Akira Ajisaka
b0738ae673
HDFS-9773. Remove dead code related to SimulatedFSDataset in tests. Contributed by Brahma Reddy Battula.
2016-02-15 15:35:06 +09:00
Akira Ajisaka
ac5da11399
HDFS-9691. TestBlockManagerSafeMode#testCheckSafeMode fails intermittently. Contributed by Mingliang Liu.
2016-02-15 14:00:09 +09:00
Arpit Agarwal
1de1641f17
HDFS-9801. ReconfigurableBase should update the cached configuration. (Arpit Agarwal)
2016-02-12 12:41:04 -08:00
Arun Suresh
9fdfb546fb
HDFS-9797. Log Standby exceptions thrown by RequestHedgingProxyProvider at DEBUG Level (Inigo Goiri via asuresh)
2016-02-12 11:18:10 -08:00
Jing Zhao
972782d956
HDFS-9754. Avoid unnecessary getBlockCollection calls in BlockManager. Contributed by Jing Zhao.
2016-02-12 11:07:52 -08:00
Zhe Zhang
b21bbe9ed1
HDFS-9644. Update encryption documentation to reflect nested EZs. (zhz)
...
Change-Id: I84ae96f4711a16ded46d501c51ca322bfca5151d
2016-02-12 10:44:15 -08:00
Akira Ajisaka
e6a7044b85
HDFS-9768. Reuse ObjectMapper instance in HDFS to improve the performance. Contributed by Lin Yiqun.
2016-02-13 01:57:24 +09:00
Arpit Agarwal
9b959f3ee7
HDFS-9790. HDFS Balancer should exit with a proper message if upgrade is not finalized (Contributed by Xiaobing Zhou)
2016-02-11 20:40:03 -08:00
Karthik Kambatla
33ce7f6c07
HDFS-9780. RollingFileSystemSink doesn't work on secure clusters. (Daniel Templeton via kasha)
2016-02-11 20:37:14 -08:00
Lei Xu
c7fcec24b8
HDFS-9795. OIV Delimited should show which files are ACL-enabled (lei)
2016-02-11 20:33:51 -08:00
Zhe Zhang
0aa8c82894
HDFS-9755. Erasure Coding: allow to use multiple EC policies in striping related tests [Part 2]. Contributed by Rui Li.
...
Change-Id: I2100bc27ad484f83c9cb2d2e5bb232f4f74fd286
2016-02-11 10:14:09 -08:00
Jing Zhao
19adb2bc64
HDFS-9789. Correctly update DataNode's scheduled block size when writing small EC file. Contributed by Jing Zhao.
2016-02-10 21:24:00 -08:00
Andrew Wang
aeb13ef2ab
HDFS-9788. Incompatible tag renumbering in HeartbeatResponseProto.
2016-02-10 15:26:08 -08:00
Karthik Kambatla
bc425a623f
HDFS-9637. Tests for RollingFileSystemSink. (Daniel Templeton via kasha)
2016-02-10 10:08:05 -08:00
Akira Ajisaka
802a7edfae
HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
2016-02-10 23:49:14 +09:00
Akira Ajisaka
5e565ce7a7
Revert "HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta."
...
This reverts commit fe124da5ff
2016-02-10 23:43:33 +09:00
Uma Maheswara Rao G
a7fce9ab41
HDFS-9779 . TestReplicationPolicyWithNodeGroup NODE variable picks wrong rack value. Contributed by Kuhu Shukla.
2016-02-09 15:14:30 -08:00
Zhe Zhang
a0fb2eff9b
HDFS-9775. Erasure Coding : Rename BlockRecoveryWork to BlockReconstructionWork. Contributed by Rakesh R.
...
Change-Id: I6dfc8efd94fa2bbb4eec0e4730a5a4f92c8a5519
2016-02-09 14:43:59 -08:00
Allen Wittenauer
401ae4ecdb
HDFS-9760. WebHDFS AuthFilter cannot be configured with custom AltKerberos auth handler (Ryan Sasson via aw)
2016-02-09 14:15:21 -08:00
Akira Ajisaka
60d2011b7c
HDFS-9784. Example usage is not correct in Transparent Encryption document. Contributed by Ohnishi Takashi.
2016-02-10 03:30:26 +09:00
zhezhang
dbe49c1bd6
HDFS-9244. Support nested encryption zones.
...
Change-Id: I43a13035a8b27956e90967ee82058efb647c3415
2016-02-08 16:30:51 -08:00
Uma Maheswara Rao G
bffaa38a91
HDFS-9777. Fix typos in DFSAdmin command line and documentation.(Wei-Chiu Chuang via umamahesh)
2016-02-08 13:47:45 -08:00
Kihwal Lee
193d27de0a
HDFS-9752. Permanent write failures may happen to slow writers during datanode rolling upgrades. Contributed by Walter Su.
2016-02-08 12:16:29 -06:00
Uma Maheswara Rao G
aea31eee78
HDFS-9713. DataXceiver#copyBlock should return if block is pinned. Contributed by Uma Maheswara Rao G.
2016-02-07 20:50:47 -08:00
Akira Ajisaka
fe124da5ff
HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
2016-02-06 19:31:01 +09:00
cnauroth
c6497949e8
HDFS-9761. Rebalancer sleeps too long between iterations. Contributed by Mingliang Liu.
2016-02-05 15:07:43 -08:00
Tsz-Wo Nicholas Sze
4e5e1c0f99
HDFS-9726. Refactor IBR code to a new class.
2016-02-05 23:17:59 +08:00
Haohui Mai
1bcfab8e7f
HDFS-9724. Degraded performance in WebHDFS listing as it does not reuse ObjectMapper. Contributed by Akira Ajisaka.
2016-02-04 11:34:26 -08:00
Kihwal Lee
496f33de0c
HDFS-9730. Storage ID update does not happen when there is a layout change. Contributed by Tsz Wo Nicholas Sze.
2016-02-04 13:19:54 -06:00
Vinayakumar B
a6728a013e
HDFS-9629. Update the footer of Web UI to show year 2016 (Contributed by Xiao Chen and Vinayakumar B)
2016-02-04 13:37:51 +05:30
Konstantin V Shvachko
81ef1a1f55
HDFS-9503. Use generic option -fs for NNThroughputBenchmark instead of -namenode. Contributed by Mingliang Liu.
2016-02-03 14:59:46 -08:00
zhezhang
a0f5e83f28
HDFS-9658. Erasure Coding: allow to use multiple EC policies in striping related tests. Contributed by Rui Li.
...
Change-Id: I5b02f5bde4d343b7529c6a7fe5df73bd99c6cb24
2016-02-03 14:41:07 -08:00
Jing Zhao
7badf15604
HDFS-9748. Avoid duplication in pendingReplications when addExpectedReplicasToPending is called twice. Contributed by Walter Su.
2016-02-03 13:30:09 -08:00
Vinayakumar B
eb2fb943fd
HDFS-9740. Use a reasonable limit in DFSTestUtil.waitForMetric() (Contributed by Chang Li)
2016-02-03 11:14:29 +05:30
Vinayakumar B
04375756a5
HDFS-9715. Check storage ID uniqueness on datanode startup (Contributed by Lei (Eddy) Xu)
2016-02-03 07:35:01 +05:30
zhezhang
6d1213860f
HDFS-9403. Erasure coding: some EC tests are missing timeout. Contributed by Gao Rui.
...
Change-Id: I13c4bc0e566c4f8af5371d26f8e18a749ed41d8b
2016-02-02 14:03:45 -08:00
zhezhang
4ae543fdcd
HDFS-9731. Erasure Coding: Rename BlockECRecoveryCommand to BlockECReconstructionCommand. Contributed by Rakesh R.
...
Change-Id: I405365a8395770e494b92bfe9651f4f0366d8f28
2016-02-02 12:32:08 -08:00
Colin Patrick Mccabe
dd9ebf6eed
HDFS-9260. Improve the performance and GC friendliness of NameNode startup and full block reports (Staffan Friberg via cmccabe)
2016-02-02 11:23:00 -08:00
Colin Patrick Mccabe
2da03b48eb
HDFS-9669. TcpPeerServer should respect ipc.server.listen.queue.size (Elliot Clark via cmccabe)
2016-02-02 10:59:10 -08:00
Lei Xu
9d494f0c0e
HDFS-9721. Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference. (Xiao Chen via lei)
2016-02-02 10:42:01 -08:00
Vinayakumar B
4d7055e0c8
HDFS-9718. HAUtil#getConfForOtherNodes should unset independent generic keys before initialize (Contributed by DENG FEI)
2016-02-02 12:51:35 +05:30
Yongjun Zhang
34ab50ea92
HDFS-9406. FSImage may get corrupted after deleting snapshot. (Contributed by Jing Zhao, Stanislav Antic, Vinayakumar B, Yongjun Zhang)
2016-02-01 13:56:55 -08:00
Lei Xu
e50aa53eed
HDFS-9701. DN may deadlock when hot-swapping under load. (Xiao Chen via lei)
2016-02-01 12:56:53 -08:00
Xiaoyu Yao
b11008d54d
HDFS-9210. Fix some misuse of %n in VolumeScanner#printStats. Contributed by Xiaoyu Yao.
2016-02-01 11:55:27 -08:00
Steve Loughran
af2dccbca5
HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames (Mingliang Liu via stevel)
2016-02-01 16:01:50 +00:00
Vinayakumar B
8f2622b6a0
HDFS-9659. EditLogTailerThread to Active Namenode RPC should timeout (Contributed by surendra singh lilhore)
2016-02-01 14:10:55 +05:30
Vinayakumar B
e418bd1fb0
HDFS-9566. Remove expensive 'BlocksMap#getStorages(Block b, final DatanodeStorage.State state)' method (Contributed by Daryn Sharp)
2016-02-01 13:24:05 +05:30
Yongjun Zhang
8ee060311c
HDFS-9706. Log more details in debug logs in BlockReceiver's constructor. (Xiao Chen via Yongjun Zhang)
2016-01-28 23:04:03 -08:00
Colin Patrick Mccabe
f67149ab08
HDFS-7764. DirectoryScanner shouldn't abort the scan if one directory had an error (Rakesh R via cmccabe)
2016-01-28 19:54:50 -08:00
Tsz-Wo Nicholas Sze
662e17b46a
HDFS-9654. Code refactoring for HDFS-8578.
2016-01-28 10:58:03 +08:00
Jing Zhao
ec25c7f9c7
HDFS-9677. Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp. Contributed by Mingliang Liu.
2016-01-27 16:34:40 -08:00
Jing Zhao
3a9571308e
Revert "HDFS-9677. Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp. Contributed by Mingliang Liu."
...
This reverts commit 8a91109d16
2016-01-27 16:31:19 -08:00
Jing Zhao
8a91109d16
HDFS-9677. Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp. Contributed by Mingliang Liu.
2016-01-27 15:48:47 -08:00
Tsz-Wo Nicholas Sze
45c763ad61
HDFS-9690. ClientProtocol.addBlock is not idempotent after HDFS-8071.
2016-01-26 11:20:13 +08:00
Tsz-Wo Nicholas Sze
bd909ed9f2
HDFS-8999. Allow a file to be closed with COMMITTED but not yet COMPLETE blocks.
2016-01-26 10:32:51 +08:00
Jitendra Pandey
e8650fea1f
HDFS-9672. o.a.h.hdfs.TestLeaseRecovery2 fails intermittently. Contributed by Mingliang Liu.
2016-01-25 16:08:46 -08:00
Arpit Agarwal
d62b4a4de7
HDFS-9094. Add command line option to ask NameNode reload configuration. (Contributed by Xiaobing Zhou)
2016-01-25 12:17:05 -08:00
Eric Yang
10a2bc0dff
HDFS-9653. Added blocks pending deletion report to dfsadmin.
...
(Weiwei Yang via eyang)
2016-01-24 14:19:49 -08:00
Allen Wittenauer
d22c4239a4
HDFS-9525. hadoop utilities need to support provided delegation tokens (HeeSoo Kim via aw)
2016-01-22 12:15:22 -08:00
Jing Zhao
95363bcc7d
HDFS-9646. ErasureCodingWorker may fail when recovering data blocks with length less than the first internal block. Contributed by Jing Zhao.
2016-01-22 09:46:02 -08:00
Kihwal Lee
2fd19b9674
HDFS-8898. Create API and command-line argument to get quota and quota usage without detailed content summary. Contributed by Ming Ma.
2016-01-22 09:10:06 -06:00
Masatake Iwasaki
ae9c61ff0a
HDFS-9618. Fix mismatch between log level and guard in BlockManager#computeRecoveryWorkForBlocks (iwasakims)
2016-01-22 14:11:48 +09:00
Masatake Iwasaki
b2ffcc2915
HDFS-9601. NNThroughputBenchmark.BlockReportStats should handle NotReplicatedYetException on adding block (iwasakims)
2016-01-22 12:28:38 +09:00
Zhe Zhang
b4a05c1fd5
HDFS-9674. The HTrace span for OpWriteBlock should record the maxWriteToDisk time. Contributed by Colin McCabe.
...
Change-Id: I9bf3f3bcd57f5880189ad7c160f3dd66f97d904b
2016-01-21 13:25:42 -08:00
Jing Zhao
c304890c8c
HDFS-9542. Move BlockIdManager from FSNamesystem to BlockManager. Contributed by Jing Zhao.
2016-01-21 11:13:17 -08:00
Kihwal Lee
3616c7b855
HDFS-9634. webhdfs client side exceptions don't provide enough details. Contributed by Eric Payne.
2016-01-21 09:32:57 -06:00
Vinayakumar B
b7372b7166
HDFS-9625. set replication for empty file failed when set storage policy (Contributed by DENG FEI)
2016-01-21 19:17:05 +05:30
Lei Xu
2ec438e8f7
HDFS-9655. NN should start JVM pause monitor before loading fsimage. (John Zhuge via Lei (Eddy) Xu)
2016-01-20 14:26:11 -08:00
Vinayakumar B
1425578690
HDFS-9661. Deadlock in DN.FsDatasetImpl between moveBlockAcrossStorage and createRbw (Contributed by ade)
2016-01-20 21:04:21 +05:30
Yongjun Zhang
57d0a94305
HDFS-6054. MiniQJMHACluster should not use static port to avoid binding failure in unit test. (Yongjun Zhang)
2016-01-19 22:54:47 -08:00
Andrew Wang
c07f7fa8ff
HDFS-9624. DataNode start slowly due to the initial DU command operations. (Lin Yiqun via wang)
2016-01-15 11:28:46 -08:00
Arpit Agarwal
e97cd4ade5
HDFS-9415. Document dfs.cluster.administrators and dfs.permissions.superusergroup. (Contributed by Xiaobing Zhou)
2016-01-14 14:17:52 -08:00
Yongjun Zhang
817cc1f02a
HDFS-9648. TestStartup.testImageChecksum is broken by HDFS-9569's message change. (Wei-Chiu Chuang via Yongjun Zhang)
2016-01-14 12:36:36 -08:00
Yongjun Zhang
25051c3bd0
HDFS-9569. Log the name of the fsimage being loaded for better supportability. (Yongjun Zhang)
2016-01-12 09:22:22 -08:00
Arpit Agarwal
de37f37543
HDFS-9639. Inconsistent Logging in BootstrapStandby. (Contributed by Xiaobing Zhou)
2016-01-11 15:08:00 -08:00
Jing Zhao
9f4bf3bdf9
HDFS-9621. getListing wrongly associates Erasure Coding policy to pre-existing replicated files under an EC directory. Contributed by Jing Zhao.
2016-01-11 11:31:59 -08:00
Lei Xu
fd8065a763
HDFS-9493. Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk. (Tony Wu via lei)
2016-01-08 15:00:30 -08:00
Zhe Zhang
0af2022e6d
HDFS-9626. TestBlockReplacement#testBlockReplacement fails occasionally. Contributed by Xiao Chen.
...
Change-Id: Ia766a5bfb89f8cf8a48b473bb27c30d5818ebfc0
2016-01-08 13:14:36 -08:00
Kihwal Lee
38c4c14472
HDFS-9574. Reduce client failures during datanode restart. Contributed by Kihwal Lee.
2016-01-08 11:13:58 -06:00
Vinayakumar B
6702e7d66a
HDFS-9619. SimulatedFSDataset sometimes can not find blockpool for the correct namenode (Contributed by Wei-Chiu Chuang)
2016-01-07 14:20:18 +05:30
Vinayakumar B
34cd7cd765
HDFS-9600. do not check replication if the block is under construction (Contributed by Phil Yang)
2016-01-07 11:27:42 +05:30
Arpit Agarwal
67c9780609
HDFS-9498. Move code that tracks blocks with future generation stamps to BlockManagerSafeMode. (Contributed by Mingliang Liu)
2016-01-06 10:30:59 -08:00
Arpit Agarwal
b9936689c9
HDFS-9615. Fix variable name typo in DFSConfigKeys. (Contributed by Ray Chiang)
2016-01-06 09:40:31 -08:00
Haohui Mai
dec8fedb65
HDFS-9605. Add links to failed volumes to explorer.html in HDFS Web UI. Contributed by Archana T.
2016-01-05 13:55:55 -08:00
Haohui Mai
cea0972fa1
HDFS-7779. Support changing ownership, group and replication in HDFS Web UI. Contributed by Ravi Prakash.
2016-01-05 13:44:16 -08:00
Uma Mahesh
778146eaae
HADOOP-12658. Clear javadoc and check style issues around DomainSocket. Contributed by Kai Zheng
2016-01-04 14:32:09 -08:00
Akira Ajisaka
64ae85fd2e
HADOOP-11252. RPC client does not time out by default. Contributed by Wilfred Spiegelenburg and Masatake Iwasaki.
2016-01-04 14:31:22 +09:00
cnauroth
84a8147791
HDFS-9376. TestSeveralNameNodes fails occasionally. Contributed by Masatake Iwasaki.
2015-12-29 14:37:03 -08:00
cnauroth
99cf2ecee9
HDFS-7553. fix the TestDFSUpgradeWithHA due to BindException. Contributed by Xiao Chen.
2015-12-29 10:56:59 -08:00
Benoy Antony
8cfd672397
HDFS-9597. BaseReplicationPolicyTest should update data node stats after adding a data node. Contributed by Wei-Chiu Chuang.
2015-12-25 08:55:39 -08:00
Benoy Antony
df83230948
HDFS-9034. StorageTypeStats Metric should not count failed storage. Contributed by Surendra Singh Lilhore.
2015-12-22 15:28:17 -08:00
Kihwal Lee
867048c3e4
HDFS-7163. WebHdfsFileSystem should retry reads according to the configured retry policy. Contributed by Eric Payne.
2015-12-22 14:08:23 -06:00
cnauroth
5c0ff69618
HDFS-9458. TestBackupNode always binds to port 50070, which can cause bind failures. Contributed by Xiao Chen.
2015-12-22 10:29:35 -08:00
Colin Patrick Mccabe
bb540ba85a
HDFS-9589. Block files which have been hardlinked should be duplicated before the DataNode appends to the them (cmccabe)
2015-12-22 09:36:50 -08:00
Tsz-Wo Nicholas Sze
70d6f20126
HDFS-9575. Use byte array for internal block indices in a striped block. Contributed by jing9
2015-12-22 14:47:57 +08:00
Zhe Zhang
0087734cc1
HDFS-9580. TestComputeInvalidateWork#testDatanodeReRegistration failed due to unexpected number of invalidate blocks. Contributed by Wei-Chiu Chuang.
...
Change-Id: I9411dfb1dd4605cbcc515ca3361218d35d7c5fe3
2015-12-21 22:38:30 -08:00
Akira Ajisaka
fa544020f6
HDFS-9505. HDFS Architecture documentation needs to be refreshed. Contributed by Masatake Iwasaki.
2015-12-21 21:23:03 +09:00
Uma Mahesh
52ad9125b8
HDFS-9582. TestLeaseRecoveryStriped file missing Apache License header and not well formatted. (umamahesh)
2015-12-20 13:10:57 -08:00
Uma Mahesh
0f82b5d878
YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh)
2015-12-19 23:10:13 -08:00
cnauroth
932956458d
HADOOP-12570. HDFS Secure Mode Documentation updates. Contributed by Arpit Agarwal.
2015-12-18 16:24:53 -08:00
Zhe Zhang
ef3f3f6bb1
HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang.
...
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
2015-12-18 16:04:47 -08:00
Zhe Zhang
61ab0440f7
HDFS-9173. Erasure Coding: Lease recovery for striped file. Contributed by Walter Su and Jing Zhao.
...
Change-Id: I51703a61c9d8454f883028f3f6acb5729fde1b15
2015-12-18 15:57:48 -08:00
Andrew Wang
058cfddebb
HDFS-8477. describe dfs.ha.zkfc.port in hdfs-default.xml. Contributed by Kanaka Kumar Avvaru.
2015-12-18 09:14:14 -08:00
Vinayakumar B
bfadf11b36
HDFS-9393. After choosing favored nodes, choosing nodes for remaining replicas should go through BlockPlacementPolicy (Contributed by J.Andreina)
2015-12-18 11:38:12 +05:30
cnauroth
6da9deec20
HDFS-9571. Fix ASF Licence warnings in Jenkins reports. Contributed by Brahma Reddy Battula.
2015-12-17 16:26:01 -08:00
Kihwal Lee
5cb1e0118b
HDFS-9533. seen_txid in the shared edits directory is modified during bootstrapping. Contributed by Kihwal Lee.
2015-12-17 17:11:16 -06:00
cnauroth
03bab8dea1
HDFS-9572. Prevent DataNode log spam if a client connects on the data transfer port but sends no data. Contributed by Chris Nauroth
2015-12-17 14:04:45 -08:00
Arpit Agarwal
2f623fb8cc
HDFS-9515. NPE when MiniDFSCluster#shutdown is invoked on uninitialized reference. (Contributed by Wei-Chiu Chuang)
2015-12-17 13:57:08 -08:00
cnauroth
db37f02dc7
Revert "HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang."
...
This reverts commit eb6939cea0
2015-12-17 13:50:23 -08:00
cnauroth
724e71f63b
HDFS-9552. Document types of permission checks performed for HDFS operations. Contributed by Chris Nauroth.
2015-12-17 10:43:48 -08:00
Arpit Agarwal
72a3f649e4
HDFS-9565. TestDistributedFileSystem.testLocatedFileStatusStorageIdsTypes is flaky due to race condition. (Contributed by Wei-Chiu Chuang)
2015-12-17 10:36:07 -08:00
Allen Wittenauer
8d278d8d2e
HDFS-9570. Minor typos, grammar, and case sensitivity cleanup in HdfsPermissionsGuide.md's (Travis Campbell via aw)
2015-12-17 08:31:44 -08:00
Kihwal Lee
eb6939cea0
HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang.
2015-12-17 10:29:41 -06:00
Uma Mahesh
f741476146
HDFS-9198. Coalesce IBR processing in the NN. (Daryn Sharp via umamahesh)
2015-12-16 18:16:39 -08:00
Andrew Wang
3c0adac88c
HDFS-9300. TestDirectoryScanner.testThrottle() is still a little flakey. Contributed by Daniel Templeton.
2015-12-16 16:21:06 -08:00
cnauroth
c470c8953d
HDFS-9557. Reduce object allocation in PB conversion. Contributed by Daryn Sharp.
2015-12-16 11:10:40 -08:00
Andrew Wang
49949a4bb0
HDFS-8894. Set SO_KEEPALIVE on DN server sockets. Contributed by Kanaka Kumar Avvaru.
2015-12-15 14:38:35 -08:00
Jing Zhao
8602692338
HDFS-9371. Code cleanup for DatanodeManager. Contributed by Jing Zhao.
2015-12-15 10:47:53 -08:00
Plamen Jeliazkov
96d307e1e3
HDFS-9516. Truncate file fails with data dirs on multiple disks. Contributed by Plamen Jeliazkov.
2015-12-15 00:32:10 -08:00
Lei Xu
de522d2cd4
Revert "Revert "HDFS-8860. Remove unused Replica copyOnWrite code (Lei (Eddy) Xu via Colin P. McCabe)""
...
This reverts commit 7f393a6f61
2015-12-14 11:25:09 -08:00
Jing Zhao
e534569814
HDFS-9535. Newly completed blocks in IBR should not be considered under-replicated too quickly. Contributed by Mingliang Liu.
2015-12-14 11:05:25 -08:00
Lei Xu
f229772f99
HDFS-9281. Change TestDeleteBlockPool to not explicitly use File to check block pool existence. (lei)
2015-12-14 11:02:07 -08:00
Yongjun Zhang
bf5295b118
HDFS-9514. TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing; exception being swallowed. (Wei-Chiu Chuang via Yongjun Zhang)
2015-12-11 19:52:38 -08:00
Yongjun Zhang
2a4c7d4fac
HDFS-9519. Some coding improvement in SecondaryNameNode#main. (Xiao Chen via Yongjun Zhang)
2015-12-11 18:44:01 -08:00
Yongjun Zhang
f5a911452f
HDFS-9532. Detailed exception info is lost in reportTo methods of ErrorReportAction and ReportBadBlockAction. (Yongjun Zhang)
2015-12-11 18:15:24 -08:00
Uma Mahesh
796a676d18
HDFS-9528. Cleanup namenode audit/log/exception messages. (szetszwo via umamahesh)
2015-12-11 17:57:35 -08:00
Allen Wittenauer
576b569b6c
Revert "HDFS-9525. hadoop utilities need to support provided delegation tokens (HeeSoo Kim via aw)"
...
This reverts commit 832b3cbde1
2015-12-11 10:59:42 -08:00
Kihwal Lee
a48301791e
HDFS-9445. Datanode may deadlock while handling a bad volume. Contributed by Walter Su.
2015-12-11 08:44:47 -06:00
Uma Mahesh
e363417e7b
HDFS-9472. concat() API does not give proper exception messages on ./reserved relative path (Rakesh R via umamahesh)
2015-12-10 23:55:29 -08:00
Lei Xu
7f393a6f61
Revert "HDFS-8860. Remove unused Replica copyOnWrite code (Lei (Eddy) Xu via Colin P. McCabe)"
...
This reverts commit a153b9601a
2015-12-10 10:57:33 -10:00
Tsz-Wo Nicholas Sze
132478e805
HDFS-9527. The return type of FSNamesystem.getBlockCollection should be changed to INodeFile.
2015-12-09 17:55:28 -08:00
Allen Wittenauer
832b3cbde1
HDFS-9525. hadoop utilities need to support provided delegation tokens (HeeSoo Kim via aw)
2015-12-08 12:56:38 -08:00
cnauroth
303f80e656
HDFS-9414. Refactor reconfiguration of ClientDatanodeProtocol for reusability. Contributed by Xiaobing Zhou.
2015-12-08 10:49:32 -08:00
Steve Loughran
65f395226b
HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072]
2015-12-06 17:43:35 +00:00
Arpit Agarwal
86c95cb31a
HDFS-9414. Refactor reconfiguration of ClientDatanodeProtocol for reusability. (Contributed by Xiaobing Zhou)
2015-12-04 20:24:08 -08:00
Lei Xu
4265a85f6d
HDFS-9491. Tests should get the number of pending async deletes via FsDatasetTestUtils. (Tony Wu via lei)
2015-12-04 18:06:16 -08:00
Arpit Agarwal
bc6d8d9814
HDFS-9214. Add missing license header
2015-12-04 15:33:09 -08:00
Arpit Agarwal
9d817fa1b1
HDFS-9214. Support reconfiguring dfs.datanode.balance.max.concurrent.moves without DN restart. (Contributed by Xiaobing Zhou)
2015-12-04 14:46:46 -08:00
Yongjun Zhang
59dbe8b3e9
HDFS-9474. TestPipelinesFailover should not fail when printing debug message. (John Zhuge via Yongjun Zhang)
2015-12-04 13:45:01 -08:00
Colin P. McCabe
e02bbeb886
HDFS-9267. TestDiskError should get stored replicas through FsDatasetTestUtils. (Lei (Eddy) Xu via Colin P. McCabe)
2015-12-04 12:51:19 -08:00
Xiaoyu Yao
cbc7b6bf97
HDFS-8831. Trash Support for deletion in HDFS encryption zone. Contributed by Xiaoyu Yao.
2015-12-04 10:39:45 -08:00
Lei Xu
0ac8fb4b33
HDFS-9490. MiniDFSCluster should change block generation stamp via FsDatasetTestUtils. (Tony Wu via lei)
2015-12-04 10:24:55 -08:00
Ming Ma
3fa33b5c2c
HDFS-9430 Remove waitForLoadingFSImage since checkNNStartup has ensured image loaded and namenode started. (Brahma Reddy Battula via mingma)
2015-12-04 09:47:57 -08:00
Konstantin V Shvachko
924a33d02d
HDFS-9484. NNThroughputBenchmark$BlockReportStats should not send empty block reports. Contributed by Mingliang Liu.
2015-12-03 15:03:34 -08:00
Konstantin V Shvachko
e71aa717d4
HDFS-9436. Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default. Contributed by Mingliang.
2015-12-03 15:03:33 -08:00
Colin Patrick Mccabe
53e3bf7e70
HDFS-9429. Tests in TestDFSAdminWithHA intermittently fail with EOFException (Xiao Chen via Colin P. McCabe)
2015-12-01 23:21:21 -08:00
Jing Zhao
a49cc74b4c
HDFS-9129. Move the safemode block count into BlockManager. Contributed by Mingliang Liu.
2015-12-01 16:09:19 -08:00
Arpit Agarwal
58f6f54eea
HDFS-6533. TestBPOfferService#testBasicFunctionalitytest fails intermittently. (Contributed by Wei-Chiu Chuang)
2015-12-01 13:32:32 -08:00
Jing Zhao
830eb252aa
HDFS-9485. Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block. Contributed by Mingliang Liu.
2015-12-01 13:05:22 -08:00
Akira Ajisaka
1c05393b51
HDFS-9336. deleteSnapshot throws NPE when snapshotname is null. Contributed by Brahma Reddy Battula.
2015-12-01 11:29:09 +08:00
Andrew Wang
9b8e50b424
HDFS-9470. Encryption zone on root not loaded from fsimage after NN restart. Xiao Chen via wang.
2015-11-30 14:32:51 -08:00
Vinayakumar B
9f256d1d71
HDFS-9426. Rollingupgrade finalization is not backward compatible (Contributed by Kihwal Lee)
...
(cherry picked from commit c62d42cd8bb09a5ffc0c5eefa2d87913e71b9e7e)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
2015-11-26 09:39:47 +05:30
Jing Zhao
e556c35b05
HDFS-9467. Fix data race accessing writeLockHeldTimeStamp in FSNamesystem. Contributed by Mingliang Liu.
2015-11-25 14:22:12 -08:00
Konstantin V Shvachko
fc799ab16c
HDFS-9407. TestFileTruncate should not use fixed NN port. Contributed by Brahma Reddy Battula.
2015-11-25 13:52:09 -08:00
Xiaoyu Yao
e3d673901b
HDFS-8512. WebHDFS : GETFILESTATUS should return LocatedBlock with storage type info. Contributed by Xiaoyu Yao.
2015-11-25 13:41:06 -08:00
Haohui Mai
b21dffb1fe
HDFS-9451. Clean up depreated umasks and related unit tests. Contributed by Wei-Chiu Chuang.
2015-11-25 12:47:24 -08:00
Yongjun Zhang
8176ea7dc6
HDFS-9438. TestPipelinesFailover assumes Linux ifconfig. (John Zhuge via Yongjun Zhang)
2015-11-25 07:40:16 -08:00
Tsz-Wo Nicholas Sze
78ec38b2ed
HDFS-8807. dfs.datanode.data.dir does not handle spaces between storageType and URI correctly. Contributed by Anu Engineer
2015-11-24 16:01:55 -08:00
Haohui Mai
977e0b3c4c
HDFS-8335. FSNamesystem should construct FSPermissionChecker only if permission is enabled. Contributed by Gabor Liptak.
2015-11-24 13:14:49 -08:00
Xiaoyu Yao
fe5624b85d
HDFS-8855. Webhdfs client leaks active NameNode connections. Contributed by Xiaobing Zhou.
2015-11-24 12:47:57 -08:00
Haohui Mai
db4cab21f4
HDFS-5165. Remove the TotalFiles metrics. Contributed by Akira AJISAKA.
2015-11-24 11:41:57 -08:00
Ming Ma
0e54b164a8
HDFS-9314. Improve BlockPlacementPolicyDefault's picking of excess replicas. (Xiao Chen via mingma)
2015-11-24 10:30:24 -08:00
cnauroth
1777608fa0
HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed by Wei-Chiu Chuang.
2015-11-24 09:39:21 -08:00
Uma Mahesh
411e2b2e7c
HDFS-9433. DFS getEZForPath API on a non-existent file should throw FileNotFoundException (Rakesh R via umamahesh)
2015-11-23 17:54:31 -08:00
Walter Su
201f14e271
HDFS-9423. Fix intermittent failure of TestEditLogTailer. Contributed by Masatake Iwasaki.
2015-11-23 18:49:40 +08:00
Walter Su
001ecf962c
HDFS-9435. TestBlockRecovery#testRBWReplicas is failing intermittently. Contributed by Rakesh R.
2015-11-23 15:49:58 +08:00
Walter Su
5aba093361
HDFS-9428. Fix intermittent failure of TestDNFencing.testQueueingWithAppend. Contributed by Masatake Iwasaki.
2015-11-23 15:37:45 +08:00
Haohui Mai
8228697857
HDFS-9024. Deprecate the TotalFiles metric. Contributed by Akira Ahisaka.
2015-11-22 23:02:50 -08:00
Haohui Mai
04c14b5dc4
HDFS-9356. Decommissioning node does not have Last Contact value in the UI. Contributed by Surendra Singh Lilhore.
2015-11-22 22:58:58 -08:00
Haohui Mai
a4bd54f9d7
HDFS-7897. Shutdown metrics when stopping JournalNode. Contributed by zhouyingchao.
2015-11-22 19:12:23 -08:00
Haohui Mai
bfbcfe7364
HDFS-6885. Fix wrong use of BytesWritable in FSEditLogOp#RenameOp. Contributed by Yi Liu.
2015-11-22 17:16:51 -08:00
Haohui Mai
2326171ea8
HDFS-3302. Review and improve HDFS trash documentation. Contributed by Harsh J.
2015-11-22 16:30:39 -08:00
Haohui Mai
38146a6cdb
HDFS-7796. Include X-editable for slick contenteditable fields in the web UI. Contributed by Ravi Prakash.
2015-11-22 16:22:06 -08:00
Haohui Mai
0c7340f377
HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke.
2015-11-22 16:10:27 -08:00
Haohui Mai
176ff5ce90
HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su.
2015-11-22 15:54:41 -08:00
Uma Mahesh
5b714a2819
HDFS-9348. Erasure Coding: DFS GetErasureCodingPolicy API on a non-existent file should be handled properly. (Rakesh R via umamahesh)
2015-11-21 22:38:36 -08:00
Tsz-Wo Nicholas Sze
8e03e855b6
HDFS-9434. Recommission a datanode with 500k blocks may pause NN for 30 seconds for printing info log messags.
2015-11-19 14:54:01 -08:00
Yongjun Zhang
31dc5c650e
HDFS-9439. Include status of closeAck into exception message in DataNode#run. (Xiao Chen via Yongjun Zhang)
2015-11-19 14:20:33 -08:00
Ming Ma
1c4951a7a0
HDFS-8056. Decommissioned dead nodes should continue to be counted as dead after NN restart. (mingma)
2015-11-19 10:04:01 -08:00
Akira Ajisaka
ac1aa6c819
HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii.
2015-11-19 20:55:01 +09:00
Colin Patrick Mccabe
dfbde3fc51
HDFS-9252. Change TestFileTruncate to use FsDatasetTestUtils to get block file size and genstamp. (Lei (Eddy) Xu via cmccabe)
2015-11-17 10:48:15 -08:00
Arpit Agarwal
007c6ce1af
HDFS-9397. Fix typo for readChecksum() LOG.warn in BlockSender.java. (Contributed by Enrique Flores)
2015-11-16 21:02:00 -08:00
Walter Su
621cbb4f69
HDFS-9358. TestNodeCount#testNodeCount timed out. Contributed by Masatake Iwasaki.
2015-11-17 10:51:29 +08:00
Xiaoyu Yao
a4f62a2d58
HDFS-9421. NNThroughputBenchmark replication test NPE with -namenode option. Contributed by Mingliang Liu.
2015-11-16 16:42:15 -08:00
Xiaoyu Yao
2701f2d255
HDFS-9387. Fix namenodeUri parameter parsing in NNThroughputBenchmark. Contributed by Mingliang Liu.
2015-11-16 16:29:47 -08:00
Ming Ma
02653add98
HDFS-9413. getContentSummary() on standby should throw StandbyException. (Brahma Reddy Battula via mingma)
2015-11-16 09:32:40 -08:00
Walter Su
cccf88480b
HDFS-9410. Some tests should always reset sysout and syserr. Contributed by Xiao Chen.
2015-11-13 14:26:43 +08:00
Zhe Zhang
7b00c8e20e
HDFS-8968. Erasure coding: a comprehensive I/O throughput benchmark tool. Contributed by Rui Li.
...
Change-Id: I01ff5b04727fd79e6373582d4815c5e7b2096c67
2015-11-12 11:30:39 -08:00
Kihwal Lee
a2a104f3d3
HDFS-9396. Total files and directories on jmx and web UI on standby is uninitialized. Contributed by Kihwal Lee.
2015-11-11 15:24:43 -06:00
Zhe Zhang
73b94d7899
HDFS-9364. Unnecessary DNS resolution attempts when creating NameNodeProxies. Contributed by Xiao Chen.
...
Change-Id: I9e42f724f27924cf73891425a832de37ce014a1e
2015-11-10 09:55:29 -08:00
Walter Su
2fda45b9dc
HDFS-9401. Fix findbugs warnings in BlockRecoveryWorker. Contributed by Brahma Reddy Battula.
2015-11-10 14:48:08 +08:00
Haohui Mai
94a1833638
HDFS-2261. AOP unit tests are not getting compiled or run. Contributed by Haohui Mai.
2015-11-09 15:25:19 -08:00
Yongjun Zhang
2741a2109b
HDFS-9249. NPE is thrown if an IOException is thrown in NameNode constructor. (Wei-Chiu Chuang via Yongjun Zhang)
2015-11-09 14:04:03 -08:00
Xiaoyu Yao
41d3f8899d
HDFS-9234. WebHdfs: getContentSummary() should give quota for storage types. Contributed by Surendra Singh Lilhore.
2015-11-09 09:57:56 -08:00
Vinayakumar B
6d2332ae37
HDFS-9057. allow/disallow snapshots via webhdfs (Contributed by Brahma Reddy Battula)
2015-11-08 16:24:27 +05:30
Allen Wittenauer
b76b0ce51e
HADOOP-12366. expose calculated paths (aw)
2015-11-07 08:32:56 -08:00
Arpit Agarwal
2801b42a7e
HDFS-9379. Make NNThroughputBenchmark support more than 10 datanodes. (Contributed by Mingliang Liu)
2015-11-06 18:58:49 -08:00
Kihwal Lee
bf6aa30a15
HDFS-9318. considerLoad factor can be improved. Contributed by Kuhu Shukla.
2015-11-06 13:30:33 -06:00
Yongjun Zhang
b64242c0d2
HDFS-9236. Missing sanity check for block size during block recovery. (Tony Wu via Yongjun Zhang)
2015-11-06 11:15:54 -08:00
Arpit Agarwal
0b18e5e8c6
HDFS-6481. DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs. (Contributed by szetszwo)
2015-11-06 10:15:23 -08:00
Yongjun Zhang
286cc6483f
HDFS-9377. Fix findbugs warnings in FSDirSnapshotOp. (Mingliang Liu via Yongjun Zhang)
2015-11-05 14:36:12 -08:00
Lei Xu
46e78a7a16
HDFS-9282. Make data directory count and storage raw capacity related tests FsDataset-agnostic. (Tony Wu via lei)
2015-11-05 10:47:33 -08:00
Haohui Mai
efc73d7896
HDFS-9372. Remove dead code in DataStorage.recoverTransitionRead. Contributed by Duo Zhang.
2015-11-05 10:00:01 -08:00
Kihwal Lee
ff47f35dee
HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee.
2015-11-05 09:25:20 -06:00
Xiaoyu Yao
ea5bb48326
HDFS-9360. Storage type usage isn't updated properly after file deletion. Contributed by Ming Ma.
2015-11-05 01:30:13 -08:00
Lei Xu
5667129276
HDFS-9363. Add fetchReplica to FsDatasetTestUtils to return FsDataset-agnostic replica. (Tony Wu via lei)
2015-11-04 10:49:28 -08:00
Lei Xu
e2a5441b06
HDFS-9331. Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use. (Tony Wu via lei)
2015-11-04 10:27:35 -08:00
Lei Xu
ec414600ed
HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma via lei)
2015-11-04 10:22:17 -08:00
Haohui Mai
88beb46cf6
Revert "HDFS-8855. Webhdfs client leaks active NameNode connections. Contributed by Xiaobing Zhou."
...
This reverts commit 84cbd72afd
2015-11-04 10:21:13 -08:00
Haohui Mai
0eed886a16
HDFS-9357. NN UI renders icons of decommissioned DN incorrectly. Contributed by Surendra Singh Lilhore.
2015-11-04 09:16:43 -08:00
Varun Vasudev
73b9c7b82b
HADOOP-10787. Rename/remove non-HADOOP_*, etc from the shell scripts. Contributed by Allen Wittenauer.
2015-11-04 15:56:17 +05:30
Yongjun Zhang
194251c852
HDFS-9351. checkNNStartup() need to be called when fsck calls FSNamesystem.getSnapshottableDirs(). (Xiao Chen via Yongjun Zhang)
2015-11-03 17:16:17 -08:00
Zhe Zhang
dac0463a4e
HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock. Contributed by Chang Li.
...
Change-Id: If5ce1b2d212bb0726bce52ad12a3de401bcec02d
2015-11-03 13:34:24 -08:00
cnauroth
7e2829662b
HDFS-9362. TestAuditLogger#testAuditLoggerWithCallContext assumes Unix line endings, fails on Windows. Contributed by Chris Nauroth.
2015-11-03 11:54:57 -08:00
cnauroth
095ac83402
HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows. Contributed by Xiaoyu Yao.
2015-11-03 11:21:08 -08:00
Ming Ma
d565480da2
HDFS-9313. Possible NullPointerException in BlockManager if no excess replica can be chosen. (mingma)
2015-11-02 19:36:37 -08:00
Lei Xu
8e05dbf2bd
HDFS-9308. Add truncateMeta() and deleteMeta() to MiniDFSCluster. (Tony Wu via lei)
2015-11-02 18:05:43 -08:00
yliu
5ba2b98d0f
HDFS-9275. Wait previous ErasureCodingWork to finish before schedule another one. (Walter Su via yliu)
2015-11-03 09:14:32 +08:00
Lei Xu
7632409482
HDFS-9312. Fix TestReplication to be FsDataset-agnostic. (lei)
2015-11-02 17:09:39 -08:00
Zhe Zhang
78d6890865
HDFS-9339. Extend full test of KMS ACLs. Contributed by Daniel Templeton.
...
Change-Id: I61eeee8fa5e85250eabc1eef3d8c11f32700d6fb
2015-11-02 13:51:45 -08:00
Zhe Zhang
259bea3b48
HDFS-9329. TestBootstrapStandby#testRateThrottling is flaky because fsimage size is smaller than IO buffer size. Contributed by Zhe Zhang.
...
Change-Id: I09896c46e9ee0718b67c64fac5acfb3f7decf0b9
2015-11-02 10:03:39 -08:00
Zhe Zhang
90e1405516
HDFS-8777. Erasure Coding: add tests for taking snapshots on EC files. Contributed by Rakesh R.
...
Change-Id: Ia3ae582405e741ca8e90d9255ab9b95d085e5fa8
2015-11-02 07:48:30 -08:00
Arpit Agarwal
3cde6931cb
HDFS-9343. Empty caller context considered invalid. (Contributed by Mingliang Liu)
2015-11-01 15:35:02 -08:00
yliu
7fd6416759
Revert "HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee."
...
This reverts commit 43539b5ff4
2015-10-31 16:20:48 +08:00
Kihwal Lee
43539b5ff4
HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee.
2015-10-30 09:29:13 -05:00
Tsz-Wo Nicholas Sze
f072eb5a20
HDFS-9323. Randomize the DFSStripedOutputStreamWithFailure tests.
2015-10-30 15:41:03 +08:00
Andrew Wang
888c6245e2
HDFS-9332. Fix Precondition failures from NameNodeEditLogRoller while saving namespace.
2015-10-29 11:30:47 -07:00
Zhe Zhang
8def51a708
HDFS-9229. Expose size of NameNode directory as a metric. Contributed by Surendra Singh Lilhore.
...
Change-Id: I985627a5d1400249d72d084283ef366d5ac6e07b
2015-10-29 11:23:08 -07:00
Walter Su
5eca6dece6
HADOOP-12327. Initialize output buffers with ZERO bytes in erasure coder. Contributed by Kai Zheng.
2015-10-29 15:04:34 +08:00
Vinayakumar B
588baab160
HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group (Contributed by J.Andreina)
2015-10-29 11:44:01 +05:30
Vinayakumar B
7d2d16f4ee
HDFS-8545. Refactor FS#getUsed() to use ContentSummary and add an API to fetch the total file length from a specific path (Contributed by J.Andreina)
2015-10-29 11:28:17 +05:30
Haohui Mai
65f53f246a
HDFS-9168. Move client side unit test to hadoop-hdfs-client. Contributed by Haohui Mai.
2015-10-28 15:54:37 -07:00
Zhe Zhang
73bc65e02b
HDFS-9295. Add a thorough test of the full KMS code path. Contributed by Daniel Templeton.
...
Change-Id: Ife1abb2f8e9a30102c4f705c47646d9ac8328786
2015-10-28 14:36:20 -07:00
Kihwal Lee
19a77f5466
HDFS-9279. Decomissioned capacity should not be considered for configured/used capacity. Contributed by Kihu Shukla .
2015-10-28 11:58:51 -05:00
Zhe Zhang
e287e7d14b
HDFS-9255. Consolidate block recovery related implementation into a single class. Contributed by Walter Su.
...
Change-Id: I7a1c03f50123d79ac0a78c981d9721617e3229d1
2015-10-28 07:34:06 -07:00
yliu
c201cf951d
HADOOP-12040. Adjust inputs order for the decode API in raw erasure coder. (Kai Zheng via yliu)
2015-10-28 16:18:23 +08:00
yliu
6ff6663f64
HDFS-9302. WebHDFS throws NullPointerException if newLength is not provided. (Jagadesh Kiran N via yliu)
2015-10-28 15:54:04 +08:00
Yongjun Zhang
97913f430c
HDFS-9231. fsck doesn't list correct file path when Bad Replicas/Blocks are in a snapshot. (Xiao Chen via Yongjun Zhang)
2015-10-27 23:31:50 -07:00
cnauroth
bf8e452982
HDFS-9311. Support optional offload of NameNode HA service health checks to a separate RPC server. Contributed by Chris Nauroth.
2015-10-27 23:07:14 -07:00
Allen Wittenauer
1396867b52
HADOOP-12494. fetchdt stores the token based on token kind instead of token service (HeeSoo Kim via aw)
2015-10-27 12:02:43 -07:00
Colin Patrick Mccabe
fe93577faf
HDFS-9299. Give ReplicationMonitor a readable thread name (Staffan Friberg via Colin P. McCabe)
2015-10-27 11:37:26 -07:00
Ming Ma
aa09880ab8
HDFS-9259. Make SO_SNDBUF size configurable at DFSClient side for hdfs write scenario. (Mingliang Liu via mingma)
2015-10-27 09:28:40 -07:00
Lei Xu
399ad00915
HDFS-9292. Make TestFileConcorruption independent to underlying FsDataset Implementation. (lei)
2015-10-26 16:09:22 -07:00
Arpit Agarwal
d8736eb9ca
HDFS-9305. Delayed heartbeat processing causes storm of subsequent heartbeats. (Contributed by Arpit Agarwal)
2015-10-26 15:54:14 -07:00
Andrew Wang
e8aefdf08b
HDFS-8945. Update the description about replica placement in HDFS Architecture documentation. Contributed by Masatake Iwasaki.
2015-10-26 15:47:59 -07:00
Lei Xu
37bf6141f1
HDFS-9291. Fix TestInterDatanodeProtocol to be FsDataset-agnostic. (lei)
2015-10-26 15:18:24 -07:00
Andrew Wang
677a936bf7
HDFS-9284. fsck command should not print exception trace when file not found. Contributed by Jagadesh Kiran N.
2015-10-26 15:15:03 -07:00
Yongjun Zhang
5e718de522
HDFS-7284. Add more debug info to BlockInfoUnderConstruction#setGenerationStampAndVerifyReplicas. (Wei-Chiu Chuang via Yongjun Zhang)
2015-10-26 13:25:31 -07:00
Haohui Mai
67e3d75aed
HDFS-9304. Add HdfsClientConfigKeys class to TestHdfsConfigFields#configurationClasses. Contributed by Mingliang Liu.
2015-10-26 12:02:52 -07:00
Arpit Agarwal
86c92227fc
HDFS-4015. Safemode should count and report orphaned blocks. (Contributed by Anu Engineer)
2015-10-23 18:07:17 -07:00
Lei Xu
5679e46b7f
HDFS-9297. Update TestBlockMissingException to use corruptBlockOnDataNodesByDeletingBlockFile(). (Tony Wu via lei)
2015-10-23 17:42:23 -07:00
Haohui Mai
15eb84b37e
HDFS-9301. HDFS clients can't construct HdfsConfiguration instances. Contributed by Mingliang Liu.
2015-10-23 14:53:34 -07:00
Zhe Zhang
ab3c4cff4a
HDFS-8808. dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby. Contributed by Zhe Zhang.
2015-10-23 14:01:49 -07:00
Lei Xu
533a2be5ac
HDFS-9264. Minor cleanup of operations on FsVolumeList#volumes. (Walter Su via lei)
2015-10-23 13:52:59 -07:00
Jitendra Pandey
600ad7bf41
HDFS-9184. Logging HDFS operation's caller context into audit logs. Contributed by Mingliang Liu.
2015-10-23 12:15:01 -07:00
Zhe Zhang
aea26bf4dd
HDFS-9280. Document NFS gateway export point parameter. Contributed by Xiao Chen.
...
Change-Id: I1cea610c31301d793db3c9ca4dae86d0e5d2d64b
2015-10-22 11:38:04 -07:00
Andrew Wang
3dadf369d5
HDFS-7087. Ability to list /.reserved. Contributed by Xiao Chen.
2015-10-21 16:58:47 -07:00
cnauroth
1b525a9c32
HDFS-9273. ACLs on root directory may be lost after NN restart. Contributed by Xiao Chen.
2015-10-21 16:39:02 -07:00
Andrew Wang
a24c6e8420
HDFS-9278. Fix preferredBlockSize typo in OIV XML output. Contributed by Nicole Pazmany.
2015-10-21 14:58:21 -07:00
Andrew Wang
3e50f478b9
HDFS-9225. Fix intermittent test failure of TestBlockManager.testBlocksAreNotUnderreplicatedInSingleRack. Contributed by Masatake Iwasaki.
2015-10-21 14:55:23 -07:00
Jing Zhao
d806a5bf07
HDFS-9070. Allow fsck display pending replica location information for being-written blocks. Contributed by Gao Rui.
2015-10-21 14:44:05 -07:00
Zhe Zhang
59ce780d53
HDFS-9274. Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec should be consistent. Contributed by Yi Liu.
...
Change-Id: Iafa2efb8f2eafd193b417690cc8794080b9d745b
2015-10-21 14:03:12 -07:00
Ming Ma
e27c2ae8ba
HDFS-8647. Abstract BlockManager's rack policy into BlockPlacementPolicy. (Brahma Reddy Battula via mingma)
2015-10-21 08:06:58 -07:00
Andrew Wang
6c8b6f3646
HDFS-3059. ssl-server.xml causes NullPointer. Contributed by Xiao Chen.
2015-10-20 13:44:53 -07:00
Colin Patrick Mccabe
6381ddc096
HDFS-9270. TestShortCircuitLocalRead should not leave socket after unit test (Masatake Iwasaki via Colin P. McCabe)
2015-10-20 10:51:18 -07:00
Lei Xu
71e533a153
HDFS-9251. Refactor TestWriteToReplica and TestFsDatasetImpl to avoid explicitly creating Files in the tests code. (lei)
2015-10-20 10:08:02 -07:00
yliu
9cb5d35353
HDFS-9208. Disabling atime may fail clients like distCp. (Kihwal Lee via yliu)
2015-10-20 11:33:04 +08:00
Andrew Wang
8175c4f6b9
HDFS-9250. Add Precondition check to LocatedBlock#addCachedLoc. Contributed by Xiao Chen.
2015-10-19 16:12:56 -07:00
Tsuyoshi Ozawa
4c9c3955ae
HDFS-9237. NPE at TestDataNodeVolumeFailureToleration#tearDown. Contributed by Brahma Reddy Battula.
2015-10-19 15:30:21 +09:00
Harsh J
52ac73f344
HDFS-9257. improve error message for "Absolute path required" in INode.java to contain the rejected path. Contributed by Marcell Szabo
2015-10-16 17:12:35 +02:00
Jing Zhao
7a98d94b7b
Add Apache license header in DatanodeStats.java
2015-10-15 10:32:20 -07:00
Kihwal Lee
c7c36cbd62
HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao.
2015-10-15 11:24:14 -05:00
Vinayakumar B
08f5de1ef5
HDFS-9157. [OEV and OIV] : Unnecessary parsing for mandatory arguements if '-h' option is specified as the only option (Contributed by nijel)
2015-10-15 17:37:06 +05:30
Tsz-Wo Nicholas Sze
5411dc559d
HDFS-9205. Do not schedule corrupt blocks for replication. (szetszwo)
2015-10-15 18:07:09 +08:00
Lei Xu
c80b3a804f
HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei)
2015-10-14 20:18:43 -07:00
Jing Zhao
be7a0add8b
HDFS-9223. Code cleanup for DatanodeDescriptor and HeartbeatManager. Contributed by Jing Zhao.
2015-10-14 16:17:49 -07:00
Xiaoyu Yao
a8070259f8
Revert "HDFS-9210. Fix some misuse of %n in VolumeScanner#printStats. Contributed by Xiaoyu Yao."
...
This reverts commit 3d50855952
2015-10-14 14:41:13 -07:00
Xiaoyu Yao
3d50855952
HDFS-9210. Fix some misuse of %n in VolumeScanner#printStats. Contributed by Xiaoyu Yao.
2015-10-14 13:22:58 -07:00
Lei Xu
ba3c197878
HDFS-9238. Update TestFileCreation.testLeaseExpireHardLimit() to avoid using DataNodeTestUtils.getFile(). (Tony Wu via lei)
2015-10-14 13:10:26 -07:00
Jing Zhao
2a98724342
HDFS-1172. Blocks in newly completed files are considered under-replicated too quickly. Contributed by Masatake Iwasaki.
2015-10-13 23:00:18 -07:00
Haohui Mai
da8441d0fe
HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu.
2015-10-13 15:24:24 -07:00
Haohui Mai
aa299ecd5e
HDFS-8779. WebUI fails to display block IDs that are larger than 2^53 - 1. Contributed by Haohui Mai.
2015-10-13 13:53:28 -07:00
Haohui Mai
d1e1925bf6
HDFS-9145. Tracking methods that hold FSNamesytemLock for too long. Contributed by Mingliang Liu.
2015-10-13 13:24:15 -07:00
Jing Zhao
da16c9b3b4
HDFS-8438. Erasure Coding: Allow concat striped files if they have the same ErasureCodingPolicy. Contributed by Walter Su.
2015-10-13 11:03:37 -07:00
Jitendra Pandey
84cbd72afd
HDFS-8855. Webhdfs client leaks active NameNode connections. Contributed by Xiaobing Zhou.
2015-10-13 09:12:04 -07:00
Kihwal Lee
5b43db47a3
HDFS-8676. Delayed rolling upgrade finalization can cause heartbeat expiration. Contributed by Walter Su.
2015-10-13 10:04:18 -05:00
Vinayakumar B
39581e3be2
HDFS-9139. Enable parallel JUnit tests for HDFS Pre-commit (Contributed by Chris Nauroth and Vinayakumar B)
2015-10-13 19:00:08 +05:30
Vinayakumar B
caa711b660
HDFS-9160. [OIV-Doc] : Missing details of 'delimited' for processor options (Contributed by nijel)
2015-10-13 18:13:01 +05:30
Vinayakumar B
69b025dbba
HDFS-9224. TestFileTruncate fails intermittently with BindException (Contributed by Brahma Reddy Battula)
2015-10-13 11:42:12 +05:30
Lei Xu
0f5f9846ed
HDFS-9006. Provide BlockPlacementPolicy that supports upgrade domain. (Ming Ma via lei)
2015-10-12 16:24:16 -07:00
yliu
73b86a5046
HDFS-8988. Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap. (yliu)
2015-10-12 14:40:14 +08:00
yliu
0ff1216100
HDFS-9221. HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array. (Staffan Friberg via yliu)
2015-10-12 14:28:15 +08:00
cnauroth
4f6e842ba9
HDFS-9085. Show renewer information in DelegationTokenIdentifier#toString. Contributed by Zhihai Xu.
2015-10-09 14:27:13 -07:00
Andrew Wang
c32614f410
HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should resolve relative path. Contributed by Rakesh R.
2015-10-09 11:57:03 -07:00
Andrew Wang
357b1fd082
HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for better efficiency. Contributed by Charlie Helin.
2015-10-09 11:49:16 -07:00
Ming Ma
de8efc65a4
HDFS-9142. Separating Configuration object for namenode(s) in MiniDFSCluster. (Siqi Li via mingma)
2015-10-09 11:10:46 -07:00
Yongjun Zhang
c11fc8a1be
HDFS-9181. Better handling of exceptions thrown during upgrade shutdown. Contributed by Wei-Chiu Chuang.
2015-10-09 09:21:29 -07:00
Jing Zhao
118a35bc2e
HDFS-9204. DatanodeDescriptor#PendingReplicationWithoutTargets is wrongly calculated. Contributed by Mingliang Liu.
2015-10-08 14:11:02 -07:00
Yongjun Zhang
1107bd399c
HDFS-8164. cTime is 0 in VERSION file for newly formatted NameNode. Contributed by Xiao Chen.
2015-10-07 23:20:10 -07:00
yliu
35affec38e
HDFS-9137. DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceeded. (Uma Maheswara Rao G via yliu)
2015-10-08 10:27:47 +08:00
Andrew Wang
66e2cfa1a0
HDFS-8632. Add InterfaceAudience annotation to the erasure coding classes. Contributed by Rakesh R.
2015-10-07 18:12:26 -07:00
Jing Zhao
fde729feeb
HDFS-9209. Erasure coding: Add apache license header in TestFileStatusWithECPolicy.java. Contributed by Surendra Singh Lilhore.
2015-10-07 15:17:50 -07:00
Lei Xu
6dd47d754c
HDFS-9176. Fix TestDirectoryScanner#testThrottling often fails. (Daniel Templeton via lei)
2015-10-07 14:33:43 -07:00
Kihwal Lee
99e5204ff5
HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee.
2015-10-07 10:17:17 -05:00
Vinayakumar B
61b3547c46
HDFS-9159. [OIV] : return value of the command is not correct if invalid value specified in '-p (processor)' option (Contributed by nijel)
2015-10-07 14:29:34 +05:30
Haohui Mai
3112f26368
HDFS-9170. Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client. Contributed by Haohui Mai.
2015-10-07 00:06:33 -07:00
Jing Zhao
8e53311ca2
HDFS-9206. Inconsistent default value of dfs.datanode.stripedread.buffer.size. Contributed by Walter Su.
2015-10-06 23:44:05 -07:00
Jing Zhao
a8b4d0ff28
HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers. Contributed by Jing Zhao.
2015-10-06 10:56:10 -07:00
Colin Patrick Mccabe
2fc2b50534
HDFS-9155. OEV should treat .XML files as XML even when the file name extension is uppercase (nijel via cmccabe)
2015-10-05 12:53:58 -07:00
Vinayakumar B
b925cf1006
HDFS-9158. [OEV-Doc] : Document does not mention about '-f' and '-r' options (Contributed by nijel)
2015-10-05 18:23:18 +05:30
Vinayakumar B
0faa4efa3d
HDFS-9151. Mover should print the exit status/reason on console like balancer tool. (Contributed by Surendra singh lilhore)
2015-10-05 13:22:44 +05:30
Tsuyoshi Ozawa
30e2f836a2
HADOOP-11098. [JDK8] Max Non Heap Memory default changed between JDK7 and 8.
2015-10-05 08:15:06 +09:00
Haohui Mai
3b85bd7c33
HDFS-9193. Fix incorrect references the usages of the DN in dfshealth.js. Contributed by Chang Li.
2015-10-03 11:44:34 -07:00
Yongjun Zhang
1037ee580f
HDFS-9100. HDFS Balancer does not respect dfs.client.use.datanode.hostname. Contributed by Casey Brotherton.
2015-10-02 12:00:14 -07:00
Lei Xu
a68b6eb0f4
HDFS-9015. Refactor TestReplicationPolicy to test different block placement policies. (Ming Ma via lei)
2015-10-02 11:42:09 -07:00
Jakob Homan
3929ac9340
HDFS-9191. Typo in Hdfs.java. NoSuchElementException is misspelled. Contributed by Catherine Palmer.
2015-10-02 11:13:24 -07:00
Jing Zhao
c6cafc77e6
HDFS-9185. Fix null tracer in ErasureCodingWorker. Contributed by Rakesh R.
2015-10-02 11:08:17 -07:00
Zhe Zhang
6c17d31528
Merge remote-tracking branch 'apache/trunk' into HDFS-7285
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
Change-Id: I53ec1c426dc988d6c4a2c87b00caef49c4057010
2015-09-30 08:39:13 -07:00
Aaron T. Myers
071733dc69
HDFS-9001. DFSUtil.getNsServiceRpcUris() can return too many entries in a non-HA, non-federated cluster. Contributed by Daniel Templeton.
2015-09-29 18:19:31 -07:00
Haohui Mai
6f335e4f0e
HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou.
2015-09-29 14:59:13 -07:00
Haohui Mai
8703301b46
HDFS-9174. Fix findbugs warnings in FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread. Contributed by Yi Liu.
2015-09-29 14:56:06 -07:00
Haohui Mai
80d33b589b
HDFS-9165. Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client. Contributed by Mingliang Liu.
2015-09-29 09:50:04 -07:00
Colin Patrick Mccabe
850d679acb
HADOOP-12447. Clean up some htrace integration issues (cmccabe)
2015-09-29 09:25:11 -07:00
yliu
715dbddf77
HDFS-9141. Thread leak in Datanode#refreshVolumes. (Uma Maheswara Rao G via yliu)
2015-09-29 22:05:34 +08:00
Zhe Zhang
8fd5520246
Merge remote-tracking branch 'apache/trunk' into HDFS-7285
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
Change-Id: Ic7946c4ea35bed587fe879ce58b959b25ecc0823
2015-09-29 01:39:16 -07:00
yliu
d6fa34e014
HDFS-8859. Improve DataNode ReplicaMap memory footprint to save about 45%. (yliu)
2015-09-29 16:20:35 +08:00
Kihwal Lee
3abbdc929b
HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu.
2015-09-28 17:31:51 -05:00
Jing Zhao
6419900ac2
HDFS-9040. Erasure coding: coordinate data streamers in DFSStripedOutputStream. Contributed by Jing Zhao and Walter Su.
2015-09-28 14:40:27 -07:00
Lei Xu
e5992ef4df
HDFS-9147. Fix the setting of visibleLength in ExternalBlockReader. (Colin P. McCabe via Lei (Eddy) Xu)
2015-09-28 13:39:00 -07:00
Lei Xu
50741cb568
HDFS-9148. Incorrect assert message in TestWriteToReplica#testWriteToTemporary (Tony Wu via Lei (Eddy) Xu)
2015-09-28 13:27:25 -07:00
Colin Patrick Mccabe
892ade689f
HDFS-9080. Update htrace version to 4.0.1 (cmccabe)
2015-09-28 07:42:12 -07:00
Haohui Mai
1c030c6e58
HDFS-8740. Move DistributedFileSystem to hadoop-hdfs-client. Contributed by Mingliang Liu.
2015-09-27 10:54:44 -07:00
Colin Patrick Mccabe
0b31c237f2
HDFS-9087. Add some jitter to DataNode.checkDiskErrorThread (Elliott Clark via Colin P. McCabe)
2015-09-26 20:36:24 -07:00
Haohui Mai
bf37d3d80e
HDFS-8053. Move DFSIn/OutputStream and related classes to hadoop-hdfs-client. Contributed by Mingliang Liu.
2015-09-26 11:08:25 -07:00
Colin Patrick Mccabe
7a3c381b39
HDFS-8873. Allow the directoryScanner to be rate-limited (Daniel Templeton via Colin P. McCabe)
2015-09-26 04:09:06 -07:00
Lei Xu
67b0e967f0
HDFS-9133. ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF. (Colin Patrick McCabe via Lei (Eddy) Xu)
2015-09-25 16:01:41 -07:00
Lei Xu
5eb237d544
HDFS-9132. Pass genstamp to ReplicaAccessorBuilder. (Colin Patrick McCabe via Lei (Eddy) Xu)
2015-09-25 15:50:19 -07:00
Colin Patrick Mccabe
4e7c6a653f
HDFS-9107. Prevent NN's unrecoverable death spiral after full GC (Daryn Sharp via Colin P. McCabe)
...
Change-Id: Ib8420310e515bb98091de86ea5c4be354878d43c
2015-09-25 15:25:42 -07:00
Jing Zhao
83e99d06d0
HDFS-9112. Improve error message for Haadmin when multiple name service IDs are configured. Contributed by Anu Engineer.
2015-09-25 14:21:23 -07:00
Yongjun Zhang
3187dbd7a8
HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang)
2015-09-24 21:36:38 -07:00
Haohui Mai
692b1a45ce
HDFS-9134. Move LEASE_{SOFTLIMIT,HARDLIMIT}_PERIOD constants from HdfsServerConstants to HdfsConstants. Contributed by Mingliang Liu.
2015-09-24 20:19:16 -07:00
Arpit Agarwal
df31c446bf
HADOOP-12437. Allow SecurityUtil to lookup alternate hostnames. (Contributed by Arpit Agarwal)
2015-09-24 11:41:48 -07:00
Haohui Mai
71a81b6257
HDFS-7529. Consolidate encryption zone related implementation into a single class. Contributed by Rakesh R.
2015-09-24 08:34:32 -07:00
Vinayakumar B
e52bc697f8
HDFS-9076. Log full path instead of inodeId in DFSClient#closeAllFilesBeingWritten() (Contributed by Surendra Singh Lilhore)
2015-09-24 17:43:23 +05:30
Haohui Mai
ead1b9e680
HDFS-9131. Move config keys used by hdfs-client to HdfsClientConfigKeys. Contributed by Mingliang Liu.
2015-09-24 00:30:01 -07:00
Haohui Mai
4893adff19
HDFS-9130. Use GenericTestUtils#setLogLevel to the logging level. Contributed by Mingliang Liu.
2015-09-23 23:59:19 -07:00
Haohui Mai
06d1c9033e
HDFS-9128. TestWebHdfsFileContextMainOperations and TestSWebHdfsFileContextMainOperations fail due to invalid HDFS path on Windows. Contributed by Chris Nauroth.
2015-09-23 19:38:09 -07:00
Zhe Zhang
c09dc258a8
HDFS-8882. Erasure Coding: Use datablocks, parityblocks and cell size from ErasureCodingPolicy. Contributed by Vinayakumar B.
...
Change-Id: Ic56da0b426f47c63dac440aef6f5fc8554f6cf13
2015-09-23 13:34:09 -07:00
Vinayakumar B
a2c76e5f26
HDFS-9013. Deprecate NameNodeMXBean#getNNStarted in branch2 and remove from trunk (Contributed by Surendra Singh Lilhore)
2015-09-23 15:28:41 +05:30
Haohui Mai
7c5c099324
HDFS-8733. Keep server related definition in hdfs.proto on server side. Contributed by Mingliang Liu.
2015-09-22 20:57:05 -07:00
Haohui Mai
63d9f1596c
HDFS-9039. Separate client and server side methods of o.a.h.hdfs.NameNodeProxies. Contributed by Mingliang Liu.
2015-09-22 20:52:37 -07:00
Kai Zheng
7bff8ca1c8
HDFS-8920. Erasure Coding: when recovering lost blocks, logs can be too verbose and hurt performance. Contributed by Rui Li
2015-09-22 13:27:24 -07:00
Zhe Zhang
1080c37300
Merge remote-tracking branch 'apache/trunk' into HDFS-7285
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
Change-Id: I8511c4d64b0959e79129febc179845a3892fedcc
2015-09-22 13:27:09 -07:00
Vinayakumar B
57003fa971
HDFS-9043. Doc updation for commands in HDFS Federation (Contributed by J.Andreina)
2015-09-22 16:27:24 +05:30
Vinayakumar B
10ab7d595e
HDFS-8780. Fetching live/dead datanode list with arg true for removeDecommissionNode,returns list with decom node. (Contributed by J.Andreina)
2015-09-22 12:25:35 +05:30
Tsuyoshi Ozawa
dfd807afab
HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.
2015-09-22 12:54:29 +09:00
Haohui Mai
06022b8fdc
HDFS-9111. Move hdfs-client protobuf convert methods from PBHelper to PBHelperClient. Contributed by Mingliang Liu.
2015-09-21 18:53:13 -07:00
Zhe Zhang
a9e6681edf
HDFS-9091. Erasure Coding: Provide DistributedFilesystem API to getAllErasureCodingPolicies. Contributed by Rakesh R.
...
Change-Id: Id30a1ce9e2ce676d00a9220a2d3f14b8d9f00527
2015-09-21 13:24:02 -07:00
Uma Mahesh
b762199adb
HDFS-9113. ErasureCodingWorker#processErasureCodingTasks should not fail to process remaining tasks due to one invalid ECTask. Contributed by Uma Maheswara Rao G.
2015-09-19 21:18:42 -07:00
Lei Xu
3a9c7076e8
HDFS-9004. Add upgrade domain to DatanodeInfo. Contributed by Ming Ma (via Lei (Eddy) Xu).
...
Change-Id: I887c66578eebd61acc34b94f18da6e6851c609f4
2015-09-19 18:08:09 -07:00
Haohui Mai
66b46d0885
HDFS-9101. Remove deprecated NameNode.getUri() static helper method. Contributed by Mingliang Liu.
2015-09-18 18:23:53 -07:00
Zhe Zhang
c457095206
HDFS-8550. Erasure Coding: Fix FindBugs Multithreaded correctness Warning. Contributed by Rakesh R.
...
Change-Id: Ic248999a7f8e5e740d49c9b10abcf16f66dd0f98
2015-09-18 11:04:06 -07:00
Zhe Zhang
82a88b92b4
HDFS-9086. Rename dfs.datanode.stripedread.threshold.millis to dfs.datanode.stripedread.timeout.millis. Contributed by Andrew Wang.
...
Change-Id: Ice86c5d46d29e94087c0f186b941d5394e7ac0e6
2015-09-18 10:45:29 -07:00
Jing Zhao
3f42753102
HDFS-9063. Correctly handle snapshot path for getContentSummary. Contributed by Jing Zhao.
2015-09-18 09:26:33 -07:00
Yongjun Zhang
2ff6faf954
HDFS-5802. NameNode does not check for inode type before traversing down a path. (Xiao Chen via Yongjun Zhang)
2015-09-18 07:17:30 -07:00
Vinayakumar B
92c1af1646
HDFS-6955. DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru)
2015-09-18 16:37:10 +05:30
Zhe Zhang
607bda2a21
HDFS-9097. Erasure coding: update EC command "-s" flag to "-p" when specifying policy. Contributed by Zhe Zhang.
...
Change-Id: I82919e55c4a0d8646d25a53aac757098a56b3029
2015-09-17 14:52:37 -07:00
Haohui Mai
9eee97508f
HDFS-9022. Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client. Contributed by Mingliang Liu.
2015-09-17 14:18:00 -07:00
Zhe Zhang
e36129b61a
HDFS-9088. Cleanup erasure coding documentation. Contributed by Andrew Wang.
...
Change-Id: Ic3ec1f29fef0e27c46fff66fd28a51f8c4c61e71
2015-09-17 09:56:32 -07:00
Haohui Mai
07f304467d
HDFS-7995. Implement chmod in the HDFS Web UI. Contributed by Ravi Prakash and Haohui Mai.
2015-09-16 11:50:28 -07:00
Steve Loughran
08024d1b0b
HDFS-9073. Fix failures in TestLazyPersistLockedMemory testReleaseOnEviction(). (J.Andreina via stevel)
2015-09-16 10:12:19 +01:00
Steve Loughran
4477643749
HDFS-9072. Fix random failures in TestJMXGet. (J.Andreina via stevel)
2015-09-16 10:08:33 +01:00
Haohui Mai
6c52be78a0
HDFS-7986. Allow files / directories to be deleted from the NameNode UI. Contributed by Ravi Prakash.
2015-09-15 16:48:15 -07:00
Vinayakumar B
ce69c9b54c
HDFS-8953. DataNode Metrics logging (Contributed by Kanaka Kumar Avvaru)
2015-09-16 00:18:29 +05:30
Zhe Zhang
ced438a4bf
HDFS-8899. Erasure Coding: use threadpool for EC recovery tasks on DataNode. Contributed by Rakesh R.
...
Change-Id: I9429706ae3c9b10a9274c07b98da6ed54cce192b
2015-09-15 10:43:13 -07:00
Andrew Wang
c2ebab67ac
HDFS-7351. Document the HDFS Erasure Coding feature. Contributed by Uma Maheswara Rao G and Zhe Zhang.
2015-09-15 10:16:45 -07:00
Ming Ma
083b44c136
HDFS-9008. Balancer#Parameters class could use a builder pattern. (Chris Trezzo via mingma)
2015-09-15 10:16:02 -07:00
Vinayakumar B
a440567491
HDFS-9067. o.a.h.hdfs.server.datanode.fsdataset.impl.TestLazyWriter is failing in trunk (Contributed by Surendra Singh Lilhore)
2015-09-15 17:30:46 +05:30
Haohui Mai
d777757d21
HDFS-9065. Include commas on # of files, blocks, total filesystem objects in NN Web UI. Contributed by Daniel Templeton.
2015-09-14 20:26:56 -07:00
Haohui Mai
76957a485b
HDFS-9010. Replace NameNode.DEFAULT_PORT with HdfsClientConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT config key. Contributed by Mingliang Liu.
2015-09-14 18:22:52 -07:00
Colin Patrick Mccabe
7b5cf5352e
HDFS-8829. Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning (He Tianyi via Colin P. McCabe)
...
Change-Id: I77dc71aaf9e14ef743f2a2cbebeec04a4f628c78
2015-09-14 16:02:10 -07:00
Colin Patrick Mccabe
53bad4eb00
HDFS-8996. Consolidate validateLog and scanLog in FJM#EditLogFile (Zhe Zhang via Colin P. McCabe)
2015-09-14 15:22:12 -07:00
Steve Loughran
6955771261
HDFS-9069. TestNameNodeMetricsLogger failing -port in use. (stevel)
2015-09-14 10:36:44 +01:00
Haohui Mai
0c7d3f4805
HDFS-9041. Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client. Contributed by Mingliang Liu.
2015-09-12 11:02:51 -07:00
Vinayakumar B
c715650385
HDFS-9036. In BlockPlacementPolicyWithNodeGroup#chooseLocalStorage , random node is selected eventhough fallbackToLocalRack is true. (Contributed by J.Andreina)
2015-09-12 17:40:16 +05:30
Vinayakumar B
ce02b5532c
HDFS-8975. Erasure coding : Fix random failure in TestSafeModeWithStripedFile (Contributed by J.Andreina)
2015-09-12 13:29:25 +05:30
Vinayakumar B
d8455479b8
HDFS-9042. Update document for the Storage policy name (Contributed by J.Andreina)
2015-09-12 12:36:12 +05:30
Arpit Agarwal
15a557fcfe
HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method. (Contributed by Mingliang Liu)
2015-09-11 10:11:52 -07:00
Zhe Zhang
96d6b516b2
HDFS-8853. Erasure Coding: Provide ECSchema validation when setting EC policy. Contributed by J.Andreina.
...
Change-Id: I9211d9728480225a407d82e6c0bea1a928adfa11
2015-09-10 16:31:37 -07:00
Kihwal Lee
a40342b0da
HDFS-6763. Initialize file system-wide quota once on transitioning to active. Contributed by Kihwal Lee
2015-09-10 09:16:29 -05:00
Akira Ajisaka
7b5b2c5822
HDFS-8974. Convert docs in xdoc format to markdown. Contributed by Masatake Iwasaki.
2015-09-10 16:45:27 +09:00
Zhe Zhang
f62237bc2f
HDFS-8833. Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones.
2015-09-09 23:07:55 -07:00
Vinayakumar B
4014ce5990
HDFS-8581. ContentSummary on / skips further counts on yielding lock (contributed by J.Andreina)
2015-09-10 00:08:19 +05:30
Vinayakumar B
0f0e897bf1
HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R)
2015-09-09 22:58:50 +05:30
Kihwal Lee
4d13335fc9
HDFS-8716. Introduce a new config specifically for safe mode block count. Contributed by Chang Li.
2015-09-09 11:08:24 -05:00
Akira Ajisaka
0113e4528d
HDFS-9033. dfsadmin -metasave prints "NaN" for cache used%. Contributed by Brahma Reddy Battula.
2015-09-09 17:20:46 +09:00
Vinayakumar B
94cf7ab9d2
HDFS-8929. Add a metric to expose the timestamp of the last journal (Contributed by surendra singh lilhore)
2015-09-09 11:03:57 +05:30
Colin Patrick Mccabe
a153b9601a
HDFS-8860. Remove unused Replica copyOnWrite code (Lei (Eddy) Xu via Colin P. McCabe)
2015-09-08 18:37:22 -07:00
Xiaoyu Yao
970daaa5e4
HDFS-9019. Adding informative message to sticky bit permission denied exception. Contributed by Xiaoyu Yao.
2015-09-08 09:57:36 -07:00
Andrew Wang
e1feaf6db0
HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao.
2015-09-04 13:42:55 -07:00
Ming Ma
30db1adac3
HDFS-8981. Adding revision to data node jmx getVersion() method. (Siqi Li via mingma)
2015-09-04 11:55:58 -07:00
Haohui Mai
715b9c6499
HDFS-8984. Move replication queues related methods in FSNamesystem to BlockManager. Contributed by Haohui Mai.
2015-09-04 11:45:31 -07:00
Jing Zhao
8928729c80
HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao.
2015-09-04 11:42:22 -07:00
Haohui Mai
d16c4eee18
HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to hadoop-hdfs-client module. Contributed by Mingliang Liu.
2015-09-04 10:41:09 -07:00
Andrew Wang
c83d13c649
HDFS-9021. Use a yellow elephant rather than a blue one in diagram.
2015-09-03 16:27:20 -07:00
Haohui Mai
ed78b14ebc
HDFS-9002. Move o.a.h.hdfs.net/*Peer classes to hdfs-client. Contributed by Mingliang Liu.
2015-09-03 15:32:53 -07:00
Jakob Homan
c2d2c1802a
HDFS-8939. Test(S)WebHdfsFileContextMainOperations failing on branch-2. Contributed by Chris Nauroth.
...
(cherry picked from commit 9871f57d41990b2763d4c23d9040aeb685b4a80f)
2015-09-03 13:26:54 -07:00
Colin Patrick Mccabe
53c38cc89a
HDFS-8964. When validating the edit log, do not read at or beyond the file offset that is being written (Zhe Zhang via Colin P. McCabe)
2015-09-03 11:22:47 -07:00
Tsuyoshi Ozawa
9a87f81ce6
HADOOP-5323. Trash documentation should describe its directory structure and configurations. Contributed by Weiwei Yang.
2015-09-03 22:34:25 +09:00
Akira Ajisaka
c92e31bd65
HDFS-8885. ByteRangeInputStream used in webhdfs does not override available(). Contributed by Shradha Revankar.
2015-09-03 19:40:34 +09:00
Jing Zhao
60bd765ac1
HDFS-8978. Erasure coding: fix 2 failed tests of DFSStripedOutputStream. Contributed by Walter Su.
2015-09-02 17:41:08 -07:00
Ming Ma
d31a41c359
HDFS-8890. Allow admin to specify which blockpools the balancer should run on. (Chris Trezzo via mingma)
2015-09-02 15:55:42 -07:00
Jing Zhao
de928d566a
HDFS-9003. ForkJoin thread pool leaks. Contributed by Kihwal Lee.
2015-09-02 15:19:04 -07:00
Andrew Wang
afc88b396f
HDFS-328. Improve fs -setrep error message for invalid replication factors. Contributed by Daniel Templeton.
2015-09-02 13:46:00 -07:00
Colin Patrick Mccabe
c006c3a1e6
HDFS-2070. Add more unit tests for FsShell getmerge (Daniel Templeton via Colin P. McCabe)
2015-09-02 12:36:14 -07:00
Jing Zhao
ddf4e78547
HDFS-8937. Erasure coding: do not throw exception when setting replication factor on EC files. Contributed by Gao Rui.
2015-09-02 11:45:45 -07:00
Akira Ajisaka
65ccf2b125
HDFS-8388. Time and Date format need to be in sync in NameNode UI page. Contributed by Surendra Singh Lilhore.
2015-09-02 14:28:38 +09:00
yliu
5652131d2e
HDFS-8995. Flaw in registration bookeeping can make DN die on reconnect. (Kihwal Lee via yliu)
2015-09-02 08:58:51 +08:00
Zhe Zhang
53358fe680
Merge remote-tracking branch 'apache/trunk' into HDFS-7285
2015-09-01 14:48:37 -07:00
Zhe Zhang
ab56fcdb12
Merge remote-tracking branch 'apache/trunk' into HDFS-7285
2015-09-01 14:30:25 -07:00
Colin Patrick Mccabe
24f6a7c956
HDFS-8965. Harden edit log reading code against out of memory errors (cmccabe)
2015-08-31 18:06:30 -07:00
yliu
8fa41d9dd4
HDFS-8946. Improve choosing datanode storage for block placement. (yliu)
2015-09-01 08:52:50 +08:00
Haohui Mai
826ae1c26d
HDFS-8990. Move RemoteBlockReader to hdfs-client module. Contributed by Mingliang Liu.
2015-08-31 13:54:14 -07:00
Jing Zhao
caa04de149
HDFS-8980. Remove unnecessary block replacement in INodeFile. Contributed by Jing Zhao.
2015-08-31 11:48:09 -07:00
Jakob Homan
837fb75e8e
HDFS-8155. Support OAuth2 in WebHDFS.
2015-08-29 18:37:05 -07:00
Arpit Agarwal
bdbe53c676
HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal)
2015-08-29 09:52:37 -07:00
Haohui Mai
e2c9b288b2
HDFS-8925. Move BlockReaderLocal to hdfs-client. Contributed by Mingliang Liu.
2015-08-28 14:38:36 -07:00
Colin Patrick Mccabe
b94b56806d
HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list (Daniel Templeton)
2015-08-28 14:21:46 -07:00
Haohui Mai
6d12cd8d60
HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu.
2015-08-28 14:14:32 -07:00
Kihwal Lee
b6ceee9bf4
HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee.
2015-08-28 13:14:35 -05:00
yliu
e166c038c0
HDFS-8963. Fix incorrect sign extension of xattr length in HDFS-8900. (Colin Patrick McCabe via yliu)
2015-08-28 10:54:55 +08:00
Haohui Mai
035ed26147
Revert "HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu."
...
This reverts commit 4e9307f26d
2015-08-27 16:09:35 -07:00
Haohui Mai
4e9307f26d
HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu.
2015-08-27 15:36:44 -07:00
Haohui Mai
f97a0f8c2c
HDFS-8969. Clean up findbugs warnings for HDFS-8823 and HDFS-8932. Contributed by Anu Engineer.
2015-08-27 13:03:16 -07:00
Haohui Mai
7e971b7315
HDFS-8962. Clean up checkstyle warnings in o.a.h.hdfs.DfsClientConf. Contributed by Mingliang Liu.
2015-08-27 09:02:34 -07:00
Harsh J
0bf285413f
HDFS-2390. dfsadmin -setBalancerBandwidth does not validate -ve value. Contributed by Gautam Gopalakrishnan.
2015-08-27 16:22:48 +05:30
Walter Su
164cbe6439
HDFS-8909. Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature. Contributed by Jing Zhao.
2015-08-27 16:02:30 +08:00
Vinayakumar B
fdb56f74f3
HDFS-8682. Should not remove decommissioned node,while calculating the number of live/dead decommissioned node. (Contributed by J. Andreina)
2015-08-27 13:06:43 +05:30
Haohui Mai
4cbbfa2220
HDFS-8248. Store INodeId instead of the INodeFile object in BlockInfoContiguous. Contributed by Haohui Mai.
2015-08-26 18:14:29 -07:00
Walter Su
067ec8c2b1
HDFS-8838. Erasure Coding: Tolerate datanode failures in DFSStripedOutputStream when the data length is small. Contributed by Tsz Wo Nicholas Sze.
2015-08-27 09:11:23 +08:00
Jing Zhao
f44b599003
HDFS-8896. DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager. Contributed by Walter Su.
2015-08-26 16:27:34 -07:00
Haohui Mai
c992bcf9c1
HDFS-8951. Move the shortcircuit package to hdfs-client. Contributed by Mingliang Liu.
2015-08-26 14:02:48 -07:00
Colin Patrick Mccabe
a4d9acc51d
HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe)
2015-08-25 14:29:53 -07:00
yliu
eee0d4563c
HDFS-8900. Compact XAttrs to optimize memory footprint. (yliu)
2015-08-25 16:16:09 +08:00
Xiaoyu Yao
3b00eaea25
HDFS-8932. NPE thrown in NameNode when try to get TotalSyncCount metric before editLogStream initialization. Contributed by Surendra Singh Lilhore
2015-08-24 16:56:24 -07:00
Haohui Mai
66d0c81d8f
HDFS-8948. Use GenericTestUtils to set log levels in TestPread and TestReplaceDatanodeOnFailure. Contributed by Mingliang Liu.
2015-08-24 16:18:15 -07:00
Jing Zhao
bdd79388f3
HDFS-8928. Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a separate class and replicas as an array. Contributed by Jing Zhao.
2015-08-24 15:53:34 -07:00
Zhe Zhang
6b6a63bbbd
Merge commit '456e901a4c5c639267ee87b8e5f1319f256d20c2' (HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.) into HDFS-7285-merge
2015-08-24 12:59:26 -07:00
Colin Patrick Mccabe
b5ce87f84d
HDFS-8930. Block report lease may leak if the 2nd full block report comes when NN is still in safemode (Colin P. McCabe via Jing Zhao)
2015-08-24 11:31:56 -07:00
Akira Ajisaka
bcaf83902a
HDFS-8942. Update hyperlink to rack awareness page in HDFS Architecture documentation. Contributed by Masatake Iwasaki.
2015-08-24 13:52:49 +09:00
Haohui Mai
490bb5ebd6
HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu.
2015-08-22 13:31:03 -07:00
Haohui Mai
745d04be59
HDFS-8823. Move replication factor into individual blocks. Contributed by Haohui Mai.
2015-08-22 00:09:40 -07:00
Lei Xu
7087e700e0
HDFS-8924. Add pluggable interface for reading replicas in DFSClient. (Colin Patrick McCabe via Lei Xu)
2015-08-21 17:02:00 -07:00
Jing Zhao
c8bca62718
HDFS-8809. HDFS fsck reports under construction blocks as CORRUPT. Contributed by Jing Zhao.
2015-08-20 16:31:24 -07:00
Colin Patrick Mccabe
7642f64c24
HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe)
2015-08-20 13:57:32 -07:00
yliu
5e8fe89437
HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu)
2015-08-20 20:15:03 +08:00
yliu
80a29906bc
HDFS-8884. Fail-fast check in BlockPlacementPolicyDefault#chooseTarget. (yliu)
2015-08-20 20:07:18 +08:00
Jing Zhao
4e14f7982a
HDFS-8917. Cleanup BlockInfoUnderConstruction from comments and tests. Contributed by Zhe Zhang.
2015-08-19 15:11:37 -07:00
Haohui Mai
3aac4758b0
HDFS-8803. Move DfsClientConf to hdfs-client. Contributed by Mingliang Liu.
2015-08-19 11:28:05 -07:00
Jing Zhao
f61120d964
HDFS-8867. Enable optimized block reports. Contributed by Daryn Sharp.
2015-08-19 10:36:56 -07:00
Arpit Agarwal
9c3571ea60
HDFS-8911. NameNode Metric : Add Editlog counters as a JMX metric. (Contributed by Anu Engineer)
2015-08-19 09:42:35 -07:00
yliu
2da5aaab33
HDFS-8908. TestAppendSnapshotTruncate may fail with IOException: Failed to replace a bad datanode. (Tsz Wo Nicholas Sze via yliu)
2015-08-19 21:55:43 +08:00
Tsz-Wo Nicholas Sze
7ecbfd44aa
HDFS-8826. In Balancer, add an option to specify the source node list so that balancer only selects blocks to move from those nodes.
2015-08-18 19:25:50 -07:00
Chris Douglas
30e342a5d3
HDFS-8435. Support CreateFlag in WebHDFS. Contributed by Jakob Homan
2015-08-18 17:32:48 -07:00
Akira Ajisaka
fc509f66d8
HDFS-8852. HDFS architecture documentation of version 2.x is outdated about append write support. Contributed by Ajith S.
2015-08-18 23:31:52 +09:00
yliu
71566e2382
HDFS-8862. BlockManager#excessReplicateMap should use a HashMap. (yliu)
2015-08-18 09:23:06 +08:00
Tsz-Wo Nicholas Sze
51a00964da
HDFS-8278. When computing max-size-to-move in Balancer, count only the storage with remaining >= default block size.
2015-08-17 17:55:25 -07:00
Andrew Wang
eee4d716b4
HDFS-8895. Remove deprecated BlockStorageLocation APIs.
2015-08-17 17:40:37 -07:00
Arpit Agarwal
a88f31ebf3
HDFS-8880. NameNode metrics logging. (Arpit Agarwal)
2015-08-17 16:55:46 -07:00
Colin Patrick Mccabe
ec183faadc
HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe)
2015-08-17 15:25:29 -07:00
Colin Patrick Mccabe
c77bd6af16
HDFS-8792. BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory (Yi Liu via Colin P. McCabe)
2015-08-17 12:00:45 -07:00
Haohui Mai
e535e0f05b
HDFS-8801. Convert BlockInfoUnderConstruction as a feature. Contributed by Jing Zhao.
2015-08-17 11:28:28 -07:00
Haohui Mai
456e901a4c
HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.
2015-08-17 11:04:00 -07:00
Andrew Wang
2e7b7e2cda
HDFS-8713. Convert DatanodeDescriptor to use SLF4J logging.
2015-08-17 10:17:06 -07:00
Xiaoyu Yao
a7862d5fe4
HDFS-8883. NameNode Metrics : Add FSNameSystem lock Queue Length. Contributed by Anu Engineer.
2015-08-17 10:15:56 -07:00
Zhe Zhang
b57c9a35f7
HDFS-8220. Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't satisfy BlockGroupSize.
2015-08-14 15:16:58 -07:00
Jing Zhao
dc7a061668
HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang.
2015-08-14 14:42:43 -07:00
Tsz-Wo Nicholas Sze
2bc0a4f299
HDFS-8824. Do not use small blocks for balancing the cluster.
2015-08-14 13:03:19 -07:00
Xiaoyu Yao
1569228ec9
HDFS-8565. Typo in dfshealth.html - Decomissioning. (nijel via xyao)
2015-08-14 12:34:31 -07:00
Arpit Agarwal
ae57d60d82
HDFS-7649. Multihoming docs should emphasize using hostnames in configurations. (Contributed by Brahma Reddy Battula)
2015-08-13 14:26:40 -07:00
Zhe Zhang
1d37a88121
HDFS-8854. Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs. Contributed by Walter Su.
2015-08-13 10:04:45 -07:00
yliu
53bef9c5b9
HADOOP-12295. Improve NetworkTopology#InnerNode#remove logic. (yliu)
2015-08-13 16:45:20 +08:00
Akira Ajisaka
40f815131e
HDFS-8622. Implement GETCONTENTSUMMARY operation for WebImageViewer. Contributed by Jagadesh Kiran N.
2015-08-13 11:31:56 +09:00
cnauroth
6cc8e38db5
HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu.
2015-08-12 16:44:53 -07:00
Xiaoyu Yao
3e715a4f4c
HDFS-8879. Quota by storage type usage incorrectly initialized upon namenode restart. Contributed by Xiaoyu Yao.
2015-08-12 15:20:54 -07:00
Andrew Wang
1ea1a8334e
HDFS-8887. Expose storage type and storage ID in BlockLocation.
2015-08-11 23:25:33 -07:00
Jing Zhao
1fc3c779a4
HDFS-8805. Archival Storage: getStoragePolicy should not need superuser privilege. Contributed by Brahma Reddy Battula.
2015-08-11 10:28:18 -07:00
Tsz-Wo Nicholas Sze
b56daff6a1
HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable.
2015-08-10 16:52:02 -07:00
Jakob Homan
8f73bdd06b
HDFS-8866. Typo in docs: Rumtime -> Runtime. Contributed by Gabor Liptak.
2015-08-07 11:38:31 -07:00
Jing Zhao
fbf7e81ca0
HDFS-8827. Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks. Contributed by Walter Su and Takuya Fukudome.
2015-08-07 11:25:20 -07:00
Andrew Wang
98a27d1101
HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su.
2015-08-07 09:51:46 -07:00
Arpit Agarwal
6d4eee718a
HDFS-8856. Make LeaseManager#countPath O(1). (Contributed by Arpit Agarwal)
2015-08-06 18:51:28 -07:00
boli2
8799363db1
HDFS-8857. Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure. Contributed by Li Bo
2015-08-05 22:11:50 -04:00
Zhe Zhang
4de48211c6
HDFS-8399. Erasure Coding: unit test the behaviour of BlockManager recovery work for the deleted blocks. Contributed by Rakesh R.
2015-08-04 15:23:39 -07:00
Jing Zhao
9312b168e2
HDFS-8804. Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation. Contributed by Jing Zhao.
2015-08-03 17:03:15 -07:00
Jing Zhao
f4c523b69b
Revert "HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang."
...
This reverts commit c17439c2dd
2015-08-06 10:21:55 -07:00
Jing Zhao
663eba0ab1
Revert "HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang."
...
This reverts commit de480d6c89
2015-08-06 10:21:54 -07:00
Vinayakumar B
cc71ad80e1
HDFS-8815. DFS getStoragePolicy implementation using single RPC call (Contributed by Surendra Singh Lilhore)
2015-08-06 11:10:48 +05:30
Colin Patrick Mccabe
c95993cbaf
HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe)
2015-08-04 12:22:06 -07:00
yliu
9a08999906
HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu)
2015-08-04 15:54:23 +08:00
Zhihai Xu
90b510405a
HDFS-8847. change TestHDFSContractAppend to not override testRenameFileBeingAppended method. Contributed by Zhihai Xu
2015-07-31 21:40:43 -07:00
Xiaoyu Yao
d311a38a6b
HDFS-6860. BlockStateChange logs are too noisy. Contributed by Chang Li and Xiaoyu Yao.
2015-07-31 16:15:50 -07:00
Colin Patrick Mccabe
21d10ccc6e
HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe)
2015-07-31 15:01:34 -07:00
Allen Wittenauer
666cafca8d
HADOOP-12249. pull argument parsing into a function (aw)
2015-07-31 14:32:21 -07:00
Zhe Zhang
ba90c02853
HDFS-8202. Improve end to end stirpping file test to add erasure recovering test. Contributed by Xinwei Qin.
2015-07-31 12:16:15 -07:00
Harsh J
c5caa25b8f
HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan.
2015-07-31 08:58:22 +05:30
Arpit Agarwal
88d8736dde
HDFS-7192. DN should ignore lazyPersist hint if the writer is not local. (Contributed by Arpit Agarwal)
2015-07-30 13:16:46 -07:00
Haohui Mai
ddc867ceb9
HDFS-8816. Improve visualization for the Datanode tab in the NN UI. Contributed by Haohui Mai.
2015-07-29 17:14:05 -07:00
Jason Lowe
6f0a35724f
HADOOP-10945. 4-digit octal umask permissions throws a parse error. Contributed by Chang Li
2015-07-29 15:25:10 +00:00
Vinayakumar B
6374ee0db4
HDFS-8670. Better to exclude decommissioned nodes for namenode NodeUsage JMX (Contributed by J.Andreina)
2015-07-29 14:47:19 +05:30
Akira Ajisaka
2a1d656196
Revert "HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)"
...
This reverts commit 02c01815ec
2015-07-29 16:48:30 +09:00
Lei Xu
f4f1b8b267
HDFS-8834. TestReplication is not valid after HDFS-6482. (Contributed by Lei Xu)
2015-07-28 23:39:26 -07:00
Vinayakumar B
975e138df3
HDFS-8822. Add SSD storagepolicy tests in TestBlockStoragePolicy#testDefaultPolicies (Contributed by Vinayakumar B)
2015-07-29 11:50:48 +05:30
Vinayakumar B
50887e5b07
HDFS-8811. Move BlockStoragePolicy name's constants from HdfsServerConstants.java to HdfsConstants.java (Contributed by Vinayakumar B)
2015-07-29 11:45:33 +05:30
Jakob Homan
0712a8103f
HDFS-8180. AbstractFileSystem Implementation for WebHdfs. Contributed by Sathosh G Nayak.
2015-07-28 21:03:31 -07:00
Arun Suresh
030fcfa99c
HDFS-7858. Improve HA Namenode Failover detection on the client. (asuresh)
2015-07-27 23:02:03 -07:00
Xiaoyu Yao
2196e39e14
HDFS-8785. TestDistributedFileSystem is failing in trunk. Contributed by Xiaoyu Yao.
2015-07-27 07:28:41 -07:00
Akira Ajisaka
1df78688c6
HDFS-8810. Correct assertions in TestDFSInotifyEventInputStream class. Contributed by Surendra Singh Lilhore.
2015-07-27 13:17:24 +09:00
Walter Su
4a72be6e0e
HDFS-8769. Erasure coding: unit test for SequentialBlockGroupIdGenerator. Contributed by Rakesh R.
2015-07-25 10:07:40 +08:00
Jing Zhao
95b499a367
HDFS-8798. Erasure Coding: fix DFSStripedInputStream/DFSStripedOutputStream re-fetch token when expired. Contributed by Walter Su.
2015-07-24 13:52:50 -07:00
Akira Ajisaka
f8f6091823
HDFS-8735. Inotify: All events classes should implement toString() API. Contributed by Surendra Singh Lilhore.
2015-07-25 02:56:55 +09:00
Akira Ajisaka
206d4933a5
HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang.
2015-07-24 18:28:44 +09:00
Akira Ajisaka
02c01815ec
HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)
2015-07-24 11:37:23 +09:00
Jing Zhao
c2c26e6ea7
HDFS-8813. Erasure Coding: Client no need to decode missing parity blocks. Contributed by Walter Su.
2015-07-23 11:42:02 -07:00
Arpit Agarwal
adfa34ff99
HADOOP-12161. Add getStoragePolicy API to the FileSystem interface. (Contributed by Brahma Reddy Battula)
2015-07-23 10:13:04 -07:00
Jing Zhao
e91ccfad07
HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao.
2015-07-22 17:42:31 -07:00
yliu
4025326288
HDFS-8795. Improve InvalidateBlocks#node2blocks. (yliu)
2015-07-22 15:16:50 +08:00
Haohui Mai
31f117138a
HDFS-8495. Consolidate append() related implementation into a single class. Contributed by Rakesh R.
2015-07-21 17:25:23 -07:00
Jing Zhao
5956d23b64
HDFS-8781. Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block. Contributed by Yi Liu.
2015-07-21 17:00:49 -07:00
Jing Zhao
f8f7a923b7
HDFS-8760. Erasure Coding: reuse BlockReader when reading the same block in pread. Contributed by Jing Zhao.
2015-07-21 15:47:26 -07:00
cnauroth
cb03768b1b
HDFS-8721. Add a metric for number of encryption zones. Contributed by Rakesh R.
2015-07-21 13:55:58 -07:00
Ravi Prakash
5137b388fc
Revert "HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak)"
...
This reverts commit e4f756260f
2015-07-21 11:29:35 -07:00
Allen Wittenauer
c9507fe6c1
HDFS-8800. hdfs --daemon stop namenode corrupts logs (John Smith via aw)
2015-07-21 09:41:28 -07:00
Vinayakumar B
29cf887b22
HDFS-7582. Enforce maximum number of ACL entries separately per access and default. (Contributed by Vinayakumar B)
2015-07-21 15:16:52 +05:30
Haohui Mai
df1e8ce44a
HDFS-7483. Display information per tier on the Namenode UI. Contributed by Benoy Antony and Haohui Mai.
2015-07-20 20:13:12 -07:00
yliu
d6d58606b8
HDFS-8794. Improve CorruptReplicasMap#corruptReplicasMap. (yliu)
2015-07-21 09:20:22 +08:00
Aaron T. Myers
ed01dc70b2
HDFS-8657. Update docs for mSNN. Contributed by Jesse Yates.
2015-07-20 16:40:06 -07:00
Ravi Prakash
e4f756260f
HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak)
2015-07-20 14:03:34 -07:00
Walter Su
06394e3760
HDFS-8433. Erasure coding: set blockToken in LocatedStripedBlock. Contributed by Walter Su.
2015-07-20 10:18:34 +08:00
Allen Wittenauer
ee36f4f9b8
HADOOP-10979. Auto-entries in hadoop_usage (aw)
2015-07-16 16:58:11 -07:00
Ming Ma
fbd88f1062
HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma)
2015-07-16 12:33:57 -07:00
Zhe Zhang
4fdd9abd7e
HDFS-8787. Erasure coding: rename BlockInfoContiguousUC and BlockInfoStripedUC to be consistent with trunk.
2015-07-15 20:13:04 -07:00
Arpit Agarwal
3ec0a0444f
HDFS-8778. TestBlockReportRateLimiting#testLeaseExpiration can deadlock. (Contributed by Arpit Agarwal)
2015-07-15 14:08:58 -07:00
Zhe Zhang
7e091de136
HDFS-8058. Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile. Contributed by Zhe Zhang and Yi Liu.
2015-07-15 09:49:32 -07:00
yliu
f32d9a1758
HDFS-8619. Erasure Coding: revisit replica counting for striped blocks. (Jing Zhao via yliu)
2015-07-15 22:35:19 +08:00
Jing Zhao
0a93712f3b
HDFS-8734. Erasure Coding: fix one cell need two packets. Contributed by Walter Su.
2015-07-14 13:10:51 -07:00
Akira Ajisaka
979c9ca2ca
HDFS-8742. Inotify: Support event for OP_TRUNCATE. Contributed by Surendra Singh Lilhore.
2015-07-15 04:42:08 +09:00
Kihwal Lee
59388a8015
HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee
2015-07-14 14:04:06 -05:00
Colin Patrick Mccabe
1d74ccecec
HDFS-7608: hdfs dfsclient newConnectedPeer has no write timeout (Xiaoyu Yao via Colin P. McCabe)
2015-07-14 10:57:59 -07:00
Jing Zhao
6ff957be88
HDFS-8702. Erasure coding: update BlockManager.blockHasEnoughRacks(..) logic for striped block. Contributed by Kai Sasaki.
2015-07-14 10:55:58 -07:00
Tsz-Wo Nicholas Sze
9ef03a4c5b
HDFS-8541. Mover should exit with NO_MOVE_PROGRESS if there is no move progress. Contributed by Surendra Singh Lilhore
2015-07-13 15:12:26 -07:00
Jing Zhao
b1e6429a6b
HDFS-8669. Erasure Coding: handle missing internal block locations in DFSStripedInputStream. Contributed by Jing Zhao.
2015-07-13 11:41:18 -07:00
boli2
f4098dfae4
HDFS-8744. Erasure Coding: the number of chunks in packet is not updated when writing parity data. Contributed by Li Bo
2015-07-13 00:41:36 -04:00
Jing Zhao
47f4c54106
HDFS-8751. Remove setBlocks API from INodeFile and misc code cleanup. Contributed by Zhe Zhang
2015-07-10 14:15:22 -07:00
Vinayakumar B
b48908033f
HDFS-2956. calling fetchdt without a --renewer argument throws NPE (Contributed by Vinayakumar B)HDFS-2956. calling fetchdt without a --renewer argument throws NPE (Contributed by Vinayakumar B)
2015-07-10 15:47:04 +05:30
Akira Ajisaka
d66302ed9b
HDFS-8749. Fix findbugs warnings in BlockManager.java. Contributed by Brahma Reddy Battula.
2015-07-10 15:04:06 +09:00
Jing Zhao
e692c7dd92
HDFS-8484. Erasure coding: Two contiguous blocks occupy IDs belong to same striped group. Contributed by Walter Su.
2015-07-09 14:07:37 -07:00
Jing Zhao
f4ca530c1c
HDFS-8729. Fix TestFileTruncate#testTruncateWithDataNodesRestartImmediately which occasionally failed. Contributed by Walter Su.
2015-07-09 13:17:52 -07:00
Jing Zhao
2470a7bf88
HDFS-8563. Erasure Coding: fsck handles file smaller than a full stripe. Contributed by Walter Su.
2015-07-09 10:55:28 -07:00
Walter Su
48f3830f21
HDFS-8719. Erasure Coding: client generates too many small packets when writing parity data. Contributed by Li Bo.
2015-07-09 16:18:05 +08:00
Zhe Zhang
42a1d0dbde
Addendum fix for HDFS-8323 (adding back a mistakenly deleted line).
2015-07-08 22:52:09 -07:00
Haohui Mai
fc6182d5ed
HDFS-8726. Move protobuf files that define the client-sever protocols to hdfs-client. Contributed by Haohui Mai.
2015-07-08 10:37:10 -07:00
Arpit Agarwal
4119ad3112
HDFS-8642. Make TestFileTruncate more reliable. (Contributed by Rakesh R)
2015-07-08 09:31:02 -07:00
Vinayakumar B
bd4e10900c
HDFS-8712. Remove 'public' and 'abstract' modifiers in FsVolumeSpi and FsDatasetSpi (Contributed by Lei (Eddy) Xu)
2015-07-08 16:34:54 +05:30
Xiaoyu Yao
b68701b7b2
HDFS-8711. setSpaceQuota command should print the available storage type when input storage type is wrong. Contributed by Brahma Reddy Battula.
2015-07-07 13:50:49 -07:00
Jing Zhao
bc99aaffe7
Revert "HDFS-8652. Track BlockInfo instead of Block in CorruptReplicasMap. Contributed by Jing Zhao."
...
This reverts commit d62b63d297
2015-07-07 10:13:41 -07:00
Jing Zhao
2c494a8436
HDFS-8461. Erasure coding: fix priority level of UnderReplicatedBlocks for striped block. Contributed by Walter Su.
2015-07-06 16:39:47 -07:00
Jing Zhao
d62b63d297
HDFS-8652. Track BlockInfo instead of Block in CorruptReplicasMap. Contributed by Jing Zhao.
2015-07-06 15:54:25 -07:00
yliu
ee01a09500
HDFS-8684. Erasure Coding: fix some block number calculation for striped block. (yliu)
2015-07-06 19:14:18 +08:00
Vinayakumar B
fc92d3e651
HDFS-8686. WebHdfsFileSystem#getXAttr(Path p, final String name) doesn't work if namespace is in capitals (Contributed by kanaka kumar avvaru)
2015-07-06 16:09:24 +05:30
Vinayakumar B
2eae130ab9
HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina)
2015-07-03 13:35:48 +05:30
Andrew Wang
5fddc5177d
HDFS-8709. Clarify automatic sync in FSEditLog#logEdit.
2015-07-02 10:26:40 -07:00
Vinayakumar B
bff5999d07
HDFS-8703. Merge refactor of DFSInputStream from ErasureCoding branch (Contributed by Vinayakumar B)
2015-07-02 16:11:50 +05:30
Colin Patrick Mccabe
a78d5074fb
HADOOP-12171. Shorten overly-long htrace span names for server (cmccabe)
2015-07-01 17:57:11 -07:00
Jing Zhao
152e5df3b6
HDFS-8666. Speedup the TestMover test. Contributed by Walter Su.
2015-07-01 14:59:50 -07:00
Arpit Agarwal
ec0dc71cab
HDFS-8706. Fix typo in datanode startup options in HDFSCommands.html. (Controibuted by Brahma Reddy Battula)
2015-07-01 10:36:04 -07:00
Colin Patrick Mccabe
68e588cbee
HADOOP-10798. globStatus() should always return a sorted list of files (cmccabe)
2015-06-30 16:40:21 -07:00
Colin Patrick Mccabe
29df73dc05
HDFS-8635. Migrate HDFS native build to new CMake framework (Alan Burlison via Colin P. McCabe)
2015-06-30 15:54:38 -07:00
Vinayakumar B
7ba5bbac02
HDFS-8627. NPE thrown if unable to fetch token from Namenode (Contributed by J.Andreina)
2015-06-30 15:42:59 +05:30
Vinayakumar B
6d99017f38
HADOOP-12154. FileSystem#getUsed() returns the file length only from root '/' (Contributed by J.Andreina)
2015-06-30 15:25:20 +05:30
Vinayakumar B
77588e1d32
HDFS-8579. Update HDFS usage with missing options (Contributed by J.Andreina)
2015-06-30 15:00:22 +05:30
Vinayakumar B
0b7af27b9a
HDFS-8468. 2 RPC calls for every file read in DFSClient#open(..) resulting in double Audit log entries (Contributed by Vinayakumar B)
2015-06-30 11:02:13 +05:30
Haohui Mai
d3797f9f3c
HDFS-8493. Consolidate truncate() related implementation in a single class. Contributed by Rakesh R.
2015-06-29 16:45:35 -07:00
Arpit Agarwal
8e333720e1
HDFS-8687. Remove the duplicate usage message from Dfsck.java. (Contributed by Brahma Reddy Battula)
2015-06-29 14:56:42 -07:00
Yongjun Zhang
43a1288fff
HDFS-8659. Block scanner INFO message is spamming logs. Contributed by Yongjun Zhang.
2015-06-29 14:43:56 -07:00
Andrew Wang
2ffd84273a
HDFS-8653. Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo. Contributed by Zhe Zhang.
2015-06-29 12:12:41 -07:00
Benoy Antony
d3fed8e653
HDFS-7390. Provide JMX metrics per storage type. (Benoy Antony)
2015-06-29 11:00:22 -07:00
Vinayakumar B
fde20ffcef
HDFS-8628. Update missing command option for fetchdt (Contributed by J.Andreina)
2015-06-29 16:50:48 +05:30
Vinayakumar B
88ceb382ef
HDFS-8586. Dead Datanode is allocated for write when client is from deadnode (Contributed by Brahma Reddy Battula)
2015-06-29 15:25:03 +05:30
Arpit Agarwal
c6793dd8cc
HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal)
2015-06-28 14:51:36 -07:00
Andrew Wang
60b858bfa6
HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization.
2015-06-26 11:30:59 -07:00
Jing Zhao
de480d6c89
HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang.
2015-06-26 10:49:01 -07:00
Colin Patrick Mccabe
1b764a01fd
HDFS-8651. Make hadoop-hdfs-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe)
2015-06-26 10:22:40 -07:00
Andrew Wang
1403b84b12
HDFS-8546. Use try with resources in DataStorage and Storage.
2015-06-25 17:50:32 -07:00
Andrew Wang
ff0e5e572f
HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication.
2015-06-25 17:29:24 -07:00
Arpit Agarwal
67a62da5c5
HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru)
2015-06-25 10:13:22 -07:00
Akira Ajisaka
bc433908d3
HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N.
2015-06-26 00:20:12 +09:00
Andrew Wang
afe9ea3c12
HDFS-8646. Prune cached replicas from DatanodeDescriptor state on replica invalidation.
2015-06-24 14:42:33 -07:00
Tsz-Wo Nicholas Sze
093907d728
HDFS-8253. DFSStripedOutputStream.closeThreads releases cellBuffers multiple times. Contributed by Kai Sasaki
2015-06-24 23:39:05 +08:00
Jing Zhao
d920cdab91
HDFS-8567. Erasure Coding: SafeMode handles file smaller than a full stripe. Contributed by Walter Su.
2015-06-23 22:29:28 -07:00
Jing Zhao
2ba6465721
HDFS-8639. Add Option for NameNode HTTP port in MiniDFSClusterManager. Contributed by Kai Sasaki.
2015-06-23 22:16:16 -07:00
Aaron T. Myers
49dfad9429
HDFS-6440. Support more than 2 NameNodes. Contributed by Jesse Yates.
2015-06-23 17:26:11 -07:00
Jakob Homan
fac4e04dd3
HDFS-8542. WebHDFS getHomeDirectory behavior does not match specification. Contributed by Kanaka Kumar Avvaru.
2015-06-22 16:30:45 -07:00
Colin Patrick Mccabe
7b424f938c
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe)
2015-06-22 14:37:10 -07:00
Arpit Agarwal
c7d022b66f
HDFS-8192. Eviction should key off used locked memory instead of ram disk free space. (Contributed by Arpit Agarwal)
2015-06-20 13:27:52 -07:00
Colin Patrick Mccabe
d112d18324
HDFS-8582. Support getting a list of reconfigurable config properties and do not generate spurious reconfig warnings (Lei (Eddy) Xu via Colin P. McCabe)
2015-06-19 19:14:05 -07:00
Arpit Agarwal
55278c012f
HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru)
2015-06-19 18:23:34 -07:00
Jing Zhao
8c423a8163
HDFS-8602. Erasure Coding: Client can't read(decode) the EC files which have corrupt blocks. Contributed by Jing Zhao and Kai Sasaki.
2015-06-19 14:07:38 -07:00
Jing Zhao
448cb7df67
HDFS-8543. Erasure Coding: processOverReplicatedBlock() handles striped block. Contributed by Walter Su.
2015-06-19 11:53:05 -07:00
Jing Zhao
3682e01984
HDFS-8254. Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure. Contributed by Tsz Wo Nicholas Sze.
2015-06-19 10:23:45 -07:00
Yongjun Zhang
b42f1ec13d
HDFS-8633. Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys. Contributed by Ray Chiang.
2015-06-19 09:47:31 -07:00
Andrew Wang
1c13519e1e
HDFS-8605. Merge Refactor of DFSOutputStream from HDFS-7285 branch. (vinayakumarb)
2015-06-18 08:48:09 -07:00
Vinayakumar B
62c5a879ef
HDFS-8466. Refactor BlockInfoContiguous and fix NPE in TestBlockInfo#testCopyConstructor() (Contributed by Vinayakumar B)
2015-06-18 14:15:01 +05:30
Akira Ajisaka
cc432885ad
HDFS-6249. Output AclEntry in PBImageXmlWriter. Contributed by surendra singh lilhore.
2015-06-17 17:41:10 -07:00
Akira Ajisaka
1a169a26bc
HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy Battula.
2015-06-17 17:13:02 -07:00
Colin Patrick Mccabe
45ced38f10
HDFS-8589. Fix unused imports in BPServiceActor and BlockReportLeaseManager (cmccabe)
2015-06-17 17:01:42 -07:00
Haohui Mai
015535dc0a
HDFS-8446. Separate safemode related operations in GetBlockLocations(). Contributed by Haohui Mai.
2015-06-17 16:38:43 -07:00
Haohui Mai
b832774488
HDFS-8238. Move ClientProtocol to the hdfs-client. Contributed by Takanobu Asanuma.
2015-06-17 16:16:36 -07:00
Jing Zhao
c12a974ccf
HDFS-8580. Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature. Contributed by Walter Su.
2015-06-17 11:35:31 -07:00
Andrew Wang
6e3fcffe29
HDFS-8608. Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks). Contributed by Zhe Zhang.
2015-06-17 08:05:44 -07:00
Arpit Agarwal
5dbc8c9cb0
HDFS-7164. Feature documentation for HDFS-6581. (Contributed by Arpit Agarwal)
2015-06-16 19:05:44 -07:00
Tsz-Wo Nicholas Sze
d4929f448f
HDFS-8606. Cleanup DFSOutputStream by removing unwanted changes from HDFS-8386. Contributed by Rakesh R
2015-06-16 18:08:29 -07:00
Xiaoyu Yao
9cd5ad9d84
HDFS-8551. Fix hdfs datanode CLI usage message. Contributed by Brahma Reddy Battula.
2015-06-16 14:12:45 -07:00
Xiaoyu Yao
6a76250b39
HDFS-8548. Minicluster throws NPE on shutdown. Contributed by surendra singh lilhore.
2015-06-16 13:57:02 -07:00
Kihwal Lee
c74517c46b
HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee.
2015-06-16 15:40:16 -05:00
Xiaoyu Yao
4e88ff5b27
HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao.
2015-06-16 12:38:07 -07:00
Andrew Wang
a3990ca414
HDFS-8591. Remove support for deprecated configuration key dfs.namenode.decommission.nodes.per.interval.
2015-06-16 10:03:34 -07:00
Haohui Mai
32e39d8a29
HDFS-8592. SafeModeException never get unwrapped. Contributed by Haohui Mai.
2015-06-15 22:07:24 -07:00
Tsz-Wo Nicholas Sze
175e6d120f
HDFS-8361. Choose SSD over DISK in block placement.
2015-06-15 17:12:01 -07:00
Tsz-Wo Nicholas Sze
321940cf19
HDFS-8540. Mover should exit with NO_MOVE_BLOCK if no block can be moved. Contributed by surendra singh lilhore
2015-06-15 16:26:53 -07:00
Tsz-Wo Nicholas Sze
2cb09e98e3
HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina
2015-06-15 16:07:38 -07:00
Arpit Agarwal
32ffda1266
HDFS-8607. TestFileCorruption doesn't work as expected. (Contributed by Walter Su)
2015-06-15 10:11:53 -07:00
Arpit Agarwal
4c5da9bd94
HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal)
2015-06-14 16:24:30 -07:00
Yongjun Zhang
b0dc291961
HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang.
2015-06-14 11:39:31 -07:00
yliu
49d5cff490
HDFS-8559. Erasure Coding: fix non-protobuf fsimage for striped blocks. (Jing Zhao via yliu)
2015-06-14 15:39:19 +08:00
cnauroth
b8341f1cd8
HDFS-8593. Calculation of effective layout version mishandles comparison to current layout version in storage. Contributed by Chris Nauroth.
2015-06-13 10:37:52 -07:00
Jing Zhao
683332b36d
HDFS-8585. Erasure Coding: Remove dataBlockNum and parityBlockNum from StripedBlockProto. Contributed by Yi Liu.
2015-06-12 14:48:53 -07:00
Haohui Mai
eef7b50e23
HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai.
2015-06-12 14:45:34 -07:00
Andrew Wang
c17439c2dd
HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang.
2015-06-12 11:38:39 -07:00
Colin Patrick Mccabe
12b5b06c06
HDFS-7923. The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages (cmccabe)
2015-06-12 11:17:51 -07:00
Akira Ajisaka
e4489d97e5
HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to hadoop-common. Contributed by Masatake Iwasaki.
2015-06-12 07:25:15 -07:00
Arpit Agarwal
83e8110f87
HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal)
2015-06-11 15:08:56 -07:00
Colin Patrick Mccabe
587309bbba
HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe)
2015-06-11 14:26:22 -07:00
Andrew Wang
b258b344bb
HDFS-8573. Move creation of restartMeta file logic from BlockReceiver to ReplicaInPipeline. Contributed by Eddy Xu.
2015-06-11 10:12:31 -07:00
Vinayakumar B
98d340745b
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related implementation into a single class (Contributed by Rakesh R)
2015-06-11 10:48:02 +05:30
Walter Su
b7d6ea8e91
HDFS-8571. Fix TestErasureCodingCli test. Contributed by Vinayakumar B.
2015-06-11 10:26:07 +08:00
Andrew Wang
a7a7768341
HDFS-8549. Abort the balancer if an upgrade is in progress.
2015-06-10 13:42:57 -07:00
Vinayakumar B
c5d4652ded
HDFS-8556. Erasure Coding: Fix usage of 'createZone' (Contributed by Vinayakumar B)
2015-06-10 10:42:45 +05:30
Kai Zheng
e299fe86b8
HDFS-8557 Allow to configure RS and XOR raw coders. Contributed by Kai Zheng
2015-06-10 15:35:26 +08:00
Arpit Agarwal
3107434031
HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal)
2015-06-09 20:58:39 -07:00
Xiaoyu Yao
3c2397cb04
HDFS-8568. TestClusterId#testFormatWithEmptyClusterIdOption is failing. Contributed by Rakesh R.
2015-06-09 12:16:24 -07:00
Xiaoyu Yao
927577c87c
HDFS-8552. Fix hdfs CLI usage message for namenode and zkfc. Contributed by Brahma Reddy Battula
2015-06-08 21:57:26 -07:00
cnauroth
d2832b3d42
HDFS-8553. Document hdfs class path options. Contributed by Brahma Reddy Battula.
2015-06-08 16:15:38 -07:00
cnauroth
0e80d51983
HDFS-8554. TestDatanodeLayoutUpgrade fails on Windows. Contributed by Chris Nauroth.
2015-06-08 08:39:02 -07:00
cnauroth
a6cb4894b4
HDFS-8539. Hdfs doesnt have class debug in windows. Contributed by Anu Engineer.
2015-06-07 13:01:43 -07:00
Xiaoyu Yao
b61b489995
HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula.
2015-06-06 18:47:45 -07:00
cnauroth
71de367c5e
HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth.
2015-06-06 09:43:47 -07:00
cnauroth
e46cb80002
HADOOP-12055. Deprecate usage of NativeIO#link. Contributed by Andrew Wang.
2015-06-06 09:17:03 -07:00
Xiaoyu Yao
3841d09765
HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. (Contributed by Xiaoyu Yao)
2015-06-05 15:09:24 -07:00
Zhe Zhang
2eee19cd1d
HDFS-8460. Erasure Coding: stateful read result doesn't match data occasionally because of flawed test. Contributed by Walter Su.
2015-06-05 13:39:37 -07:00
Zhe Zhang
c0929ab3c2
HDFS-8319. Erasure Coding: support decoding for stateful read. Contributed by Jing Zhao.
2015-06-04 18:16:41 -07:00
Andrew Wang
b2540f486e
HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu.
2015-06-04 15:35:07 -07:00
Andrew Wang
5149dc7b97
HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable to private. Contributed by Rakesh R.
2015-06-04 11:09:19 -07:00
Kihwal Lee
ade6d9a61e
HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki.
2015-06-04 12:51:00 -05:00
yliu
a31eada33a
HDFS-8328. Follow-on to update decode for DataNode striped blocks reconstruction. (yliu)
2015-06-04 14:41:38 +08:00
Zhe Zhang
673280df24
HDFS-7621. Erasure Coding: update the Balancer/Mover data migration logic. Contributed by Walter Su.
2015-06-03 11:51:58 -07:00
Vinayakumar B
c59e745630
HDFS-3716. Purger should remove stale fsimage ckpt files (Contributed by J.Andreina)
2015-06-03 15:30:40 +05:30
Vinayakumar B
e965dcec37
HDFS-8470. fsimage loading progress should update inode, delegation token and cache pool count. (Contributed by surendra singh lilhore)
2015-06-03 14:24:55 +05:30
Vinayakumar B
54f83d9bd9
HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina)
2015-06-03 12:11:46 +05:30
Colin Patrick Mccabe
806e407ac8
HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe)
2015-06-02 20:06:28 -07:00
Jing Zhao
5f15084bd5
HDFS-8453. Erasure coding: properly handle start offset for internal blocks in a block group. Contributed by Zhe Zhang.
2015-06-02 16:14:08 -07:00
Andrew Wang
c1d50a91f7
HDFS-8513. Rename BlockPlacementPolicyRackFaultTolarent to BlockPlacementPolicyRackFaultTolerant. (wang)
2015-06-02 15:48:26 -07:00
Andrew Wang
efc510a570
HDFS-8386. Improve synchronization of 'streamer' reference in DFSOutputStream. Contributed by Rakesh R.
2015-06-02 15:39:24 -07:00
Jing Zhao
71329e817b
HDFS-8517. Fix a decoding issue in stripped block recovering in client side. Contributed by Kai Zheng.
2015-06-02 15:35:49 -07:00
Colin Patrick Mccabe
03fb5c6425
HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe)
2015-06-02 11:40:37 -07:00
Vinayakumar B
2d847e7d62
HDFS-8444. Erasure Coding: fix cannot rename a zone dir (Contributed by Walter Su)
2015-06-02 16:41:19 +05:30
Vinayakumar B
9b54e66f3e
HDFS-8336. Expose some administrative erasure coding operations to HdfsAdmin (Contributed by Uma Maheswara Rao G)
2015-06-02 16:38:43 +05:30
Vinayakumar B
a2bd6217eb
HDFS-8256. '-storagepolicies , -blockId ,-replicaDetails ' options are missed out in usage and from documentation (Contributed by J.Andreina)
2015-06-02 16:07:18 +05:30
Jing Zhao
cdc13efb1a
HDFS-8489. Subclass BlockInfo to represent contiguous blocks. Contributed by Zhe Zhang.
2015-06-01 11:42:58 -07:00
Zhe Zhang
014bd32c58
HDFS-8481. Erasure coding: remove workarounds in client side stripped blocks recovering. Contributed by Zhe Zhang.
2015-06-01 09:37:29 -07:00
Tsuyoshi Ozawa
c2356cb0bc
HDFS-8490. Typo in trace enabled log in ExceptionHandler of WebHDFS. Contributed by Archana T.
2015-06-01 00:41:50 +09:00
Jing Zhao
7817674a3a
HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma.
2015-05-29 11:05:13 -07:00
Vinayakumar B
b75df697e0
HDFS-7401. Add block info to DFSInputStream' WARN message when it adds node to deadNodes (Contributed by Arshad Mohammad)
2015-05-29 16:36:19 +05:30
Akira Ajisaka
d725dd8af6
HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina.
2015-05-29 11:36:08 +09:00
Colin Patrick Mccabe
a1140959da
HADOOP-11894. Bump the version of Apache HTrace to 3.2.0-incubating (Masatake Iwasaki via Colin P. McCabe)
2015-05-28 12:00:55 -07:00
Colin Patrick Mccabe
d2d95bfe88
HDFS-8407. libhdfs hdfsListDirectory must set errno to 0 on success (Masatake Iwasaki via Colin P. McCabe)
2015-05-28 11:38:06 -07:00
cnauroth
50eeea1300
HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer.
2015-05-27 22:54:10 -07:00
Allen Wittenauer
bf500d9798
HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw)
2015-05-27 16:51:34 -07:00
Andrew Wang
4928f54733
HDFS-8482. Rename BlockInfoContiguous to BlockInfo. Contributed by Zhe Zhang.
2015-05-27 15:42:07 -07:00
Akira Ajisaka
bb1816328a
HADOOP-11242. Record the time of calling in tracing span of IPC server. Contributed by Mastake Iwasaki.
2015-05-27 18:04:40 +09:00
Jing Zhao
1299357a05
HDFS-8479. Erasure coding: fix striping related logic in FSDirWriteFileOp to sync with HDFS-8421. Contributed by Zhe Zhang.
2015-05-26 16:06:50 -07:00
Zhe Zhang
c9e0268216
Addendum fix for HDFS-7912.
2015-05-26 12:32:16 -07:00
Vinayakumar B
80c56c2d31
HDFS-8408. Revisit and refactor ErasureCodingInfo (Contributed by Vinayakumar B)
...
Added missed file
2015-05-26 12:07:15 -07:00
Vinayakumar B
9a18598e2d
HDFS-8408. Revisit and refactor ErasureCodingInfo (Contributed by Vinayakumar B)
2015-05-26 12:07:15 -07:00
Kai Zheng
b30e96bfb4
HDFS-8382 Remove chunkSize and initialize from erasure coder. Contributed by Kai Zheng
2015-05-26 12:07:15 -07:00
Tsz-Wo Nicholas Sze
0ed92e5b13
HDFS-7768. Change fsck to support EC files. Contributed by Takanobu Asanuma
2015-05-26 12:07:15 -07:00
Walter Su
3d734df24c
HDFS-8441. Erasure Coding: make condition check earlier for setReplication. (waltersu4549)
2015-05-26 12:07:09 -07:00
Zhe Zhang
7af05a3db4
HDFS-8294. Erasure Coding: Fix Findbug warnings present in erasure coding. Contributed by Rakesh R.
2015-05-26 12:02:34 -07:00
Zhe Zhang
e53fa769c9
HDFS-8186. Erasure coding: Make block placement policy for EC file configurable. Contributed by Walter Su.
2015-05-26 12:02:34 -07:00
Jing Zhao
744ef17792
HDFS-8427. Remove dataBlockNum and parityBlockNum from BlockInfoStriped. Contributed by Kai Sasaki.
2015-05-26 12:02:34 -07:00
Jing Zhao
45db1a0b72
HDFS-8323. Bump GenerationStamp for write faliure in DFSStripedOutputStream. Contributed by Tsz Wo Nicholas Sze.
2015-05-26 12:02:34 -07:00
Zhe Zhang
47ef869fa7
HDFS-8428. Erasure Coding: Fix the NullPointerException when deleting file. Contributed by Yi Liu.
2015-05-26 12:02:33 -07:00
Zhe Zhang
91c81fdc24
HDFS-8375. Add cellSize as an XAttr to ECZone. Contributed by Vinayakumar B.
2015-05-26 12:02:33 -07:00
Walter Su
914580934c
HDFS-8378. Erasure Coding: Few improvements for the erasure coding worker. Contributed by Rakesh R.
2015-05-26 12:02:33 -07:00
boli2
8c95673db4
HDFS-8366. Erasure Coding: Make the timeout parameter of polling blocking queue configurable in DFSStripedOutputStream. Contributed by Li Bo
2015-05-26 12:02:32 -07:00
Zhe Zhang
7434c44b16
HDFS-8320. Erasure coding: consolidate striping-related terminologies. Contributed by Zhe Zhang and Jing Zhao.
2015-05-26 12:02:32 -07:00
Jing Zhao
b008348dbf
HDFS-8418. Fix the isNeededReplication calculation for Striped block in NN. Contributed by Yi Liu.
2015-05-26 12:02:32 -07:00
Jing Zhao
6c310db159
HDFS-8417. Erasure Coding: Pread failed to read data starting from not-first stripe. Contributed by Walter Su.
2015-05-26 12:02:32 -07:00
waltersu4549
a919726914
HDFS-8352. Erasure Coding: test webhdfs read write stripe file. (waltersu4549)
2015-05-26 12:02:30 -07:00
Kai Zheng
c9103e9cac
HDFS-8367 BlockInfoStriped uses EC schema. Contributed by Kai Sasaki
2015-05-26 12:02:30 -07:00
Jing Zhao
d8ea443af0
Merge HDFS-8394 from trunk: Move getAdditionalBlock() and related functionalities into a separate class.
2015-05-26 12:02:26 -07:00
Uma Maheswara Rao G
c99c337928
HDFS-8391. NN should consider current EC tasks handling count from DN while assigning new tasks. Contributed by Uma Maheswara Rao G.
2015-05-26 12:01:51 -07:00
Vinayakumar B
bba15e06d2
HDFS-8364. Erasure coding: fix some minor bugs in EC CLI (Contributed by Walter Su)
2015-05-26 12:01:51 -07:00
Zhe Zhang
9798065cbb
HDFS-8195. Erasure coding: Fix file quota change when we complete/commit the striped blocks. Contributed by Takuya Fukudome.
2015-05-26 12:01:50 -07:00
yliu
8da9e18dad
HDFS-8363. Erasure Coding: DFSStripedInputStream#seekToNewSource. (yliu)
2015-05-26 12:01:50 -07:00
Zhe Zhang
54d2827522
HDFS-8368. Erasure Coding: DFS opening a non-existent file need to be handled properly. Contributed by Rakesh R.
2015-05-26 12:01:50 -07:00
Jing Zhao
97a2396af6
HDFS-8372. Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327. Contributed by Zhe Zhang.
2015-05-26 12:01:50 -07:00
Zhe Zhang
8d3030f064
HDFS-7678. Erasure coding: DFSInputStream with decode functionality (pread). Contributed by Zhe Zhang.
2015-05-26 12:01:49 -07:00
Zhe Zhang
6bacaa9a52
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit is for HDFS-8327 and HDFS-8357). Contributed by Zhe Zhang.
2015-05-26 12:01:49 -07:00
Jing Zhao
51ea117f88
HDFS-8355. Erasure Coding: Refactor BlockInfo and BlockInfoUnderConstruction. Contributed by Tsz Wo Nicholas Sze.
2015-05-26 12:01:49 -07:00
Jing Zhao
9da927540f
HDFS-8289. Erasure Coding: add ECSchema to HdfsFileStatus. Contributed by Yong Zhang.
2015-05-26 12:01:49 -07:00
Jing Zhao
a17cedb44c
HDFS-8203. Erasure Coding: Seek and other Ops in DFSStripedInputStream. Contributed by Yi Liu.
2015-05-26 12:01:48 -07:00
Uma Maheswara Rao G
ac97edd1ab
HDFS-8129. Erasure Coding: Maintain consistent naming for Erasure Coding related classes - EC/ErasureCoding. Contributed by Uma Maheswara Rao G
2015-05-26 12:01:48 -07:00
Zhe Zhang
cea46f79b0
HDFS-8334. Erasure coding: rename DFSStripedInputStream related test classes. Contributed by Zhe Zhang.
2015-05-26 12:01:48 -07:00
Zhe Zhang
6616de24cb
HDFS-7348. Erasure Coding: DataNode reconstruct striped blocks. Contributed by Yi Liu.
2015-05-26 12:01:47 -07:00
Tsz-Wo Nicholas Sze
220ca960bc
HDFS-7672. Handle write failure for stripping blocks and refactor the existing code in DFSStripedOutputStream and StripedDataStreamer.
2015-05-26 12:00:45 -07:00
Uma Maheswara Rao G
e849be2d31
HDFS-8324. Add trace info to DFSClient#getErasureCodingZoneInfo(..). Contributed by Vinayakumar B
2015-05-26 11:59:58 -07:00
Vinayakumar B
4392325546
HDFS-8242. Erasure Coding: XML based end-to-end test for ECCli commands (Contributed by Rakesh R)
2015-05-26 11:59:58 -07:00
Uma Maheswara Rao G
436c14855a
HDFS-8137. Send the EC schema to DataNode via EC encoding/recovering command. Contributed by Uma Maheswara Rao G
2015-05-26 11:59:57 -07:00
Jing Zhao
6dea01f1ee
HDFS-8281. Erasure Coding: implement parallel stateful reading for striped layout. Contributed by Jing Zhao.
2015-05-26 11:59:57 -07:00
Jing Zhao
ea6c66ed57
HDFS-8316. Erasure coding: refactor EC constants to be consistent with HDFS-8249. Contributed by Zhe Zhang.
2015-05-26 11:59:57 -07:00
Zhe Zhang
77d94dd5ec
HDFS-7949. WebImageViewer need support file size calculation with striped blocks. Contributed by Rakesh R.
2015-05-26 11:59:56 -07:00
Jing Zhao
5059958bf4
HDFS-8308. Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog. Contributed by Jing Zhao.
2015-05-26 11:59:56 -07:00
Zhe Zhang
1a31f1c303
HDFS-8183. Erasure Coding: Improve DFSStripedOutputStream closing of datastreamer threads. Contributed by Rakesh R.
2015-05-26 11:59:56 -07:00
Zhe Zhang
f0628280c3
HDFS-8282. Erasure coding: move striped reading logic to StripedBlockUtil. Contributed by Zhe Zhang.
2015-05-26 11:59:56 -07:00
Zhe Zhang
b00c663877
HDFS-8272. Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read). Contributed by Jing Zhao
2015-05-26 11:59:55 -07:00
Jing Zhao
1a8139e6ad
Fix merge conflicts.
2015-05-26 11:59:55 -07:00
Jing Zhao
9593776e34
HDFS-8235. Erasure Coding: Create DFSStripedInputStream in DFSClient#open. Contributed by Kai Sasaki.
2015-05-26 11:59:55 -07:00
Vinayakumar B
5eb17e5726
HDFS-8189. ClientProtocol#createErasureCodingZone API was wrongly annotated as Idempotent (Contributed by Vinayakumar B)
2015-05-26 11:59:55 -07:00
Vinayakumar B
f9eb95c31d
HDFS-8230. Erasure Coding: Ignore DatanodeProtocol#DNA_ERASURE_CODING_RECOVERY commands from standbynode if any (Contributed by Vinayakumar B)
2015-05-26 11:59:54 -07:00
Zhe Zhang
89d3378578
HDFS-8033. Erasure coding: stateful (non-positional) read from files in striped layout. Contributed by Zhe Zhang.
2015-05-26 11:59:54 -07:00
Zhe Zhang
f5d4a95ef5
HDFS-8228. Erasure Coding: SequentialBlockGroupIdGenerator#nextValue may cause block id conflicts. Contributed by Jing Zhao.
2015-05-26 11:59:54 -07:00
Jing Zhao
3f2c6938f1
HDFS-8223. Should calculate checksum for parity blocks in DFSStripedOutputStream. Contributed by Yi Liu.
2015-05-26 11:59:54 -07:00
Jing Zhao
eb612b0b70
HDFS-8233. Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last stripe is at the block group boundary. Contributed by Jing Zhao.
2015-05-26 11:59:53 -07:00
Kai Zheng
fcd54ecce2
HDFS-8136. Client gets and uses EC schema when reads and writes a stripping file. Contributed by Kai Sasaki
2015-05-26 11:59:53 -07:00
Zhe Zhang
e8df2581c3
HDFS-8156. Add/implement necessary APIs even we just have the system default schema. Contributed by Kai Zheng.
2015-05-26 11:59:53 -07:00
Uma Maheswara Rao G
014d8675c5
HDFS-8024. Erasure Coding: ECworker frame, basics, bootstraping and configuration. (Contributed by Uma Maheswara Rao G)
2015-05-26 11:59:52 -07:00
Zhe Zhang
2f11109bb0
HDFS-8212. DistributedFileSystem.createErasureCodingZone should pass schema in FileSystemLinkResolver. Contributed by Tsz Wo Nicholas Sze.
2015-05-26 11:59:52 -07:00
Zhe Zhang
e9d85bbf30
HDFS-8216. TestDFSStripedOutputStream should use BlockReaderTestUtil to create BlockReader. Contributed by Tsz Wo Nicholas Sze.
2015-05-26 11:59:52 -07:00
Tsz-Wo Nicholas Sze
e107886d6f
HDFS-8190. StripedBlockUtil.getInternalBlockLength may have overflow error.
2015-05-26 11:59:52 -07:00
Zhe Zhang
922631f04f
HDFS-8188. Erasure coding: refactor client-related code to sync with HDFS-8082 and HDFS-8169. Contributed by Zhe Zhang.
2015-05-26 11:59:51 -07:00
Vinayakumar B
dfba46ab57
HDFS-8181. createErasureCodingZone sets retryCache state as false always (Contributed by Uma Maheswara Rao G)
2015-05-26 11:59:51 -07:00
Vinayakumar B
a1c9425265
HDFS-8146. Protobuf changes for BlockECRecoveryCommand and its fields for making it ready for transfer to DN (Contributed by Uma Maheswara Rao G)
2015-05-26 11:59:51 -07:00
Jing Zhao
f6e1160ef1
HDFS-8145. Fix the editlog corruption exposed by failed TestAddStripedBlocks. Contributed by Jing Zhao.
2015-05-26 11:59:46 -07:00
Jing Zhao
ab76e1fe36
HDFS-7937. Erasure Coding: INodeFile quota computation unit tests. Contributed by Kai Sasaki.
2015-05-26 11:59:13 -07:00
Jing Zhao
cd458c38a0
HDFS-8166. DFSStripedOutputStream should not create empty blocks. Contributed by Jing Zhao.
2015-05-26 11:59:13 -07:00
Zhe Zhang
64f8f0a145
HDFS-8167. BlockManager.addBlockCollectionWithCheck should check if the block is a striped block. Contributed by Hui Zheng.
2015-05-26 11:59:12 -07:00
Tsz-Wo Nicholas Sze
4c039b0876
HDFS-7994. Detect if resevered EC Block ID is already used during namenode startup. Contributed by Hui Zheng
2015-05-26 11:59:12 -07:00
Jing Zhao
5e8837dd6c
HDFS-8120. Erasure coding: created util class to analyze striped block groups. Contributed by Zhe Zhang and Li Bo.
2015-05-26 11:59:12 -07:00
Vinayakumar B
ceb3d1c170
HDFS-7349. Support DFS command for the EC encoding (Contributed by Vinayakumar B)
2015-05-26 11:59:11 -07:00
Vinayakumar B
04c23d95d0
HDFS-8123. Erasure Coding: Better to move EC related proto messages to a separate erasurecoding proto file (Contrubuted by Rakesh R)
2015-05-26 11:59:11 -07:00
Zhe Zhang
3b56a5aa9e
HDFS-8114. Erasure coding: Add auditlog FSNamesystem#createErasureCodingZone if this operation fails. Contributed by Rakesh R.
2015-05-26 11:59:10 -07:00
Zhe Zhang
a32c4dc38a
HDFS-8122. Erasure Coding: Support specifying ECSchema during creation of ECZone. Contributed by Vinayakumar B.
2015-05-26 11:59:06 -07:00
Zhe Zhang
35797b0889
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit mainly addresses HDFS-8081 and HDFS-8048. Contributed by Zhe Zhang.
2015-05-26 11:58:40 -07:00
Vinayakumar B
f53e402635
HDFS-8090. Erasure Coding: Add RPC to client-namenode to list all ECSchemas loaded in Namenode. (Contributed by Vinayakumar B)
2015-05-26 11:58:40 -07:00
Kai Zheng
4970f2a2ab
HDFS-7889 Subclass DFSOutputStream to support writing striping layout files. Contributed by Li Bo
2015-05-26 11:58:40 -07:00
Jing Zhao
4d0bc724f2
HDFS-8077. Erasure coding: fix bugs in EC zone and symlinks. Contributed by Jing Zhao and Zhe Zhang.
2015-05-26 11:58:32 -07:00
Kai Zheng
c4a0d88366
HDFS-8104 Make hard-coded values consistent with the system default schema first before remove them. Contributed by Kai Zheng
2015-05-26 11:55:17 -07:00
Kai Zheng
1a37b0fe9a
HDFS-8074 Define a system-wide default EC schema. Contributed by Kai Zheng
2015-05-26 11:55:16 -07:00
Vinayakumar B
2ca3e38feb
HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B)
...
Added missed file
2015-05-26 11:55:16 -07:00
Vinayakumar B
16484f0030
HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B)
2015-05-26 11:55:16 -07:00
Zhe Zhang
bff59392ee
HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao
2015-05-26 11:55:15 -07:00
Zhe Zhang
91c741a2a1
HDFS-7782. Erasure coding: pread from files in striped layout. Contributed by Zhe Zhang and Jing Zhao
2015-05-26 11:55:15 -07:00
Zhe Zhang
146ce7a978
HDFS-7969. Erasure coding: NameNode support for lease recovery of striped block groups. Contributed by Zhe Zhang.
2015-05-26 11:55:14 -07:00
Zhe Zhang
c243319eab
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging trunk changes (this commit is for HDFS-8035). Contributed by Zhe Zhang
2015-05-26 11:55:13 -07:00
Zhe Zhang
1af8c14862
HDFS-7839. Erasure coding: implement facilities in NameNode to create and manage EC zones. Contributed by Zhe Zhang
2015-05-26 11:55:07 -07:00
Zhe Zhang
578019d6a2
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging (this commit is for conflicts from HDFS-6945). Contributed by Zhe Zhang.
2015-05-26 11:44:47 -07:00
Zhe Zhang
9edc73d123
HDFS-7617. Add unit tests for editlog transactions for EC. Contributed by Hui Zheng.
2015-05-26 11:44:47 -07:00
Jing Zhao
cd655ee817
HDFS-8005. Erasure Coding: simplify striped block recovery work computation and add tests. Contributed by Jing Zhao.
2015-05-26 11:44:42 -07:00
Jing Zhao
abf833a7b2
HDFS-7907. Erasure Coding: track invalid, corrupt, and under-recovery striped blocks in NameNode. Contributed by Jing Zhao.
2015-05-26 11:43:44 -07:00
Zhe Zhang
97378e4cd0
HDFS-7936. Erasure coding: resolving conflicts in the branch when merging (this commit is for HDFS-7742)
2015-05-26 11:43:43 -07:00
Tsz-Wo Nicholas Sze
9d1175b8fb
HDFS-7716. Add a test for BlockGroup support in FSImage. Contributed by Takuya Fukudome
2015-05-26 11:43:43 -07:00
Zhe Zhang
ea2e60fbcc
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903, HDFS-7435, HDFS-7930, HDFS-7960 (this commit is for HDFS-7960)
2015-05-26 11:43:42 -07:00
Jing Zhao
d0d75a8339
HDFS-7827. Erasure Coding: support striped blocks in non-protobuf fsimage. Contributed by Hui Zheng.
2015-05-26 11:43:42 -07:00
Jing Zhao
46dac3595f
HDFS-7864. Erasure Coding: Update safemode calculation for striped blocks. Contributed by GAO Rui.
2015-05-26 11:43:42 -07:00
Zhe Zhang
544f75d651
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903, HDFS-7435 and HDFS-7930 (this commit is for HDFS-7930 only)
2015-05-26 11:43:42 -07:00
Zhe Zhang
57a84c0d14
HDFS-7369. Erasure coding: distribute recovery work for striped blocks to DataNode. Contributed by Zhe Zhang.
2015-05-26 11:43:37 -07:00
Jing Zhao
a38a37c634
HDFS-7912. Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks. Contributed by Jing Zhao.
2015-05-26 11:41:59 -07:00
Jing Zhao
26773d9d6c
HDFS-7826. Erasure Coding: Update INodeFile quota computation for striped blocks. Contributed by Kai Sasaki.
2015-05-26 11:32:26 -07:00
Jing Zhao
11585883a9
HDFS-7936. Erasure coding: resolving conflicts when merging with HDFS-7903 and HDFS-7435. Contributed by Zhe Zhang.
2015-05-26 11:32:26 -07:00
Jing Zhao
f05c21285e
HDFS-7853. Erasure coding: extend LocatedBlocks to support reading from striped files. Contributed by Jing Zhao.
2015-05-26 11:32:25 -07:00
Jing Zhao
39a0a85fb7
HDFS-7872. Erasure Coding: INodeFile.dumpTreeRecursively() supports to print striped blocks. Contributed by Takuya Fukudome.
2015-05-26 11:32:25 -07:00
Jing Zhao
1e1e930407
HDFS-7837. Erasure Coding: allocate and persist striped blocks in NameNode. Contributed by Jing Zhao.
2015-05-26 11:32:20 -07:00
Zhe Zhang
9af1f4779b
Addendum fix for HDFS-7749 to be compatible with HDFS-7993
2015-05-26 11:07:16 -07:00
Jing Zhao
9f2f583f40
HDFS-7749. Erasure Coding: Add striped block support in INodeFile. Contributed by Jing Zhao.
2015-05-26 11:07:15 -07:00
Jing Zhao
ba93714920
HDFS-7716. Erasure Coding: extend BlockInfo to handle EC info. Contributed by Jing Zhao.
2015-05-26 11:07:11 -07:00
Jing Zhao
ec48013426
Fix Compilation Error in TestAddBlockgroup.java after the merge
2015-05-26 11:03:34 -07:00
Zhe Zhang
0c1da5a030
HDFS-7652. Process block reports for erasure coded blocks. Contributed by Zhe Zhang
2015-05-26 11:03:34 -07:00
Zhe Zhang
bc2833b1c9
HDFS-7339. Allocating and persisting block groups in NameNode. Contributed by Zhe Zhang
2015-05-26 11:03:34 -07:00
Vinayakumar B
f166e67a23
HDFS-7347. Configurable erasure coding policy for individual files and directories ( Contributed by Zhe Zhang )
2015-05-26 11:03:34 -07:00
Haohui Mai
ada233b7cd
HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang.
2015-05-24 22:30:32 -07:00
Vinayakumar B
0c6638c2ea
HDFS-8268. Port conflict log for data node server is not sufficient (Contributed by Mohammad Shahid Khan)
2015-05-22 16:15:15 +05:30
Arpit Agarwal
cf2b5694d6
HDFS-8454. Remove unnecessary throttling in TestDatanodeDeath. (Arpit Agarwal)
2015-05-21 20:43:57 -07:00
Xiaoyu Yao
05e04f34f2
HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran.
2015-05-21 11:58:00 -07:00
Haohui Mai
2b6bcfdafa
HDFS-8421. Move startFile() and related functions into FSDirWriteFileOp. Contributed by Haohui Mai.
2015-05-21 08:08:28 -07:00
Akira Ajisaka
0e4f1081c7
HADOOP-10366. Add whitespaces between classes for values in core-default.xml to fit better in browser. Contributed by kanaka kumar avvaru.
2015-05-21 17:52:03 +09:00
Akira Ajisaka
dc8434ab2b
HDFS-4383. Document the lease limits. Contributed by Arshad Mohammad.
2015-05-21 17:30:43 +09:00
Kihwal Lee
8860e352c3
HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts.
2015-05-19 13:05:15 -05:00
Colin Patrick Mccabe
470c87dbc6
HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe)
2015-05-19 10:50:15 -07:00
Kihwal Lee
de30d66b26
HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui.
2015-05-19 08:04:38 -05:00
Vinayakumar B
d4a2830b63
HADOOP-11103. Clean up RemoteException (Contributed by Sean Busbey)
2015-05-19 14:41:05 +05:30
Vinayakumar B
93972a332a
HDFS-6348. SecondaryNameNode not terminating properly on runtime exceptions (Contributed by Rakesh R)
2015-05-19 12:24:25 +05:30
Ravi Prakash
cdfae446ad
HDFS-4185. Add a metric for number of active leases (Rakesh R via raviprak)
2015-05-18 12:37:21 -07:00
Tsz-Wo Nicholas Sze
0c590e1c09
HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma
2015-05-19 02:57:54 +08:00
Arpit Agarwal
a2190bf15d
HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal)
2015-05-18 11:36:29 -07:00
Arpit Agarwal
e453989a57
HDFS-8157. Writes to RAM DISK reserve locked memory for block files. (Arpit Agarwal)
2015-05-16 09:05:35 -07:00
Haohui Mai
ac742c762d
HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal.
2015-05-15 19:12:40 -07:00
Haohui Mai
e5afac5896
HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai.
2015-05-15 19:09:59 -07:00
Jing Zhao
8f37873342
HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze.
2015-05-15 16:14:54 -07:00
Vinayakumar B
3bef7c80a9
HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He)
2015-05-15 11:07:51 +05:30
Akira Ajisaka
ee7beda6e3
HDFS-8350. Remove old webhdfs.xml and other outdated documentation stuff. Contributed by Brahma Reddy Battula.
2015-05-15 13:54:35 +09:00
Akira Ajisaka
cbc01ed08e
HDFS-8371. Fix test failure in TestHdfsConfigFields for spanreceiver properties. Contributed by Ray Chiang.
2015-05-15 12:14:47 +09:00
Vinayakumar B
def9136e02
HDFS-8150. Make getFileChecksum fail for blocks under construction (Contributed by J.Andreina)
2015-05-14 15:54:51 +05:30
Haohui Mai
b2c85db86c
HDFS-7728. Avoid updating quota usage while loading edits. Contributed by Jing Zhao.
2015-05-13 21:50:35 -07:00
Arpit Agarwal
54fa9b4217
HDFS-8243. Files written by TestHostsFiles and TestNameNodeMXBean are causing Release Audit Warnings. (Contributed by Ruth Wisniewski)
2015-05-13 19:43:53 -07:00
Colin Patrick Mccabe
281d47a969
HDFS-8380. Always call addStoredBlock on blocks which have been shifted from one storage to another (cmccabe)
2015-05-13 14:29:05 -07:00
Kihwal Lee
f9a46a00d2
HDFS-8358. TestTraceAdmin fails. Contributed by Masatake Iwasaki.
2015-05-13 14:15:27 -05:00
Tsz-Wo Nicholas Sze
cdec12d1b8
HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore
2015-05-13 11:57:49 -07:00
Vinayakumar B
065d8f2a34
HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R)
2015-05-13 17:27:34 +05:30
Akira Ajisaka
2463666ecb
HDFS-6184. Capture NN's thread dump when it fails over. Contributed by Ming Ma.
2015-05-13 11:37:22 +09:00
yliu
6d5da94841
HDFS-8255. Rename getBlockReplication to getPreferredBlockReplication. (Contributed by Zhe Zhang)
2015-05-12 21:29:22 +08:00
Vinayakumar B
987abc99b0
HDFS-8362. Java Compilation Error in TestHdfsConfigFields.java (Contributed by Arshad Mohammad)
2015-05-12 12:09:13 +05:30
Kihwal Lee
ea11590aad
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah.
2015-05-11 14:31:58 -05:00
Akira Ajisaka
1dd79ffaca
HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula.
2015-05-12 00:18:18 +09:00
Junping Du
7e543c27fa
YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak.
2015-05-11 05:27:07 -07:00
Akira Ajisaka
3fa2efc09f
HDFS-8351. Remove namenode -finalize option from document. (aajisaka)
2015-05-11 15:34:44 +09:00
Haohui Mai
4536399d47
HDFS-8357. Consolidate parameters of INode.CleanSubtree() into a parameter objects. Contributed by Li Lu.
2015-05-09 22:51:58 -07:00
Haohui Mai
02a4a22b9c
HDFS-8327. Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate(). Contributed by Haohui Mai.
2015-05-08 23:09:25 -07:00
Haohui Mai
00fe1ed3a4
HDFS-6757. Simplify lease manager with INodeID. Contributed by Haohui Mai.
2015-05-08 23:04:31 -07:00
Arpit Agarwal
59995cec4a
HDFS-8097. TestFileTruncate is failing intermittently. (Contributed by Rakesh R)
2015-05-08 16:28:24 -07:00
Allen Wittenauer
c945c20483
HADOOP-11590. Update sbin commands and documentation to use new --slaves option (aw)
2015-05-08 15:27:25 -07:00
Xiaoyu Yao
d0e75e60fb
HDFS-8326. Documentation about when checkpoints are run is out of date. (Misty Stanley-Jones via xyao)
2015-05-08 14:46:37 -07:00
Kihwal Lee
2d4ae3d18b
HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah.
2015-05-08 16:36:41 -05:00
Colin Patrick Mccabe
f523e963e4
HDFS-8113. Add check for null BlockCollection pointers in BlockInfoContiguous structures (Chengbing Liu via Colin P. McCabe)
2015-05-08 13:50:51 -07:00
Kihwal Lee
7a7960be41
HDFS-7433. Optimize performance of DatanodeManager's node map. Contributed by Daryn Sharp.
2015-05-08 15:18:29 -05:00
Colin Patrick Mccabe
8f7c2364d7
HDFS-8284. Update documentation about how to use HTrace with HDFS (Masatake Iwasaki via Colin P. McCabe)
2015-05-08 12:30:03 -07:00
Yongjun Zhang
730f9930a4
HDFS-8311. DataStreamer.transfer() should timeout the socket InputStream. (Esteban Gutierrez via Yongjun Zhang)
2015-05-08 00:11:46 -07:00
cnauroth
26f61d41df
HDFS-5640. Add snapshot methods to FileContext. Contributed by Rakesh R.
2015-05-08 11:20:55 -07:00
Arun Suresh
3cefc02af7
HDFS-7559. Create unit test to automatically compare HDFS related classes and hdfs-default.xml. (Ray Chiang via asuresh)
2015-05-08 10:29:38 -07:00
Arpit Agarwal
a2d40bced9
HDFS-8340. Fix NFS documentation of nfs.wtmax. (Contributed by Ajith S)
2015-05-08 10:15:26 -07:00
Haohui Mai
f4ebbc6afc
HDFS-8346. libwebhdfs build fails during link due to unresolved external symbols. Contributed by Chris Nauroth.
2015-05-08 09:58:17 -07:00
Kihwal Lee
6f622672b6
HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula.
2015-05-08 09:30:38 -05:00
Uma Maheswara Rao G
7b1ea9c481
HDFS-6285. tidy an error log inside BlockReceiver. Contributed by Liang Xie.
2015-05-08 17:22:52 +05:30
Vinayakumar B
c648317a68
HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G)
2015-05-08 17:18:14 +05:30
Vinayakumar B
e01c1a979c
HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina)
2015-05-08 15:24:36 +05:30
Vinayakumar B
4c6816faf8
HDFS-8209. Support different number of datanode directories in MiniDFSCluster. (Contributed by surendra singh lilhore)
2015-05-08 15:03:44 +05:30
Tsuyoshi Ozawa
888e630990
HDFS-8207. Improper log message when blockreport interval compared with initial delay. Contributed by Brahma Reddy Battula and Ashish Singhi.
2015-05-08 18:13:09 +09:00
Vinayakumar B
6232235df3
HDFS-8175. Provide information on snapshotDiff for supporting the comparison between snapshot and current status (Contributed by J.Andreina)
2015-05-08 14:37:50 +05:30
Vinayakumar B
e642eff0e0
HDFS-8187. Remove usage of '-setStoragePolicy' and '-getStoragePolicy' using dfsadmin cmd (as it is not been supported) (Contributed by J.Andreina)
2015-05-08 14:16:30 +05:30
Uma Maheswara Rao G
4b00ae5c11
HDFS-8108. Fsck should provide the info on mandatory option to be used along with -blocks ,-locations and -racks. Contributed by J.Andreina.
2015-05-08 14:00:45 +05:30
Akira Ajisaka
fd605997ee
HDFS-8222. Remove usage of "dfsadmin -upgradeProgress" from document which is no longer supported. Contributed by J.Andreina.
2015-05-08 16:27:40 +09:00
Vinayakumar B
26da2ddf38
HDFS-7998. HDFS Federation : Command mentioned to add a NN to existing federated cluster is wrong (Contributed by Ajith S)
2015-05-08 12:33:49 +05:30
Akira Ajisaka
c5afe444df
HDFS-8349. Remove .xml and documentation references to dfs.webhdfs.enabled. Contributed by Ray Chiang.
2015-05-08 15:57:13 +09:00
Uma Maheswara Rao G
e16f4b7f70
HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R.
2015-05-08 12:26:47 +05:30
Vinayakumar B
e5e492a963
HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun() ( Contributed by Sanghyun Yun)
2015-05-08 11:25:24 +05:30
Uma Maheswara Rao G
2ea0f2fc93
HDFS-8174. Update replication count to live rep count in fsck report. Contributed by J.Andreina
2015-05-08 11:01:51 +05:30
Vinayakumar B
66988476d0
HDFS-8067. haadmin prints out stale help messages (Contributed by Ajith S)
2015-05-08 10:45:21 +05:30
Vinayakumar B
c7c26a1e4a
HDFS-8257. Namenode rollingUpgrade option is incorrect in document (Contributed by J.Andreina)
2015-05-08 10:35:25 +05:30
Haohui Mai
4d9f9e546f
HDFS-8037. CheckAccess in WebHDFS silently accepts malformed FsActions parameters. Contributed by Walter Su.
2015-05-07 15:25:26 -07:00
Haohui Mai
767b91cd83
HDFS-8321. CacheDirectives and CachePool operations should throw RetriableException in safemode. Contributed by Haohui Mai.
2015-05-07 15:17:11 -07:00
Tsz-Wo Nicholas Sze
f9427f1760
HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su
2015-05-07 11:36:35 -07:00
Akira Ajisaka
449e4426a5
HDFS-8325. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula.
2015-05-07 15:14:20 +09:00
cnauroth
6633a8474d
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu.
2015-05-06 21:11:30 -07:00
Konstantin V Shvachko
c75cfa29cf
HDFS-2484. checkLease should throw FileNotFoundException when file does not exist. Contributed by Rakesh R.
2015-05-06 16:06:04 -07:00
Haohui Mai
4da8490b51
HDFS-8314. Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users. Contributed by Li Lu.
2015-05-05 15:41:22 -07:00
Xiaoyu Yao
0100b15501
HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao)
2015-05-05 13:41:14 -07:00
Colin Patrick Mccabe
ffce9a3413
HDFS-7847. Modify NNThroughputBenchmark to be able to operate on a remote NameNode (Charles Lamb via Colin P. McCabe)
2015-05-05 11:34:58 -07:00
Colin Patrick Mccabe
24d3a2d4fd
HDFS-7758. Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead (Lei (Eddy) Xu via Colin P. McCabe)
2015-05-05 11:08:59 -07:00
Colin Patrick Mccabe
fcd4cb7516
HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe)
2015-05-05 10:50:09 -07:00
Vinayakumar B
318081ccd7
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B)
...
Reverted earlier commit
2015-05-05 11:05:37 +05:30
Haohui Mai
0d6aa5d609
HDFS-8237. Move all protocol classes used by ClientProtocol to hdfs-client. Contributed by Haohui Mai.
2015-05-04 15:00:29 -07:00
Colin Patrick Mccabe
3fe79e1db8
HDFS-7397. Add more detail to the documentation for the conf key "dfs.client.read.shortcircuit.streams.cache.size" (Brahma Reddy Battula via Colin P. McCabe)
2015-05-04 12:50:29 -07:00
cnauroth
c4578760b6
HDFS-8290. WebHDFS calls before namesystem initialization can cause NullPointerException. Contributed by Chris Nauroth.
2015-05-04 11:35:04 -07:00
Xiaoyu Yao
a319771d1d
HDFS-8309. Skip unit test using DataNodeTestUtils#injectDataDirFailure() on Windows. (xyao)
2015-05-02 22:15:24 -07:00
Haohui Mai
6ae2a0d048
HDFS-8249. Separate HdfsConstants into the client and the server side class. Contributed by Haohui Mai.
2015-05-02 10:03:23 -07:00
Arpit Agarwal
6f541edce0
HDFS-8229. LAZY_PERSIST file gets deleted after NameNode restart. (Contributed by Surendra Singh Lilhore)
2015-05-01 16:42:19 -07:00
Tsz-Wo Nicholas Sze
d3d019c337
HDFS-8086. Move LeaseRenewer to the hdfs.client.impl package. Contributed by Takanobu
2015-05-01 15:12:18 -07:00
Arpit Agarwal
64d30a6186
HDFS-8276. LazyPersistFileScrubber should be disabled if scrubber interval configured zero. (Contributed by Surendra Singh Lilhore)
2015-05-01 14:43:09 -07:00
Colin Patrick Mccabe
b82567d455
HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe)
2015-05-01 11:19:40 -07:00
Yongjun Zhang
279958b772
HDFS-7281. Missing block is marked as corrupted block (Ming Ma via Yongjun Zhang)
2015-05-01 08:42:00 -07:00
Jing Zhao
98a6176628
HDFS-8300. Fix unit test failures and findbugs warning caused by HDFS-8283. Contributed by Jing Zhao.
2015-04-30 19:27:07 -07:00
Andrew Wang
87e9978235
HDFS-8292. Move conditional in fmt_time from dfs-dust.js to status.html. Contributed by Charles Lamb.
2015-04-30 18:23:13 -07:00
Haohui Mai
c55d609053
HDFS-8200. Refactor FSDirStatAndListingOp. Contributed by Haohui Mai.
2015-04-30 13:41:46 -07:00
Akira Ajisaka
de9404f02f
HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao.
2015-04-30 20:12:32 +09:00
Akira Ajisaka
e89fc53a1d
HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang.
2015-04-30 19:11:22 +09:00
Andrew Wang
aa22450442
HDFS-8214. Secondary NN Web UI shows wrong date for Last Checkpoint. Contributed by Charles Lamb.
2015-04-29 17:37:56 -07:00
Haohui Mai
3dd6395bb2
HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai.
2015-04-29 11:12:45 -07:00
Jing Zhao
7947e5b53b
HDFS-8283. DataStreamer cleanup and some minor improvement. Contributed by Tsz Wo Nicholas Sze.
2015-04-29 10:41:46 -07:00
Haohui Mai
439614b0c8
HDFS-8280. Code Cleanup in DFSInputStream. Contributed by Jing Zhao.
2015-04-28 18:11:59 -07:00
Haohui Mai
c79e7f7d99
HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai.
2015-04-28 18:05:46 -07:00
Tsz-Wo Nicholas Sze
5639bf02da
HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su
2015-04-28 13:05:33 -07:00
cnauroth
feb68cb547
HDFS-8232. Missing datanode counters when using Metrics2 sink interface. Contributed by Anu Engineer.
2015-04-27 16:48:13 -07:00
Arpit Agarwal
0d5b0143cc
HDFS-8205. CommandFormat#parse() should not parse option as value of option. (Contributed by Peter Shi and Xiaoyu Yao)
2015-04-27 12:23:34 -07:00
Allen Wittenauer
8b69c825e5
HADOOP-11865. Incorrect path mentioned in document for accessing script files (J.Andreina via aw)
2015-04-26 09:55:46 -07:00
cnauroth
a00e001a1a
HDFS-8247. TestDiskspaceQuotaUpdate#testAppendOverTypeQuota is failing. Contributed by Xiaoyu Yao.
2015-04-25 11:37:27 -07:00
Allen Wittenauer
f83c55a6be
HDFS-7673. synthetic load generator docs give incorrect/incomplete commands (Brahma Reddy Battula via aw)
2015-04-25 15:05:43 +01:00
Arpit Agarwal
dcc5455e07
HDFS-8211. DataNode UUID is always null in the JMX counter. (Contributed by Anu Engineer)
2015-04-24 16:47:48 -07:00
Andrew Wang
c7d9ad68e3
HDFS-8191. Fix byte to integer casting in SimulatedFSDataset#simulatedByte. Contributed by Zhe Zhang.
2015-04-24 11:54:25 -07:00
Jing Zhao
cf6c8a1b4e
HDFS-8176. Record from/to snapshots in audit log for snapshot diff report. Contributed by J. Andreina.
2015-04-24 10:23:32 -07:00
Akira Ajisaka
91b97c21c9
HDFS-8110. Remove unsupported 'hdfs namenode -rollingUpgrade downgrade' from document. Contributed by J.Andreina.
2015-04-24 20:32:55 +09:00
Vinayakumar B
c8d72907ff
HDFS-8231. StackTrace displayed at client while QuotaByStorageType exceeds (Contributed by J.Andreina and Xiaoyu Yao)
2015-04-24 12:51:04 +05:30
Vinayakumar B
262c1bc339
HDFS-8217. During block recovery for truncate Log new Block Id in case of copy-on-truncate is true. (Contributed by Vinayakumar B)
2015-04-24 12:16:41 +05:30
Colin Patrick Mccabe
a8898445dc
HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe)
2015-04-23 19:13:27 -07:00
Colin Patrick Mccabe
a0e0a63209
HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe)
2015-04-23 19:00:24 -07:00
Haohui Mai
bcf89ddc7d
HDFS-8052. Move WebHdfsFileSystem into hadoop-hdfs-client. Contributed by Haohui Mai.
2015-04-23 17:33:45 -07:00
Tsz-Wo Nicholas Sze
ef4e9963b2
HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma
2015-04-23 14:19:33 -07:00
Tsz-Wo Nicholas Sze
416b84354e
HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore
2015-04-23 11:55:06 -07:00
Andrew Wang
26971e52ae
HADOOP-11852. Disable symlinks in trunk.
2015-04-23 11:47:01 -07:00
Arun Suresh
baf8bc6c48
HDFS-4448. Allow HA NN to start in secure mode with wildcard address configured (atm via asuresh)
2015-04-23 01:44:17 -07:00
Haohui Mai
12f4df043f
HDFS-8218. Move classes that used by ClientProtocol into hdfs-client. Contributed by Haohui Mai.
2015-04-22 11:33:21 -07:00
Haohui Mai
6f8003dc7b
HDFS-8185. Separate client related routines in HAUtil into a new class. Contributed by Haohui Mai.
2015-04-21 21:59:47 -07:00
Colin Patrick Mccabe
997408eaac
HDFS-8133. Improve readability of deleted block check (Daryn Sharp via Colin P. McCabe)
2015-04-21 11:43:24 -07:00
Arpit Agarwal
dfc1c4c303
HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal)
2015-04-21 10:58:05 -07:00
Vinayakumar B
8ddbb8dd43
HDFS-7993. Provide each Replica details in fsck (Contributed by J.Andreina)
2015-04-21 15:24:49 +05:30
Arpit Agarwal
c92f6f3605
HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao)
2015-04-20 15:42:42 -07:00
Vinayakumar B
c17cd4f7c1
HDFS-8173. NPE thrown at DataNode shutdown when HTTP server was not able to create (Contributed by surendra singh lilhore)
2015-04-20 14:58:04 +05:30
Haohui Mai
5c97db07fb
HDFS-8169. Move LocatedBlocks and related classes to hdfs-client. Contributed by Haohui Mai.
2015-04-20 00:36:46 -07:00
Arpit Agarwal
8511d80804
HDFS-8152. Refactoring of lazy persist storage cases. (Arpit Agarwal)
2015-04-19 16:09:06 -07:00
Tsuyoshi Ozawa
5459b241c8
HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula.
2015-04-19 18:19:36 +09:00
Tsuyoshi Ozawa
5112477d9e
HDFS-7863. Missing description of some methods and parameters in javadoc of FSDirDeleteOp. Contributed by Brahma Reddy Battula.
2015-04-19 18:06:56 +09:00
Haohui Mai
76e7264e8d
HDFS-8165. Move GRANDFATHER_GENERATION_STAMP and GRANDFATER_INODE_ID to hdfs-client. Contributed by Haohui Mai.
2015-04-16 22:49:30 -07:00
Jitendra Pandey
369ddc67bd
HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer.
2015-04-16 22:13:09 -07:00
Tsz-Wo Nicholas Sze
9595cc003c
HDFS-7891. A block placement policy with best rack failure tolerance. Contributed by Walter Su
2015-04-16 18:25:53 -07:00
Akira Ajisaka
4308910eed
Revert "HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema"
...
This reverts commit eb1d4c39e0
2015-04-16 16:54:01 -05:00
Tsz-Wo Nicholas Sze
75bbcc8bf3
HDFS-8082. Move dfs.client.read.*, dfs.client.short.circuit.*, dfs.client.mmap.* and dfs.client.hedged.read.* conf from DFSConfigKeys to HdfsClientConfigKeys.
2015-04-16 13:22:31 -07:00
Haohui Mai
80a2a12423
HDFS-8135. Remove the deprecated FSConstants class. Contributed by Li Lu.
2015-04-16 11:26:05 -07:00
Andrew Wang
2e8ea780a4
HDFS-8142. DistributedFileSystem encryption zone commands should resolve relative paths. Contributed by Rakesh R.
2015-04-16 07:14:44 -07:00
Akira Ajisaka
de0f1700c1
HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula.
2015-04-15 15:31:04 -05:00
Jing Zhao
b172d03595
HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina.
2015-04-15 13:10:38 -07:00
Arpit Agarwal
9e8309a1b2
HDFS-8144. Split TestLazyPersistFiles into multiple tests. (Arpit Agarwal)
2015-04-15 10:25:04 -07:00
Jing Zhao
fddd55279d
HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao.
2015-04-14 15:05:23 -07:00
cnauroth
fef596df03
HDFS-8055. NullPointerException when topology script is missing. Contributed by Anu Engineer.
2015-04-14 10:19:30 -07:00
cnauroth
d45aa7647b
HDFS-6666. Abort NameNode and DataNode startup if security is enabled but block access token is not enabled. Contributed by Vijay Bhat.
2015-04-14 09:59:01 -07:00
Andrew Wang
d60e22152a
HDFS-8117. More accurate verification in SimulatedFSDataset: replace DEFAULT_DATABYTE with patterned data. Contributed by Zhe Zhang.
2015-04-13 13:01:10 -07:00
Tsz-Wo Nicholas Sze
7fc50e2525
HDFS-8083. Move dfs.client.write.* conf from DFSConfigKeys to HdfsClientConfigKeys.Write.
2015-04-13 11:43:35 -07:00
Vinayakumar B
14384f5b51
HDFS-8111. NPE thrown when invalid FSImage filename given for 'hdfs oiv_legacy' cmd ( Contributed by surendra singh lilhore )
2015-04-13 17:16:05 +05:30
cnauroth
f8f5887209
HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao.
2015-04-11 13:23:18 -07:00
Tsz-Wo Nicholas Sze
60da0e49e7
HDFS-8084. Move dfs.client.failover.* confs from DFSConfigKeys to HdfsClientConfigKeys.Failover and fix typos in the dfs.http.client.* configuration keys.
2015-04-10 19:38:43 -07:00
Haohui Mai
36e4cd3be6
HDFS-8103. Move BlockTokenSecretManager.AccessMode into BlockTokenIdentifier. Contributed by Haohui Mai.
2015-04-10 16:36:45 -07:00
Tsz-Wo Nicholas Sze
2cc9514ad6
HDFS-8100. Refactor DFSClient.Conf to a standalone class and separates short-circuit related conf to ShortCircuitConf.
2015-04-10 14:48:45 -07:00
Arun Suresh
7660da95cb
HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh)
2015-04-10 11:40:25 -07:00
Vinayakumar B
987c9e12e1
HDFS-7939. Two fsimage_rollback_* files are created which are not deleted after rollback. (Contributed by J.Andreina)
2015-04-10 11:12:07 +05:30
Konstantin V Shvachko
0959b67f1a
HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko
2015-04-09 22:00:20 -07:00
Haohui Mai
af9d4fede5
HDFS-8102. Separate webhdfs retry configuration keys from DFSConfigKeys. Contributed by Haohui Mai.
2015-04-09 14:36:27 -07:00
Arun Suresh
922b7ed21d
HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh)
2015-04-09 12:38:33 -07:00
Colin Patrick Mccabe
61dc2ea3fe
HDFS-8063: Fix intermittent test failures in TestTracing (Masatake Iwasaki via Colin P. McCabe)
2015-04-09 11:28:02 -07:00
Colin Patrick Mccabe
30acb7372a
HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe)
2015-04-09 11:22:39 -07:00
Aaron T. Myers
3fe61e0bb0
HDFS-8101. DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS classes at runtime. Contributed by Sean Busbey.
2015-04-09 09:40:08 -07:00
Vinayakumar B
9d8952f97f
HDFS-8096. DatanodeMetrics#blocksReplicated will get incremented early and even for failed transfers (Contributed by Vinayakumar B)
2015-04-09 11:58:00 +05:30
Andrew Wang
b1e059089d
HDFS-7979. Initialize block report IDs with a random number.
2015-04-08 21:43:42 -07:00
Haohui Mai
cc25823546
HDFS-8089. Move o.a.h.hdfs.web.resources.* to the client jars. Contributed by Haohui Mai.
2015-04-08 16:30:08 -07:00
Andrew Wang
5a540c3d31
HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma.
2015-04-08 16:09:17 -07:00
Haohui Mai
82d56b337d
Revert HDFS-7813.
2015-04-08 16:02:45 -07:00
Haohui Mai
bd4c99bece
Revert HDFS-7808.
2015-04-08 15:59:55 -07:00
Andrew Wang
6af0d74a75
HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma.
2015-04-08 15:52:06 -07:00
Andrew Wang
a42bb1cd91
HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang.
2015-04-08 15:41:48 -07:00
Kihwal Lee
285b31e75e
HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee.
2015-04-08 15:38:29 -05:00
Tsz-Wo Nicholas Sze
c931a3c776
HDFS-8079. Move CorruptFileBlockIterator to a new hdfs.client.impl package.
2015-04-08 11:50:52 -07:00
Arpit Agarwal
608c499841
HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal)
2015-04-08 11:38:21 -07:00
Haohui Mai
ab04ff9efe
HDFS-8080. Separate JSON related routines used by WebHdfsFileSystem to a package local class. Contributed by Haohui Mai.
2015-04-07 21:30:57 -07:00
Vinayakumar B
d505c8acd3
HDFS-8073. Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided. (Contributed by Walter Su)
2015-04-08 09:56:37 +05:30
Tsz-Wo Nicholas Sze
4be648b55c
HDFS-8079. Move dfs.client.retry.* confs from DFSConfigKeys to HdfsClientConfigKeys.Retry.
2015-04-07 19:48:57 -07:00
Yongjun Zhang
66763bb06f
HDFS-5215. dfs.datanode.du.reserved is not considered while computing
...
available space ( Brahma Reddy Battula via Yongjun Zhang)
2015-04-07 19:31:58 -07:00
cnauroth
1e72d98c69
HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao.
2015-04-07 14:23:01 -07:00
Tsz-Wo Nicholas Sze
571a1ce9d0
HDFS-8049. Add @InterfaceAudience.Private annotation to hdfs client implementation. Contributed by Takuya Fukudome
2015-04-07 13:59:48 -07:00
Konstantin V Shvachko
75c5454860
HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko.
2015-04-06 22:20:44 -07:00
Colin Patrick Mccabe
28bebc81db
HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe)
2015-04-06 08:56:52 -07:00
Colin Patrick Mccabe
ef591b1d6a
HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe)
2015-04-03 16:34:23 -07:00
Colin Patrick Mccabe
023133cef9
HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe)
2015-04-03 14:19:46 -07:00
cnauroth
db80e42891
HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao.
2015-04-03 10:44:02 -07:00
cnauroth
dc5596c36a
HDFS-8034. Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows. Contributed by Xiaoyu Yao.
2015-04-03 10:10:11 -07:00
Jing Zhao
72f6bd4893
HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao.
2015-04-02 21:20:30 -07:00
Haohui Mai
16b74f978a
HDFS-7893. Update the POM to create a separate hdfs-client jar. Contributed by Haohui Mai.
2015-04-02 15:29:18 -07:00
Haohui Mai
96649c38f9
HDFS-8035. Move checkBlocksProperlyReplicated() in FSNamesystem to BlockManager. Contributed by Haohui Mai.
2015-04-02 11:36:51 -07:00
Tsz-Wo Nicholas Sze
4f412f487c
HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao
2015-04-02 11:11:34 -07:00
Tsz-Wo Nicholas Sze
9ed43f2189
HDFS-7888. Change DFSOutputStream and DataStreamer for convenience of subclassing. Contributed by Li Bo
2015-04-02 10:59:26 -07:00
Vinayakumar B
867d5d2675
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B)
2015-04-02 08:12:00 +05:30
Haohui Mai
6ccf4fbf8a
HDFS-8008. Support client-side back off when the datanodes are congested. Contributed by Haohui Mai.
2015-04-01 16:54:46 -07:00
Colin Patrick Mccabe
3c7adaaf35
HDFS-7922. ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors (Rakesh R via Colin P. McCabe)
2015-04-01 16:02:39 -07:00
Colin Patrick Mccabe
c94d594a57
HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk (cmccabe)
2015-04-01 14:10:10 -07:00
Andrew Wang
ed72daa5df
HDFS-7978. Add LOG.isDebugEnabled() guard for some LOG.debug(..). Contributed by Walter Su.
2015-04-01 12:54:28 -07:00
Haohui Mai
796fb26871
Add the missing files for HDFS-8009.
2015-04-01 10:58:32 -07:00
Haohui Mai
53471d462c
HDFS-8009. Signal congestion on the DataNode. Contributed by Haohui Mai.
2015-04-01 10:56:53 -07:00
Akira Ajisaka
18a91fe4df
HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka)
2015-04-01 09:07:28 +09:00
Akira Ajisaka
859cab2f22
HDFS-7671. hdfs user guide should point to the common rack awareness doc. Contributed by Kai Sasaki.
2015-04-01 00:26:16 +09:00
yliu
3df61f303a
HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu)
2015-03-31 21:17:44 +08:00
Arpit Agarwal
85dc3c14b2
HDFS-7944. Minor cleanup of BlockPoolManager#getAllNamenodeThreads. (Arpit Agarwal)
2015-03-30 21:24:51 -07:00
Harsh J
cce66ba3c9
HDFS-3918. EditLogTailer shouldn't log WARN when other node is in standby mode. Contributed by Todd Lipcon.
2015-03-31 08:04:18 +05:30
Arpit Agarwal
1a495fbb48
HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara)
2015-03-30 15:25:16 -07:00
Haohui Mai
0967b1d99d
Addendum for HDFS-7748.
2015-03-30 13:11:09 -07:00
Haohui Mai
b80457158d
HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai.
2015-03-30 11:59:21 -07:00
Colin Patrick Mccabe
1feb9569f3
HDFS-7261. storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState() (Brahma Reddy Battula via Colin P. McCabe)
2015-03-30 10:46:21 -07:00
Akira Ajisaka
e7ea2a8e8f
HDFS-8002. Website refers to /trash directory. Contributd by Brahma Reddy Battula.
2015-03-31 00:27:50 +09:00
Kihwal Lee
04ee18ed48
HDFS-7742. Favoring decommissioning node for replication can cause a block to stay
...
underreplicated for long periods. Contributed by Nathan Roberts.
2015-03-30 10:10:11 -05:00
Harsh J
ae3e8c61ff
HDFS-4396. Add START_MSG/SHUTDOWN_MSG for ZKFC. Contributed by Liang Xie.
2015-03-30 15:21:18 +05:30
Vinayakumar B
1ed9fb7664
HDFS-7890. Improve information on Top users for metrics in RollingWindowsManager and lower log level (Contributed by J.Andreina)
2015-03-30 10:02:48 +05:30
Akira Ajisaka
232eca944a
HDFS-6408. Remove redundant definitions in log4j.properties. Contributed by Abhiraj Butala.
2015-03-30 11:25:35 +09:00
Akira Ajisaka
257c77f895
HDFS-6263. Remove DRFA.MaxBackupIndex config from log4j.properties. Contributed by Abhiraj Butala.
2015-03-30 10:52:15 +09:00
Harsh J
3d9132d434
HDFS-7501. TransactionsSinceLastCheckpoint can be negative on SBNs. Contributed by Gautam Gopalakrishnan.
2015-03-29 00:45:01 +05:30
Arpit Agarwal
89fb0f57ef
HDFS-7700. Document quota support for storage types. (Contributed by Xiaoyu Yao)
2015-03-27 19:49:26 -07:00
Arun Suresh
e97f8e44af
HDFS-8004. Use KeyProviderCryptoExtension#warmUpEncryptedKeys when creating an encryption zone. (awang via asuresh)
2015-03-27 19:23:45 -07:00
Kihwal Lee
60882ab26d
HDFS-7990. IBR delete ack should not be delayed. Contributed by Daryn Sharp.
2015-03-27 09:05:17 -05:00
Colin Patrick Mccabe
61df1b27a7
HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe)
2015-03-26 13:21:09 -07:00
Arpit Agarwal
3d0708bdb0
HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao)
2015-03-26 10:24:11 -07:00
Kihwal Lee
222845632b
HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki.
2015-03-26 08:44:58 -05:00
cnauroth
4528eb9fb2
HADOOP-11524. hadoop_do_classpath_subcommand throws a shellcheck warning. Contributed by Chris Nauroth.
2015-03-25 22:36:09 -07:00
Kihwal Lee
fc1031af74
HDFS-7928. Scanning blocks from disk during rolling upgrade startup takes a lot of time if disks are busy. Contributed by Rushabh Shah.
2015-03-25 14:42:59 -05:00
Jing Zhao
5e21e4ca37
HDFS-6353. Check and make checkpoint before stopping the NameNode. Contributed by Jing Zhao.
2015-03-25 10:38:00 -07:00
Haohui Mai
80278a5f85
HDFS-7985. WebHDFS should be always enabled. Contributed by Li Lu.
2015-03-24 21:55:56 -07:00
Jitendra Pandey
53a28afe29
HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh.
2015-03-24 16:02:40 -07:00
Haohui Mai
e38ef70fbc
HDFS-7713. Implement mkdirs in the HDFS Web UI. Contributed by Ravi Prakash.
2015-03-24 15:48:52 -07:00
Jing Zhao
a16bfff71b
HDFS-7854. Separate class DataStreamer out of DFSOutputStream. Contributed by Li Bo.
2015-03-24 11:06:13 -07:00
Brandon Li
570a83ae80
HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li
2015-03-24 10:49:16 -07:00
Harsh J
eda02540ce
HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel.
2015-03-24 23:06:18 +05:30
Brandon Li
51f1f49374
HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li
2015-03-24 10:28:38 -07:00
Andrew Wang
6413d34986
HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu.
2015-03-24 09:07:02 -07:00
Vinayakumar B
c6c396fcd6
HDFS-3325. When configuring 'dfs.namenode.safemode.threshold-pct' to a value greater or equal to 1 there is mismatch in the UI report (Contributed by J.Andreina)
2015-03-24 12:12:01 +05:30
Plamen Jeliazkov
970ee3fc56
HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov.
2015-03-23 23:05:21 -07:00
Andrew Wang
50ee8f4e67
HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu.
2015-03-23 22:00:34 -07:00
Tsz-Wo Nicholas Sze
d7e3c3364e
HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula
2015-03-24 13:49:17 +09:00
cnauroth
2c238ae4e0
HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu.
2015-03-23 16:29:51 -07:00
Brandon Li
36af4a913c
HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li
2015-03-23 10:06:47 -07:00
Kihwal Lee
75ead273be
HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B
2015-03-20 14:02:09 -05:00
Jing Zhao
d368d3647a
HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao.
2015-03-20 11:50:24 -07:00
Jing Zhao
a6a5aae472
HDFS-7829. Code clean up for LocatedBlock. Contributed by Takanobu Asanuma.
2015-03-20 10:50:19 -07:00
Yongjun Zhang
15612313f5
HDFS-7835. make initial sleeptime in locateFollowingBlock configurable for DFSClient. Contributed by Zhihai Xu.
2015-03-20 09:12:42 -07:00
yliu
978ef11f26
HDFS-7962. Remove duplicated logs in BlockManager. (yliu)
2015-03-20 05:29:25 +08:00
yliu
e37ca221bf
HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu)
2015-03-19 23:23:19 +08:00
Kihwal Lee
61a4c7fc98
HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee.
2015-03-19 12:27:03 -05:00
Vinayakumar B
5be5129d93
HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina )
2015-03-19 19:05:42 +05:30
Kihwal Lee
e79be0ee12
HDFS-7816. Unable to open webhdfs paths with "+". Contributed by Haohui Mai
2015-03-19 08:01:01 -05:00
Vinayakumar B
338b477ff7
HDFS-7869. Update action param from 'start' to 'prepare' in rolling upgrade ( Contributed by J.Andreina)
2015-03-19 11:59:31 +05:30
Jing Zhao
c7c71cdba5
HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao.
2015-03-18 18:53:50 -07:00
Colin Patrick Mccabe
43b41f2241
HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe)
2015-03-18 18:48:54 -07:00
Jing Zhao
bee5a6a64a
HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao.
2015-03-18 18:40:59 -07:00
Colin Patrick Mccabe
8234fd0e10
HDFS-7054. Make DFSOutputStream tracing more fine-grained (cmccabe)
2015-03-18 18:14:54 -07:00
Haohui Mai
8c40e88d5d
HDFS-7945. The WebHdfs system on DN does not honor the length parameter. Contributed by Haohui Mai.
2015-03-18 16:19:46 -07:00
Arpit Agarwal
c239b6d376
HDFS-7914. TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use. (Contributed by Xiaoyu Yao)
2015-03-18 14:04:53 -07:00
Haohui Mai
f0dea037ff
HDFS-7697. Mark the PB OIV tool as experimental. Contributed by Lei (Eddy) Xu.
2015-03-18 13:33:41 -07:00
Arpit Agarwal
e817cedcdc
HDFS-7951. Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux. (Contributed by Xiaoyu Yao)
2015-03-18 12:45:46 -07:00
Arpit Agarwal
d462c62755
HDFS-7950. Fix TestFsDatasetImpl#testAddVolumes failure on Windows. (Contributed by Xiaoyu Yao)
2015-03-18 12:35:30 -07:00
Arpit Agarwal
fdd58aa32f
HDFS-7948. TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows. (Contributed by Xiaoyu Yao)
2015-03-18 12:26:00 -07:00
Haohui Mai
402817cd9b
HDFS-7953. NN Web UI fails to navigate to paths that contain #. Contributed by kanaka kumar avvaru.
2015-03-18 11:08:08 -07:00
Arpit Agarwal
5b322c6a82
HDFS-7946. TestDataNodeVolumeFailureReporting NPE on Windows. (Contributed by Xiaoyu Yao)
2015-03-17 21:34:31 -07:00
Colin Patrick Mccabe
d8846707c5
HDFS-7940. Add tracing to DFSClient#setQuotaByStorageType (Rakesh R via Colin P. McCabe)
2015-03-17 10:47:21 -07:00
Vinayakumar B
018893e81e
HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R)
2015-03-17 15:32:34 +05:30
yliu
48c2db34ef
HDFS-7838. Expose truncate API for libhdfs. (yliu)
2015-03-17 07:22:17 +08:00
Harsh J
046521cd65
HDFS-2360. Ugly stacktrce when quota exceeds. (harsh)
2015-03-17 10:28:17 +05:30
Konstantin V Shvachko
ce5de93a58
HDFS-7886. Fix TestFileTruncate falures. Contributed by Plamen Jeliazkov and Konstantin Shvachko.
2015-03-16 12:54:04 -07:00
Colin Patrick Mccabe
bc9cb3e271
HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe)
2015-03-14 22:36:46 -07:00
Colin Patrick Mccabe
32741cf3d2
Revert "HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe)" (jenkins didn't run yet)
...
This reverts commit 5aa892ed48
2015-03-13 18:40:20 -07:00
Colin Patrick Mccabe
5aa892ed48
HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe)
2015-03-13 18:29:49 -07:00
Konstantin V Shvachko
6acb7f2110
HDFS-7903. Cannot recover block after truncate and delete snapshot. Contributed by Plamen Jeliazkov.
2015-03-13 13:12:51 -07:00
Kihwal Lee
d324164a51
HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp.
2015-03-13 14:23:37 -05:00
Brandon Li
f446669afb
HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contributed by Tsz Wo Nicholas Sze
2015-03-13 10:42:22 -07:00
Colin Patrick Mccabe
b49c3a1813
HDFS-7722. DataNode#checkDiskError should also remove Storage when error is found. (Lei Xu via Colin P. McCabe)
2015-03-12 12:00:18 -07:00
Tsz-Wo Nicholas Sze
6dae6d12ec
HDFS-6833. DirectoryScanner should not register a deleting block with memory of DataNode. Contributed by Shinichi Yamashita
2015-03-13 02:25:32 +08:00
cnauroth
fb34f45727
HDFS-7491. Add incremental blockreport latency to DN metrics. Contributed by Ming Ma.
2015-03-11 14:11:19 -07:00
Colin Patrick Mccabe
5c1036d598
HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu via Colin P. McCabe)
2015-03-10 18:20:58 -07:00
Akira Ajisaka
82db3341bf
HDFS-6806. HDFS Rolling upgrade document should mention the versions available. Contributed by J.Andreina.
2015-03-10 11:22:11 +09:00
Jing Zhao
e43882e84a
HDFS-7898. Change TestAppendSnapshotTruncate to fail-fast. Contributed by Tsz Wo Nicholas Sze.
2015-03-09 10:52:17 -07:00
Chris Douglas
6ee0d32b98
HDFS-7411. Change decommission logic to throttle by blocks rather
...
than nodes in each interval. Contributed by Andrew Wang
2015-03-08 18:31:04 -07:00
Brandon Li
0f8ecb1d0c
HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li
2015-03-06 15:19:45 -08:00
Haohui Mai
c79710302e
HDFS-7818. OffsetParam should return the default value instead of throwing NPE when the value is unspecified. Contributed by Eric Payne.
2015-03-06 14:26:23 -08:00
Allen Wittenauer
01bfe6f05b
HADOOP-11653. shellprofiles should require .sh extension (Brahma Reddy Battula via aw)
2015-03-06 13:54:11 -08:00
Jing Zhao
24db0812be
HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze.
2015-03-06 10:55:56 -08:00
Jing Zhao
952640fa4c
HDFS-7855. Separate class Packet from DFSOutputStream. Contributed by Li Bo.
2015-03-05 10:58:53 -08:00
Tsuyoshi Ozawa
74a4754d1c
HADOOP-11648. Set DomainSocketWatcher thread name explicitly. Contributed by Liang Xie.
2015-03-05 16:05:44 +09:00
Tsz-Wo Nicholas Sze
ded0200e9c
HDFS-7746. Add a test randomly mixing append, truncate and snapshot operations.
2015-03-05 10:21:29 +08:00
Dongming Liang
430b537188
HDFS-1522. Combine two BLOCK_FILE_PREFIX constants into one. Contributed by Dongming Liang.
2015-03-04 17:51:12 -08:00
Kihwal Lee
722b479469
HDFS-7434. DatanodeID hashCode should not be mutable. Contributed by Daryn Sharp.
2015-03-04 17:21:51 -06:00
Haohui Mai
f717dc51b2
HDFS-7879. hdfs.dll does not export functions of the public libhdfs API. Contributed by Chris Nauroth.
2015-03-04 09:17:21 -08:00
Vinayakumar B
3560180b6e
HDFS-7869. Inconsistency in the return information while performing rolling upgrade ( Contributed by J.Andreina )
2015-03-04 14:38:38 +05:30
Aaron T. Myers
f2d7a67a2c
HDFS-7682. {{DistributedFileSystem#getFileChecksum}} of a snapshotted file includes non-snapshotted content. Contributed by Charles Lamb.
2015-03-03 18:08:59 -08:00
Haohui Mai
e2262d3d18
HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA.
2015-03-03 17:54:13 -08:00
Arpit Agarwal
1004473aa6
HDFS-7757. Misleading error messages in FSImage.java. (Contributed by Brahma Reddy Battula)
2015-03-03 10:55:22 -08:00
Tsuyoshi Ozawa
d1c6accb6f
HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)
2015-03-03 14:17:52 +09:00
Jing Zhao
b442aeec95
HDFS-7871. NameNodeEditLogRoller can keep printing 'Swallowing exception' message. Contributed by Jing Zhao.
2015-03-02 20:22:04 -08:00
Tsz-Wo Nicholas Sze
431e7d84c7
HDFS-7302. Remove "downgrade" from "namenode -rollingUpgrade" startup option since it may incorrectly finalize an ongoing rolling upgrade.
...
Contributed by Kai Sasaki
2015-03-03 10:04:08 +08:00
Haohui Mai
c5eac9c6fe
HDFS-7785. Improve diagnostics information for HttpPutFailedException. Contributed by Chengbing Liu.
2015-03-02 15:35:02 -08:00
Gera Shegalov
cbb492578e
HDFS-7789. DFSck should resolve the path to support cross-FS symlinks. (gera)
2015-03-02 00:55:35 -08:00
Tsz-Wo Nicholas Sze
67ed59348d
HDFS-7439. Add BlockOpResponseProto's message to the exception messages. Contributed by Takanobu Asanuma
2015-03-02 15:03:58 +08:00
Akira Ajisaka
30e73ebc77
HADOOP-11657. Align the output of hadoop fs -du to be more Unix-like. (aajisaka)
2015-03-01 21:09:15 -08:00
Akira Ajisaka
aa55fd3096
HDFS-5853. Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml (aajisaka)
2015-03-01 01:16:36 -08:00
Allen Wittenauer
dbc9b6433e
HDFS-4681. TestBlocksWithNotEnoughRacks#testCorruptBlockRereplicatedAcrossRacks fails using IBM java (Ayappan via aw)
2015-02-28 23:32:09 -08:00
Tsz-Wo Nicholas Sze
cf51ff2fe8
recommit "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir."
...
(cherry picked from commit 7c6b6547ee
2015-02-27 14:30:41 -08:00
Akira Ajisaka
8719cdd4f6
HDFS-7685. Document dfs.namenode.heartbeat.recheck-interval in hdfs-default.xml. Contributed by Kai Sasaki.
2015-02-27 12:17:34 -08:00
Tsz-Wo Nicholas Sze
a979f3b58f
HDFS-7308. Change the packet chunk size computation in DFSOutputStream in order to enforce packet size <= 64kB. Contributed by Takuya Fukudome
2015-02-27 23:45:37 +08:00
Vinayakumar B
4f75b15628
HDFS-6753. Initialize checkDisk when DirectoryScanner not able to get files list for scanning (Contributed by J.Andreina)
2015-02-27 16:36:28 +05:30
Tsz-Wo Nicholas Sze
2954e65467
Revert "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir."
...
This reverts commit 7c6b6547ee
2015-02-27 18:25:32 +08:00
cnauroth
c6d5b37a1c
HDFS-7774. Unresolved symbols error while compiling HDFS on Windows 7/32 bit. Contributed by Kiran Kumar M R.
2015-02-26 13:37:46 -08:00
Colin Patrick Mccabe
f0c980abed
HDFS-7819. Log WARN message for the blocks which are not in Block ID based layout (Rakesh R via Colin P. McCabe)
2015-02-26 11:58:29 -08:00
Vinayakumar B
166eecf687
HDFS-7832. Show 'Last Modified' in Namenode's 'Browse Filesystem' (Contributed by Vinayakumar B)
2015-02-26 14:36:09 +05:30
Tsz-Wo Nicholas Sze
725cc499f0
HDFS-7537. Add "UNDER MIN REPL'D BLOCKS" count to fsck. Contributed by GAO Rui
2015-02-26 11:45:56 +08:00
Tsz-Wo Nicholas Sze
606f5b517f
HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade.
2015-02-26 10:14:40 +08:00
Benoy Antony
d140d76a43
HDFS-7467. Provide storage tier information for a directory via fsck. (Benoy Antony)
2015-02-25 16:19:35 -08:00
Colin Patrick Mccabe
caa42adf20
HDFS-7495. Remove updatePosition argument from DFSInputStream#getBlockAt() (cmccabe)
2015-02-25 13:30:01 -08:00
Andrew Wang
7105ebaa9f
HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed by Liang Xie.
2015-02-24 15:31:13 -08:00
Jing Zhao
73bcfa99af
HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko.
2015-02-24 10:31:32 -08:00
Tsuyoshi Ozawa
1aea440560
Fix build because of 946456c6d8.
2015-02-25 00:48:51 +09:00
Tsuyoshi Ozawa
9cedad11d8
Revert "HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)"
...
This reverts commit 946456c6d8
2015-02-25 00:32:04 +09:00
Tsuyoshi Ozawa
b53fd7163b
HDFS-7008. xlator should be closed upon exit from DFSAdmin#genericRefresh(). (ozawa)
2015-02-24 23:59:34 +09:00
Colin Patrick Mccabe
faaddb6ecb
HDFS-7805. NameNode recovery prompt should be printed on console (Surendra Singh Lilhore via Colin P. McCabe)
2015-02-23 19:00:10 -08:00
Colin Patrick Mccabe
585768667e
HDFS-7807. libhdfs htable.c: fix htable resizing, add unit test (cmccabe)
2015-02-23 18:44:53 -08:00
cnauroth
769507bd7a
HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma.
2015-02-23 15:12:27 -08:00
Arpit Agarwal
8b465b4b8c
HDFS-7806. Refactor: move StorageType from hadoop-hdfs to hadoop-common. (Contributed by Xiaoyu Yao)
2015-02-21 15:38:35 -08:00
yliu
737bad02d4
HDFS-7740. Test truncate with DataNodes restarting. (yliu)
2015-02-21 06:32:34 +08:00
cnauroth
0d6af574e0
HDFS-7813. TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently. Contributed by Rakesh R.
2015-02-20 17:01:08 -08:00
cnauroth
02e7dec79d
HDFS-7773. Additional metrics in HDFS to be accessed via jmx. Contributed by Anu Engineer.
2015-02-20 12:37:48 -08:00
cnauroth
8c6ae0d619
HDFS-7814. Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota". Contributed by Xiaoyu Yao.
2015-02-20 12:21:46 -08:00
Kihwal Lee
7ae5255a16
HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah.
2015-02-20 09:09:02 -06:00
Harsh J
b9a17909ba
HDFS-7752. Improve description for "dfs.namenode.num.extra.edits.retained" and "dfs.namenode.num.checkpoints.retained" properties on hdfs-default.xml. Contributed by Wellington Chevreuil.
2015-02-20 19:20:41 +05:30
Tsuyoshi Ozawa
946456c6d8
HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)
2015-02-19 13:06:53 +09:00
Haohui Mai
9a3e292087
HDFS-7808. Remove obsolete -ns options in in DFSHAAdmin.java. Contributed by Arshad Mohammad.
2015-02-18 15:15:08 -08:00
cnauroth
2aa9979a71
HDFS-7772. Document hdfs balancer -exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao.
2015-02-18 12:03:07 -08:00
Uma Maheswara Rao G
2ecea5ab74
HDFS-7804. correct the haadmin command usage in #HDFSHighAvailabilityWithQJM.html (Brahma Reddy Battula via umamahesh)
2015-02-18 19:24:45 +05:30
Akira Ajisaka
6dc8812a95
HDFS-7780. Update use of Iterator to Iterable in DataXceiverServer and SnapshotDiffInfo. Contributed by Ray Chiang.
2015-02-17 14:47:53 -08:00
Haohui Mai
043e44bc36
HDFS-6662. WebHDFS cannot open a file if its path contains "%". Contributed by Gerson Carlos.
2015-02-17 13:04:38 -08:00
Kihwal Lee
db6606223c
HDFS-7795. Show warning if not all favored nodes were chosen by namenode. Contributed by Kihwal Lee.
2015-02-17 13:05:43 -06:00
Allen Wittenauer
34b78d51b5
HDFS-7803. Wrong command mentioned in HDFSHighAvailabilityWithQJM documentation (Arshad Mohammad via aw)
2015-02-17 06:46:37 -10:00
Uma Maheswara Rao G
f0412de1c1
HDFS-4266. BKJM: Separate write and ack quorum (Rakesh R via umamahesh)
2015-02-17 21:28:49 +05:30
Uma Maheswara Rao G
f24a56787a
HDFS-7797. Add audit log for setQuota operation (Rakesh R via umamahesh)
2015-02-17 20:11:11 +05:30
Tsuyoshi Ozawa
2f0f756b26
HADOOP-11600. Fix up source codes to be compiled with Guava 17.0. (ozawa)
2015-02-17 21:56:20 +09:00
yliu
500e6a0f46
HDFS-7798. Checkpointing failure caused by shared KerberosAuthenticator. (Chengbing Liu via yliu)
2015-02-17 07:46:33 +08:00
cnauroth
9729b244de
HDFS-7604. Track and display failed DataNode storage locations in NameNode. Contributed by Chris Nauroth.
2015-02-16 14:43:02 -08:00
Arpit Agarwal
f2231cebcd
HDFS-7775. Use consistent naming for NN-internal quota related types and functions. (Contributed bu Xiaoyu Yao)
2015-02-13 21:01:33 -08:00
cnauroth
2efb2347a9
HDFS-7778. Rename FsVolumeListTest to TestFsVolumeList and commit it to branch-2. Contributed by Lei (Eddy) Xu.
2015-02-13 16:30:28 -08:00
Akira Ajisaka
0d521e3326
HDFS-4265. BKJM doesn't take advantage of speculative reads. Contributed by Rakesh R.
2015-02-13 15:20:52 -08:00
Colin Patrick Mccabe
8bb9a5000e
HDFS-7686. Re-add rapid rescan of possibly corrupt block feature to the block scanner (cmccabe)
2015-02-13 14:35:49 -08:00
Arpit Agarwal
0a33a98eb4
HDFS-7776. Adding additional unit tests for Quota By Storage Type. (Contributed by Xiaoyu Yao)
2015-02-13 11:56:33 -08:00
Allen Wittenauer
a126ac3edb
HDFS-7791. dfs count -v should be added to quota documentation (Akira AJISAKA via aw)
2015-02-13 10:56:30 -08:00
Allen Wittenauer
2f1e5dc628
HDFS-7668. Convert site documentation from apt to markdown (Masatake Iwasaki via aw)
2015-02-12 18:19:45 -08:00
Allen Wittenauer
93b941c637
HADOOP-11565. Add --slaves shell option (aw)
2015-02-12 18:01:28 -08:00
Colin Patrick Mccabe
871cb56152
HDFS-7790. Do not create optional fields in DFSInputStream unless they are needed (cmccabe)
2015-02-12 17:48:51 -08:00
Akira Ajisaka
46b6d23e8f
HDFS-7684. The host:port settings of the deamons should be trimmed before use. Contributed by Anu Engineer.
2015-02-12 17:40:30 -08:00
