big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,879 Commits 2 Branches 2 Tags 264 MiB
cef2815cf4
Commit Graph

3608 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
aebb9127ba HDFS-11160. VolumeScanner reports write-in-progress replicas as corrupt incorrectly. Contributed by Wei-Chiu Chuang and Yongjun Zhang. 2016-12-15 16:32:50 -08:00
Mingliang Liu
ac0d402c6a HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger 2016-12-15 14:53:53 -08:00
Arpit Agarwal
6ba9587d37 HDFS-10958. Add instrumentation hooks around Datanode disk IO. 2016-12-14 11:18:58 -08:00
Kai Zheng
1f14f6d038 HDFS-8411. Add bytes count metrics to datanode for ECWorker. Contributed by Sammi Chen and Andrew Wang 2016-12-14 14:50:50 +08:00
Andrew Wang
fbdbbd57cd HDFS-10684. WebHDFS DataNode calls fail without parameter createparent. Contributed by John Zhuge. 2016-12-13 18:01:31 -08:00
Uma Maheswara Rao G
e24a923db5 HDFS-11164: Mover should avoid unnecessary retries if the block is pinned. Contributed by Rakesh R 2016-12-13 17:09:58 -08:00
Akira Ajisaka
2d4731c067 HDFS-11233. Fix javac warnings related to the deprecated APIs after upgrading Jackson. Contributed by Yiqun Lin. 2016-12-13 11:11:15 +09:00
Vinayakumar B
d1d4aba71b HDFS-11224. Lifeline message should be ignored for dead nodes (Contributed by Vinayakumar B) 2016-12-09 14:53:50 +05:30
Kihwal Lee
9f8344db50 Revert "HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger" 
This reverts commit 8c4680852b.
 2016-12-08 15:36:08 -06:00
Xiao Chen
401c731872 HDFS-11197. Listing encryption zones fails when deleting a EZ that is on a snapshotted directory. Contributed by Wellington Chevreuil. 2016-12-08 12:40:20 -08:00
Andrew Wang
ea2895f4ed HDFS-8630. WebHDFS : Support get/set/unset StoragePolicy. Contributed by Surendra Singh Lilhore. 2016-12-07 15:52:16 -08:00
Ming Ma
c73e08a6da HDFS-10206. Datanodes not sorted properly by distance when the reader isn't a datanode. (Nandakumar via mingma) 2016-12-07 08:26:09 -08:00
Karthik Kambatla
c8d0a049b0 HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha) 2016-12-06 14:10:58 -08:00
Andrew Wang
f53b481ffe HDFS-11172. Support an erasure coding policy using RS 10 + 4. Contributed by Wei Zhou. 2016-12-06 12:49:38 -08:00
Xiaoyu Yao
df983b524a HDFS-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao. 2016-12-06 11:05:47 -08:00
Brahma Reddy Battula
ed89856755 HDFS-11178. TestAddStripedBlockInFBR#testAddBlockInFullBlockReport fails frequently in trunk. Contributed By Yiqun Lin. 2016-12-06 22:30:43 +05:30
Andrew Wang
08a7253bc0 Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang" 
This reverts commit c7ff34f8dc.
 2016-12-05 23:09:35 -08:00
Xiaoyu Yao
dcedb72af4 Revert "HADOOP-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao." 
This reverts commit aeecfa24f4.
 2016-12-05 12:44:20 -08:00
Mingliang Liu
8c4680852b HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger 2016-12-05 11:48:58 -08:00
Mingliang Liu
c7ff34f8dc HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang 2016-12-02 11:10:13 -08:00
Akira Ajisaka
e0fa49234f HDFS-11180. Intermittent deadlock in NameNode when failover happens. 2016-12-01 23:08:59 +09:00
Andrew Wang
7226a71b1f HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers and Andrew Wang. 2016-11-30 15:58:31 -08:00
Kai Zheng
51e6c1cc3f HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen 2016-11-30 15:52:56 +08:00
Kai Zheng
cfd8076f81 Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen" 
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
 2016-11-30 15:44:52 +08:00
Xiaoyu Yao
aeecfa24f4 HADOOP-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao. 2016-11-29 20:52:36 -08:00
Arpit Agarwal
eaaa32950c HDFS-11149. Support for parallel checking of FsVolumes. 2016-11-29 20:31:02 -08:00
Andrew Wang
a2b1ff0257 HADOOP-13828. Implement getFileChecksum(path, length) for ViewFileSystem. Contributed by Manoj Govindassamy. 2016-11-28 11:54:43 -08:00
Kai Zheng
5d5614f847 HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen 2016-11-28 14:34:44 +08:00
Andrew Wang
de4894936a HDFS-11368. Erasure Coding: Deprecate replication-related config keys. Contributed by Rakesh R. 2016-11-23 16:42:06 -08:00
Andrew Wang
dd98a8005a HDFS-11058. Implement 'hadoop fs -df' command for ViewFileSystem. Contributed by Manoj Govindassamy. 2016-11-23 16:40:39 -08:00
Lei Xu
3a09e5970d HDFS-11145. Implement getTrashRoot() for ViewFileSystem. (Manoj Govindassamy via lei) 2016-11-22 13:52:09 -08:00
Arpit Agarwal
613b902b98 HDFS-11148. Update DataNode to use StorageLocationChecker at startup. 2016-11-22 10:50:25 -08:00
Kihwal Lee
49a09179e3 HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang. 2016-11-21 10:13:24 -06:00
Brahma Reddy Battula
f922067748 HDFS-11101. TestDFSShell#testMoveWithTargetPortEmpty fails intermittently. Contributed By Brahma Reddy Battula. 2016-11-21 19:34:52 +05:30
Brahma Reddy Battula
c68dad18ab HDFS-11144. TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exception. Contributed By Brahma Reddy Battula. 2016-11-21 10:58:34 +05:30
Akira Ajisaka
d232625f73 HDFS-11113. Document dfs.client.read.striped configuration in hdfs-default.xml. Contributed by Rakesh R. 2016-11-21 14:05:15 +09:00
Kai Zheng
f6ffa11635 HDFS-11117. Refactor striped file tests to allow flexibly test erasure coding policy. Contributed by Sammi Chen 2016-11-18 16:08:59 +06:00
Wei-Chiu Chuang
c0b1a44f6c HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory. 2016-11-17 22:48:35 -06:00
Brahma Reddy Battula
b2d4b7b1b9 HDFS-11134. Fix bind exception threw in TestRenameWhileOpen. Contributed By Yiqun Lin. 2016-11-17 10:22:30 +05:30
Mingliang Liu
c90891e7b3 HDFS-11105. TestRBWBlockInvalidation#testRWRInvalidation fails intermittently. Contributed by Yiqun Lin 2016-11-16 13:02:10 -08:00
Kihwal Lee
4fcea8a0c8 HDFS-8870. Lease is leaked on write failure. Contributed by Kuhu Shukla. 2016-11-15 14:47:53 -06:00
Zhe Zhang
ff0b99eafe HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 2016-11-14 11:05:29 -08:00
Brahma Reddy Battula
6efb8c9c60 HDFS-11135. The tests in TestBalancer run fails due to NPE. Contributed By Yiqun Lin 2016-11-14 20:39:39 +05:30
Mingliang Liu
96f43921ba HADOOP-13813. TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failing. Contributed by Mingliang Liu 2016-11-12 10:35:22 -08:00
Mingliang Liu
2ee18fc15e HDFS-11129. TestAppendSnapshotTruncate fails with bind exception. Contributed by Brahma Reddy Battula 2016-11-11 22:12:27 -08:00
Arpit Agarwal
3d26717777 HDFS-11119. Support for parallel checking of StorageLocations on DataNode startup. 
This closes #155.
 2016-11-11 15:02:52 -08:00
Mingliang Liu
aa6010ccca HDFS-11122. TestDFSAdmin#testReportCommand fails due to timed out. Contributed by Yiqun Lin 2016-11-11 12:29:47 -08:00
Akira Ajisaka
8848a8a76c HDFS-11116. Fix javac warnings caused by deprecation of APIs in TestViewFsDefaultValue. Contributed by Yiqun Lin. 2016-11-11 13:45:30 +09:00
Vinayakumar B
ca68f9cb5b HDFS-9337. Validate required params for WebHDFS requests (Contributed by Jagadesh Kiran N) 2016-11-10 16:51:33 +05:30
Xiao Chen
71adf44c3f HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge. 2016-11-09 17:16:07 -08:00
Wei-Chiu Chuang
c619e9b43f HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang. 2016-11-09 09:16:50 -08:00
Brahma Reddy Battula
ed0bebabaa HDFS-9482. Replace DatanodeInfo constructors with a builder pattern. Contributed by Brahma Reddy Battula. 2016-11-08 18:17:07 -08:00
Mingliang Liu
62d8c17dfd HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou 2016-11-08 17:04:30 -08:00
Arpit Agarwal
3fff158587 HDFS-11114. Support for running async disk checks in DataNode. 
This closes #153.
 2016-11-07 18:45:53 -08:00
Akira Ajisaka
049e7d27be HDFS-10970. Update jackson from 1.9.13 to 2.x in hadoop-hdfs. 2016-11-07 11:16:31 +09:00
Xiao Chen
d8bab3dcb6 HDFS-10756. Expose getTrashRoot to HTTPFS and WebHDFS. Contributed by Yuanbo Liu. 2016-11-04 18:06:55 -07:00
Mingliang Liu
0c0ab102ab HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou 2016-11-04 14:34:49 -07:00
Mingliang Liu
7534aee09a HDFS-11076. Add unit test for extended Acls. Contributed by Chen Liang 2016-11-03 13:49:10 -07:00
Arpit Agarwal
b71907b2ae HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin. 2016-11-03 10:27:52 -07:00
Anu Engineer
730cb0cff6 HDFS-10566. Submit plan request should throw exception if Datanode is in non-REGULAR status. Contributed by Xiaobing Zhou. 2016-11-02 17:46:56 -07:00
Mingliang Liu
9b0c17f852 HDFS-11095. BlockManagerSafeMode should respect extension period default config value (30s). Contributed by Mingliang Liu 2016-11-02 14:58:24 -07:00
Brahma Reddy Battula
cb5cc0dc53 HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu. 2016-11-02 10:35:47 +05:30
Mingliang Liu
fa1512a740 HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou 2016-11-01 14:42:03 -07:00
Brahma Reddy Battula
808f5d8985 HDFS-10499. TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently. Contributed by Yiqun Lin. 2016-11-01 21:32:52 +05:30
Mingliang Liu
0c49f73a6c HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu 2016-10-31 16:00:33 -07:00
Arpit Agarwal
a9d68d2e8e HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru. 2016-10-31 12:39:30 -07:00
Brahma Reddy Battula
e9c7a97089 HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 2016-10-31 00:22:51 +05:30
Mingliang Liu
8a9388e5f6 HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger 2016-10-28 10:53:14 -07:00
Robert Kanter
5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Erik Krogen
f3ac1f41b8 HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen. 2016-10-27 15:58:25 -07:00
Vinayakumar B
1cf6ec4ad4 HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su) 2016-10-27 16:44:00 +05:30
Vinayakumar B
77f2684dd3 HDFS-10769. BlockIdManager.clear doesn't reset the counter for blockGroupIdGenerator (Contributed by Yiqun Lin and Rakesh R) 2016-10-27 14:16:03 +05:30
Vinayakumar B
a895c66255 HDFS-9929. Duplicate keys in NAMENODE_SPECIFIC_KEYS (Contributed by Akira Ajisaka) 2016-10-27 12:20:15 +05:30
Anu Engineer
9f32364d28 HDFS-11038. DiskBalancer: support running multiple commands in single test. Contributed by Xiaobing Zhou. 2016-10-26 19:39:02 -07:00
Mingliang Liu
55e1fb8e32 HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger 2016-10-26 14:11:38 -07:00
Lei Xu
f209e93566 HDFS-10638. Modifications to remove the assumption that StorageLocation is associated with java.io.File in Datanode. (Virajith Jalaparti via lei) 2016-10-26 10:32:35 -07:00
Xiaoyu Yao
9a8a386458 HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou. 2016-10-25 11:24:03 -07:00
Kihwal Lee
9d175853b0 HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp. 2016-10-24 17:14:51 -05:00
Kihwal Lee
b18f35f097 HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla. 2016-10-24 08:28:44 -05:00
Wei-Chiu Chuang
df857f0d10 HDFS-10975. fsck -list-corruptfileblocks does not report corrupt EC files. Contributed by Takanobu Asanuma. 2016-10-21 15:43:12 -07:00
Andrew Wang
61e30cf83c HDFS-8410. Add computation time metrics to datanode for ECWorker. Contributed by SammiChen. 2016-10-21 13:12:41 -07:00
Chris Douglas
ae8bccd509 HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du 2016-10-21 12:59:54 -07:00
Brahma Reddy Battula
f63cd78f60 HDFS-10730. Fix some failed tests due to BindException. Contributed by Yiqun Lin 2016-10-21 18:16:39 +05:30
Mingliang Liu
d7d87deece HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou 2016-10-20 19:51:48 -07:00
Wei-Chiu Chuang
5e83a21cb6 HDFS-10976. Report erasure coding policy of EC files in Fsck. Contributed by Wei-Chiu Chuang. 2016-10-20 13:06:43 -07:00
Brahma Reddy Battula
f872c6bc03 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 2016-10-20 21:47:17 +05:30
Brahma Reddy Battula
6fb6b651e8 HDFS-10699. Log object instance get incorrectly in TestDFSAdmin. Contributed by Yiqun Lin 2016-10-20 21:28:16 +05:30
Brahma Reddy Battula
73504b1bdc HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin 2016-10-20 10:41:18 +05:30
Xiao Chen
c5573e6a75 HDFS-11009. Add a tool to reconstruct block meta file from CLI. 2016-10-18 22:42:28 -07:00
Xiaoyu Yao
c62ae7107f HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru. 2016-10-18 14:05:43 -07:00
Kai Zheng
d26a1bb9d6 HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R 2016-10-18 14:51:08 +06:00
Kai Zheng
c023c74886 HADOOP-13061. Refactor erasure coders. Contributed by Kai Sasaki 2016-10-18 12:02:53 +06:00
Akira Ajisaka
bedfec0c10 HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini. 2016-10-18 15:00:44 +09:00
Ming Ma
b61fb267b9 HDFS-9390. Block management for maintenance states. 2016-10-17 17:45:41 -07:00
Xiaoyu Yao
8fd4c37c45 HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang. 2016-10-17 14:21:36 -07:00
Brahma Reddy Battula
5f4ae85bd8 HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula 2016-10-15 22:28:33 +05:30
Vinitha Reddy Gankidi
391ce535a7 HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi. 2016-10-14 18:13:54 -07:00
Anu Engineer
30bb1970cc HDFS-10558. DiskBalancer: Print the full path to plan file. Contributed by Xiaobing Zhou. 2016-10-14 17:07:59 -07:00
Mingliang Liu
76cc84e6d4 HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu 2016-10-14 14:29:02 -07:00
Lei Xu
8c520a27cb HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei) 2016-10-14 13:41:59 -07:00
Jing Zhao
adb96e109f HDFS-10827. When there are unrecoverable ec block groups, Namenode Web UI doesn't show the block names. Contributed by Takanobu Asanuma. 2016-10-14 13:21:53 -07:00
Andrew Wang
0007360c33 HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. Contributed by Yuanbo Liu. 2016-10-14 11:41:29 -07:00
Kihwal Lee
8a9f6635a3 HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin. 2016-10-14 11:38:48 -05:00
Andrew Wang
8c721aa00a Revert "HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin." 
This reverts commit fdce515091.
 2016-10-13 13:23:28 -07:00
Andrew Wang
fdce515091 HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin. 2016-10-13 11:41:37 -07:00
Anu Engineer
b371c56365 HDFS-10949. DiskBalancer: deprecate TestDiskBalancer#setVolumeCapacity. Contributed by Xiaobing Zhou. 2016-10-13 10:26:07 -07:00
Brahma Reddy Battula
1291254042 HDFS-10986. DFSAdmin should log detailed error message if any. Contributed by MingLiang Liu 2016-10-13 22:05:00 +05:30
Kihwal Lee
85cd06f663 HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 2016-10-12 15:11:42 -05:00
Mingliang Liu
7ba7092bbc HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou 2016-10-11 17:23:54 -07:00
Mingliang Liu
3c9a01062e HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang 2016-10-11 16:29:30 -07:00
Xiaoyu Yao
61f0490a73 HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle. 2016-10-11 15:55:02 -07:00
Lei Xu
96b12662ea HDFS-10637. Modifications to remove the assumption that FsVolumes are backed by java.io.File. (Virajith Jalaparti via lei) 2016-10-10 15:30:03 -07:00
Mingliang Liu
b963818621 HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula 2016-10-10 13:19:17 -07:00
Mingliang Liu
3441c746b5 HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou 2016-10-10 11:33:37 -07:00
Jing Zhao
4d106213c0 HDFS-10968. BlockManager#isInNewRack should consider decommissioning nodes. Contributed by Jing Zhao. 2016-10-07 22:44:54 -07:00
Xiao Chen
6a38d118d8 HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory. 2016-10-07 17:37:15 -07:00
Kihwal Lee
e57fa81d95 HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp. 2016-10-07 17:20:15 -05:00
Wei-Chiu Chuang
3059b251d8 HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma. 2016-10-07 10:17:50 -07:00
Kihwal Lee
f32e9fc8f7 HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn sharp. 2016-10-06 15:11:14 -05:00
Mingliang Liu
202325485d HDFS-10893. Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test. Contributed by Mingliang Liu 2016-10-05 22:33:29 -07:00
Andrew Wang
e68c7b96c7 HDFS-10962. TestRequestHedgingProxyProvider is flaky. 2016-10-05 15:08:56 -07:00
Jing Zhao
8867762256 HDFS-10826. Correctly report missing EC blocks in FSCK. Contributed by Takanobu Asanuma. 2016-10-05 10:52:50 -07:00
Kihwal Lee
44f48ee96e HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp. 2016-10-04 15:05:09 -05:00
Kihwal Lee
382307cbdd HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah. 2016-10-04 11:09:03 -05:00
Mingliang Liu
f61e3d13e9 HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger 2016-10-03 22:34:41 -04:00
Mingliang Liu
8078a5efd0 HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula 2016-10-03 21:50:16 -04:00
Xiao Chen
853d65a157 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 2016-10-03 16:01:54 -07:00
Xiaoyu Yao
607705c488 HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu. 2016-10-03 10:53:21 -07:00
Kihwal Lee
744208431f HDFS-10940. Reduce performance penalty of block caching when not used. Contributed by Daryn Sharp. 2016-10-03 11:27:23 -05:00
Arpit Agarwal
fe9ebe20ab Revert "HDFS-10923. Make InstrumentedLock require ReentrantLock." 
This reverts commit c7ce6fdc20.
 2016-09-30 23:11:51 -07:00
Arpit Agarwal
c7ce6fdc20 HDFS-10923. Make InstrumentedLock require ReentrantLock. 2016-09-30 23:00:06 -07:00
Zhe Zhang
434c5ea75d HDFS-10896. Move lock logging logic from FSNamesystem into FSNamesystemLock. Contributed by Erik Krogen. 2016-09-30 13:16:08 -07:00
Kihwal Lee
a0730aa5ce HDFS-10851. FSDirStatAndListingOp: stop passing path as string. Contributed by Daryn Sharp. 2016-09-30 13:03:24 -05:00
Kihwal Lee
0670149c88 HDFS-10850. getEZForPath should NOT throw FNF. Contributed by Andrew Wang. 2016-09-30 08:44:18 -05:00
Mingliang Liu
84c6264079 HDFS-10892. Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'. Contributed by Mingliang Liu 2016-09-28 16:03:51 -07:00
Arpit Agarwal
c3b235e565 HDFS-10824. MiniDFSCluster#storageCapacities has no effects on real capacity. Contributed by Xiaobing Zhou. 2016-09-28 11:47:37 -07:00
Anu Engineer
03f519a757 HDFS-9850. DiskBalancer: Explore removing references to FsVolumeSpi. Contributed by Manoj Govindassamy. 2016-09-27 21:35:06 -07:00
Mingliang Liu
6437ba18c5 HDFS-10915. Fix time measurement bug in TestDatanodeRestart. Contributed by Xiaobing Zhou 2016-09-27 20:40:17 -07:00
Yongjun Zhang
2acfb1e1e4 HDFS-10376. Enhance setOwner testing. (John Zhuge via Yongjun Zhang) 2016-09-27 14:55:28 -07:00
Arpit Agarwal
8ae4729107 HDFS-10828. Fix usage of FsDatasetImpl object lock in ReplicaMap. (Arpit Agarwal) 2016-09-27 10:02:15 -07:00
Masatake Iwasaki
f55eb981dd HDFS-10426. TestPendingInvalidateBlock failed in trunk. Contributed by Yiqun Lin. 2016-09-28 01:29:24 +09:00
Brahma Reddy Battula
edf0d0f8b2 HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki 2016-09-27 07:04:37 +05:30
Arpit Agarwal
059058f961 HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru. 2016-09-26 17:09:47 -07:00
Wei-Chiu Chuang
8e06d865c4 Addendum patch for HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. 2016-09-26 14:41:04 -07:00
Wei-Chiu Chuang
3ae652f821 HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. Contributed by Wei-Chiu Chuang. 2016-09-26 13:11:32 -07:00
Konstantin V Shvachko
6eb700ee74 HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko. 2016-09-23 14:10:11 -07:00
Konstantin V Shvachko
a5bb88c8e0 HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen. 2016-09-23 10:37:46 -07:00
Andrew Wang
40acacee08 HDFS-10877. Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers. Contributed by Sean Mackrory. 2016-09-22 11:43:11 -07:00
Masatake Iwasaki
964e546ab1 HDFS-9333. Some tests using MiniDFSCluster errored complaining port in use. (iwasakims) 2016-09-21 10:35:25 +09:00
Xiao Chen
0e918dff59 HDFS-10879. TestEncryptionZonesWithKMS#testReadWrite fails intermittently. Contributed by Xiao Chen. 2016-09-20 16:56:52 -07:00
Kai Zheng
734d54c1a8 HDFS-10861. Refactor StripeReaders and use ECChunk version decode API. Contributed by Sammi Chen 2016-09-21 21:34:48 +08:00
Andrew Wang
c54f6ef30f HDFS-10868. Remove stray references to DFS_HDFS_BLOCKS_METADATA_ENABLED. 2016-09-19 11:17:03 -07:00
Xiao Chen
ea839bd48e HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen. 2016-09-17 22:25:39 -07:00
Arpit Agarwal
ec3ea18875 HDFS-9895. Remove unnecessary conf cache from DataNode. Contributed by Xiaobing Zhou. 2016-09-15 15:25:47 -07:00
Arpit Agarwal
2a8f55a0cf HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak. 2016-09-14 09:31:17 -07:00
Kai Zheng
ea0c2b8b05 HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng 2016-09-15 10:46:00 +08:00
Lei Xu
86c9862bec HDFS-10636. Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File. (Virajith Jalaparti via lei) 2016-09-13 12:54:14 -07:00
Anu Engineer
e3f7f58a5f HDFS-10599. DiskBalancer: Execute CLI via Shell. Contributed by Manoj Govindassamy. 2016-09-13 09:38:12 -07:00
Yongjun Zhang
d2466ec3e0 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 2016-09-12 21:25:06 -07:00
Jing Zhao
72dfb048a9 HDFS-10858. FBR processing may generate incorrect reportedBlock-blockGroup mapping. Contributed by Jing Zhao. 2016-09-12 16:40:11 -07:00
Anu Engineer
8a93f45a80 HDFS-10821. DiskBalancer: Report command support with multiple nodes. Contributed by Yiqun Lin. 2016-09-12 15:45:26 -07:00
Arpit Agarwal
a99bf26a08 HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy) 2016-09-10 18:22:25 -07:00
Anu Engineer
bee9f57f5c HDFS-10808. DiskBalancer does not execute multi-steps plan-redux. Contributed by Anu Engineer. 2016-09-09 15:00:39 -07:00
Xiao Chen
a0b0383677 Addendum patch for HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy. 2016-09-09 10:54:14 -07:00
Anu Engineer
35c5943b8b HDFS-10553. DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLI. Contributed by Manoj Govindassamy. 2016-09-08 19:26:56 -07:00
Chris Douglas
011f3b24d4 HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang 2016-09-08 17:53:40 -07:00
Zhe Zhang
401db4fc65 HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng. 2016-09-08 11:54:33 -07:00
Akira Ajisaka
63f594892e HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer. 2016-09-08 15:13:43 +09:00
Kai Zheng
d355573f56 Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng" 
This reverts commit 62a9667136
 2016-09-08 05:50:17 +08:00
Arpit Agarwal
5f23abfa30 HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula) 2016-09-06 13:37:21 -07:00
Chris Nauroth
f0d5382ff3 HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth. 2016-09-06 11:02:39 -07:00
Kai Zheng
62a9667136 HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng 2016-09-07 17:05:33 +08:00
Xiao Chen
07650bc37a HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy. 2016-09-02 15:33:11 -07:00
Zhe Zhang
6c600360ca Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2. 2016-08-31 23:43:59 -07:00
Zhe Zhang
6f4b0d33ca HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen. 2016-08-31 15:40:09 -07:00
Andrew Wang
85bab5fb57 HDFS-10784. Implement WebHdfsFileSystem#listStatusIterator. 2016-08-31 14:29:37 -07:00
Anu Engineer
20ae1fa259 HDFS-10813. DiskBalancer: Add the getNodeList method in Command. Contributed by Yiqun Lin. 2016-08-30 18:42:55 -07:00
Ming Ma
9dcbdbdb5a HDFS-9392. Admins support for maintenance state. Contributed by Ming Ma. 2016-08-30 14:00:13 -07:00
Zhe Zhang
4bd45f54ee HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi. 2016-08-29 23:37:26 -07:00
Xiao Chen
cd5e10ccca HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge. 2016-08-29 17:48:08 -07:00
Yongjun Zhang
c25817159a HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar B., Wei-Chiu Chuang, Yongjun Zhang. 2016-08-27 22:51:31 -07:00
Zhe Zhang
e806db7190 HDFS-10584. Allow long-running Mover tool to login with keytab. Contributed by Rakesh R. 2016-08-26 16:43:45 -07:00
Zhe Zhang
407b519fb1 HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen. 2016-08-26 14:19:55 -07:00
Kihwal Lee
8b7adf4ddf HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp. 2016-08-26 15:39:56 -05:00
Kihwal Lee
13fb1b50e6 HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki. 2016-08-26 09:53:10 -05:00
Xiaoyu Yao
4da5000dd3 HDFS-10748. TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout. Contributed by Yiqun Lin. 2016-08-25 09:00:44 -07:00
Kai Zheng
793447f799 HDFS-8905. Refactor DFSInputStream#ReaderStrategy. Contributed by Kai Zheng and Sammi Chen 2016-08-24 21:57:23 +08:00
Daryn Sharp
ec252ce0fc HDFS-10762. Pass IIP for file status related methods 2016-08-24 08:46:47 -05:00
Jason Lowe
126d165efd HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen 2016-08-23 14:46:58 +00:00
Wei-Chiu Chuang
f0efea490e HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-23 04:14:24 -07:00
Akira Ajisaka
e90f3359de HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin. 2016-08-23 19:57:23 +09:00
Eric Yang
c49333becf HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang) 2016-08-22 18:29:56 -07:00
Kihwal Lee
3ca4d6ddfd Revert "HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp." 
This reverts commit 22fc46d765.
 2016-08-22 16:57:45 -05:00
Kihwal Lee
22fc46d765 HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp. 2016-08-22 15:37:02 -05:00
Akira Ajisaka
8179f9a493 HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu. 2016-08-19 16:15:00 +09:00
Xiao Chen
2aa5e2c403 HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin. 2016-08-17 15:52:38 -07:00
Kihwal Lee
869393643d HDFS-10745. Directly resolve paths into INodesInPath. Contributed by Daryn Sharp. 2016-08-17 15:53:03 -05:00
Mingliang Liu
4d4d95fdd5 HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp) 2016-08-15 20:28:40 -07:00
Mingliang Liu
382d615260 HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu) 2016-08-15 20:22:14 -07:00
Anu Engineer
5628b36c08 HDFS-10559. DiskBalancer: Use SHA1 for Plan ID. Contributed by Xiaobing Zhou. 2016-08-15 20:10:21 -07:00