big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,524 Commits 2 Branches 2 Tags 264 MiB
182fc1986a
Commit Graph

5500 Commits

Author SHA1 Message Date
Masatake Iwasaki
8eadd7145e HDFS-10325. TestDFSShell#testMoveWithTargetPortEmpty fails. Contributed by Takanobu Asanuma. 2016-04-26 07:11:13 +09:00
Tsz-Wo Nicholas Sze
f308561f1d HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma 2016-04-25 12:01:48 -07:00
Kihwal Lee
10f0f7851a HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor. 2016-04-25 11:38:14 -05:00
Masatake Iwasaki
6fcde2e38d HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims) 2016-04-23 23:37:56 +09:00
Jing Zhao
63e5412f1a HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou. 2016-04-22 15:14:40 -07:00
Ravi Prakash
d6402faded Revert "HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano" 
This reverts commit 4838b735f0.
 2016-04-22 10:17:20 -07:00
Kihwal Lee
5d4255a801 HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 2016-04-22 11:08:44 -05:00
Jing Zhao
4a043b3fec HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou. 2016-04-21 20:17:27 -07:00
Tsz-Wo Nicholas Sze
14ab7a81e2 HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand 2016-04-21 15:41:50 -07:00
Tsz-Wo Nicholas Sze
bbce1d525e HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:20:12 -07:00
Ravi Prakash
4838b735f0 HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano 2016-04-21 11:32:39 -07:00
Xiaoyu Yao
b4be288c5d HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou. 2016-04-21 10:18:48 -07:00
Xiaoyu Yao
5bd7b592e5 HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao. 2016-04-20 23:10:52 -07:00
Chris Nauroth
63ac2db59a HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth. 2016-04-20 13:39:44 -07:00
Colin Patrick Mccabe
ad36fa6f42 HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe) 2016-04-20 11:33:11 -07:00
Arpit Agarwal
af9bdbe447 HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:26:30 -07:00
Walter Su
90d1af1416 HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun. 2016-04-19 18:54:11 +08:00
Walter Su
4b4df2cd48 HDFS-10284. o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently. Contributed by Mingliang Liu. 2016-04-19 18:07:46 +08:00
Steve Loughran
438efc3634 HDFS-10291 TestShortCircuitLocalRead failing (stevel) 
(cherry picked from commit dd64769699ea4c85abdf9c590b9db680837e8b27)
 2016-04-19 17:48:20 +08:00
Kai Zheng
b5d4c7dc76 HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li. 2016-04-20 15:47:01 +08:00
Jing Zhao
be0bce1b71 HDFS-10306. SafeModeMonitor should not leave safe mode if name system is starting active service. Contributed by Mingliang Liu. 2016-04-18 18:05:49 -07:00
Colin Patrick Mccabe
cb3ca460ef HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe) 2016-04-18 13:47:56 -07:00
Kihwal Lee
d8b729e16f HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun. 2016-04-18 07:58:55 -05:00
Walter Su
ab903029a9 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 2016-04-18 20:29:29 +08:00
Walter Su
67523ffcf4 HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. 2016-04-18 09:28:02 +08:00
Jing Zhao
55e19b7f0c HDFS-10293. StripedFileTestUtil#readAll flaky. Contributed by Mingliang Liu. 2016-04-15 10:53:40 -07:00
Jing Zhao
89a838769f HDFS-10283. o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently. Contributed by Mingliang Liu. 2016-04-15 10:49:21 -07:00
Kihwal Lee
b9c9d03591 HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu. 2016-04-14 15:24:39 -05:00
Kihwal Lee
2c155afe27 HDFS-10292. Add block id when client got Unable to close file exception. Contributed by Brahma Reddy Battula. 2016-04-14 14:25:11 -05:00
Xiaoyu Yao
809226752d HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. Contributed by Xiaobing Zhou. 2016-04-14 10:57:02 -07:00
Kihwal Lee
c970f1d005 HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang. 2016-04-14 12:45:47 -05:00
Kihwal Lee
0d1c1152f1 HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe. 2016-04-14 07:58:24 -05:00
Xiaoyu Yao
5566177c9a HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou. 2016-04-13 16:51:13 -07:00
Xiaoyu Yao
4895c73dd4 Revert "HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode" 
This reverts commit 192112d5a2.
 2016-04-13 16:45:18 -07:00
Andrew Wang
314aa21a89 HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun. 2016-04-13 16:39:50 -07:00
Xiaobing Zhou
192112d5a2 HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode 2016-04-13 15:34:41 -07:00
Kihwal Lee
d2f3bbc290 HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák 2016-04-13 11:22:36 -05:00
Walter Su
903428bf94 HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 2016-04-13 17:49:45 +08:00
Colin Patrick Mccabe
35f0770555 HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe) 2016-04-12 16:28:42 -07:00
Zhe Zhang
6ef42873a0 HDFS-9918. Erasure Coding: Sort located striped blocks based on decommissioned states. Contributed by Rakesh R. 2016-04-12 13:38:58 -07:00
Colin Patrick Mccabe
600d129bb8 HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe) 2016-04-12 10:52:39 -07:00
Akira Ajisaka
209303be3a HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang. 2016-04-12 15:26:50 +09:00
Vinayakumar B
a9a607f8fc HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 2016-04-12 11:44:34 +05:30
Akira Ajisaka
a409508b3f HDFS-10277. PositionedReadable test testReadFullyZeroByteFile failing in HDFS. Contributed by Steve Loughran. 2016-04-12 14:55:01 +09:00
Chris Nauroth
843ee8d59d HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran. 2016-04-08 13:36:58 -07:00
Akira Ajisaka
9c32f8785e HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang. 2016-04-08 15:32:13 +09:00
Uma Maheswara Rao G
3c18a53cbd HDFS-9719. Refactoring ErasureCodingWorker into smaller reusable constructs. Contributed by Kai Zheng. 2016-04-06 22:50:24 -07:00
Tsz-Wo Nicholas Sze
654cd1d0c0 HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:38:42 +08:00
Colin Patrick Mccabe
4bd7cbc29d HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose 2016-04-06 21:07:31 -07:00
Eric Payne
aede8c10ec HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee 2016-04-06 20:20:14 +00:00
Colin Patrick Mccabe
188f65287d HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe) 2016-04-06 11:28:34 -07:00
Jing Zhao
221b3a8722 HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula. 2016-04-06 10:42:59 -07:00
Kihwal Lee
9ba1e5af06 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 2016-04-05 16:26:18 -05:00
Ravi Prakash
0cd320a846 HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula. 2016-04-05 13:41:19 -07:00
Masatake Iwasaki
85ec5573eb HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims) 2016-04-06 03:22:48 +09:00
Kihwal Lee
917464505c HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 2016-04-05 09:07:24 -05:00
Vinayakumar B
818d6b799e HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 2016-04-05 09:49:39 +08:00
Colin Patrick Mccabe
f6b1a81812 HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe) 2016-04-04 18:02:15 -07:00
Kihwal Lee
a7d1fb0cd2 HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 2016-04-04 16:40:00 -05:00
Masatake Iwasaki
154d2532cf HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims) 2016-04-05 03:19:48 +09:00
Vinayakumar B
54b2e78fd2 HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao) 2016-04-03 13:27:49 +05:30
Allen Wittenauer
0a74610d1c HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw) 2016-03-31 07:51:05 -07:00
Akira Ajisaka
32c0c3ecdf HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma. 2016-03-31 09:06:14 +09:00
Colin Patrick Mccabe
37e23ce45c HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe) 2016-03-30 13:37:37 -07:00
Vinayakumar B
09d63d5a19 HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B) 2016-03-30 14:22:00 +08:00
Arpit Agarwal
ddfe6774c2 HDFS-9439. Support reconfiguring fs.protected.directories without NN restart. (Contributed by Xiaobing Zhou) 2016-03-29 13:55:00 -07:00
Arpit Agarwal
c9307e48b7 HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA) 2016-03-29 13:50:08 -07:00
Andrew Wang
f2aec4eb82 HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun. 2016-03-29 11:54:42 -07:00
Vinayakumar B
1f004b3367 HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla) 2016-03-29 11:25:41 +08:00
Walter Su
d8383c687c HDFS-10182. Hedged read might overwrite user's buf. Contributed by zhouyingchao. 2016-03-28 15:44:25 +08:00
Uma Maheswara Rao G
3a4ff7776e HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng 2016-03-26 19:58:09 -07:00
Arpit Agarwal
a337ceb74e Revert "HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng" 
This reverts commit e5ff0ea7ba.
 2016-03-26 09:20:01 -07:00
Uma Maheswara Rao G
e5ff0ea7ba HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng 2016-03-26 00:52:50 -07:00
Lei Xu
fde8ac5d85 Add missing files from HDFS-9005. (lei) 2016-03-25 17:11:35 -07:00
Lei Xu
4fcfea71bf HDFS-9005. Provide support for upgrade domain script. (Ming Ma via Lei Xu) 2016-03-25 17:09:12 -07:00
Allen Wittenauer
738155063e HADOOP-12857. rework hadoop-tools (aw) 2016-03-23 13:46:38 -07:00
Andrew Wang
8f85e5d212 HDFS-10200. Docs for WebHDFS still describe GETDELEGATIONTOKENS operation. Contributed by Wellington Chevreuil. 2016-03-23 12:55:15 -07:00
Colin Patrick Mccabe
0d19a0ce98 HDFS-10193. fuse_dfs segfaults if uid cannot be resolved to a username (John Thiltges via cmccabe) 2016-03-23 11:19:18 -07:00
Colin Patrick Mccabe
a7d8f2b396 HDFS-10189. PacketResponder#toString should include the downstreams for PacketResponderType.HAS_DOWNSTREAM_IN_PIPELINE (Joe Pallas via cmccabe) 2016-03-22 14:56:02 -07:00
Colin Patrick Mccabe
680716f31e HDFS-9951. Use string constants for XML tags in OfflineImageReconstructor (Lin Yiqun via cmccabe) 2016-03-21 11:40:37 -07:00
Andrew Wang
e3bb38d625 HDFS-9405. Warmup NameNode EDEK caches in background thread. Contributed by Xiao Chen. 2016-03-21 11:39:05 -07:00
Haohui Mai
8a3f0cb255 HDFS-7166. SbNN Web UI shows #Under replicated blocks and #pending deletion blocks. Contributed by Wei-Chiu Chuang. 2016-03-20 17:21:25 -07:00
Sangjin Lee
cd8b6889a7 HDFS-9579. Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level (Ming Ma via sjlee) 2016-03-19 14:02:04 -07:00
Akira Ajisaka
9b623fbaf7 HDFS-3677. dfs.namenode.edits.dir.required is missing from hdfs-default.xml. Contributed by Mark Yang. 2016-03-19 01:11:15 +09:00
Kihwal Lee
63c966a3fb HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah. 2016-03-18 10:24:59 -05:00
Colin Patrick Mccabe
dc951e606f HDFS-9949. Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared (Harsh J via cmccabe) 2016-03-17 10:37:42 -07:00
Andrew Wang
02a250db9f HDFS-10173. Typo in DataXceiverServer. Contributed by Michael Han. 2016-03-16 18:49:57 -07:00
Zhe Zhang
32d043d9c5 HDFS-9857. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-1]. Contributed by Rakesh R. 2016-03-16 16:53:58 -07:00
Kihwal Lee
d457401784 HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun. 2016-03-15 10:52:47 -05:00
Masatake Iwasaki
5de848cd5d HDFS-9928. Make HDFS commands guide up to date (Wei-Chiu Chuang via iwasakims) 2016-03-15 20:42:05 +09:00
Colin Patrick Mccabe
9a43094e12 HDFS-9947. Block#toString should not output information from derived classes (cmccabe) 2016-03-14 14:59:11 -07:00
Ravi Prakash
bd5556ba12 Download File from UI broken after pagination. Contributed by Brahma Reddy Battula 2016-03-14 14:30:13 -07:00
Chris Nauroth
5644137ada HDFS-9941. Do not log StandbyException on NN, other minor logging fixes. Contributed by Arpit Agarwal. 2016-03-14 09:54:54 -07:00
Arpit Agarwal
e01c6ea688 HDFS-1477. Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart. (Contributed by Xiaobing Zhou) 2016-03-10 19:03:55 -08:00
Colin Patrick Mccabe
79961ecea8 HDFS-9933. ReverseXML should be capitalized in oiv usage message (cmccabe) 2016-03-10 13:43:27 -08:00
Colin Patrick Mccabe
bd49354c6d HDFS-9934. ReverseXML oiv processor should bail out if the XML file's layoutVersion doesn't match oiv's (cmccabe) 2016-03-10 13:41:06 -08:00
Colin Patrick Mccabe
500875dfcc HDFS-9927. Document the new OIV ReverseXML processor (Wei-Chiu Chuang via cmccabe) 2016-03-10 13:38:00 -08:00
Zhe Zhang
7600e3c48f HDFS-7866. Erasure coding: NameNode manages multiple erasure coding policies. Contributed by Rui Li. 2016-03-08 22:30:30 -08:00
Chris Nauroth
89b16d27e2 HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth. 2016-03-08 21:22:06 -08:00
Andrew Wang
a14a6f08ee HDFS-9919. TestStandbyCheckpoints#testNonPrimarySBNUploadFSImage waitForCheckpoint incorrectly. Contributed by Lin Yiqun. 2016-03-08 11:54:50 -08:00