big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,776 Commits 2 Branches 2 Tags 264 MiB
b762199adb
Commit Graph

2719 Commits

Author SHA1 Message Date
Zhe Zhang
1d37a88121 HDFS-8854. Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs. Contributed by Walter Su. 2015-08-13 10:04:45 -07:00
yliu
53bef9c5b9 HADOOP-12295. Improve NetworkTopology#InnerNode#remove logic. (yliu) 2015-08-13 16:45:20 +08:00
Akira Ajisaka
40f815131e HDFS-8622. Implement GETCONTENTSUMMARY operation for WebImageViewer. Contributed by Jagadesh Kiran N. 2015-08-13 11:31:56 +09:00
cnauroth
6cc8e38db5 HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu. 2015-08-12 16:44:53 -07:00
Xiaoyu Yao
3e715a4f4c HDFS-8879. Quota by storage type usage incorrectly initialized upon namenode restart. Contributed by Xiaoyu Yao. 2015-08-12 15:20:54 -07:00
Andrew Wang
1ea1a8334e HDFS-8887. Expose storage type and storage ID in BlockLocation. 2015-08-11 23:25:33 -07:00
Tsz-Wo Nicholas Sze
b56daff6a1 HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. 2015-08-10 16:52:02 -07:00
Jing Zhao
fbf7e81ca0 HDFS-8827. Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks. Contributed by Walter Su and Takuya Fukudome. 2015-08-07 11:25:20 -07:00
Andrew Wang
98a27d1101 HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. 2015-08-07 09:51:46 -07:00
Arpit Agarwal
6d4eee718a HDFS-8856. Make LeaseManager#countPath O(1). (Contributed by Arpit Agarwal) 2015-08-06 18:51:28 -07:00
boli2
8799363db1 HDFS-8857. Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure. Contributed by Li Bo 2015-08-05 22:11:50 -04:00
Zhe Zhang
4de48211c6 HDFS-8399. Erasure Coding: unit test the behaviour of BlockManager recovery work for the deleted blocks. Contributed by Rakesh R. 2015-08-04 15:23:39 -07:00
Jing Zhao
f4c523b69b Revert "HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang." 
This reverts commit c17439c2dd.
 2015-08-06 10:21:55 -07:00
Jing Zhao
663eba0ab1 Revert "HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang." 
This reverts commit de480d6c89.
 2015-08-06 10:21:54 -07:00
Vinayakumar B
cc71ad80e1 HDFS-8815. DFS getStoragePolicy implementation using single RPC call (Contributed by Surendra Singh Lilhore) 2015-08-06 11:10:48 +05:30
Colin Patrick Mccabe
c95993cbaf HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) 2015-08-04 12:22:06 -07:00
Zhihai Xu
90b510405a HDFS-8847. change TestHDFSContractAppend to not override testRenameFileBeingAppended method. Contributed by Zhihai Xu 2015-07-31 21:40:43 -07:00
Colin Patrick Mccabe
21d10ccc6e HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 2015-07-31 15:01:34 -07:00
Zhe Zhang
ba90c02853 HDFS-8202. Improve end to end stirpping file test to add erasure recovering test. Contributed by Xinwei Qin. 2015-07-31 12:16:15 -07:00
Arpit Agarwal
88d8736dde HDFS-7192. DN should ignore lazyPersist hint if the writer is not local. (Contributed by Arpit Agarwal) 2015-07-30 13:16:46 -07:00
Jason Lowe
6f0a35724f HADOOP-10945. 4-digit octal umask permissions throws a parse error. Contributed by Chang Li 2015-07-29 15:25:10 +00:00
Vinayakumar B
6374ee0db4 HDFS-8670. Better to exclude decommissioned nodes for namenode NodeUsage JMX (Contributed by J.Andreina) 2015-07-29 14:47:19 +05:30
Akira Ajisaka
2a1d656196 Revert "HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)" 
This reverts commit 02c01815ec.
 2015-07-29 16:48:30 +09:00
Lei Xu
f4f1b8b267 HDFS-8834. TestReplication is not valid after HDFS-6482. (Contributed by Lei Xu) 2015-07-28 23:39:26 -07:00
Vinayakumar B
975e138df3 HDFS-8822. Add SSD storagepolicy tests in TestBlockStoragePolicy#testDefaultPolicies (Contributed by Vinayakumar B) 2015-07-29 11:50:48 +05:30
Vinayakumar B
50887e5b07 HDFS-8811. Move BlockStoragePolicy name's constants from HdfsServerConstants.java to HdfsConstants.java (Contributed by Vinayakumar B) 2015-07-29 11:45:33 +05:30
Jakob Homan
0712a8103f HDFS-8180. AbstractFileSystem Implementation for WebHdfs. Contributed by Sathosh G Nayak. 2015-07-28 21:03:31 -07:00
Arun Suresh
030fcfa99c HDFS-7858. Improve HA Namenode Failover detection on the client. (asuresh) 2015-07-27 23:02:03 -07:00
Xiaoyu Yao
2196e39e14 HDFS-8785. TestDistributedFileSystem is failing in trunk. Contributed by Xiaoyu Yao. 2015-07-27 07:28:41 -07:00
Akira Ajisaka
1df78688c6 HDFS-8810. Correct assertions in TestDFSInotifyEventInputStream class. Contributed by Surendra Singh Lilhore. 2015-07-27 13:17:24 +09:00
Walter Su
4a72be6e0e HDFS-8769. Erasure coding: unit test for SequentialBlockGroupIdGenerator. Contributed by Rakesh R. 2015-07-25 10:07:40 +08:00
Jing Zhao
95b499a367 HDFS-8798. Erasure Coding: fix DFSStripedInputStream/DFSStripedOutputStream re-fetch token when expired. Contributed by Walter Su. 2015-07-24 13:52:50 -07:00
Akira Ajisaka
f8f6091823 HDFS-8735. Inotify: All events classes should implement toString() API. Contributed by Surendra Singh Lilhore. 2015-07-25 02:56:55 +09:00
Akira Ajisaka
02c01815ec HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka) 2015-07-24 11:37:23 +09:00
Arpit Agarwal
adfa34ff99 HADOOP-12161. Add getStoragePolicy API to the FileSystem interface. (Contributed by Brahma Reddy Battula) 2015-07-23 10:13:04 -07:00
Jing Zhao
e91ccfad07 HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. 2015-07-22 17:42:31 -07:00
Jing Zhao
5956d23b64 HDFS-8781. Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block. Contributed by Yi Liu. 2015-07-21 17:00:49 -07:00
Jing Zhao
f8f7a923b7 HDFS-8760. Erasure Coding: reuse BlockReader when reading the same block in pread. Contributed by Jing Zhao. 2015-07-21 15:47:26 -07:00
cnauroth
cb03768b1b HDFS-8721. Add a metric for number of encryption zones. Contributed by Rakesh R. 2015-07-21 13:55:58 -07:00
Ravi Prakash
5137b388fc Revert "HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak)" 
This reverts commit e4f756260f.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-07-21 11:29:35 -07:00
Vinayakumar B
29cf887b22 HDFS-7582. Enforce maximum number of ACL entries separately per access and default. (Contributed by Vinayakumar B) 2015-07-21 15:16:52 +05:30
Haohui Mai
df1e8ce44a HDFS-7483. Display information per tier on the Namenode UI. Contributed by Benoy Antony and Haohui Mai. 2015-07-20 20:13:12 -07:00
yliu
d6d58606b8 HDFS-8794. Improve CorruptReplicasMap#corruptReplicasMap. (yliu) 2015-07-21 09:20:22 +08:00
Ravi Prakash
e4f756260f HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak) 2015-07-20 14:03:34 -07:00
Walter Su
06394e3760 HDFS-8433. Erasure coding: set blockToken in LocatedStripedBlock. Contributed by Walter Su. 2015-07-20 10:18:34 +08:00
Ming Ma
fbd88f1062 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 2015-07-16 12:33:57 -07:00
Zhe Zhang
4fdd9abd7e HDFS-8787. Erasure coding: rename BlockInfoContiguousUC and BlockInfoStripedUC to be consistent with trunk. 2015-07-15 20:13:04 -07:00
Arpit Agarwal
3ec0a0444f HDFS-8778. TestBlockReportRateLimiting#testLeaseExpiration can deadlock. (Contributed by Arpit Agarwal) 2015-07-15 14:08:58 -07:00
Zhe Zhang
7e091de136 HDFS-8058. Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile. Contributed by Zhe Zhang and Yi Liu. 2015-07-15 09:49:32 -07:00
yliu
f32d9a1758 HDFS-8619. Erasure Coding: revisit replica counting for striped blocks. (Jing Zhao via yliu) 2015-07-15 22:35:19 +08:00
Akira Ajisaka
979c9ca2ca HDFS-8742. Inotify: Support event for OP_TRUNCATE. Contributed by Surendra Singh Lilhore. 2015-07-15 04:42:08 +09:00
Colin Patrick Mccabe
1d74ccecec HDFS-7608: hdfs dfsclient newConnectedPeer has no write timeout (Xiaoyu Yao via Colin P. McCabe) 2015-07-14 10:57:59 -07:00
Tsz-Wo Nicholas Sze
9ef03a4c5b HDFS-8541. Mover should exit with NO_MOVE_PROGRESS if there is no move progress. Contributed by Surendra Singh Lilhore 2015-07-13 15:12:26 -07:00
Jing Zhao
b1e6429a6b HDFS-8669. Erasure Coding: handle missing internal block locations in DFSStripedInputStream. Contributed by Jing Zhao. 2015-07-13 11:41:18 -07:00
Jing Zhao
47f4c54106 HDFS-8751. Remove setBlocks API from INodeFile and misc code cleanup. Contributed by Zhe Zhang 2015-07-10 14:15:22 -07:00
Vinayakumar B
b48908033f HDFS-2956. calling fetchdt without a --renewer argument throws NPE (Contributed by Vinayakumar B)HDFS-2956. calling fetchdt without a --renewer argument throws NPE (Contributed by Vinayakumar B) 2015-07-10 15:47:04 +05:30
Jing Zhao
f4ca530c1c HDFS-8729. Fix TestFileTruncate#testTruncateWithDataNodesRestartImmediately which occasionally failed. Contributed by Walter Su. 2015-07-09 13:17:52 -07:00
Jing Zhao
2470a7bf88 HDFS-8563. Erasure Coding: fsck handles file smaller than a full stripe. Contributed by Walter Su. 2015-07-09 10:55:28 -07:00
Arpit Agarwal
4119ad3112 HDFS-8642. Make TestFileTruncate more reliable. (Contributed by Rakesh R) 2015-07-08 09:31:02 -07:00
Xiaoyu Yao
b68701b7b2 HDFS-8711. setSpaceQuota command should print the available storage type when input storage type is wrong. Contributed by Brahma Reddy Battula. 2015-07-07 13:50:49 -07:00
Jing Zhao
bc99aaffe7 Revert "HDFS-8652. Track BlockInfo instead of Block in CorruptReplicasMap. Contributed by Jing Zhao." 
This reverts commit d62b63d297.
 2015-07-07 10:13:41 -07:00
Jing Zhao
2c494a8436 HDFS-8461. Erasure coding: fix priority level of UnderReplicatedBlocks for striped block. Contributed by Walter Su. 2015-07-06 16:39:47 -07:00
Jing Zhao
d62b63d297 HDFS-8652. Track BlockInfo instead of Block in CorruptReplicasMap. Contributed by Jing Zhao. 2015-07-06 15:54:25 -07:00
Vinayakumar B
fc92d3e651 HDFS-8686. WebHdfsFileSystem#getXAttr(Path p, final String name) doesn't work if namespace is in capitals (Contributed by kanaka kumar avvaru) 2015-07-06 16:09:24 +05:30
Vinayakumar B
2eae130ab9 HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) 2015-07-03 13:35:48 +05:30
Vinayakumar B
bff5999d07 HDFS-8703. Merge refactor of DFSInputStream from ErasureCoding branch (Contributed by Vinayakumar B) 2015-07-02 16:11:50 +05:30
Colin Patrick Mccabe
a78d5074fb HADOOP-12171. Shorten overly-long htrace span names for server (cmccabe) 2015-07-01 17:57:11 -07:00
Jing Zhao
152e5df3b6 HDFS-8666. Speedup the TestMover test. Contributed by Walter Su. 2015-07-01 14:59:50 -07:00
Colin Patrick Mccabe
68e588cbee HADOOP-10798. globStatus() should always return a sorted list of files (cmccabe) 2015-06-30 16:40:21 -07:00
Vinayakumar B
7ba5bbac02 HDFS-8627. NPE thrown if unable to fetch token from Namenode (Contributed by J.Andreina) 2015-06-30 15:42:59 +05:30
Vinayakumar B
6d99017f38 HADOOP-12154. FileSystem#getUsed() returns the file length only from root '/' (Contributed by J.Andreina) 2015-06-30 15:25:20 +05:30
Vinayakumar B
0b7af27b9a HDFS-8468. 2 RPC calls for every file read in DFSClient#open(..) resulting in double Audit log entries (Contributed by Vinayakumar B) 2015-06-30 11:02:13 +05:30
Haohui Mai
d3797f9f3c HDFS-8493. Consolidate truncate() related implementation in a single class. Contributed by Rakesh R. 2015-06-29 16:45:35 -07:00
Benoy Antony
d3fed8e653 HDFS-7390. Provide JMX metrics per storage type. (Benoy Antony) 2015-06-29 11:00:22 -07:00
Vinayakumar B
88ceb382ef HDFS-8586. Dead Datanode is allocated for write when client is from deadnode (Contributed by Brahma Reddy Battula) 2015-06-29 15:25:03 +05:30
Arpit Agarwal
c6793dd8cc HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal) 2015-06-28 14:51:36 -07:00
Andrew Wang
60b858bfa6 HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization. 2015-06-26 11:30:59 -07:00
Jing Zhao
de480d6c89 HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang. 2015-06-26 10:49:01 -07:00
Andrew Wang
ff0e5e572f HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. 2015-06-25 17:29:24 -07:00
Arpit Agarwal
67a62da5c5 HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru) 2015-06-25 10:13:22 -07:00
Akira Ajisaka
bc433908d3 HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N. 2015-06-26 00:20:12 +09:00
Andrew Wang
afe9ea3c12 HDFS-8646. Prune cached replicas from DatanodeDescriptor state on replica invalidation. 2015-06-24 14:42:33 -07:00
Jing Zhao
d920cdab91 HDFS-8567. Erasure Coding: SafeMode handles file smaller than a full stripe. Contributed by Walter Su. 2015-06-23 22:29:28 -07:00
Jing Zhao
2ba6465721 HDFS-8639. Add Option for NameNode HTTP port in MiniDFSClusterManager. Contributed by Kai Sasaki. 2015-06-23 22:16:16 -07:00
Aaron T. Myers
49dfad9429 HDFS-6440. Support more than 2 NameNodes. Contributed by Jesse Yates. 2015-06-23 17:26:11 -07:00
Jakob Homan
fac4e04dd3 HDFS-8542. WebHDFS getHomeDirectory behavior does not match specification. Contributed by Kanaka Kumar Avvaru. 2015-06-22 16:30:45 -07:00
Colin Patrick Mccabe
7b424f938c HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe) 2015-06-22 14:37:10 -07:00
Arpit Agarwal
c7d022b66f HDFS-8192. Eviction should key off used locked memory instead of ram disk free space. (Contributed by Arpit Agarwal) 2015-06-20 13:27:52 -07:00
Colin Patrick Mccabe
d112d18324 HDFS-8582. Support getting a list of reconfigurable config properties and do not generate spurious reconfig warnings (Lei (Eddy) Xu via Colin P. McCabe) 2015-06-19 19:14:05 -07:00
Arpit Agarwal
55278c012f HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru) 2015-06-19 18:23:34 -07:00
Jing Zhao
8c423a8163 HDFS-8602. Erasure Coding: Client can't read(decode) the EC files which have corrupt blocks. Contributed by Jing Zhao and Kai Sasaki. 2015-06-19 14:07:38 -07:00
Jing Zhao
448cb7df67 HDFS-8543. Erasure Coding: processOverReplicatedBlock() handles striped block. Contributed by Walter Su. 2015-06-19 11:53:05 -07:00
Jing Zhao
3682e01984 HDFS-8254. Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure. Contributed by Tsz Wo Nicholas Sze. 2015-06-19 10:23:45 -07:00
Akira Ajisaka
cc432885ad HDFS-6249. Output AclEntry in PBImageXmlWriter. Contributed by surendra singh lilhore. 2015-06-17 17:41:10 -07:00
Haohui Mai
015535dc0a HDFS-8446. Separate safemode related operations in GetBlockLocations(). Contributed by Haohui Mai. 2015-06-17 16:38:43 -07:00
Jing Zhao
c12a974ccf HDFS-8580. Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature. Contributed by Walter Su. 2015-06-17 11:35:31 -07:00
Andrew Wang
6e3fcffe29 HDFS-8608. Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks). Contributed by Zhe Zhang. 2015-06-17 08:05:44 -07:00
Andrew Wang
a3990ca414 HDFS-8591. Remove support for deprecated configuration key dfs.namenode.decommission.nodes.per.interval. 2015-06-16 10:03:34 -07:00
Haohui Mai
32e39d8a29 HDFS-8592. SafeModeException never get unwrapped. Contributed by Haohui Mai. 2015-06-15 22:07:24 -07:00
Tsz-Wo Nicholas Sze
175e6d120f HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:12:01 -07:00