big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,596 Commits 2 Branches 2 Tags 264 MiB
1050d424a2
Commit Graph

2179 Commits

Author SHA1 Message Date
Tsz-Wo Nicholas Sze
70dfe9cfab Merge branch 'trunk' into HDFS-6584 2014-09-11 15:23:38 +08:00
Jing Zhao
db41a1b7b9 Merge changes from trunk 2014-09-09 22:47:50 -07:00
Alejandro Abdelnur
bbff44cb03 HDFS-6776. Using distcp to copy data between insecure and secure cluster via webdhfs doesn't work. (yzhangal via tucu) 2014-09-09 22:16:42 -07:00
arp
0d2f6b4600 Merge branch 'trunk' into HDFS-6581 2014-09-09 18:07:18 -07:00
Konstantin V Shvachko
05af0ff4be Revert HDFS-6940. 2014-09-09 17:30:10 -07:00
cnauroth
2749fc677c Newly moved block replica been invalidated and deleted in TestBalancer. Contributed by Binglin Chang. 2014-09-09 11:32:54 -07:00
arp
61ccd50362 Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
 2014-09-08 22:44:04 -07:00
arp
f949f6b548 HDFS-6981. Fix DN upgrade with layout version change. (Arpit Agarwal) 2014-09-08 21:20:55 -07:00
Andrew Wang
04915a0814 HDFS-6951. Correctly persist raw namespace xattrs to edit log and fsimage. Contributed by Charles Lamb. 2014-09-08 16:59:30 -07:00
Jing Zhao
6997c1c658 Merge remote-tracking branch 'origin/trunk' into HDFS-6584 2014-09-08 15:15:34 -07:00
arp
4ff128af12 Merge branch 'trunk' into HDFS-6581 2014-09-08 15:00:01 -07:00
Jing Zhao
867f0f141c HDFS-7029. Archival Storage: fix TestDFSInotifyEventInputStream and TestDistributedFileSystem. Contributed by Tsz Wo Nicholas Sze. 2014-09-08 14:57:27 -07:00
arp
a18caf7753 HDFS-6991. Notify NN of evicted block before deleting it from RAM disk. (Arpit Agarwal) 2014-09-08 14:29:30 -07:00
Jing Zhao
74a7e227c8 HDFS-7027. Archival Storage: Mover does not terminate when some storage type is out of space. Contributed by Tsz Wo Nicholas Sze. 2014-09-08 14:21:26 -07:00
Jing Zhao
2b5c528a73 HDFS-6875. Archival Storage: support migration for a list of specified paths. Contributed by Jing Zhao. 2014-09-08 14:10:00 -07:00
Colin Patrick Mccabe
cad14aa916 HDFS-6036. Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long. (cmccabe) 2014-09-08 12:51:56 -07:00
Kihwal Lee
6a84f88c11 HDFS-7005. DFS input streams do not timeout. Contributed by Daryn Sharp. 2014-09-08 14:41:44 -05:00
arp
ccdf0054a3 HDFS-6977. Delete all copies when a block is deleted from the block space. (Arpit Agarwal) 2014-09-08 10:35:54 -07:00
Tsz-Wo Nicholas Sze
f1432e2424 Merge branch 'trunk' into HDFS-6584 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2014-09-08 10:54:48 +08:00
arp
eb8284d50e Merge trunk into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
 2014-09-07 14:46:46 -07:00
cnauroth
a23144fd8a HDFS Credential Provider related Unit Test Failure. Contributed by Xiaoyu Yao. 2014-09-07 08:39:20 -07:00
arp
d1fa58292e HDFS-6898. DN must reserve space for a full block when an RBW block is created. (Contributed by Arpit Agarwal) 2014-09-06 21:04:29 -07:00
Tsz-Wo Nicholas Sze
22a41dce4a HDFS-6997: add more tests for data migration and replicaion. 2014-09-07 07:44:28 +08:00
arp
31bbeaf383 Merge remote-tracking branch 'apache-commit/trunk' into HDFS-6581 2014-09-06 12:28:22 -07:00
Konstantin V Shvachko
88209ce181 HDFS-6940. Refactoring to allow ConsensusNode implementation. 
Contributed by Konstantin Shvachko.
 2014-09-06 12:07:52 -07:00
Alejandro Abdelnur
3b35f81603 HDFS-6986. DistributedFileSystem must get delegation tokens from configured KeyProvider. (zhz via tucu) 2014-09-05 22:33:48 -07:00
arp
339d21f273 Merge trunk into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
 2014-09-05 21:36:25 -07:00
arp
c5ef9c9e69 HDFS-6862. Add missing timeout annotations to tests. (Contributed by Xiaoyu Yao) 2014-09-05 13:50:42 -07:00
Jing Zhao
54767d3ff4 Fix TestOfflineEditsViewer after merging. 2014-09-05 12:05:22 -07:00
Jing Zhao
300cb12bef Merge trunk into HDFS-6584 2014-09-05 11:37:22 -07:00
arp
9609b7303a HDFS-6862. Add missing timeout annotations to tests. (Contributed by Xiaoyu Yao) 2014-09-05 11:10:58 -07:00
arp
9e941d9f99 HDFS-6831. Inconsistency between 'hdfs dfsadmin' and 'hdfs dfsadmin -help'. (Contributed by Xiaoyu Yao) 2014-09-05 10:45:54 -07:00
Jing Zhao
c6107f566f HDFS-6376. Distcp data between two HA clusters requires another configuration. Contributed by Dave Marion and Haohui Mai. 2014-09-05 10:40:02 -07:00
Vinayakumar B
772d1fb01e HDFS-6714. TestBlocksScheduledCounter#testBlocksScheduledCounter should shutdown cluster (vinayakumarb) 2014-09-05 10:25:09 +05:30
Jing Zhao
185200e709 HDFS-6876. Archival Storage: support set/get storage policy in DFSAdmin. Contributed by Jing Zhao. 2014-09-04 20:14:46 -07:00
Jing Zhao
6104520369 HDFS-6886. Use single editlog record for creating file + overwrite. Contributed by Yi Liu. 2014-09-04 18:54:38 -07:00
Jing Zhao
f7df24bb1b HDFS-6996. SnapshotDiff report can hit IndexOutOfBoundsException when there are nested renamed directory/file. Contributed by Jing Zhao. 2014-09-04 16:12:44 -07:00
Jing Zhao
e08701ec71 HDFS-6961. Archival Storage: BlockPlacementPolicy#chooseTarget should check each valid storage type in each choosing round. 2014-09-04 14:19:32 -07:00
Colin Patrick Mccabe
f4caedfcbf HDFS-6959 Make the HDFS home directory location customizable. Contributed by Yongjun Zhang 2014-09-03 19:30:30 -07:00
Jing Zhao
45d5b13256 Merge changes from trunk 2014-09-03 15:36:47 -07:00
arp
08a8f2be81 Merge trunk into HDFS-6581 2014-09-03 10:53:37 -07:00
arp
762b04e994 HDFS-6950. Add Additional unit tests for HDFS-6581. (Contributed by Xiaoyu Yao) 2014-09-03 10:51:26 -07:00
Uma Maheswara Rao G
3425ae5d7e HDFS-2975. Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart). (Yi Liu via umamahesh) 2014-09-03 18:53:51 +05:30
arp
a6b32a3e78 Merge trunk into HDFS-6581 2014-09-02 16:25:32 -07:00
Haohui Mai
08a9ac7098 HDFS-6942. Fix typos in log messages. Contributed by Ray Chiang. 2014-09-02 16:22:18 -07:00
Colin Patrick Mccabe
727331becc HDFS-4257. The ReplaceDatanodeOnFailure policies could have a forgiving option. Contributed by szetszwo. 2014-09-02 15:14:33 -07:00
Andrew Wang
faa4455be5 HDFS-6634. inotify in HDFS. Contributed by James Thomas. 2014-09-02 14:02:29 -07:00
Jing Zhao
3e2a0b5446 HDFS-6969. Archival Storage: INode#getStoragePolicyID should always return the latest storage policy. Contributed by Jing Zhao. 2014-09-01 17:56:04 -07:00
arp
217dea7794 Merge remote-tracking branch 'apache-commit/trunk' into HDFS-6581 2014-09-01 11:52:59 -07:00
Andrew Wang
6066b1a455 HDFS-6972. TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration doesn't decode url correctly. Contributed by Yongjun Zhang. 2014-08-29 15:47:01 -07:00
arp
f65183eba3 Merge remote-tracking branch 'apache-commit/trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-08-29 13:37:17 -07:00
Aaron T. Myers
7eab2a29a5 HDFS-6774. Make FsDataset and DataStore support removing volumes. Contributed by Lei Xu. 2014-08-29 13:00:17 -07:00
arp
c92837aeab HDFS-6931. Move lazily persisted replicas to finalized directory on DN startup. (Arpit Agarwal) 2014-08-28 23:13:46 -07:00
arp
4cf9afacbe HDFS-6960. Bugfix in LazyWriter, fix test case and some refactoring. (Arpit Agarwal) 2014-08-28 23:05:32 -07:00
arp
7e32be8768 Merge remote-tracking branch 'apache-commit/trunk' into HDFS-6581 
Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-08-28 19:06:46 -07:00
Todd Lipcon
ab638e77b8 HDFS-6865. Byte array native checksumming on client side. Contributed by James Thomas. 2014-08-28 16:44:09 -07:00
arp
bbaa7dc28d HDFS-6928. 'hdfs put' command should accept lazyPersist flag for testing. (Arpit Agarwal) 2014-08-28 15:53:12 -07:00
arp
2e987148e0 HDFS-6929. NN periodically unlinks lazy persist files with missing replicas from namespace. (Arpit Agarwal) 2014-08-27 21:47:19 -07:00
arp
3f64c4aaf0 HDFS-6927. Initial unit tests for Lazy Persist files. (Arpit Agarwal) 2014-08-27 21:47:19 -07:00
arp
eb448e1439 HDFS-6926. DN support for saving replicas to persistent storage and evicting in-memory replicas. (Arpit Agarwal) 2014-08-27 21:47:19 -07:00
arp
a317bd7b02 HDFS-6925. DataNode should attempt to place replicas on transient storage first if lazyPersist flag is received. (Arpit Agarwal) 2014-08-27 21:47:19 -07:00
arp
c2354a7f81 HDFS-6923. Propagate LazyPersist flag to DNs via DataTransferProtocol. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
arp
042b33f20b HDFS-6922. Add LazyPersist flag to INodeFile, save it in FsImage and edit logs. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
arp
7f49537ba1 HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
arp
a7bcc95358 HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
Colin Patrick Mccabe
7a16731191 HADOOP-10957. The globber will sometimes erroneously return a permission denied exception when there is a non-terminal wildcard. 2014-08-27 19:47:10 -07:00
Jing Zhao
a26aa6bd07 HDFS-6944. Archival Storage: add retry and termination logic for Mover. Contributed by Jing Zhao. 2014-08-27 14:20:54 -07:00
Colin Patrick Mccabe
6962510f72 HDFS-6879. Adding tracing to Hadoop RPC. Contributed by Masatake Iwasaki. 2014-08-27 14:12:05 -07:00
Jing Zhao
b7ded466b0 HDFS-6920. Archival Storage: check the storage type of delNodeHintStorage when deleting a replica. Contributed by Tsz Wo Nicholas Sze. 2014-08-27 14:08:02 -07:00
Colin Patrick Mccabe
d805cc27a9 HDFS-6773. MiniDFSCluster should skip edit log fsync by default. Contributed by Stephen Chu. 2014-08-27 13:33:02 -07:00
Jing Zhao
8ea20b53a8 HDFS-6911. Archival Storage: check if a block is already scheduled in Mover. Contributed by Tsz Wo Nicholas Sze. 2014-08-27 10:38:10 -07:00
Jing Zhao
555900a9dc Merge branch 'trunk' into HDFS-6584 2014-08-27 10:32:30 -07:00
Jing Zhao
6b441d227a HDFS-6908. Incorrect snapshot directory diff generated by snapshot deletion. Contributed by Juan Yu and Jing Zhao. 2014-08-27 10:26:22 -07:00
arp
c5d9a4a91e HDFS-6694. TestPipelinesFailover.testPipelineRecoveryStress tests fail intermittently with various symptoms - debugging patch (Contributed by Yongjun Zhang) 2014-08-27 09:52:33 -07:00
Arpit Agarwal
e871955765 HDFS-6899. Allow changing MiniDFSCluster volumes per DN and capacity per volume. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619970 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-23 06:01:17 +00:00
Jing Zhao
e69954d22c HDFS-6906. Archival Storage: Add more tests for BlockStoragePolicy. Contributed by Tsz Wo Nicholas Sze. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1619628 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 23:42:33 +00:00
Tsz-wo Sze
c92d869d02 Merge r1609845 through r1619277 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1619293 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 05:22:10 +00:00
Arpit Agarwal
6824abc19e HDFS-6758. Block writer should pass the expected block size to DataXceiverServer (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619275 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 01:13:49 +00:00
Arpit Agarwal
8f00897a4b HDFS-6878. Change MiniDFSCluster to support StorageType configuration for individual directories. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619271 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 00:42:27 +00:00
Andrew Wang
834fccf02b Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1619194 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 18:19:13 +00:00
Jing Zhao
f20d363126 HDFS-6870. Blocks and INodes could leak for Rename with overwrite flag. Contributed by Yi Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619192 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 18:13:03 +00:00
Andrew Wang
d2a39b61aa Merge from trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1619018 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 01:22:42 +00:00
Tsz-wo Sze
4e324bcd3d Merge r1609845 through r1618763 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618764 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 23:41:53 +00:00
Brandon Li
2fb04d2a30 HDFS-6569. OOB message can't be sent to the client when DataNode shuts down for upgrade. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618742 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 21:23:08 +00:00
Andrew Wang
0cc08f6da4 Merge from trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618700 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:41:31 +00:00
Andrew Wang
bd616552cc HDFS-6825. Edit log corruption due to delayed block removal. Contributed by Yongjun Zhang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618684 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:08:42 +00:00
Tsz-wo Sze
cb75b6b07d Merge r1609845 through r1618416 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618417 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-16 21:02:21 +00:00
Tsz-wo Sze
9b250d74f0 HDFS-6847. Support storage policy on directories and include storage policy in HdfsFileStatus. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618416 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-16 20:58:20 +00:00
Charles Lamb
20dcb841ce HDFS-6546. Add non-superuser capability to get the encryption zone for a specific path. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618022 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 19:11:06 +00:00
Vinayakumar B
195961a7c1 HDFS-6247. Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer (vinayakumarb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617799 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 18:43:05 +00:00
Vinayakumar B
6554994fab Reverted 
Merged revision(s) 1617784 from hadoop/common/trunk:
HDFS-6847. Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer (Contributed by Vinayakumar B.)
........


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617794 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 18:36:51 +00:00
Vinayakumar B
471b1368e2 HDFS-6847. Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer (Contributed by Vinayakumar B.) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617784 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 18:06:33 +00:00
Allen Wittenauer
a34dafe325 HADOOP-8944. Shell command fs -count should include human readable option (Jonathan Allen via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617775 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 17:30:26 +00:00
Arpit Agarwal
662c5bb3af HDFS-6830. BlockInfo.addStorage fails when DN changes the storage for a block replica. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617598 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 21:42:21 +00:00
Jing Zhao
37207b75d4 HDFS-6835. Archival Storage: Add a new API to set storage policy. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1617568 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 19:32:09 +00:00
Charles Lamb
74f7be0887 merge from trunk r1617527 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1617532 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 17:02:07 +00:00
Jing Zhao
04884cd530 Merging r1616894 through r1617376 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1617377 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 22:27:50 +00:00
Jing Zhao
e60673697d HDFS-6837. Code cleanup for Balancer and Dispatcher. Contributed by Tsz Wo Nicholas Sze. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617337 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 18:01:14 +00:00
Jing Zhao
9d5f8fa68d Merging r1616428 through r1616893 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1616897 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 22:21:35 +00:00
Jing Zhao
c3cf331dc9 HDFS-6828. Separate block replica dispatching from Balancer. Contributed by Tsz Wo Nicholas Sze. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616889 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:33:57 +00:00
Arpit Agarwal
05d1bf4157 HDFS-6830. Revert accidental checkin 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616884 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:22:48 +00:00
Arpit Agarwal
a70c9de3f1 HDFS-6830. BlockInfo.addStorage fails when DN changes the storage for a block replica. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616883 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:20:19 +00:00
Charles Lamb
e6bdb33784 HADOOP-10919. Copy command should preserve raw.* namespace extended attributes. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616840 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 17:58:01 +00:00
Arpit Agarwal
d3a2fe2807 HDFS-6772. Get DN storages out of blockContentsStale state faster after NN restarts. (Contributed by Ming Ma) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616680 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 05:41:38 +00:00
Haohui Mai
2ac640ec75 HDFS-6722. Display readable last contact time for dead nodes on NN webUI. Contributed by Ming Ma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616669 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 04:27:47 +00:00
Aaron Myers
d758be1f35 HDFS-6740. Make FSDataset support adding data volumes dynamically. Contributed by Lei Xu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616623 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 22:59:59 +00:00
Aaron Myers
eca80dca3e HDFS-6728. Dynamically add new volumes to DataStorage, formatted if necessary. Contributed by Lei Xu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616615 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 22:46:51 +00:00
Andrew Wang
311d2f0773 Merge from trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616589 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 20:26:52 +00:00
Tsz-wo Sze
b50cb5e056 Merge r1609845 through r1616427 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1616428 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 07:38:23 +00:00
Tsz-wo Sze
0ed8732fee HDFS-6812. Remove addBlock and replaceBlock from DatanodeDescriptor. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616426 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 07:30:53 +00:00
Jing Zhao
efc73a0f13 HDFS-6791. A block could remain under replicated if all of its replicas are on decommissioned nodes. Contributed by Ming Ma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616306 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 19:03:07 +00:00
Brandon Li
812ac91add HDFS-6790. DFSUtil Should Use configuration.getPassword for SSL passwords. Contributed by Larry McCay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616058 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 00:50:07 +00:00
Andrew Wang
ac73d416f3 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615844 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 02:30:54 +00:00
Uma Maheswara Rao G
10db613389 HADOOP-10886. CryptoCodec#getCodecclasses throws NPE when configurations not loaded. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615523 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-04 09:56:57 +00:00
Vinayakumar B
33518e5613 HDFS-5185. DN fails to startup if one of the data dir is full. Contributed by Vinayakumar B. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615504 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-04 08:43:51 +00:00
Vinayakumar B
30a9ec26f7 HDFS-5723. Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction. Contributed by Vinayakumar B. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615491 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-04 06:44:06 +00:00
Colin McCabe
1ba3f89714 HDFS-6482. Use block ID-based block layout on datanodes (James Thomas via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615223 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 20:41:05 +00:00
Andrew Wang
70c99278a9 HDFS-6780. Batch the encryption zones listing API. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615189 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 18:47:01 +00:00
Charles Lamb
3d9e39e51f HDFS-6807. Fix TestReservedRawPaths. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615188 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 18:36:32 +00:00
Andrew Wang
da2ce46fbe HDFS-6802. Some tests in TestDFSClientFailover are missing @Test annotation. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615183 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 17:58:14 +00:00
Arpit Agarwal
45db4d204b HDFS-6794. Update BlockManager methods to use DatanodeStorageInfo where possible. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615169 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 16:58:06 +00:00
Tsz-wo Sze
7e12b1912f HDFS-6796. Improve the argument check during balancer command line parsing. Contributed by Benoy Antony 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615107 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 14:12:35 +00:00
Arpit Agarwal
397c882362 HDFS-6798. Add test case for incorrect data node condition during balancing. (Contributed by Benoy Antony) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615044 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 07:12:39 +00:00
Tsz-wo Sze
dc7744d2e5 Merge r1609845 through r1615019 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1615020 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 01:29:49 +00:00
Tsz-wo Sze
b8597e6a10 HDFS-6685. Balancer should preserve storage type of replicas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615015 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 01:05:33 +00:00
Arpit Agarwal
b8b8f3f5e7 HDFS-6441. Add ability to exclude/include specific datanodes while balancing. (Contributed by Benoy Antony and Yu Li) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614812 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 06:02:46 +00:00
Charles Lamb
ab47b666d0 HDFS-6785. Should not be able to create encryption zone using path to a non-directory file. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614755 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 20:00:24 +00:00
Andrew Wang
7405953692 HDFS-6692. Add more HDFS encryption tests. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614735 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 18:39:48 +00:00
Arpit Agarwal
18360e71f1 HDFS-6768. Fix a few unit tests that use hard-coded port numbers. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614732 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 18:37:41 +00:00
Chris Nauroth
535fe14ded HDFS-6570. add api that enables checking if a user has certain permissions on a file. Contributed by Jitendra Pandey. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614723 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 17:49:09 +00:00
Andrew Wang
c3fa0b5884 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614553 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 06:09:33 +00:00
Andrew Wang
12241908ad HDFS-6665. Add tests for XAttrs in combination with viewfs. Contributed by Stephen Chu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614545 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 05:53:20 +00:00
Charles Lamb
9ca2f34c8f HDFS-6730. Create a .RAW extended attribute namespace. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614535 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 01:35:04 +00:00
Andrew Wang
415223548d HDFS-6771. Require specification of an encryption key when creating an encryption zone. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614519 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 23:39:38 +00:00
Andrew Wang
3a90228c30 HDFS-6509 addendum, extra file 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614491 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 21:11:52 +00:00
Andrew Wang
407bb3d3e4 HDFS-6509. Create a special /.reserved/raw directory for raw access to encrypted data. Contributed by Charles Lamb. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614490 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 21:11:26 +00:00
Tsz-wo Sze
10d6210322 Merge r1609845 through r1614231 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1614234 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 00:49:14 +00:00
Tsz-wo Sze
4f9ffc7455 HDFS-6739. Add getDatanodeStorageReport to ClientProtocol. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614215 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-28 23:43:39 +00:00
Charles Lamb
1d3e9ec935 merge from trunk r1613787 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1613788 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 13:32:51 +00:00
Chris Nauroth
9bce3eca42 HDFS-6749. FSNamesystem methods should call resolvePath. Contributed by Charles Lamb. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613561 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-26 04:18:28 +00:00
Colin McCabe
e85a3fecc6 HDFS-6750. The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613537 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 23:56:47 +00:00
Andrew Wang
b9e4be4523 HDFS-6724. Decrypt EDEK before creating CryptoInputStream/CryptoOutputStream. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1613490 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 18:33:10 +00:00
Vinayakumar B
cc0464f4fe HDFS-6752. Avoid Address bind errors in TestDatanodeConfig#testMemlockLimit (vinayakumarb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613486 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 18:17:50 +00:00
Vinayakumar B
2bb650146d HDFS-5919. FileJournalManager doesn't purge empty and corrupt inprogress edits files (vinayakumarb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613355 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 07:02:53 +00:00
Andrew Wang
e670641bbf Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1613334 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 23:55:15 +00:00
Andrew Wang
e171254d56 Name node cannot start if the path of a file under construction contains .snapshot. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613329 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 23:42:06 +00:00
Jing Zhao
8c6e172a0a HDFS-6715. Webhdfs wont fail over when it gets java.io.IOException: Namenode is in startup mode. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613237 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 18:28:00 +00:00
Todd Lipcon
2054453a39 HADOOP-10882. Move DirectBufferPool into common util. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613006 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 06:22:02 +00:00
Andrew Wang
552b4fb9f9 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612928 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 19:26:29 +00:00
Uma Maheswara Rao G
5343b43fd9 HDFS-6422. getfattr in CLI doesn't throw exception or return non-0 return code when xattr doesn't exist. (Charles Lamb via umamahesh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612922 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 19:05:11 +00:00
Tsz-wo Sze
48dc486672 Merge r1609845 through r1612880 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612883 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 17:30:06 +00:00
Charles Lamb
687ce1a5fc HDFS-6733. Creating encryption zone results in NPE when KeyProvider is null. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612843 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 14:51:06 +00:00
Andrew Wang
69b75fca7a HDFS-6720. Remove KeyProvider in EncryptionZoneManager. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612632 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 17:57:06 +00:00
Andrew Wang
c83c5b868e HDFS-6701. Make seed optional in NetworkTopology#sortByDistance. Contributed by Ashwin Shankar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612625 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 17:47:49 +00:00
Tsz-wo Sze
ee13f8bf3e Merge r1609845 through r1612502 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612505 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 08:23:34 +00:00
Tsz-wo Sze
25b0e8471e HDFS-6702. Change DFSClient to pass the StorageType from the namenode to datanodes and change datanode to write block replicas using the specified storage type. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612493 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 07:41:24 +00:00
Andrew Wang
7b466b3b70 HDFS-6716. Update usage of KeyProviderCryptoExtension APIs on NameNode. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612438 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 00:27:51 +00:00
Tsz-wo Sze
3de6c61f86 Merge r1609845 through r1612431 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612432 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 23:37:54 +00:00
Tsz-wo Sze
44d9bb26d6 HDFS-6680. BlockPlacementPolicyDefault does not choose favored nodes correctly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612427 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 23:21:12 +00:00
Andrew Wang
cb20bb6117 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612403 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 21:44:50 +00:00
Tsz-wo Sze
ee3a32784f Merge r1609845 through r1612268 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612270 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 13:48:02 +00:00
Tsz-wo Sze
0a736fbc17 HDFS-6679. Bump NameNodeLayoutVersion and update editsStored test files. Contributed by vinayakumarb 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612266 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 13:38:06 +00:00
Tsz-wo Sze
014be2510f HDFS-6710. Change BlockPlacementPolicy to consider block storage policy in replica deletion. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612265 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 13:34:10 +00:00
Alejandro Abdelnur
64ed72a047 HADOOP-10817. ProxyUsers configuration should support configurable prefixes. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611780 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 19:42:01 +00:00
Tsz-wo Sze
7c18f8d55b HDFS-6616. Add exclude-datanodes feature to WebHDFS redirection so that it will not redirect retries to the same datanode. Contributed by zhaoyunjiong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611750 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 18:20:20 +00:00
Daryn Sharp
551024915d HDFS-6599. 2.4 addBlock is 10 to 20 times slower compared to 0.23 (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611737 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 17:58:07 +00:00
Tsz-wo Sze
2b07af0c59 Merge r1609845 through r1611734 from tunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1611736 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 17:57:51 +00:00
Tsz-wo Sze
08466eaa00 HDFS-6700. BlockPlacementPolicy shoud choose storage but not datanode for deletion. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611731 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 17:40:03 +00:00
Chris Nauroth
5f9e52f745 HDFS-6597. Add a new option to NN upgrade to terminate the process after upgrade on NN is completed. Contributed by Danilo Vunjak. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611723 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 17:14:22 +00:00
Tsz-wo Sze
04fd2012fd Merge r1609845 through r1611528 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1611531 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 02:21:21 +00:00
Jing Zhao
7ba5913797 HDFS-6667. In HDFS HA mode, Distcp/SLive with webhdfs on secure cluster fails with Client cannot authenticate via:[TOKEN, KERBEROS] error. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611508 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 23:11:27 +00:00
Vinayakumar B
ce1659f070 HDFS-6693. TestDFSAdminWithHA fails on windows ( Contributed by Vinayakumar B ) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611441 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 19:17:51 +00:00
Jing Zhao
f4151bbf4f HDFS-6478. RemoteException can't be retried properly for non-HA scenario. Contributed by Ming Ma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611410 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 17:37:22 +00:00
Tsz-wo Sze
38af6101d8 HDFS-6671. Change BlockPlacementPolicy to consider block storage policy in replicaiton. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1611334 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 11:56:22 +00:00
Andrew Wang
77f0e2cca5 HDFS-6405. Test Crypto streams in HDFS. (yliu via wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1611140 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 19:04:06 +00:00
Chris Nauroth
43fe48d9e2 HDFS-5624. Add HDFS tests for ACLs in combination with viewfs. Contributed by Stephen Chu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611068 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 16:27:38 +00:00
Andrew Wang
56c0bd4d37 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1610853 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 21:10:24 +00:00
Andrew Wang
b066be8115 HDFS-6619. Clean up encryption-related tests. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1610849 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 20:59:40 +00:00
Colin McCabe
7fe6ab7f91 HDFS-5809. BlockPoolSliceScanner and high speed hdfs appending make datanode to drop into infinite loop (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610790 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 18:17:13 +00:00
Chris Nauroth
97b5fbc281 HDFS-6678. MiniDFSCluster may still be partially running after initialization fails. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610549 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-14 22:49:08 +00:00
Tsz-wo Sze
395c763aef Merge r1609845 through r1610532 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1610533 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-14 21:23:20 +00:00
Tsz-wo Sze
4f1fae88f8 HDFS-6670. Add block storage policy support with default HOT, WARM and COLD policies. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1610529 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-14 21:11:22 +00:00
Tsz-wo Sze
6f41baa623 HDFS-6677. Change INodeFile and FSImage to support storage policy ID. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1610525 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-14 20:57:15 +00:00
Chris Nauroth
3b54223c0f HDFS-2856. Fix block protocol so that Datanodes don't require root or jsvc. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610474 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-14 18:10:03 +00:00
Andrew Wang
d79f27b429 HDFS-6474. Namenode needs to get the actual keys and iv from the KeyProvider. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1609833 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-11 20:54:47 +00:00
Chris Nauroth
81930b75bd HDFS-5202. Support Centralized Cache Management on Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609618 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-11 03:06:11 +00:00
Kihwal Lee
8f520386fb HDFS-6647. Edit log corruption when pipeline recovery occurs for deleted file present in snapshot. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609543 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 18:18:22 +00:00
Chris Nauroth
cce02e3bef HDFS-6631. TestPread#testHedgedReadLoopTooManyTimes fails intermittently. Contributed by Liang Xie. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609534 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 18:03:32 +00:00
Tsz-wo Sze
e67b80ec3a HDFS-6643. Refactor INodeWithAdditionalFields.PermissionStatusFormat and INodeFile.HeaderFormat. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609401 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 06:16:34 +00:00
Jing Zhao
36492f084b HDFS-6645. Add test for successive Snapshots between XAttr modifications. Contributed by Stephen Chu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609388 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 05:01:41 +00:00
Colin McCabe
d9c5f20333 HDFS-6622. Rename and AddBlock may race and produce invalid edits (kihwal via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609384 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 04:03:02 +00:00
Chris Nauroth
f62e93d591 HDFS-6638. Shorten test run time with a smaller retry timeout setting. Contributed by Liang Xie. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608905 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-08 19:16:48 +00:00
Chris Nauroth
3470dd9a2f HDFS-6614. Addendum patch to shorten TestPread run time with smaller retry timeout setting. Contributed by Liang Xie. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608846 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-08 16:43:16 +00:00
Haohui Mai
76a621ffd2 HDFS-6609. Use DirectorySnapshottableFeature to represent a snapshottable directory. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608631 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-08 00:08:18 +00:00
Andrew Wang
dda85637df Merge trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1608603 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:43:56 +00:00
Chris Nauroth
93c1bc8114 HDFS-6617. Flake TestDFSZKFailoverController.testManualFailoverWithDFSHAAdmin due to a long edit log sync op. Contributed by Liang Xie. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608522 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 17:37:04 +00:00
Andrew Wang
f881f0e923 HDFS-6493. Change dfs.namenode.startup.delay.block.deletion to second instead of millisecond. Contributed by Juan Yu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607732 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 20:02:14 +00:00
Andrew Wang
93e23a9915 HDFS-6613. Improve logging in caching classes. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607697 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 17:13:59 +00:00
Uma Maheswara Rao G
160c912ee6 HDFS-6620. Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607685 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 16:08:44 +00:00
Steve Loughran
b6fbaba8e9 HADOOP-9361: Strictly define FileSystem APIs - HDFS portion 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607597 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 12:09:45 +00:00
Charles Lamb
2a3bccddd9 HDFS-6625. Remove the Delete Encryption Zone function (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1607507 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 00:58:45 +00:00
Andrew Wang
51b97a1396 HDFS-6605.Client server negotiation of cipher suite. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1607499 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 23:08:29 +00:00
Andrew Wang
9b5d551ad9 HDFS-6610. TestShortCircuitLocalRead tests sometimes timeout on slow machines. Contributed by Charles Lamb. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607496 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 23:02:24 +00:00
Colin McCabe
297e3c72fb HDFS-6604. The short-circuit cache doesn't correctly time out replicas that haven't been used in a while (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607456 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 19:43:36 +00:00
Chris Nauroth
130182df1b HDFS-6614. shorten TestPread run time with a smaller retry timeout setting. Contributed by Liang Xie. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607447 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 18:57:25 +00:00
Andrew Wang
9dfe9cf400 HDFS-6612. MiniDFSNNTopology#simpleFederatedTopology(int) always hardcode nameservice ID. Contributed by Juan Yu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607442 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 18:37:00 +00:00
Chris Nauroth
c89bf0184d HDFS-6603. Add XAttr with ACL test. Contributed by Stephen Chu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607239 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 03:20:25 +00:00
Chris Nauroth
0ca41a8f35 HDFS-6591. while loop is executed tens of thousands of times in Hedged Read. Contributed by Liang Xie. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606927 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-30 20:46:44 +00:00
Kihwal Lee
94a1462bd5 HDFS-6601. Issues in finalizing rolling upgrade when there is a layout version change. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606371 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-28 15:21:46 +00:00
Uma Maheswara Rao G
78cafe34e6 HDFS-6556. Refine XAttr permissions. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606320 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-28 11:40:02 +00:00
Charles Lamb
31617733ac HDFS-6389. Rename restrictions for encryption zones. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1606253 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 21:45:18 +00:00
Andrew Wang
2efea95213 HDFS-6391. Get the Key/IV from the NameNode for encrypted files in DFSClient. Contributed by Charles Lamb and Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1606220 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 20:43:41 +00:00
Chris Nauroth
280bdb9a16 HDFS-6572. Add an option to the NameNode that prints the software and on-disk image versions. Contributed by Charles Lamb. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605928 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-26 23:12:45 +00:00
Chris Nauroth
c3e26735a6 HADOOP-9705. FsShell cp -p does not preserve directory attibutes. Contributed by Akira AJISAKA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605672 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-26 05:36:54 +00:00
Tsz-wo Sze
e3612e4428 HDFS-6595. Allow the maximum threads for balancing on datanodes to be configurable. Contributed by Benoy Antony 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605565 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-25 19:27:29 +00:00
Charles Lamb
2676365799 HDFS-6387. HDFS CLI admin tool for creating & deleting an encryption zone. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1605518 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-25 17:42:03 +00:00
Aaron Myers
55a1defa50 HDFS-6475. WebHdfs clients fail without retry because incorrect handling of StandbyException. Contributed by Yongjun Zhang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605217 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 23:07:49 +00:00
Arpit Agarwal
9ddedbf3df HDFS-6587. Bug in TestBPOfferService can cause test failure. (Contributed by Zhilei Xu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604899 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-23 18:21:00 +00:00
Vinayakumar B
e8ca648005 HDFS-6507. Improve DFSAdmin to support HA cluster better. (Contributd by Zesheng Wu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604692 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-23 05:16:05 +00:00
Haohui Mai
1e89eba47d HDFS-6583. Remove clientNode in FileUnderConstructionFeature. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604541 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-22 07:39:36 +00:00
Jing Zhao
8a83bb7ad6 HDFS-4667. Capture renamed files/directories in snapshot diff report. Contributed by Jing Zhao and Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604488 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-21 22:59:34 +00:00
Chris Nauroth
0c51289695 HDFS-6222. Remove background token renewer from webhdfs. Contributed by Rushabh Shah and Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604300 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-20 23:58:23 +00:00
Haohui Mai
9ca79e8d32 HDFS-6557. Move the reference of fsimage to FSNamesystem. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604242 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-20 18:54:19 +00:00
Jing Zhao
d9eb18bb2e HDFS-6535. HDFS quota update is wrong when file is appended. Contributed by George Wong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604226 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-20 18:25:09 +00:00
Kihwal Lee
0da148dae7 HDFS-3848. A Bug in recoverLeaseInternal method of FSNameSystem class. Contributed by Hooman Peiro Sajjad and Chen He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604011 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 19:05:25 +00:00
Andrew Wang
97583dbb0a Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1603993 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 18:32:13 +00:00
Andrew Wang
d417e49ce4 HDFS-6492. Support create-time xattrs and atomically setting multiple xattrs. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603971 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 17:37:31 +00:00
Aaron Myers
3f82484218 HDFS-6563. NameNode cannot save fsimage in certain circumstances when snapshots are in use. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603712 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 05:17:58 +00:00
Haohui Mai
a4e0ff5e05 HDFS-6480. Move waitForReady() from FSDirectory to FSNamesystem. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603705 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 04:13:56 +00:00
Andrew Wang
5e5e1a6202 Merge trunk r1603663 to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1603664 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 23:15:04 +00:00
Charles Lamb
6ef3a9e746 HDFS-6386. HDFS Encryption Zones (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1603658 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 22:33:50 +00:00
Jing Zhao
4cf94aaf80 HDFS-6551. Rename with OVERWRITE option may throw NPE when the target file/directory is a reference INode. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603612 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 19:57:00 +00:00
Arpit Agarwal
52d18aa217 HDFS-6552. add DN storage to a BlockInfo will not replace the different storage from same DN. (Contributed by Amir Langer) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603602 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 19:37:18 +00:00
Jing Zhao
8e8a769e7f HDFS-6527. Edit log corruption due to defered INode removal. Contributed by Kihwal Lee and Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603340 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 01:00:32 +00:00
Chris Nauroth
6e37dd331b HADOOP-10557. FsShell -cp -pa option for preserving extended ACLs. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603222 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-17 17:08:07 +00:00
Andrew Wang
f4b1c44415 HDFS-6528. Add XAttrs to TestOfflineImageViewer. Contributed by Stephen Chu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603020 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 23:07:31 +00:00
Andrew Wang
45c52ef924 HDFS-6518. TestCacheDirectives#testExceedsCapacity should take FSN read lock when accessing pendingCached list. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603016 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 22:47:58 +00:00
Charles Lamb
c5b7236d9c merge from trunk r1602933 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1602947 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 18:13:57 +00:00
Andrew Wang
70db2ec417 HDFS-6470. TestBPOfferService.testBPInitErrorHandling is flaky. Contributed by Ming Ma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602490 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-13 19:00:03 +00:00
Haohui Mai
02f3ff3387 HDFS-6330. Move mkdirs() to FSNamesystem. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602484 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-13 18:04:45 +00:00
Andrew Wang
be01103af7 HDFS-3493. Invalidate corrupted blocks as long as minimum replication is satisfied. Contributed by Juan Yu and Vinayakumar B. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602291 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 21:06:31 +00:00
Andrew Wang
befa4bb1ed HDFS-6395. Skip checking xattr limits for non-user-visible namespaces. Contributed by Yi Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602288 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 20:55:06 +00:00
Chris Nauroth
2368332138 HADOOP-10561. Copy command with preserve option should handle Xattrs. Contributed by Yi Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602225 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 16:34:02 +00:00
Konstantin Boudnik
2f4f261cda HDFS-6471. Make moveFromLocal CLI testcases to be non-disruptive (Contributed by Dasha Boudnik) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602064 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 05:48:26 +00:00
Arpit Agarwal
34e9173c00 HADOOP-10376. Refactor refresh*Protocols into a single generic refreshConfigProtocol. (Contributed by Chris Li) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602055 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 01:27:39 +00:00
Haohui Mai
e98529858e HDFS-6315. Decouple recording edit logs from FSDirectory. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601960 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-11 17:22:44 +00:00
Andrew Wang
123c563fe6 HDFS-6460. Ignore stale and decommissioned nodes in NetworkTopology#sortByDistance. Contributed by Yongjun Zhang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601535 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-10 00:39:20 +00:00
Colin McCabe
8b2b7ff0ae HDFS-6257. TestCacheDirectives#testExceedsCapacity fails occasionally (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601473 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-09 18:39:37 +00:00
Junping Du
cc9bc8eef7 HDFS-6500. Snapshot shouldn't be removed silently after renaming to an existing snapshot. (Contributed by Nicholas SZE) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601199 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-08 07:06:35 +00:00
Konstantin Boudnik
fb7ea6d8bc HDFS-6297. Add CLI testcases to reflect new features of dfs and dfsadmin (Contributed by Dasha Boudnik) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601059 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-07 05:22:34 +00:00