big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,557 Commits 2 Branches 2 Tags 264 MiB
46612c7a51
Commit Graph

3277 Commits

Author SHA1 Message Date
arp
032e0eba6b Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-25 19:50:28 -07:00
Andrew Wang
e96ce6f3e3 HDFS-7077. Separate CipherSuite from crypto protocol version. (wang) 2014-09-25 18:40:40 -07:00
Jing Zhao
e9c37de485 HDFS-7131. During HA upgrade, JournalNode should create a new committedTxnId file in the current directory. Contributed by Jing Zhao. 2014-09-25 17:15:33 -07:00
cnauroth
9f9a2222a2 HDFS-7119. Split error checks in AtomicFileOutputStream#close into separate conditions to improve diagnostics. Contributed by Chris Nauroth. 2014-09-25 13:33:37 -07:00
Colin Patrick Mccabe
1861b32eb5 HDFS-6808. Add command line option to ask DataNode reload configuration. (Lei Xu via Colin McCabe) 2014-09-25 13:11:59 -07:00
arp
5a5a7da3a3 Merge branch 'trunk' into HDFS-6581 2014-09-25 12:55:59 -07:00
cnauroth
b607c3c363 HDFS-7127. TestLeaseRecovery leaks MiniDFSCluster instances. Contributed by Chris Nauroth. 2014-09-25 11:24:31 -07:00
cnauroth
dc2ebf88a8 HDFS-7111. TestSafeMode assumes Unix line endings in safe mode tip. Contributed by Chris Nauroth. 2014-09-25 11:17:46 -07:00
arp
364e60b169 HDFS-7144. Fix findbugs warnings in RamDiskReplicaTracker. (Contributed by Tsz Wo Nicholas Sze) 2014-09-25 11:14:24 -07:00
arp
e9edafc731 Merge branch 'trunk' into HDFS-6581 2014-09-25 11:06:25 -07:00
cnauroth
e0b1dc514f HDFS-7118. Improve diagnostics on storage directory rename operations by using NativeIO#renameTo in Storage#rename. Contributed by Chris Nauroth. 2014-09-25 10:03:59 -07:00
arp
e8e7fbe81a HDFS-6932. Balancer and Mover tools should ignore replicas on RAM_DISK. (Contributed by Xiaoyu Yao) 2014-09-24 21:08:59 -07:00
arp
feda4733a8 HDFS-7143. Fix findbugs warnings in HDFS-6581 branch. (Contributed by Tsz Wo Nicholas Sze) 2014-09-24 21:06:56 -07:00
arp
b1000fbba4 Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
 2014-09-24 20:08:32 -07:00
Jing Zhao
428a76663a HDFS-7140. Add a tool to list all the existing block storage policies. Contributed by Jing Zhao. 2014-09-24 19:11:16 -07:00
Andrew Wang
63198bfba7 HDFS-7139. Unit test for creating encryption zone on root path. (Zhe Zhang via wang) 2014-09-24 16:40:20 -07:00
Jing Zhao
073bbd805c HDFS-7081. Add new DistributedFileSystem API for getting all the existing storage policies. Contributed by Jing Zhao. 2014-09-24 10:05:40 -07:00
Allen Wittenauer
cb74f39697 HDFS-6534. Fix build on macosx: HDFS parts (Binglin Chang via aw) 2014-09-24 08:22:02 -07:00
arp
bfc7b7e6a0 Merge branch 'trunk' into HDFS-6581 2014-09-23 16:40:35 -07:00
cnauroth
b93d9603a2 HDFS-7130. TestDataTransferKeepalive fails intermittently on Windows. Contributed by Chris Nauroth. 2014-09-23 14:35:33 -07:00
Andrew Wang
f48686a1ad HDFS-7132. hdfs namenode -metadataVersion command does not honor configured name dirs. Contributed by Charles Lamb. 2014-09-23 14:10:02 -07:00
Allen Wittenauer
3dc28e2052 HADOOP-11092. hadoop shell commands should print usage if not given a class (aw) 2014-09-23 12:24:23 -07:00
cnauroth
a1fd804a31 HDFS-7126. TestEncryptionZonesWithHA assumes Unix path separator for KMS key store path. Contributed by Xiaoyu Yao. 2014-09-23 08:21:46 -07:00
arp
8b139b0800 HDFS-6990. Add unit test for evict/delete RAM_DISK block with open handle. (Contributed by Xiaoyu Yao) 2014-09-22 18:10:28 -07:00
arp
889122bd55 Merge branch 'trunk' into HDFS-6581 2014-09-22 18:00:58 -07:00
Colin Patrick Mccabe
7b8df93ce1 HDFS-7001. Tests in TestTracing should not depend on the order of execution. (iwasakims via cmccabe) 2014-09-22 17:21:58 -07:00
Colin Patrick Mccabe
912ad32b03 HDFS-7106. Reconfiguring DataNode volumes does not release the lock files in removed volumes. (cnauroth via cmccabe) 2014-09-22 11:51:31 -07:00
cnauroth
26cba7f35f HDFS-7115. TestEncryptionZones assumes Unix path separator for KMS key store path. Contributed by Xiaoyu Yao. 2014-09-22 09:07:07 -07:00
Andrew Wang
1737950d0f HDFS-6987. Move CipherSuite xattr information up to the encryption zone root. Contributed by Zhe Zhang. 2014-09-21 21:29:46 -07:00
cnauroth
b6fd5ccbce HDFS-7110. Skip tests related to short-circuit read on platforms that do not currently implement short-circuit read. Contributed by Chris Nauroth. 2014-09-21 12:36:57 -07:00
cnauroth
0795fe3721 HDFS-7109. TestDataStorage does not release file locks between tests. Contributed by Chris Nauroth. 2014-09-20 21:01:13 -07:00
cnauroth
47491f9a11 HDFS-7107. Avoid Findbugs warning for synchronization on AbstractNNFailoverProxyProvider#fallbackToSimpleAuth. Contributed by Chris Nauroth. 2014-09-20 20:53:09 -07:00
arp
8c9860f7c9 HDFS-7071. Undo accidental commit of binary file editsStored. 2014-09-20 17:11:11 -07:00
arp
486a76a39b HDFS-7071. Updated editsStored and editsStored.xml to bump layout version and add LazyPersist flag. (Contributed by Xiaoyu Yao and Arpit Agarwal) 2014-09-20 16:58:58 -07:00
arp
e257b6dbb1 Merge branch 'trunk' into HDFS-6581 2014-09-20 16:13:17 -07:00
arp
50b321068d HDFS-7108. Fix unit test failures in SimulatedFsDataset. (Arpit Agarwal) 2014-09-20 16:12:17 -07:00
Jing Zhao
84a0a629d3 HDFS-7095. Archival Storage: TestStorageMover often fails in Jenkins. Contributed by Jing Zhao. 2014-09-20 13:44:53 -07:00
arp
b2d5ed36bc HDFS-7100. Make eviction scheme pluggable. (Arpit Agarwal) 2014-09-20 13:25:23 -07:00
arp
09dab88d3e Merge branch 'trunk' into HDFS-6581 2014-09-20 10:34:45 -07:00
cnauroth
f85cc14eb4 HDFS-7073. Allow falling back to a non-SASL connection on DataTransferProtocol in several edge cases. Contributed by Chris Nauroth. 2014-09-19 21:23:26 -07:00
cnauroth
332e2e23ba HDFS-7105. Fix TestJournalNode#testFailToStartWithBadConfig to match log output change. Contributed by Ray Chiang. 2014-09-19 21:04:55 -07:00
Kihwal Lee
9e35571965 HDFS-7046. HA NN can NPE upon transition to active. Contributed by 
Kihwal Lee.
 2014-09-19 17:07:06 -05:00
arp
222bf0fe67 Merge branch 'trunk' into HDFS-6581 2014-09-19 10:02:27 -07:00
Kihwal Lee
bf27b9ca57 HDFS-7065. Pipeline close recovery race can cause block corruption. 
Contributed by Kihwal Lee.
 2014-09-19 09:00:13 -05:00
Kihwal Lee
f02d934fed HDFS-6948. DN rejects blocks if it has older UC block. Contributed by 
Eric Payne.
 2014-09-19 08:50:43 -05:00
arp
f8bbf80067 Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsLocatedFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-09-18 22:26:42 -07:00
arp
e79c98c11f HDFS-7091. Add forwarding constructor for INodeFile for existing callers. (Arpit Agarwal) 2014-09-18 19:20:25 -07:00
Andrew Wang
8e73084491 HDFS-6840. Clients are always sent to the same datanode when read is off rack. (wang) 2014-09-18 17:49:17 -07:00
Andrew Wang
20a076bafc HDFS-6970. Move startFile EDEK retries to the DFSClient. (wang) 2014-09-18 17:35:24 -07:00
Colin Patrick Mccabe
fe38d2e9b5 HDFS-6727. Refresh data volumes on DataNode based on configuration changes (Lei Xu via Colin Patrick McCabe) 2014-09-18 16:52:46 -07:00
Tsz-Wo Nicholas Sze
98f7bfe993 Merge branch 'HDFS-6584' into trunk 2014-09-19 07:11:19 +08:00
arp
22295b4783 Merge branch 'trunk' into HDFS-6581 2014-09-18 15:59:37 -07:00
Andrew Wang
70be56d093 HDFS-7003. Add NFS Gateway support for reading and writing to encryption zones. (clamb via wang) 2014-09-18 14:57:25 -07:00
arp
0b10e5a0f7 Merge branch 'trunk' into HDFS-6581 2014-09-18 12:10:45 -07:00
Colin Patrick Mccabe
570b8b468e HDFS-7047. Expose FileStatus#isEncrypted in libhdfs (cmccabe) 2014-09-18 11:42:23 -07:00
arp
5e4627d0fb HDFS-7084. FsDatasetImpl#copyBlockFiles debug log can be improved. (Contributed by Xiaoyu Yao) 2014-09-18 11:13:53 -07:00
arp
849ccfa699 Merge branch 'trunk' into HDFS-6581 2014-09-18 11:06:40 -07:00
Jing Zhao
76ccb80fad HDFS-7088. Archival Storage: fix TestBalancer and TestBalancerWithMultipleNameNodes. Contributed by Tsz Wo Nicholas Sze. 2014-09-18 10:54:01 -07:00
Tsz-Wo Nicholas Sze
2d2b0009e6 Merge branch 'trunk' into HDFS-6584 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2014-09-18 13:00:29 +08:00
Andrew Wang
0ecefe6017 HDFS-7078. Fix listEZs to work correctly with snapshots. (wang) 2014-09-17 21:28:05 -07:00
Andrew Wang
10e8602f32 HDFS-7004. Update KeyProvider instantiation to create by URI. (wang) 2014-09-17 20:14:40 -07:00
arp
4eab083b1b HDFS-7080. Fix finalize and upgrade unit test failures. (Arpit Agarwal) 2014-09-17 15:25:04 -07:00
arp
a186d514bd Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2014-09-17 15:03:55 -07:00
Jing Zhao
911979c8ab Merge changes from trunk 2014-09-17 13:12:45 -07:00
Colin Patrick Mccabe
e3803d002c HDFS-6843. Create FileStatus isEncrypted() method (clamb via cmccabe) 2014-09-17 12:55:35 -07:00
arp
24f8156886 Merge branch 'trunk' into HDFS-6581 2014-09-17 12:10:50 -07:00
Andrew Wang
ea4e2e843e HDFS-6705. Create an XAttr that disallows the HDFS admin from accessing a file. (clamb via wang) 2014-09-17 11:23:47 -07:00
Jing Zhao
b014e83bc5 HDFS-6864. Archival Storage: add user documentation. Contributed by Tsz Wo Nicholas Sze. 2014-09-17 09:40:17 -07:00
Alejandro Abdelnur
e14e71d5fe HADOOP-11096. KMS: KeyAuthorizationKeyProvider should verify the keyversion belongs to the keyname on decrypt. (tucu) 2014-09-16 23:20:35 -07:00
Tsz-Wo Nicholas Sze
bd2e409de1 HDFS-7072. Fix TestBlockManager and TestStorageMover. Contributed by Jing Zhao 2014-09-17 13:48:32 +08:00
arp
a24a9331d4 Merge branch 'trunk' into HDFS-6581 2014-09-16 18:18:09 -07:00
Jing Zhao
432b863719 Merge changes from trunk 2014-09-16 17:58:39 -07:00
Jing Zhao
2e4033e872 HDFS-7052. Archival Storage: Add Mover into hdfs script. Contributed by Jing Zhao. 2014-09-16 17:52:28 -07:00
Andrew Wang
a044407b5c HDFS-6789. TestDFSClientFailover.testFileContextDoesntDnsResolveLogicalURI and TestDFSClientFailover.testDoesntDnsResolveLogicalURI failing on jdk7. Contributed by Akira Ajisaka. 2014-09-16 17:02:06 -07:00
Andrew Wang
33ce887c20 HDFS-6851. Refactor EncryptionZoneWithId and EncryptionZone. (clamb via wang) 2014-09-16 16:41:09 -07:00
Benoy Antony
02adf7185d HDFS-6799. The invalidate method in SimulatedFSDataset failed to remove (invalidate) blocks from the file system. Contributed by Megasthenis Asteris. 2014-09-16 16:24:23 -07:00
Allen Wittenauer
7971c97ec1 HADOOP-11022. User replaced functions get lost 2-3 levels deep (e.g., sbin) (aw) 2014-09-16 16:06:23 -07:00
arp
4603e4481f HDFS-7064. Fix unit test failures in HDFS-6581 branch. (Contributed by Xiaoyu Yao) 2014-09-16 15:45:07 -07:00
Alejandro Abdelnur
3e85f5b605 HDFS-7006. Test encryption zones with KMS. (Anthony Young-Garner and tucu) 2014-09-16 14:36:07 -07:00
arp
e0d7fb48fb Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
 2014-09-16 14:32:57 -07:00
Colin Patrick Mccabe
56119fec96 HDFS-6880. Adding tracing to DataNode data transfer protocol (iwasakims via cmccabe) 2014-09-16 13:59:32 -07:00
Kihwal Lee
0c26412be4 HDFS-6965. NN continues to issue block locations for DNs with full 
disks. Contributed by Rushabh Shah.
 2014-09-16 09:04:54 -05:00
arp
a7814e1aa4 Merge branch 'trunk' into HDFS-6581 2014-09-15 15:30:19 -07:00
arp
a4dcbaa332 HDFS-7066. LazyWriter#evictBlocks misses a null check for replicaState. (Contributed by Xiaoyu Yao) 2014-09-15 15:28:17 -07:00
Kihwal Lee
2f847b3357 HDFS-7059. HAadmin transtionToActive with forceActive option can show 
confusing message. Contributed by Rushabh Shah.
 2014-09-15 16:27:22 -05:00
Jing Zhao
ece3ecabdf Merge remote-tracking branch 'origin/trunk' into HDFS-6584 2014-09-15 10:45:45 -07:00
Andrew Wang
43b0303008 HDFS-7032. Add WebHDFS support for reading and writing to encryption zones. Contributed by Charles Lamb. 2014-09-15 10:23:57 -07:00
Jing Zhao
2689b6ca72 HDFS-7062. Archival Storage: skip under construction block for migration. Contributed by Jing Zhao. 2014-09-15 10:16:56 -07:00
Andrew Wang
fc741b5d78 HDFS-7061. Add test to verify encryption zone creation after NameNode restart without saving namespace. Contributed by Stephen Chu. 2014-09-14 23:49:01 -07:00
Tsz-Wo Nicholas Sze
dba52ce0bb Merge branch 'trunk' into HDFS-6584 2014-09-13 14:56:48 +08:00
arp
9f22fb8c9a HDFS-6978. Directory scanner should correctly reconcile blocks on RAM disk. (Arpit Agarwal) 2014-09-12 22:13:18 -07:00
arp
7aeda47bd9 Merge branch 'trunk' into HDFS-6581 2014-09-12 11:45:11 -07:00
cnauroth
d6ce26986e HDFS-7051. TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator. Contributed by Chris Nauroth. 2014-09-11 20:58:15 -07:00
Jing Zhao
dc27ee9ae9 HDFS-7039. Archival Storage: fix Balancer tests. Contributed by Tsz Wo Nicholas Sze. 2014-09-11 14:19:18 -07:00
Jing Zhao
8de20a1987 Merge trunk into HDFS-6584 2014-09-11 13:04:35 -07:00
Jing Zhao
0d85f7e591 HDFS-7034. Archival Storage: Fix TestBlockPlacement and TestStorageMover. Contributed by Jing Zhao. 2014-09-11 13:00:43 -07:00
cnauroth
80ac6aabce HDFS-7042. Upgrade fails for Windows HA cluster due to file locks held during rename in JournalNode. Contributed by Chris Nauroth. 2014-09-11 12:16:19 -07:00
Andrew Wang
1e684995d7 HDFS-7045. Fix NameNode deadlock when opening file under /.reserved path. Contributed by Yi Liu. 2014-09-11 10:59:09 -07:00
Andrew Wang
d1dab6cc63 HDFS-6621. Hadoop Balancer prematurely exits iterations. Contributed by Rafal Wodjdyla and Benjamin Bowman. 2014-09-11 10:32:20 -07:00
Andrew Wang
bf64fce78c HDFS-6966. Add additional unit tests for encryption zones. Contributed by Stephen Chu. 2014-09-11 10:26:13 -07:00
Tsz-Wo Nicholas Sze
70dfe9cfab Merge branch 'trunk' into HDFS-6584 2014-09-11 15:23:38 +08:00
Jing Zhao
db41a1b7b9 Merge changes from trunk 2014-09-09 22:47:50 -07:00
Alejandro Abdelnur
bbff44cb03 HDFS-6776. Using distcp to copy data between insecure and secure cluster via webdhfs doesn't work. (yzhangal via tucu) 2014-09-09 22:16:42 -07:00
Tsz-Wo Nicholas Sze
4124905aa5 HDFS-7028: FSDirectory should not get storage policy id from symlinks 2014-09-10 12:34:15 +08:00
arp
0d2f6b4600 Merge branch 'trunk' into HDFS-6581 2014-09-09 18:07:18 -07:00
Konstantin V Shvachko
05af0ff4be Revert HDFS-6940. 2014-09-09 17:30:10 -07:00
cnauroth
2749fc677c Newly moved block replica been invalidated and deleted in TestBalancer. Contributed by Binglin Chang. 2014-09-09 11:32:54 -07:00
arp
61ccd50362 Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
 2014-09-08 22:44:04 -07:00
arp
f949f6b548 HDFS-6981. Fix DN upgrade with layout version change. (Arpit Agarwal) 2014-09-08 21:20:55 -07:00
Andrew Wang
04915a0814 HDFS-6951. Correctly persist raw namespace xattrs to edit log and fsimage. Contributed by Charles Lamb. 2014-09-08 16:59:30 -07:00
Jing Zhao
6997c1c658 Merge remote-tracking branch 'origin/trunk' into HDFS-6584 2014-09-08 15:15:34 -07:00
arp
4ff128af12 Merge branch 'trunk' into HDFS-6581 2014-09-08 15:00:01 -07:00
Jing Zhao
867f0f141c HDFS-7029. Archival Storage: fix TestDFSInotifyEventInputStream and TestDistributedFileSystem. Contributed by Tsz Wo Nicholas Sze. 2014-09-08 14:57:27 -07:00
arp
a18caf7753 HDFS-6991. Notify NN of evicted block before deleting it from RAM disk. (Arpit Agarwal) 2014-09-08 14:29:30 -07:00
Jing Zhao
74a7e227c8 HDFS-7027. Archival Storage: Mover does not terminate when some storage type is out of space. Contributed by Tsz Wo Nicholas Sze. 2014-09-08 14:21:26 -07:00
Jing Zhao
2b5c528a73 HDFS-6875. Archival Storage: support migration for a list of specified paths. Contributed by Jing Zhao. 2014-09-08 14:10:00 -07:00
Allen Wittenauer
d989ac0444 MAPREDUCE-5972. Fix typo 'programatically' in job.xml (and a few other places) (Akira AJISAKA via aw) 2014-09-08 13:15:46 -07:00
Allen Wittenauer
6c790a7ab0 HDFS-6893. crypto subcommand is not sorted properly in hdfs\'s hadoop_usage (David Luo via aw) 2014-09-08 13:10:53 -07:00
Colin Patrick Mccabe
cad14aa916 HDFS-6036. Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long. (cmccabe) 2014-09-08 12:51:56 -07:00
Kihwal Lee
6a84f88c11 HDFS-7005. DFS input streams do not timeout. Contributed by Daryn Sharp. 2014-09-08 14:41:44 -05:00
Haohui Mai
c1f8323234 HDFS-6943. Improve NN allocateBlock log to include replicas' datanode IPs. Contributed by Ming Ma. 2014-09-08 11:07:23 -07:00
arp
ccdf0054a3 HDFS-6977. Delete all copies when a block is deleted from the block space. (Arpit Agarwal) 2014-09-08 10:35:54 -07:00
Tsz-Wo Nicholas Sze
f1432e2424 Merge branch 'trunk' into HDFS-6584 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2014-09-08 10:54:48 +08:00
arp
eb8284d50e Merge trunk into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
 2014-09-07 14:46:46 -07:00
cnauroth
a23144fd8a HDFS Credential Provider related Unit Test Failure. Contributed by Xiaoyu Yao. 2014-09-07 08:39:20 -07:00
arp
d1fa58292e HDFS-6898. DN must reserve space for a full block when an RBW block is created. (Contributed by Arpit Agarwal) 2014-09-06 21:04:29 -07:00
Tsz-Wo Nicholas Sze
22a41dce4a HDFS-6997: add more tests for data migration and replicaion. 2014-09-07 07:44:28 +08:00
arp
31bbeaf383 Merge remote-tracking branch 'apache-commit/trunk' into HDFS-6581 2014-09-06 12:28:22 -07:00
Konstantin V Shvachko
88209ce181 HDFS-6940. Refactoring to allow ConsensusNode implementation. 
Contributed by Konstantin Shvachko.
 2014-09-06 12:07:52 -07:00
Alejandro Abdelnur
3b35f81603 HDFS-6986. DistributedFileSystem must get delegation tokens from configured KeyProvider. (zhz via tucu) 2014-09-05 22:33:48 -07:00
arp
339d21f273 Merge trunk into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
 2014-09-05 21:36:25 -07:00
Allen Wittenauer
a149afd9e6 HADOOP-11052. hadoop_verify_secure_prereq's results aren't checked in bin/hdfs (aw) 2014-09-05 13:50:42 -07:00
arp
c5ef9c9e69 HDFS-6862. Add missing timeout annotations to tests. (Contributed by Xiaoyu Yao) 2014-09-05 13:50:42 -07:00
Jing Zhao
54767d3ff4 Fix TestOfflineEditsViewer after merging. 2014-09-05 12:05:22 -07:00
Jing Zhao
300cb12bef Merge trunk into HDFS-6584 2014-09-05 11:37:22 -07:00
Allen Wittenauer
7a62515c86 HADOOP-11052. hadoop_verify_secure_prereq's results aren't checked in bin/hdfs (aw) 2014-09-05 11:31:49 -07:00
arp
9609b7303a HDFS-6862. Add missing timeout annotations to tests. (Contributed by Xiaoyu Yao) 2014-09-05 11:10:58 -07:00
arp
9e941d9f99 HDFS-6831. Inconsistency between 'hdfs dfsadmin' and 'hdfs dfsadmin -help'. (Contributed by Xiaoyu Yao) 2014-09-05 10:45:54 -07:00
Jing Zhao
c6107f566f HDFS-6376. Distcp data between two HA clusters requires another configuration. Contributed by Dave Marion and Haohui Mai. 2014-09-05 10:40:02 -07:00
Vinayakumar B
772d1fb01e HDFS-6714. TestBlocksScheduledCounter#testBlocksScheduledCounter should shutdown cluster (vinayakumarb) 2014-09-05 10:25:09 +05:30
Jing Zhao
185200e709 HDFS-6876. Archival Storage: support set/get storage policy in DFSAdmin. Contributed by Jing Zhao. 2014-09-04 20:14:46 -07:00
Jing Zhao
6104520369 HDFS-6886. Use single editlog record for creating file + overwrite. Contributed by Yi Liu. 2014-09-04 18:54:38 -07:00
Jing Zhao
f7df24bb1b HDFS-6996. SnapshotDiff report can hit IndexOutOfBoundsException when there are nested renamed directory/file. Contributed by Jing Zhao. 2014-09-04 16:12:44 -07:00
Jing Zhao
e08701ec71 HDFS-6961. Archival Storage: BlockPlacementPolicy#chooseTarget should check each valid storage type in each choosing round. 2014-09-04 14:19:32 -07:00
Colin Patrick Mccabe
f4caedfcbf HDFS-6959 Make the HDFS home directory location customizable. Contributed by Yongjun Zhang 2014-09-03 19:30:30 -07:00
arp
3a0142bd77 HDFS-6848. Lack of synchronization on access to datanodeUuid in DataStorage#format. (Contributed by Xiaoyu Yao) 2014-09-03 17:17:06 -07:00
Jing Zhao
45d5b13256 Merge changes from trunk 2014-09-03 15:36:47 -07:00
arp
cb9b485075 HDFS-6930. Improve replica eviction from RAM disk. (Arpit Agarwal) 2014-09-03 13:53:01 -07:00
arp
08a8f2be81 Merge trunk into HDFS-6581 2014-09-03 10:53:37 -07:00
arp
762b04e994 HDFS-6950. Add Additional unit tests for HDFS-6581. (Contributed by Xiaoyu Yao) 2014-09-03 10:51:26 -07:00
Uma Maheswara Rao G
3425ae5d7e HDFS-2975. Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart). (Yi Liu via umamahesh) 2014-09-03 18:53:51 +05:30
arp
a6b32a3e78 Merge trunk into HDFS-6581 2014-09-02 16:25:32 -07:00
Haohui Mai
08a9ac7098 HDFS-6942. Fix typos in log messages. Contributed by Ray Chiang. 2014-09-02 16:22:18 -07:00
Colin Patrick Mccabe
727331becc HDFS-4257. The ReplaceDatanodeOnFailure policies could have a forgiving option. Contributed by szetszwo. 2014-09-02 15:14:33 -07:00
Andrew Wang
a0ccf83dfd HDFS-6954. With crypto, no native lib systems are too verbose. Contributed by Charles Lamb. 2014-09-02 14:22:20 -07:00
Andrew Wang
faa4455be5 HDFS-6634. inotify in HDFS. Contributed by James Thomas. 2014-09-02 14:02:29 -07:00
Jing Zhao
3e2a0b5446 HDFS-6969. Archival Storage: INode#getStoragePolicyID should always return the latest storage policy. Contributed by Jing Zhao. 2014-09-01 17:56:04 -07:00
arp
217dea7794 Merge remote-tracking branch 'apache-commit/trunk' into HDFS-6581 2014-09-01 11:52:59 -07:00
Andrew Wang
6066b1a455 HDFS-6972. TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration doesn't decode url correctly. Contributed by Yongjun Zhang. 2014-08-29 15:47:01 -07:00
arp
f65183eba3 Merge remote-tracking branch 'apache-commit/trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-08-29 13:37:17 -07:00
Aaron T. Myers
7eab2a29a5 HDFS-6774. Make FsDataset and DataStore support removing volumes. Contributed by Lei Xu. 2014-08-29 13:00:17 -07:00
arp
4ae8178c56 HDFS-6800. Support Datanode layout changes with rolling upgrade. (Contributed by James Thomas) 2014-08-29 00:26:13 -07:00
arp
c92837aeab HDFS-6931. Move lazily persisted replicas to finalized directory on DN startup. (Arpit Agarwal) 2014-08-28 23:13:46 -07:00
arp
4cf9afacbe HDFS-6960. Bugfix in LazyWriter, fix test case and some refactoring. (Arpit Agarwal) 2014-08-28 23:05:32 -07:00
arp
7e32be8768 Merge remote-tracking branch 'apache-commit/trunk' into HDFS-6581 
Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-08-28 19:06:46 -07:00
Allen Wittenauer
d8774cc577 HADOOP-11013. CLASSPATH handling should be consolidated, debuggable (aw) 2014-08-28 18:09:25 -07:00
Todd Lipcon
ab638e77b8 HDFS-6865. Byte array native checksumming on client side. Contributed by James Thomas. 2014-08-28 16:44:09 -07:00
arp
bbaa7dc28d HDFS-6928. 'hdfs put' command should accept lazyPersist flag for testing. (Arpit Agarwal) 2014-08-28 15:53:12 -07:00
arp
2e987148e0 HDFS-6929. NN periodically unlinks lazy persist files with missing replicas from namespace. (Arpit Agarwal) 2014-08-27 21:47:19 -07:00
arp
3f64c4aaf0 HDFS-6927. Initial unit tests for Lazy Persist files. (Arpit Agarwal) 2014-08-27 21:47:19 -07:00
arp
eb448e1439 HDFS-6926. DN support for saving replicas to persistent storage and evicting in-memory replicas. (Arpit Agarwal) 2014-08-27 21:47:19 -07:00
arp
a317bd7b02 HDFS-6925. DataNode should attempt to place replicas on transient storage first if lazyPersist flag is received. (Arpit Agarwal) 2014-08-27 21:47:19 -07:00
arp
c2354a7f81 HDFS-6923. Propagate LazyPersist flag to DNs via DataTransferProtocol. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
arp
042b33f20b HDFS-6922. Add LazyPersist flag to INodeFile, save it in FsImage and edit logs. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
arp
7f49537ba1 HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
arp
a7bcc95358 HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
Colin Patrick Mccabe
7a16731191 HADOOP-10957. The globber will sometimes erroneously return a permission denied exception when there is a non-terminal wildcard. 2014-08-27 19:47:10 -07:00
Jing Zhao
a26aa6bd07 HDFS-6944. Archival Storage: add retry and termination logic for Mover. Contributed by Jing Zhao. 2014-08-27 14:20:54 -07:00
Colin Patrick Mccabe
6962510f72 HDFS-6879. Adding tracing to Hadoop RPC. Contributed by Masatake Iwasaki. 2014-08-27 14:12:05 -07:00
Jing Zhao
b7ded466b0 HDFS-6920. Archival Storage: check the storage type of delNodeHintStorage when deleting a replica. Contributed by Tsz Wo Nicholas Sze. 2014-08-27 14:08:02 -07:00
Colin Patrick Mccabe
b6b95ff667 HDFS-6902. FileWriter should be closed in finally block in BlockReceiver#receiveBlock() (Tsuyoshi OZAWA via Colin Patrick McCabe) 2014-08-27 13:49:31 -07:00
Colin Patrick Mccabe
225569ece2 HDFS-4486. Add log category for long-running DFSClient notices. Contributed by Zhe Zhang. 2014-08-27 13:39:40 -07:00
Colin Patrick Mccabe
d805cc27a9 HDFS-6773. MiniDFSCluster should skip edit log fsync by default. Contributed by Stephen Chu. 2014-08-27 13:33:02 -07:00
Haohui Mai
26ebdd849b HDFS-6938. Cleanup javac warnings in FSNamesystem. Contributed by Charles Lamb. 2014-08-27 11:14:39 -07:00
Jing Zhao
8ea20b53a8 HDFS-6911. Archival Storage: check if a block is already scheduled in Mover. Contributed by Tsz Wo Nicholas Sze. 2014-08-27 10:38:10 -07:00
Jing Zhao
555900a9dc Merge branch 'trunk' into HDFS-6584 2014-08-27 10:32:30 -07:00
Jing Zhao
6b441d227a HDFS-6908. Incorrect snapshot directory diff generated by snapshot deletion. Contributed by Juan Yu and Jing Zhao. 2014-08-27 10:26:22 -07:00
arp
c5d9a4a91e HDFS-6694. TestPipelinesFailover.testPipelineRecoveryStress tests fail intermittently with various symptoms - debugging patch (Contributed by Yongjun Zhang) 2014-08-27 09:52:33 -07:00
Allen Wittenauer
9ec4a930f5 HADOOP-10996. Stop violence in the *_HOME (aw) 2014-08-27 07:00:31 -07:00
Arpit Agarwal
e871955765 HDFS-6899. Allow changing MiniDFSCluster volumes per DN and capacity per volume. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619970 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-23 06:01:17 +00:00
Chris Nauroth
a83d055f25 HDFS-4852. libhdfs documentation is out of date. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619967 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-23 05:30:21 +00:00
Arpit Agarwal
e6c3650070 HDFS-6829. DFSAdmin refreshSuperUserGroupsConfiguration failed in security cluster. (Contributed by zhaoyunjiong) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619882 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-22 18:14:55 +00:00
Alejandro Abdelnur
3aa3b0abc2 HDF-6905. fs-encryption merge triggered release audit failures. (clamb via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619667 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-22 05:17:22 +00:00
Jing Zhao
e69954d22c HDFS-6906. Archival Storage: Add more tests for BlockStoragePolicy. Contributed by Tsz Wo Nicholas Sze. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1619628 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 23:42:33 +00:00
Jing Zhao
603cbe5eea HDFS-6863. Archival Storage: Support migration for snapshot paths. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1619627 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 23:41:10 +00:00
Tsz-wo Sze
c92d869d02 Merge r1609845 through r1619277 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1619293 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 05:22:10 +00:00
Arpit Agarwal
6824abc19e HDFS-6758. Block writer should pass the expected block size to DataXceiverServer (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619275 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 01:13:49 +00:00
Arpit Agarwal
8f00897a4b HDFS-6878. Change MiniDFSCluster to support StorageType configuration for individual directories. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619271 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 00:42:27 +00:00
Chris Nauroth
8304001861 HDFS-6858. Allow dfs.data.transfer.saslproperties.resolver.class default to hadoop.security.saslproperties.resolver.class. Contributed by Benoy Antony. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619256 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 22:46:35 +00:00
Andrew Wang
834fccf02b Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1619194 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 18:19:13 +00:00
Jing Zhao
f20d363126 HDFS-6870. Blocks and INodes could leak for Rename with overwrite flag. Contributed by Yi Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619192 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 18:13:03 +00:00
Andrew Wang
d2a39b61aa Merge from trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1619018 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 01:22:42 +00:00
Brandon Li
fbc1ee4d6f HDFS-6868. portmap and nfs3 are documented as hadoop commands instead of hdfs. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619001 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-19 22:31:57 +00:00
Allen Wittenauer
31467453ec HADOOP-9902. Shell script rewrite (aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618847 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-19 12:11:17 +00:00
Tsz-wo Sze
4e324bcd3d Merge r1609845 through r1618763 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618764 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 23:41:53 +00:00
Brandon Li
2fb04d2a30 HDFS-6569. OOB message can't be sent to the client when DataNode shuts down for upgrade. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618742 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 21:23:08 +00:00
Arpit Agarwal
1b5139b29a HDFS-6188. An ip whitelist based implementation of TrustedChannelResolver. (Contributed by Benoy Antony) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618727 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 20:50:06 +00:00
Andrew Wang
0cc08f6da4 Merge from trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618700 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:41:31 +00:00
Andrew Wang
bd616552cc HDFS-6825. Edit log corruption due to delayed block removal. Contributed by Yongjun Zhang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618684 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:08:42 +00:00
Jing Zhao
5d5aae0694 HDFS-6801. Archival Storage: Add a new data migration tool. Contributed by Tsz Wo Nicholas Sze. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618675 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 17:51:18 +00:00
Arpit Agarwal
1c77c80410 HDFS-6783. Addendum patch to fix test failures. (Contributed by Yi Liu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618635 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 15:34:58 +00:00
Tsz-wo Sze
cb75b6b07d Merge r1609845 through r1618416 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618417 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-16 21:02:21 +00:00
Tsz-wo Sze
9b250d74f0 HDFS-6847. Support storage policy on directories and include storage policy in HdfsFileStatus. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618416 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-16 20:58:20 +00:00
Charles Lamb
20dcb841ce HDFS-6546. Add non-superuser capability to get the encryption zone for a specific path. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618022 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 19:11:06 +00:00
Uma Maheswara Rao G
44864c68b5 HDFS-6783. Fix HDFS CacheReplicationMonitor rescan logic. Contributed by Yi Liu and Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617872 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 04:20:00 +00:00
Vinayakumar B
195961a7c1 HDFS-6247. Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer (vinayakumarb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617799 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 18:43:05 +00:00
Vinayakumar B
6554994fab Reverted 
Merged revision(s) 1617784 from hadoop/common/trunk:
HDFS-6847. Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer (Contributed by Vinayakumar B.)
........


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617794 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 18:36:51 +00:00
Vinayakumar B
471b1368e2 HDFS-6847. Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer (Contributed by Vinayakumar B.) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617784 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 18:06:33 +00:00
Haohui Mai
03d8ff496a HDFS-6567. Normalize the order of public final in HdfsFileStatus. Contributed by Tassapol Athiapinya. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617779 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 17:45:47 +00:00
Allen Wittenauer
a34dafe325 HADOOP-8944. Shell command fs -count should include human readable option (Jonathan Allen via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617775 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 17:30:26 +00:00
Arpit Agarwal
a72fba5853 HDFS-6836. HDFS INFO logging is verbose & uses file appenders. (Contributed by Xiaoyu Yao) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617603 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 21:49:48 +00:00
Arpit Agarwal
662c5bb3af HDFS-6830. BlockInfo.addStorage fails when DN changes the storage for a block replica. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617598 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 21:42:21 +00:00
Jing Zhao
37207b75d4 HDFS-6835. Archival Storage: Add a new API to set storage policy. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1617568 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 19:32:09 +00:00
Charles Lamb
74f7be0887 merge from trunk r1617527 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1617532 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 17:02:07 +00:00
Jing Zhao
04884cd530 Merging r1616894 through r1617376 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1617377 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 22:27:50 +00:00
Haohui Mai
80691b073f HDFS-6838. Code cleanup for unnecessary INode replacement. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617361 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 21:28:33 +00:00
Jing Zhao
e60673697d HDFS-6837. Code cleanup for Balancer and Dispatcher. Contributed by Tsz Wo Nicholas Sze. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617337 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 18:01:14 +00:00
Jing Zhao
9d5f8fa68d Merging r1616428 through r1616893 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1616897 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 22:21:35 +00:00
Jing Zhao
c3cf331dc9 HDFS-6828. Separate block replica dispatching from Balancer. Contributed by Tsz Wo Nicholas Sze. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616889 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:33:57 +00:00
Arpit Agarwal
05d1bf4157 HDFS-6830. Revert accidental checkin 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616884 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:22:48 +00:00
Arpit Agarwal
a70c9de3f1 HDFS-6830. BlockInfo.addStorage fails when DN changes the storage for a block replica. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616883 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:20:19 +00:00
Ravi Prakash
6da830cfdf HDFS-6823. dfs.web.authentication.kerberos.principal shows up in logs for insecure HDFS 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616845 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 18:21:18 +00:00
Charles Lamb
e6bdb33784 HADOOP-10919. Copy command should preserve raw.* namespace extended attributes. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616840 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 17:58:01 +00:00
Chris Nauroth
70d453602c HDFS-573. Porting libhdfs to Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616814 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 16:26:45 +00:00
Uma Maheswara Rao G
1fa2d6c4ba HDFS-6834. Improve the configuration guidance in DFSClient when there are no Codec classes found in configs. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616721 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 11:35:16 +00:00
Arpit Agarwal
d3a2fe2807 HDFS-6772. Get DN storages out of blockContentsStale state faster after NN restarts. (Contributed by Ming Ma) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616680 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 05:41:38 +00:00
Haohui Mai
2ac640ec75 HDFS-6722. Display readable last contact time for dead nodes on NN webUI. Contributed by Ming Ma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616669 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 04:27:47 +00:00
Aaron Myers
d758be1f35 HDFS-6740. Make FSDataset support adding data volumes dynamically. Contributed by Lei Xu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616623 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 22:59:59 +00:00
Aaron Myers
eca80dca3e HDFS-6728. Dynamically add new volumes to DataStorage, formatted if necessary. Contributed by Lei Xu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616615 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 22:46:51 +00:00
Andrew Wang
311d2f0773 Merge from trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616589 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 20:26:52 +00:00
Arpit Agarwal
b98400df7b HDFS-6781. Separate HDFS commands from CommandsManual.apt.vm. (Contributed by Akira Ajisaka) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616575 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 19:42:09 +00:00
Tsz-wo Sze
b50cb5e056 Merge r1609845 through r1616427 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1616428 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 07:38:23 +00:00
Tsz-wo Sze
0ed8732fee HDFS-6812. Remove addBlock and replaceBlock from DatanodeDescriptor. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616426 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 07:30:53 +00:00
Tsz-wo Sze
83b9933db3 HDFS-6809. Move Balancer's inner classes MovedBlocks and Matcher as to standalone classes and separates KeyManager from NameNodeConnector. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616422 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 07:18:48 +00:00
Jing Zhao
efc73a0f13 HDFS-6791. A block could remain under replicated if all of its replicas are on decommissioned nodes. Contributed by Ming Ma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616306 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 19:03:07 +00:00
Allen Wittenauer
6fa66f2357 HDFS-6517. Remove hadoop-metrics2.properties from hdfs project (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616262 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 16:07:52 +00:00
Yi Liu
2b5e044424 HDFS-6817. Fix findbugs and other warnings. (yliu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616092 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 04:43:58 +00:00
Brandon Li
812ac91add HDFS-6790. DFSUtil Should Use configuration.getPassword for SSL passwords. Contributed by Larry McCay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616058 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 00:50:07 +00:00
Andrew Wang
ac47ad11de HDFS-6394. HDFS encryption documentation. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616016 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 21:49:31 +00:00
Andrew Wang
ac73d416f3 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615844 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 02:30:54 +00:00