big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8,810 Commits 2 Branches 2 Tags 264 MiB
e257b6dbb1
Commit Graph

2377 Commits

Author SHA1 Message Date
Suresh Srinivas
c00703dd08 HDFS-6150. Add inode id information in the logs to make debugging easier. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581914 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 16:32:44 +00:00
Haohui Mai
b5935fab44 HDFS-6130. NPE when upgrading namenode from fsimages older than -32. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581713 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 07:30:20 +00:00
Arpit Agarwal
1fbb04e367 HDFS-5910. Enhance DataTransferProtocol to allow per-connection choice of encryption/plain-text. (Contributed by Benoy Antony) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581688 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 04:11:18 +00:00
Haohui Mai
867e0f8ea9 HDFS-5196. Provide more snapshot information in WebUI. Contributed by Shinichi Yamashita. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581427 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 17:53:51 +00:00
Jing Zhao
56205ca7d7 HDFS-5840. Follow-up to HDFS-5138 to improve error handling during partial upgrade failures. Contributed by Aaron T. Myers, Suresh Srinivas, and Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581260 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 06:48:15 +00:00
Suresh Srinivas
3a61d25457 HDFS-6125. Cleanup unnecessary cast in HDFS code base. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581242 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 05:51:11 +00:00
Arpit Agarwal
c2ef7e239e HDFS-6124. Add final modifier to class members. (Contributed by Suresh Srinivas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581124 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 23:32:37 +00:00
Chris Nauroth
328fc86bdb HDFS-5846. Shuffle phase is slow in Windows - FadviseFileRegion::transferTo does not read disks efficiently. Contributed by Nikola Vujic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581091 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 22:16:48 +00:00
Jing Zhao
8076ff31eb HDFS-6135. In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581070 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 21:24:31 +00:00
Kihwal Lee
7a18c4a199 HDFS-3087. Decomissioning on NN restart can complete without blocks being replicated. Contributed by Rushabh S Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580886 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 15:39:00 +00:00
Chris Nauroth
00ceaade05 HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580173 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 05:32:24 +00:00
Arpit Agarwal
fc53af9c4c HDFS-6120. Fix and improve safe mode log messages. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580047 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 20:57:13 +00:00
Jing Zhao
9dab514b22 HDFS-6038. Allow JournalNode to handle editlog produced by new release with future layoutversion. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579813 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-20 23:06:06 +00:00
Andrew Wang
4f6397b4ac HDFS-6089. Standby NN while transitioning to active throws a connection refused error when the prior active NN process is suspended. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579692 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-20 17:43:16 +00:00
Tsz-wo Sze
7ec4308f82 HDFS-6129. When a replica is not found for deletion, do not throw an exception. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579670 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-20 15:50:38 +00:00
Haohui Mai
aa4a045925 HDFS-6127. WebHDFS tokens cannot be renewed in HA setup. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579546 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-20 06:47:58 +00:00
Tsz-wo Sze
cfb468332e HDFS-6123. Do not log stack trace for ReplicaAlreadyExistsException and SocketTimeoutException. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579396 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-19 21:04:21 +00:00
Jing Zhao
7817245d88 HDFS-6100. DataNodeWebHdfsMethods does not failover in HA mode. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579301 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-19 17:29:23 +00:00
Chris Nauroth
2b03ae9421 HDFS-6099. HDFS file system limits not enforced on renames. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579122 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-19 03:46:52 +00:00
Suresh Srinivas
d0f7985f5c HDFS-6117. Print file path information in FileNotFoundException on INode ID mismatch. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579104 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-19 00:59:45 +00:00
Tsz-wo Sze
02b25ce4ef HDFS-6068. Disallow snapshot names that are also invalid directory names. Contributed by sathish 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579004 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-18 18:34:19 +00:00
Chris Nauroth
6318afe3b9 HDFS-5516. WebHDFS does not require user name when anonymous http requests are disallowed. Contributed by Miodrag Radulovic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578549 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 20:25:10 +00:00
Colin McCabe
d265dd9eb0 HDFS-6107. When a block cannot be cached due to limited space on the DataNode, it becomes uncacheable (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578508 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 18:46:33 +00:00
Arpit Agarwal
809e8bf5b7 HDFS-6094. The same block can be counted twice towards safe mode threshold. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578478 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 17:37:55 +00:00
Colin McCabe
9a65a9aadc HDFS-6106. Reduce default for dfs.namenode.path.based.cache.refresh.interval.ms (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577798 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-15 07:52:19 +00:00
Andrew Wang
d37c31a2db HDFS-6102. Lower the default maximum items per directory to fix PB fsimage loading. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577426 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 04:52:03 +00:00
Colin McCabe
f730fa919e HDFS-6097. Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577350 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 22:30:42 +00:00
Andrew Wang
b849dd5df9 HDFS-6079. Timeout for getFileBlockStorageLocations does not work. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576979 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 22:54:57 +00:00
Colin McCabe
a3616c58dd HDFS-6086. Fix a case where zero-copy or no-checksum reads were not allowed even when the block was cached. (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576533 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 22:41:05 +00:00
Haohui Mai
b027ef8858 HDFS-6072. Clean up dead code of FSImage. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576513 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 21:44:38 +00:00
Chris Nauroth
8d29768704 HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576405 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 16:13:43 +00:00
Colin McCabe
bab90b2222 HDFS-6085. Improve CacheReplicationMonitor log messages a bit (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576194 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 06:24:52 +00:00
Jing Zhao
b674dfd480 HDFS-6077. running slive with webhdfs on secure HA cluster fails with unkown host exception. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576076 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 20:48:53 +00:00
Andrew Wang
daaa8f03f4 HDFS-6070. Cleanup use of ReadStatistics in DFSInputStream. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576047 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 19:04:05 +00:00
Colin McCabe
d59dbc9e38 HDFS-6071. BlockReaderLocal does not return -1 on EOF when doing a zero-length read on a short file. (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575797 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 02:41:45 +00:00
Andrew Wang
94a1632fcb HDFS-3405. Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages. Contributed by Vinayakumar B. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575611 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 21:25:13 +00:00
Andrew Wang
dbd22b23c2 Revert HDFS-3405 for recommit with correct renamed files 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575610 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 21:15:41 +00:00
Andrew Wang
98594ab787 Remove extra file from HDFS-3405. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575609 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 21:11:04 +00:00
Andrew Wang
0f595915a3 HDFS-3405. Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages. Contributed by Vinayakumar B. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575457 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 00:39:23 +00:00
Tsz-wo Sze
1f6c2b09c6 HDFS-6076. DataNode with SimulatedDataSet should not create DatanodeRegistration with namenode layout version and namenode node type. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575452 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 23:47:43 +00:00
Andrew Wang
90b399c4bd HDFS-5064. Standby checkpoints should not block concurrent readers. Contributed by Aaron Twining Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575448 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 23:35:47 +00:00
Chris Nauroth
b1f87bbabd HDFS-5986. Capture the number of blocks pending deletion on namenode webUI. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575384 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 19:37:15 +00:00
Chris Nauroth
b55c1aab7b HDFS-6069. Quash stack traces when ACLs are disabled. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575184 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 06:06:12 +00:00
Colin McCabe
173c115951 HDFS-6065. HDFS zero-copy reads should return null on EOF when doing ZCR (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575109 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 01:18:32 +00:00
Haohui Mai
90b1e25c7f HDFS-6084. DFSConfigKeys.DFS_BLOCKREPORT_INTERVAL_MSEC_DEFAULT is not updated with latest block report interval of 6 hrs. Contributed by Vinayakumar B. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575086 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-06 22:54:36 +00:00
Brandon Li
8955f718f1 HDFS-6060. NameNode should not check DataNode layout version. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575021 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-06 19:29:11 +00:00
Colin McCabe
5a3f614794 HDFS-6061. Allow dfs.datanode.shared.file.descriptor.path to contain multiple entries and fall back when needed (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574796 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-06 08:21:16 +00:00
Colin McCabe
95e0f616b9 HDFS-6057. DomainSocketWatcher.watcherThread should be marked as a daemon thread (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574787 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-06 07:57:19 +00:00
Haohui Mai
ad61eec072 HADOOP-10386. Log proxy hostname in various exceptions being thrown in a HA setup. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574716 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 23:09:47 +00:00
Aaron Myers
6adf7a0ecb HDFS-5898. Allow NFS gateway to login/relogin from its kerberos keytab. Contributed by Abin Shahab. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574693 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 22:20:39 +00:00
Haohui Mai
b46fbd0275 HDFS-5857. TestWebHDFS#testNamenodeRestart fails intermittently with NPE. Contributed By Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574683 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 21:51:24 +00:00
Jing Zhao
e1162ebb1f HDFS-5167. Add metrics about the NameNode retry cache. Contributed by Tsuyoshi OZAWA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574603 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 18:22:04 +00:00
Haohui Mai
e90687f90e HDFS-5321. Clean up the HTTP-related configuration in HDFS. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574270 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 00:28:21 +00:00
Tsz-wo Sze
291dbc3d27 svn merge --reintegrate https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535 back to trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574259 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 23:53:26 +00:00
Colin McCabe
1da076fd13 HDFS-6051. HDFS cannot run on Windows since short-circuit memory segment changes (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574246 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 23:33:29 +00:00
Michael Stack
c0a903da22 HDFS-6047 TestPread NPE inside in DFSInputStream hedgedFetchBlockByteRange (stack) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574205 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 21:07:03 +00:00
Jing Zhao
f9ebdea446 Merging r1573814 through r1574170 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1574171 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 18:22:26 +00:00
Colin McCabe
c94e43c6df HDFS-6046. add dfs.client.mmap.enabled (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573887 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 06:46:08 +00:00
Colin McCabe
33500fd304 HDFS-6040. fix DFSClient issue without libhadoop.so and some other ShortCircuitShm cleanups (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573883 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 06:31:43 +00:00
Chris Nauroth
2cfe554980 HDFS-6039. Uploading a File under a Dir with default acls throws "Duplicated ACLFeature". Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573879 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 06:08:25 +00:00
Jing Zhao
5d29338017 HDFS-6041. Downgrade/Finalize should rename the rollback image instead of purging it. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1573851 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 04:42:58 +00:00
Tsz-wo Sze
9ad8dd1e8a Merge r1569890 through r1573813 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1573821 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 23:51:58 +00:00
Arpit Agarwal
98983b2788 HDFS-6042. Fix rolling upgrade documentation and error messages. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1573785 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 22:31:17 +00:00
Haohui Mai
aa8a6e498a HDFS-6013. add rollingUpgrade information to latest UI. Contributed by Vinayakumar B. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1573700 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 19:51:33 +00:00
Haohui Mai
146bf6c01e HDFS-5866. '-maxSize' and '-step' option fail in OfflineImageViewer. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573694 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 19:40:45 +00:00
Colin McCabe
6465b0b55f add missing file for HDFS-5950 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573434 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 04:01:26 +00:00
Colin McCabe
dd049a2f60 HDFS-5950. The DFSClient and DataNode should use shared memory segments to communicate short-circuit information (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573433 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 03:58:37 +00:00
Tsz-wo Sze
0a7db7f179 Merge r1569890 through r1573119 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1573120 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-01 01:14:38 +00:00
Tsz-wo Sze
c69cc31adc HDFS-6034. Use DataNodeLayoutVersion for DN registration check and do not verify layout version if there is a rolling upgrade in progress. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1573119 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-01 01:14:02 +00:00
Haohui Mai
fec1e2eed9 HDFS-5956. A file size is multiplied by the replication factor in 'hdfs oiv -p FileDistribution' option. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573078 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 21:07:03 +00:00
Arpit Agarwal
d00605f8f0 HADOOP-10285. Admin interface to swap callqueue at runtime. (Contributed by Chris Li) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573052 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 19:48:31 +00:00
Aaron Myers
8e809b8c81 HDFS-6033. PBImageXmlWriter incorrectly handles processing cache directives. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573050 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 19:45:37 +00:00
Tsz-wo Sze
ee663fad14 Merge r1569890 through r1573042 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1573043 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 19:37:47 +00:00
Tsz-wo Sze
1bab14bb49 HDFS-6031. Add back the "-rollingUpgrade started" namenode startup option; otherwise, namenode cannot start when the layout version is changed. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1573040 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 19:33:08 +00:00
Colin McCabe
2333388e47 HDFS-6030. Remove an unused constructor in INode.java (yzhang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573031 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 18:47:31 +00:00
Jing Zhao
5c381ade4e HDFS-5339. WebHDFS URI does not accept logical nameservices when security is enabled. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573026 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 18:39:01 +00:00
Jing Zhao
5409908026 HDFS-6032. -rollingUpgrade query hits NPE after the NN restarts. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1572801 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 01:23:52 +00:00
Jing Zhao
e9a17c8ce0 HDFS-6029. Secondary NN fails to checkpoint after -rollingUpgrade prepare. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1572800 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 01:21:53 +00:00
Chris Nauroth
1fe2bd5534 HDFS-6028. Print clearer error message when user attempts to delete required mask entry from ACL. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572753 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 22:12:34 +00:00
Tsz-wo Sze
58bccdbc04 Merge r1569890 through r1572417 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1572418 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 03:11:07 +00:00
Jing Zhao
ccf0744243 HDFS-6023. Test whether the standby NN continues to checkpoint after the prepare stage. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1572337 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 23:17:53 +00:00
Andrew Wang
c080fec82b HDFS-3969. Small bug fixes and improvements for disk locations API. Contributed by Todd Lipcon and Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572284 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 21:55:47 +00:00
Tsz-wo Sze
9cc0d5d497 Merge r1569890 through r1572250 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1572251 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:55:50 +00:00
Jing Zhao
4ba2bdf340 HDFS-6018. Exception recorded in LOG when IPCLoggerChannel#close is called. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572183 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 18:23:10 +00:00
Jing Zhao
dedcc09e71 HDFS-6019. Standby NN might not checkpoint when processing the rolling upgrade marker. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1572182 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 18:20:53 +00:00
Kihwal Lee
c8182ea764 HDFS-6020. Fix the five findbugs warnings. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1572165 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 17:07:56 +00:00
Chris Nauroth
7be2c002b3 HDFS-5908. Change AclFeature to capture list of ACL entries in an ImmutableList. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572142 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 16:22:56 +00:00
Chris Nauroth
4449247410 HDFS-6008. Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571881 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 00:25:29 +00:00
Jing Zhao
e3d2e4c156 HDFS-6000. Avoid saving namespace when starting rolling upgrade. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571840 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 21:58:53 +00:00
Tsz-wo Sze
abc9a6dad5 Merge r1569890 through r1571813 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571814 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 20:24:57 +00:00
Chris Nauroth
39c09c43bd HDFS-6006. Remove duplicate code in FSNameSystem#getFileInfo. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571813 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 20:20:21 +00:00
Kihwal Lee
fa6e59891c HDFS-5498. Improve datanode startup time. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571797 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 19:27:53 +00:00
Kihwal Lee
57b28693ee HDFS-5924. Utilize OOB upgrade message processing for writes. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571792 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 19:24:15 +00:00
Kihwal Lee
6780b086d8 HDFS-6015. Fix TestBlockRecovery#testRaceBetweenReplicaRecoveryAndFinalizeBlock. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571785 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 18:51:14 +00:00
Kihwal Lee
f52fe68a2b HDFS-6014. Fix findbug warnings introduced by HDFS-5583. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571784 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 18:48:30 +00:00
Jing Zhao
df6e1ab491 HDFS-5939. WebHdfs returns misleading error code and logs nothing if trying to create a file with no DNs in cluster. Contributed by Yongjun Zhang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571781 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 18:36:46 +00:00
Chris Nauroth
6c9c3144dd HDFS-5623. NameNode: add tests for skipping ACL enforcement when permission checks are disabled, user is superuser or user is member of supergroup. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571745 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 17:02:59 +00:00
Tsz-wo Sze
2c8a6f5712 HDFS-6003. Add the new -rollingUpgrade startup option to the namenode usage message. Contributed by Vinayakumar B 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571596 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 07:50:38 +00:00
Tsz-wo Sze
98db64dd49 Merge r1569890 through r1571553 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571554 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 03:30:24 +00:00
Arpit Agarwal
440c3cd105 HDFS-5922. DN heartbeat thread can get stuck in tight loop. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571542 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 02:16:29 +00:00
Tsz-wo Sze
0809f32f4b Merge r1569890 through r1571508 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571509 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 01:16:30 +00:00
Kihwal Lee
1c6b5d2b58 HDFS-5583. Make DN send an OOB Ack on shutdown before restarting. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571491 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-24 23:38:04 +00:00
Michael Stack
036af54387 HDFS-5776 Support 'hedged' reads in DFSClient 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571467 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-24 22:34:55 +00:00
Michael Stack
17db74a1c1 HDFS-5776 Support 'hedged' reads in DFSClient 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571466 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-24 22:34:41 +00:00
Arpit Agarwal
c066cef587 HDFS-6004. Change DFSAdmin for rolling upgrade commands. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571463 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-24 22:05:07 +00:00
Arpit Agarwal
dbf14320c0 HDFS-6005. Simplify Datanode rollback and downgrade. (Contributed by Suresh Srinivas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571431 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-24 20:46:35 +00:00
Jing Zhao
8e7a2b8d5d HDFS-5999. Do not create rollback fsimage when it already exists. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1571096 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 21:35:07 +00:00
Tsz-wo Sze
3f7852bd27 HDFS-5994. Fix TestDataNodeRollingUpgrade. Contributed by Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570734 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 22:40:06 +00:00
Tsz-wo Sze
78325b08c5 Merge r1569890 through r1570692 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570694 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 20:08:08 +00:00
Tsz-wo Sze
0ad6cdd031 HDFS-5992. Fix NPE in MD5FileUtils and update editsStored for TestOfflineEditsViewer. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570690 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 20:02:27 +00:00
Chris Nauroth
4f8a487fcd HDFS-5981. loadGenerator exit code is not reliable. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570468 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 06:39:08 +00:00
Andrew Wang
b00817b54a HDFS-5988. Bad fsimage always generated after upgrade. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570429 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 02:48:40 +00:00
Jing Zhao
4da6de1ca3 HDFS-5982. Need to update snapshot manager when applying editlog for deleting a snapshottable directory. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570395 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 00:15:11 +00:00
Tsz-wo Sze
37afb4d683 Merge r1569890 through r1570391 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570392 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 23:47:58 +00:00
Arpit Agarwal
329c705181 HDFS-5987. Fix findbugs warnings in Rolling Upgrade branch. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570389 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 23:21:11 +00:00
Brandon Li
7568e2f193 HDFS-5944. LeaseManager:findLeaseWithPrefixPath can't handle path like /a/b/ and cause SecondaryNameNode failed do checkpoint. Contributed by Yunjiong Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570366 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 22:27:13 +00:00
Tsz-wo Sze
016d3b8171 HDFS-5984. Fix TestEditLog and TestStandbyCheckpoints. Contributed by jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570361 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 21:42:10 +00:00
Jing Zhao
8b196816d8 HADOOP-10348. Deprecate hadoop.ssl.configuration in branch-2, and remove it in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570295 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 18:34:24 +00:00
Kihwal Lee
b23f6cc1f2 HDFS-5962. Mtime and atime are not persisted for symbolic links. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570252 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 16:09:20 +00:00
Tsz-wo Sze
55aec006f4 Merge r1569890 through r1570083 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570084 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 04:38:30 +00:00
Tsz-wo Sze
2088584864 HDFS-5980. Rollback does not need to load edits. Contributed by jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570078 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 03:57:59 +00:00
Andrew Wang
077adb25b7 HDFS-5979. Typo and logger fix for fsimage PB code. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570070 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 03:25:08 +00:00
Arpit Agarwal
e2c0af8e7b HDFS-5742. DatanodeCluster (mini cluster of DNs) fails to start. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570067 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 03:20:29 +00:00
Arpit Agarwal
b95294416c HDFS-5483. NN should gracefully handle multiple block replicas on same DN. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570040 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 01:43:12 +00:00
Arpit Agarwal
50a0bedead HDFS-5976. Create unit tests for downgrade and finalize rolling upgrade. (Contributed by Haohui Mai) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1570019 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-20 00:47:15 +00:00
Arpit Agarwal
e891c55f8b HDFS-5963. TestRollingUpgrade#testSecondaryNameNode causes subsequent tests to fail. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1569993 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 23:56:02 +00:00
Arpit Agarwal
0369aff403 HDFS-5868. Make hsync implementation pluggable. (Contributed by Buddy Taylor) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569978 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 23:38:46 +00:00
Arpit Agarwal
2f341414dd HDFS-5318. Support read-only and read-write paths to shared replicas. (Contributed by Eric Sirianni) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569951 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 22:59:37 +00:00
Tsz-wo Sze
d69985d90b HDFS-5974. Fix compilation error, NameNodeLayoutVersion and DataNodeLayoutVersion after merge from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1569931 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 21:40:42 +00:00
Tsz-wo Sze
72c214c89b Merge r1555021 through r1569889 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1569890 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 19:57:36 +00:00
Tsz-wo Sze
377424e36a HDFS-5966. Fix rollback of rolling upgrade in NameNode HA setup. Contributed by jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1569885 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 19:41:22 +00:00
Chris Nauroth
b2832e8d94 HDFS-4685. Correct minor whitespace difference in FSImageSerialization.java in preparation for trunk merge. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1569860 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 18:05:27 +00:00
Chris Nauroth
909eb7e4f1 Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1569833 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:24:02 +00:00
Kihwal Lee
7e8d85ebb3 HDFS-5961. OIV cannot load fsimages containing a symbolic link. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569789 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 15:28:57 +00:00
Tsz-wo Sze
99b4caa888 Merge r1555021 through r1569522 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1569524 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 20:42:26 +00:00
Tsz-wo Sze
bc962d6df4 HDFS-5945. Add rolling upgrade information to fsimage; and disallow upgrade and rolling upgrade to be started simultaneously. Contributed by szetszwo & jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1569515 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 20:12:17 +00:00
Jing Zhao
a845a18c67 HDFS-5893. HftpFileSystem.RangeHeaderUrlOpener uses the default URLConnectionFactory which does not import SSL certificates. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569477 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 19:06:31 +00:00
Suresh Srinivas
bf5971b86a HDFS-5959. Fix typo at section name in FSImageFormatProtobuf.java. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569156 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 01:06:44 +00:00
Chris Nauroth
e288316acf Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1569142 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-17 23:22:32 +00:00
Chris Nauroth
2c7dfa29bd Fix javadoc in OfflineImageViewer. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569124 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-17 22:28:57 +00:00
Jing Zhao
470d4253b2 HDFS-5920. Support rollback of rolling upgrade in NameNode and JournalNodes. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1568563 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-15 00:28:21 +00:00
Brandon Li
ef90940594 HDFS-5716. Allow WebHDFS to use pluggable authentication filter. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568547 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-15 00:04:44 +00:00
Kihwal Lee
9830ef0d32 HDFS-5585. Provide admin commands for data node upgrade. Contributed by Kihwal Lee. (missing file) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1568524 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 21:20:34 +00:00
Kihwal Lee
1a68f13521 HDFS-5585. Provide admin commands for data node upgrade. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1568523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 21:20:02 +00:00
Arpit Agarwal
a795bc42d0 HDFS-5494. Merge Protobuf-based-FSImage code from trunk - fix build break after merge. (Contributed by Jing Zhao) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1568517 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:56:24 +00:00
Chris Nauroth
d8fd23fc46 Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1568497 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 19:30:52 +00:00
Jing Zhao
ba4b10354c Merging r1566359 through r1568420 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1568437 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 18:32:37 +00:00
Suresh Srinivas
72f63c8957 HDFS-5943. 'dfs.namenode.https-address' property is not loaded from configuration in federation setup. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568412 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 18:07:04 +00:00
Arpit Agarwal
5df82fa01d HDFS-5907. BlockPoolSliceStorage trash to handle block deletions during rolling upgrade. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1568346 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 16:37:42 +00:00
Chris Nauroth
a49dee06fb Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1567994 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 18:21:46 +00:00
Chris Nauroth
e291c7905f HDFS-5937. Fix TestOfflineEditsViewer on HDFS-4685 branch. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1567986 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 17:52:26 +00:00
Tsz-wo Sze
a4f152db96 Merge r1555021 through r1566358 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1567874 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 09:53:52 +00:00
Tsz-wo Sze
4f9bbaa301 HDFS-5889. When starting rolling upgrade, create a fs image for rollback so that the standby namenode can create checkpoints during upgrade. Contributed by szetszwo & jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1567861 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 08:04:58 +00:00
Colin McCabe
f0d64a078d HDFS-5940. Minor cleanups to ShortCircuitReplica, FsDatasetCache, and DomainSocketWatcher (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567835 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 03:10:48 +00:00
Jing Zhao
2624b20291 HDFS-5847. Consolidate INodeReference into a separate section. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567812 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 00:00:42 +00:00
Brandon Li
256adb2106 HDFS-5891. webhdfs should not try connecting the DN during redirection. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567810 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 23:57:35 +00:00
Chris Nauroth
ea0b21af15 HDFS-5933. Optimize the FSImage layout for ACLs. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1567785 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 22:56:32 +00:00
Chris Nauroth
fc14360b03 HDFS-5923. Do not persist the ACL bit in the FsPermission. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1567784 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 22:54:49 +00:00
Colin McCabe
b3af8edc26 HDFS-5938. Make BlockReaderFactory#BlockReaderPeer a static class to avoid a findbugs warning (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567767 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 22:01:53 +00:00
Colin McCabe
beb0d25d2a HDFS-5810. Unify mmap cache and short-circuit file descriptor cache (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567720 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 19:08:52 +00:00
Konstantin Shvachko
fa78a7a99a HDFS-4858. HDFS DataNode to NameNode RPC should timeout. Contributed by Henry Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567535 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 07:01:29 +00:00
Chris Nauroth
6d2a0aa1db HDFS-5925. ACL configuration flag must only reject ACL API calls, not ACLs present in fsimage or edits. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1567450 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-11 23:13:48 +00:00
Suresh Srinivas
93eafc8590 HDFS-5929. Add blockpool % usage to HDFS federated nn page. Contributed by Siqi Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567411 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-11 22:04:15 +00:00
Chris Nauroth
d03acc7560 HDFS-5914. Incorporate ACLs with the changes from HDFS-5698. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1566991 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-11 06:25:13 +00:00
Chris Nauroth
3bf2f04bac Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1566988 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-11 06:00:56 +00:00
Chris Nauroth
5c978a43c3 HDFS-5915. Refactor FSImageFormatProtobuf to simplify cross section reads. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566824 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 23:13:06 +00:00
Konstantin Shvachko
ff24753aa7 HDFS-4370. Fix typo Blanacer in DataNode. Contributed by Chu Tong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566422 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-09 21:07:28 +00:00
Konstantin Shvachko
204704a92d HDFS-5837. dfs.namenode.replication.considerLoad should consider decommissioned nodes. Contributed by Tao Luo. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566410 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-09 20:49:18 +00:00
Haohui Mai
2ed075e1bb Merging r1566042 through r1566364 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1566375 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-09 19:45:00 +00:00
Jing Zhao
a2edb11b68 HDFS-5698. Use protobuf to serialize / deserialize FSImage. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566359 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-09 19:18:51 +00:00
Chris Nauroth
c6505f0499 Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1566100 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 19:05:12 +00:00
Chris Nauroth
dd2eb97ddd HDFS-5899. Add configuration flag to disable/enable support for ACLs. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1566041 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 16:20:29 +00:00
Chris Nauroth
c89c516b95 HDFS-5616. NameNode: implement default ACL handling. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1565845 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 23:11:27 +00:00
Andrew Wang
60eca33e83 HDFS-5900. Cannot set cache pool limit of unlimited via CacheAdmin. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565841 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 22:58:23 +00:00
Brandon Li
300e4906af HDFS-5874. Should not compare DataNode current layout version with that of NameNode in DataStrorage. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1565750 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 18:25:41 +00:00
Tsz-wo Sze
ccf08d9dc8 Merge r1555021 through r1565516 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1565519 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:43:04 +00:00
Tsz-wo Sze
c780454413 HDFS-5869. When starting rolling upgrade or NN restarts, NN should create a checkpoint right before the upgrade marker. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1565516 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:08:05 +00:00
Chris Nauroth
12c2582c1f HDFS-5895. HDFS cacheadmin -listPools has exit_code of 1 when the command returns 0 result. Contributed by Tassapol Athiapinya. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565440 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 21:16:01 +00:00
Colin McCabe
e0cda48959 HDFS-4911. Reduce PeerCache timeout to be commensurate with dfs.datanode.socket.reuse.keepalive (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565435 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 21:08:09 +00:00
Kihwal Lee
ab96a0838d HDFS-5881. Fix skip() of the short-circuit local reader(legacy). Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565310 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 15:45:47 +00:00
Brandon Li
b2ce764093 HDFS-5890. Avoid NPE in Datanode heartbeat. Contributed by Vinay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1565023 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 00:40:24 +00:00
Jing Zhao
d598b6ef9f HDFS-5873. dfs.http.policy should have higher precedence over dfs.https.enable. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564973 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 22:48:06 +00:00
Jing Zhao
69dbf0b225 HDFS-5876. SecureDataNodeStarter does not pick up configuration in hdfs-site.xml. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564897 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 19:43:56 +00:00
Aaron Myers
d6bd920bba HDFS-5709. Improve NameNode upgrade with existing reserved paths and path components. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564645 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 06:48:00 +00:00
Todd Lipcon
0aa09f6d5a HDFS-5399. Revisit SafeModeException and corresponding retry policies. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564629 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:18:31 +00:00
Junping Du
94b512bc51 HDFS-5828. BlockPlacementPolicyWithNodeGroup can place multiple replicas on the same node group when dfs.namenode.avoid.write.stale.datanode is true. (Buddy via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563640 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-02 15:39:02 +00:00
Tsz-wo Sze
35e59eeaa2 Merge r1555021 through r1563384 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1563385 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 09:01:41 +00:00
Tsz-wo Sze
4f92eb2f61 HDFS-5848. Add rolling upgrade status to heartbeat response. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1563384 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 08:58:55 +00:00
Chris Nauroth
3fd9f29d56 HDFS-5860. Refactor INodeDirectory getDirectoryXFeature methods to use common getFeature helper method. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1563374 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 05:55:20 +00:00
Colin McCabe
140246bad8 HDFS-5746. Add ShortCircuitSharedMemorySegment (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563362 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 02:25:33 +00:00
Colin McCabe
f6bfe98639 HDFS-5859. DataNode#checkBlockToken should check block tokens even if security is not enabled (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563328 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 23:08:23 +00:00
Chris Nauroth
9e8cabf8ce Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1563327 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 23:05:23 +00:00
Chris Nauroth
14f1f76bf6 HDFS-5614. NameNode: implement handling of ACLs in combination with snapshots. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1563304 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 22:21:19 +00:00
Arpit Agarwal
5beeb30169 HDFS-5153. Datanode should send block reports for each storage in a separate message. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563254 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 21:00:33 +00:00
Chris Nauroth
c654c2e8b9 HDFS-5849. Removing ACL from an inode fails if it has only a default ACL. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1563205 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 18:33:56 +00:00
Suresh Srinivas
7039b776c6 HDFS-5856. DataNode.checkDiskError might throw NPE. Contributed by Josh Elser. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563064 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 05:35:30 +00:00
Tsz-wo Sze
f2972402b7 Merge r1555021 through r1563041 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1563042 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 03:24:24 +00:00
Tsz-wo Sze
00067895a0 HDFS-5754. Split LayoutVerion into NameNodeLayoutVersion and DataNodeLayoutVersion. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1563041 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 03:21:45 +00:00
Chris Nauroth
7a03dc0482 HDFS-5608. WebHDFS: implement ACL APIs. Contributed by Sachin Jose and Renil Joseph. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1563013 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 23:13:24 +00:00
Todd Lipcon
fd2ba4911b HDFS-5790. LeaseManager.findPath is very slow when many leases need recovery. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562970 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 21:21:15 +00:00
Jing Zhao
3d9ad8e3b6 HDFS-5843. DFSClient.getFileChecksum() throws IOException if checksum is disabled. Contributed by Laurent Goujon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562927 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 19:15:13 +00:00
Chris Nauroth
10ef8a4b56 Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1562670 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 01:55:14 +00:00
Andrew Wang
799ad3d670 HDFS-5841. Update HDFS caching documentation with new changes. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562649 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 00:17:16 +00:00
Andrew Wang
c96d078033 HDFS-5845. SecondaryNameNode dies when checkpointing with cache pools. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562644 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 23:59:21 +00:00
Jing Zhao
8c505adddb HDFS-5842. Cannot create hftp filesystem when using a proxy user ugi and a doAs on a secure cluster. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562603 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 21:53:32 +00:00
Suresh Srinivas
33a47743a5 HADOOP-10255. Rename HttpServer to HttpServer2 to retain older HttpServer in branch-2 for compatibility. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561959 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 07:32:52 +00:00
Tsz-wo Sze
efcdf81ca9 Merge r1555021 through r1561943 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1561944 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 05:37:13 +00:00
Tsz-wo Sze
917502ef31 HDFS-5835. Add a new option for starting Namenode when rolling upgrade is in progress. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1561943 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 05:35:47 +00:00
Arpit Agarwal
b8776ee65b HDFS-5833. Fix incorrect javadoc in SecondaryNameNode. (Contributed by Bangtao Zhou) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561938 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 05:09:57 +00:00
Colin McCabe
07c004db06 HDFS-5830. WebHdfsFileSystem.getFileBlockLocations throws IllegalArgumentException when accessing another cluster. (Yongjun Zhang via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561885 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-27 22:19:30 +00:00
Jing Zhao
4b61ff2283 HDFS-5781. Use an array to record the mapping between FSEditLogOpCode and the corresponding byte value. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561788 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-27 18:53:39 +00:00
Chris Nauroth
ad240d6b3a Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1561771 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-27 18:09:22 +00:00
Tsz-wo Sze
a9110e1788 Merge r1555021 through r1561510 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1561516 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 16:34:30 +00:00
Todd Lipcon
edb6dc5f30 HDFS-5138. Support HDFS upgrade in HA. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561381 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-25 20:01:26 +00:00
Uma Maheswara Rao G
befb254e61 HDFS-5343. When cat command is issued on snapshot files, getting unexpected result. Contributed by Sathish. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561325 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-25 15:50:42 +00:00
Kihwal Lee
dd1bc7e1c7 HDFS-5728. Block recovery will fail if the metafile does not have crc for all chunks of the block. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561223 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-24 22:56:05 +00:00
Chris Nauroth
738b076cc6 Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1560768 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 17:49:24 +00:00
Chris Nauroth
cd7cd94e3f HDFS-5799. Make audit logging consistent across ACL APIs. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1560766 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 17:41:53 +00:00
Daryn Sharp
a4e66e09e9 HDFS-5241. Provide alternate queuing audit logger to reduce logging contention (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560761 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 17:15:27 +00:00
Kihwal Lee
ff43021946 HDFS-5788. listLocatedStatus response can be very large. Contributed by Nathan Roberts. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560750 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 17:01:30 +00:00
Uma Maheswara Rao G
824dcdbd42 HDFS-5789. Some of snapshot APIs missing checkOperation double check in fsn. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560731 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 16:35:05 +00:00
Andrew Wang
1c602f7198 HDFS-5806. Balancer should set soTimeout to avoid indefinite hangs. Contributed by Nathan Roberts. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560548 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:33:00 +00:00
Arpit Agarwal
83280a45de HDFS-5434. Change block placement policy constructors from package private to protected. (Buddy Taylor via Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560217 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-21 23:11:15 +00:00
Tsz-wo Sze
d8bc523754 HDFS-5753. Add new NN startup options for downgrade and rollback using upgrade marker. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1559907 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-21 02:38:46 +00:00
Chris Nauroth
2fbe8235dc Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1559794 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-20 18:02:27 +00:00
Tsz-wo Sze
602f71a8da HDFS-5800. Fix a typo in DFSClient.renewLease(). Contributed by Kousuke Saruta 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559701 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-20 14:08:00 +00:00
Chris Nauroth
8a9a6dbd7e Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1559324 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-18 06:50:19 +00:00
Tsz-wo Sze
1025121874 Merge r1555021 through r1559304 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1559305 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-18 02:24:38 +00:00
Tsz-wo Sze
09b8ce5b44 HDFS-5786. Support QUERY and FINALIZE actions of rolling upgrade. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1559304 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-18 02:07:09 +00:00
Haohui Mai
6a52febfbd HDFS-5612. NameNode: change all permission checks to enforce ACLs in addition to permissions. Contributued by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1559281 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 23:28:45 +00:00
Haohui Mai
5300294fbe HDFS-5758. NameNode: complete implementation of inode modifications for ACLs. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1559267 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 22:05:47 +00:00
Jing Zhao
fb01200153 HDFS-5794. Fix the inconsistency of layout version number of ADD_DATANODE_AND_STORAGE_UUIDS between trunk and branch-2. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559209 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 18:18:36 +00:00
Colin McCabe
a7ec44d41b HDFS-5784. Reserve space in edit log header and fsimage header for feature flag section (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558974 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 01:32:05 +00:00
Jing Zhao
e02ded811f HDFS-5777. Update LayoutVersion for the new editlog op OP_ADD_BLOCK. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558675 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 02:28:10 +00:00
Brandon Li
a506df8e48 HDFS-5775. Consolidate the code for serialization in CacheManager. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558599 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 23:15:24 +00:00
Brandon Li
ca5d73d1ab HDFS-5768. Consolidate the serialization code in DelegationTokenSecretManager. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558598 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 23:11:01 +00:00
Colin McCabe
c3b236ce5d HDFS-5766. In DFSInputStream, do not add datanode to deadNodes after InvalidEncryptionKeyException in fetchBlockByteRange (Liang Xie via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558536 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 19:23:31 +00:00
Colin McCabe
037a89abc5 HDFS-5762. BlockReaderLocal does not return -1 on EOF when doing zero-length reads (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558526 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 19:17:50 +00:00
Tsz-wo Sze
4a1abe5a3d Merge r1555021 through r1558254 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1558303 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 06:06:31 +00:00
Tsz-wo Sze
5fe73a986e HDFS-5752. Add a new DFSAdmin command to query, start and finalize rolling upgrade. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1558299 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 05:56:17 +00:00
Jing Zhao
578dae9ef3 HDFS-5704. Change OP_UPDATE_BLOCKS with a new OP_ADD_BLOCK. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558255 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-15 01:06:53 +00:00
Konstantin Boudnik
4b16c2b65e HDFS-5677. Need error checking for HA cluster configuration. Contributed by Vincent Sheffer. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558217 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-14 21:59:51 +00:00
Jing Zhao
e210519d32 HDFS-5579. Under construction files make DataNode decommission take very long hours. Contributed by zhaoyunjiong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557904 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-14 02:02:15 +00:00
Jing Zhao
e06ae2d567 HDFS-5710. FSDirectory#getFullPathName should check inodes against null. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557803 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-13 18:37:56 +00:00
Chris Nauroth
c90211af7a Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1557294 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 00:37:23 +00:00
Arpit Agarwal
9d382a4174 HDFS-5747. Fix NPEs in BlockManager. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557289 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 00:05:55 +00:00
Tsz-wo Sze
1c7aa44d62 Merge r1555021 through r1557038 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1557039 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 06:07:24 +00:00
Tsz-wo Sze
4563ba08e7 HDFS-5645. Support upgrade marker in editlog streams. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1557038 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 06:06:10 +00:00
Kihwal Lee
b4d6c5823b HDFS-5449. WebHdfs compatibility broken between 2.2 and 1.x / 23.x. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556927 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-09 19:24:07 +00:00
Junping Du
7186000367 HDFS-5721. sharedEditsImage in Namenode#initializeSharedEdits() should be closed before method returns. (Ted Yu via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556803 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-09 13:33:08 +00:00
Chris Nauroth
17ab35314f Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1556665 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-08 23:10:02 +00:00
Chris Nauroth
21d4167179 HDFS-5739. ACL RPC must allow null name or unspecified permissions in ACL entries. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1556663 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-08 22:47:57 +00:00
Chris Nauroth
023c11ec7e HDFS-5737. Replacing only the default ACL can fail to copy unspecified base entries from the access ACL. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1556652 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-08 22:02:53 +00:00
Tsz-wo Sze
f3ee35ab28 Merge r1555021 through r1556550 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1556552 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-08 14:36:09 +00:00
Jing Zhao
076ecf79ca HDFS-5726. Fix compilation error in AbstractINodeDiff for JDK7. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556433 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-08 03:19:41 +00:00
Colin McCabe
53b27fc60b HDFS-5724. modifyCacheDirective logging audit log command wrongly as addCacheDirective (Uma Maheswara Rao G via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556386 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 22:27:06 +00:00
Jing Zhao
70cff9e2f0 HDFS-5715. Use Snapshot ID to indicate the corresponding Snapshot for a FileDiff/DirectoryDiff. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556353 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 20:52:48 +00:00
Chris Nauroth
2fbb3d694e Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1556097 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 01:07:33 +00:00
Chris Nauroth
c658567571 HDFS-5673. Implement logic for modification of ACLs. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1556090 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 00:47:49 +00:00
Brandon Li
bfd227bf09 HDFS-5719. FSImage#doRollback() should close prevState before return. Contributed by Ted Yu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556057 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 22:58:18 +00:00
Colin McCabe
8deb7a6057 HDFS-5589. Namenode loops caching and uncaching when data should be uncached. (awang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555996 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 19:45:02 +00:00
Colin McCabe
2a1ecd00da HDFS-5220. Expose group resolution time as metric (jxiang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555976 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 18:59:10 +00:00
Arpit Agarwal
f8a9329f2b HDFS-5667. Include DatanodeStorage in StorageReport. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555929 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 17:28:23 +00:00
Brandon Li
b80343a55f HDFS-5705. TestSecondaryNameNodeUpgrade#testChangeNsIDFails may fail due to ConcurrentModificationException. Contributed by Ted Yu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555190 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-03 18:29:44 +00:00
Tsz-wo Sze
498f9674ff Merge r1550130 through r1555020 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1555021 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-03 07:26:52 +00:00
Andrew Wang
d85c017d04 HDFS-5651. Remove dfs.namenode.caching.enabled and improve CRM locking. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555002 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-03 02:45:53 +00:00
Andrew Wang
b4eb963c3c HDFS-5659. dfsadmin -report doesn't output cache information properly. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554893 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-02 20:11:41 +00:00
Andrew Wang
07e4fb1455 HDFS-5708. The CacheManager throws a NPE in the DataNode logs when processing cache reports that refer to a block not known to the BlockManager. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554594 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-01 00:01:12 +00:00
Uma Maheswara Rao G
97e881b955 HDFS-5671. Fix socket leak in DFSInputStream#getBlockReader. Contributed by JamesLi 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554553 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-31 16:24:02 +00:00
Andrew Wang
7f86c8114e HDFS-5701. Fix the CacheAdmin -addPool -maxTtl option name. Contributed by Stephen Chu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554305 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-30 20:02:36 +00:00
Colin McCabe
e596f17734 HDFS-5582. hdfs getconf -excludeFile or -includeFile always failed (satish via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554295 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-30 19:35:43 +00:00
Chris Nauroth
eaa5321619 Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1553226 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-23 22:19:12 +00:00
Chris Nauroth
c3b56ed1c8 HDFS-5619. NameNode: record ACL modifications to edit log. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1553224 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-23 22:12:45 +00:00
Chris Nauroth
25e9d2e895 HDFS-5618. NameNode: persist ACLs in fsimage. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1553221 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-23 21:45:03 +00:00
Jing Zhao
e8de319789 HDFS-5690. DataNode fails to start in secure mode when dfs.http.policy equals to HTTP_ONLY. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553167 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-23 18:23:43 +00:00
Colin McCabe
b9ae3087c0 HDFS-5636. Enforce a max TTL per cache pool (awang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552841 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-20 23:27:20 +00:00
Kihwal Lee
04d139e2a0 HDFS-5681. renewLease should not hold fsn write lock. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552711 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-20 16:05:03 +00:00
Chris Nauroth
72b0b1881f Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1552467 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-20 01:01:18 +00:00
Chris Nauroth
c552d5d72e HDFS-5685. Implement ACL as a INode feature. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1552465 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-20 00:48:24 +00:00
Michael Stack
b0947f9305 HDFS-5663 make the retry time and interval value configurable in openInfo() 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552232 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-19 05:21:46 +00:00
Jing Zhao
a9d80ae59d HDFS-5661. Browsing FileSystem via web ui, should use datanode's fqdn instead of ip address. Contributed by Benoy Antony. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552177 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-19 00:27:16 +00:00
Colin McCabe
90122f25e1 HDFS-5676. fix inconsistent synchronization of CachingStrategy (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552162 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-18 23:29:05 +00:00
Brandon Li
fc966461e0 HDFS-5662. Can't decommission a DataNode due to file's replication factor larger than the rest of the cluster size. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552131 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-18 22:26:33 +00:00
Jing Zhao
788fca4124 HDFS-5496. Make replication queue initialization asynchronous. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1552109 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-18 21:30:37 +00:00
Tsz-wo Sze
504bd0bca3 HDFS-5674. Editlog code cleanup: remove @SuppressWarnings("deprecation") in FSEditLogOp; change FSEditLogOpCodes.fromByte(..) to be more efficient; and change Some fields in FSEditLog to final. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551812 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-18 02:46:00 +00:00
Colin McCabe
124e507674 HDFS-5634. Allow BlockReaderLocal to switch between checksumming and not (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551701 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 20:57:00 +00:00
Colin McCabe
991c453ca3 HDFS-5431. Support cachepool-based limit management in path-based caching. (awang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551651 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 18:47:04 +00:00
Andrew Wang
0fc2929d13 HDFS-5350. Name Node should report fsimage transfer time as a metric. Contributed by Jimmy Xiang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551415 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 00:30:06 +00:00
Chris Nauroth
740605cde6 Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1551332 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 20:08:58 +00:00
Colin McCabe
8e32e6aff1 HDFS-5666. Fix inconsistent synchronization in BPOfferService (jxiang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551301 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 18:35:40 +00:00
Arpit Agarwal
89a374afcb HDFS-5454. DataNode UUID should be assigned prior to FsDataset initialization. (Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551296 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 18:16:56 +00:00
Andrew Wang
6c641339d1 HDFS-5665. Remove the unnecessary writeLock while initializing CacheManager in FsNameSystem Ctor. (Uma Maheswara Rao G via Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551270 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 17:12:55 +00:00
Uma Maheswara Rao G
1dabd42999 HDFS-5592. statechangeLog of completeFile should be logged only in case of success. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551145 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 09:53:57 +00:00
Arpit Agarwal
938565925a HDFS-5406. Send incremental block reports for all storages in a single call. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551093 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 00:58:40 +00:00
Tsz-wo Sze
44a6560b5d HDFS-5632. Flatten INodeDirectory hierarchy: Replace INodeDirectoryWithSnapshot with DirectoryWithSnapshotFeature. Contributed by jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550917 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-14 10:13:30 +00:00
Chris Nauroth
f24e0c1563 Merge trunk to HDFS-4685. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1550774 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 17:28:14 +00:00
Junping Du
a31a3a68ea HDFS-5652. Refactor invalid block token exception handling in DFSInputStream. (Liang Xie via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550620 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 02:47:04 +00:00
Chris Nauroth
9263f3bb14 HDFS-5596. Implement RPC stubs. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1550544 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 21:47:46 +00:00
Jing Zhao
5fd7230671 HDFS-5647. Merge INodeDirectory.Feature and INodeFile.Feature. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550469 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 18:26:40 +00:00
Arpit Agarwal
fba994ffe2 HDFS-5648. Get rid of FsDatasetImpl#perVolumeReplicaMap. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1550357 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 07:01:24 +00:00
Arpit Agarwal
c656562556 Merging r1550313 through r1550355 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1550356 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 06:51:54 +00:00
Junping Du
8373c4134a HDFS-5637. Try to refeatchToken while local read InvalidToken occurred. (Liang Xie via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550335 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 03:04:43 +00:00
Arpit Agarwal
cd083aa807 Merging r1549949 through r1550312 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1550313 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 23:18:29 +00:00
Colin McCabe
9678020e59 HDFS-4201. NPE in BPServiceActor#sendHeartBeat (jxiang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550269 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 21:30:16 +00:00
Junping Du
327a995561 HDFS-5580. Fix infinite loop in Balancer.waitForMoveCompletion. (Binglin Chang via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550074 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 08:51:12 +00:00
Aaron Myers
512a18a8d9 HDFS-5074. Allow starting up from an fsimage checkpoint in the middle of a segment. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550016 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 01:32:22 +00:00
Arpit Agarwal
f3cef1c673 Merging r1549626 through r1549948 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1549949 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-10 19:50:02 +00:00
Jing Zhao
87a6db45b7 HDFS-5629. Support HTTPS in JournalNode and SecondaryNameNode. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1549692 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-09 21:58:47 +00:00
Arpit Agarwal
43c33491fc Merging r1548329 through r1549625 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1549626 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-09 17:38:20 +00:00
Jing Zhao
155020a6f4 HDFS-5353. Short circuit reads fail when dfs.encrypt.data.transfer is enabled. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548987 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 03:52:18 +00:00
Jing Zhao
91d0b47270 HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by Yongjun Zhang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548968 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-07 21:10:53 +00:00
Tsz-wo Sze
4c87a27ad8 HDFS-5554. Flatten INodeFile hierarchy: Replace INodeFileWithSnapshot with FileWithSnapshotFeature. Contributed by jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548796 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-07 06:17:46 +00:00
Jing Zhao
045dc880e1 HDFS-5312. Generate HTTP/HTTPS URL in DFSUtil#getInfoServer() based on the configured http policy. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548629 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-06 18:01:58 +00:00
Jing Zhao
9df84c35d5 HDFS-5590. Block ID and generation stamp may be reused when persistBlocks is set to false. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548368 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-06 01:55:13 +00:00
Jing Zhao
6b459506c3 HDFS-5633. Improve OfflineImageViewer to use less memory. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548359 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-06 01:30:22 +00:00
Arpit Agarwal
00718c2ffa Merging r1547658 through r1548328 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1548329 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 23:41:09 +00:00
Andrew Wang
55e5b0653c HDFS-5630. Hook up cache directive and pool usage statistics. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548309 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 21:09:30 +00:00
Daryn Sharp
99aed805f8 Neglected to add new file in HDFS-5514 (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548167 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 15:31:35 +00:00
Daryn Sharp
6828337d85 HDFS-5514. FSNamesystem's fsLock should allow custom implementation (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548161 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 15:28:12 +00:00
Colin McCabe
950e0644b7 HDFS-5626. dfsadmin report shows incorrect values (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548000 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 02:59:14 +00:00
Andrew Wang
859e425dfa Revert HDFS-4983 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547970 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 00:05:10 +00:00
Andrew Wang
f5e83a0b3e HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by Yongjun Zhang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547935 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-04 22:24:45 +00:00
Jing Zhao
d02baff9a0 HDFS-5536. Implement HTTP policy for Namenode and DataNode. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547925 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-04 21:40:57 +00:00
Colin McCabe
f791e291ca HDFS-5555. CacheAdmin commands fail when first listed NameNode is in Standby (jxiang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547895 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-04 20:06:25 +00:00
Arpit Agarwal
674d51e62e Merging r1547121 through r1547473 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1547492 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-03 17:33:46 +00:00
Arpit Agarwal
a1aa1836fb HDFS-5484. StorageType and State in DatanodeStorageInfo in NameNode is not accurate. (Contributed by Eric Sirianni) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1547462 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-03 16:30:29 +00:00
Kihwal Lee
7af12ab207 HDFS-5558. LeaseManager monitor thread can crash if the last block is complete but another block is not. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547393 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-03 14:12:43 +00:00
Kihwal Lee
38a04a3042 HDFS-5557. Write pipeline recovery for the last packet in the block may cause rejection of valid replicas. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547173 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 20:18:52 +00:00
Arpit Agarwal
18159be495 Merging r1544666 through r1547120 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1547122 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 17:41:44 +00:00
Colin McCabe
13331a6863 HDFS-5581. NameNodeFsck should use only one instance of BlockPlacementPolicy (vinay via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547088 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 16:31:03 +00:00
Andrew Wang
9da451cac5 HDFS-5430. Support TTL on CacheDirectives. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546301 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-28 07:20:21 +00:00
Andrew Wang
bb11d47758 HDFS-5562. TestCacheDirectives and TestFsDatasetCache should stub out native mlock. Contributed by Colin Patrick McCabe and Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546246 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-28 00:44:54 +00:00
Tsz-wo Sze
8df119da21 HDFS-5537. Remove FileWithSnapshot interface. Contributed by jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546184 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 20:32:16 +00:00
Jing Zhao
2214871d91 HDFS-5545. Allow specifying endpoints for listeners in HttpServer. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546151 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 18:20:14 +00:00
Colin McCabe
13edb391d0 HDFS-5556. Add some more NameNode cache statistics, cache pool stats (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546143 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 17:55:52 +00:00
Uma Maheswara Rao G
b6d483b122 HDFS-5568. Support includeSnapshots option with Fsck command. Contributed by Vinay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545987 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 10:59:54 +00:00
Colin McCabe
05a9a80bd4 HDFS-5565. CacheAdmin help should match against non-dashed commands (wang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545850 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-26 21:38:46 +00:00
Jing Zhao
740cf232bd HDFS-5561. FSNameSystem#getNameJournalStatus() in JMX should return plain text instead of HTML. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545791 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-26 19:28:09 +00:00
Tsz-wo Sze
82ff2d3f2e HDFS-5286. Flatten INodeDirectory hierarchy: Replace INodeDirectoryWithQuota with DirectoryWithQuotaFeature. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545768 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-26 18:33:22 +00:00
Jing Zhao
d8a2383461 HDFS-5538. URLConnectionFactory should pick up the SSL related configuration by default. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545491 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-26 01:16:51 +00:00
Jing Zhao
eb3c1607b0 HDFS-5533. Symlink delete/create should be treated as DELETE/CREATE in snapshot diff report. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545357 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-25 18:05:36 +00:00
Kihwal Lee
65ee88b0de HDFS-5526. Datanode cannot roll back to previous layout version. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545322 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-25 15:33:52 +00:00
Arpit Agarwal
f48ea1b516 Merging r1544304 through r1544665 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1544672 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-22 20:51:06 +00:00
Arpit Agarwal
97acde2d33 HDFS-5542. Fix TODO and clean up the code in HDFS-2832. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1544664 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-22 20:07:10 +00:00
Tsz-wo Sze
ce68f410b0 HDFS-5285. Flatten INodeFile hierarchy: Replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot with FileUnderContructionFeature. Contributed by jing9 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544389 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-22 01:39:02 +00:00
Arpit Agarwal
b33ff7b5f2 HDFS-5547. Fix build break after merge from trunk to HDFS-2832. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1544342 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 22:07:46 +00:00
Arpit Agarwal
16f7fb80d9 Merging r1543902 through r1544303 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1544306 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 20:06:09 +00:00
Colin McCabe
f91a45a96c HDFS-5473. Consistent naming of user-visible caching classes and methods (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544252 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 17:12:58 +00:00
Jing Zhao
39950cee7f HDFS-5532. Enable the webhdfs by default to support new HDFS web UI. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544047 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 07:16:16 +00:00
Tsz-wo Sze
5f458ef23f HDFS-5531. Combine the getNsQuota() and getDsQuota() methods in INode. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544018 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 03:17:47 +00:00
Jing Zhao
3cae2ba63f HDFS-3987. Support webhdfs over HTTPS. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543962 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 21:51:58 +00:00
Andrew Wang
916ab9286b HDFS-5451. Add byte and file statistics to PathBasedCacheEntry. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543958 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 21:31:41 +00:00
Arpit Agarwal
75747754d2 Merging r1543710 through r1543901 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1543902 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 18:37:22 +00:00
Arpit Agarwal
2d9692a36d HDFS-5527. Fix TestUnderReplicatedBlocks on branch HDFS-2832. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1543885 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 17:13:35 +00:00
Uma Maheswara Rao G
04cf2a768c HDFS-5014. Process register commands with out holding BPOfferService lock. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543861 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 16:27:28 +00:00
Uma Maheswara Rao G
f26d2adbf9 HDFS-4516. Client crash after block allocation and NN switch before lease recovery for the same file can cause readers to fail forever. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543829 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 14:43:11 +00:00
Arpit Agarwal
59393e0162 Merging r1543613 through r1543709 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1543710 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 05:31:02 +00:00
Colin McCabe
efea68dc35 HDFS-5511. improve CacheManipulator interface to allow better unit testing (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543676 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 00:48:05 +00:00
Andrew Wang
00d9b8eedb HDFS-5513. CacheAdmin commands fail when using . as the path. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543670 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-20 00:29:59 +00:00
Jonathan Turner Eagles
587f68b160 HDFS-1386. TestJMXGet fails in jdk7 (jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543612 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 22:44:13 +00:00
Arpit Agarwal
3c9cbc6893 Merging r1543111 through r1543509 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1543510 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 17:26:23 +00:00
Jing Zhao
aa002344d0 HDFS-5428. Under construction files deletion after snapshot+checkpoint+nn restart leads nn safemode. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543329 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 07:31:40 +00:00
Colin McCabe
70234e2213 HDFS-5512. CacheAdmin -listPools fails with NPE when user lacks permissions to view all pools (awang via cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543293 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 02:35:34 +00:00
Colin McCabe
4f15d0af4f HDFS-5520. loading cache path directives from edit log doesnt update nextEntryId (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543286 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 02:01:28 +00:00
Arpit Agarwal
bd5ab41a90 Merging r1542123 through r1543110 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1543115 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-18 19:22:42 +00:00
Uma Maheswara Rao G
e3d7ef36ef HDFS-5372. In FSNamesystem, hasReadLock() returns false if the current thread holds the write lock (Contributed by Vinay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542887 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-18 04:18:44 +00:00
Jing Zhao
fef8f49c58 HDFS-5502. Fix HTTPS support in HsftpFileSystem. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542438 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-16 01:00:42 +00:00
Arpit Agarwal
cd768489f3 HDFS-5515. Fix TestDFSStartupVersions for HDFS-2832. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1542176 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-15 05:21:58 +00:00
Jing Zhao
620890fcc0 HDFS-5489. Use TokenAspect in WebHDFSFileSystem. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542158 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-15 02:11:38 +00:00
Arpit Agarwal
132a8ff7c7 Merging r1541618 through r1542122 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1542125 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 23:56:56 +00:00
Kihwal Lee
ceea91c9cd HDFS-5438. Flaws in block report processing can cause data loss. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542054 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 20:11:50 +00:00
Jing Zhao
43fa41fdee HDFS-5506. Use URLConnectionFactory in DelegationTokenFetcher. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542011 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 18:40:36 +00:00
Arpit Agarwal
ea1c5ea086 HDFS-5510. Fix a findbug warning in DataStorage.java on HDFS-2832 branch. (Contributed by Junping Du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1542006 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 18:16:02 +00:00
Kihwal Lee
fe67e30bc2 HDFS-4995. Make getContentSummary less expensive. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541971 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 16:49:31 +00:00
Jing Zhao
735aae32e4 HDFS-5504. In HA mode, OP_DELETE_SNAPSHOT is not decrementing the safemode threshold, leads to NN safemode. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541773 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 01:03:34 +00:00
Jing Zhao
ced740e580 HDFS-5474. Deletesnapshot can make Namenode in safemode on NN restarts. Contributed by Sathish. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541685 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 19:59:55 +00:00
Jing Zhao
1c211f6749 HDFS-5440. Extract the logic of handling delegation tokens in HftpFileSystem to the TokenAspect class. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541665 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 19:25:09 +00:00
Colin McCabe
3c591aa442 HDFS-5366. recaching improvements (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541647 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 18:18:37 +00:00
Arpit Agarwal
bf9f8511fa Merging r1541342 through r1541617 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1541620 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 17:15:36 +00:00
Colin McCabe
39def49661 HDFS-5485. add command-line support for modifyDirective (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541377 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 03:29:02 +00:00
Arpit Agarwal
d06a782c1e HDFS-5501. Fix pendingReceivedRequests tracking in BPServiceActor. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1541371 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 02:59:46 +00:00
Arpit Agarwal
46cbce9af1 HDFS-5508. Fix compilation error after merge. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1541352 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 02:00:41 +00:00
Tsz-wo Sze
ba98e8f737 merge r1535792 through r1541341 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1541342 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 01:14:37 +00:00
Andrew Wang
9d06631719 HDFS-5450. better API for getting the cached blocks locations. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541338 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-13 00:31:55 +00:00
Colin McCabe
ce35e0950c HDFS-5471. CacheAdmin -listPools fails when user lacks permissions to view all pools (Andrew Wang via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541323 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-12 23:52:23 +00:00
Jing Zhao
8162fdcdbc HDFS-5425. Renaming underconstruction file with snapshots can make NN failure on restart. Contributed by Vinay and Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541261 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-12 22:10:09 +00:00
Arpit Agarwal
e5b3171c3b HDFS-5494. Fix findbugs warnings for HDFS-2832. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1540924 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-12 02:36:24 +00:00
Arpit Agarwal
ec5eebc450 Merging r1540548 through r1540909 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1540910 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-12 01:16:10 +00:00
Arpit Agarwal
26f2bd67f2 HDFS-5491. Update editsStored for HDFS-2832. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1540903 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-12 00:48:55 +00:00
Andrew Wang
9673baa7e8 HDFS-5320. Add datanode caching metrics. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540796 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-11 18:30:07 +00:00
Arpit Agarwal
bb5bd30df6 Merging r1540239 through r1540547 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1540549 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-10 21:01:12 +00:00
Arpit Agarwal
907fb15ee8 HDFS-5480. Update Balancer for HDFS-2832. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1540547 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-10 20:59:32 +00:00
Chris Nauroth
25ebe6bfd7 HDFS-5482. DistributedFileSystem#listPathBasedCacheDirectives must support relative paths. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540257 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-09 05:54:19 +00:00
Tsz-wo Sze
fccbb5072c merge of r1535792 through r1540238 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1540239 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-09 01:34:22 +00:00
Jing Zhao
37bc8e0c14 HDFS-5325. Remove WebHdfsFileSystem#ConnRunner. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540235 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-09 01:19:54 +00:00
Jing Zhao
f2ad787a27 HDFS-5476. Snapshot: clean the blocks/files/directories under a renamed file/directory while deletion. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540142 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-08 18:18:03 +00:00
Colin McCabe
97199baea1 HDFS-5394: Fix race conditions in DN caching and uncaching (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539909 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-08 03:00:19 +00:00
Arpit Agarwal
6b0611ed24 Merging r1539737 through r1539896 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1539898 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-08 01:44:24 +00:00
Arpit Agarwal
06d24efed0 HDFS-5475. NN incorrectly tracks more than one replica per DN. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1539890 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-08 01:01:18 +00:00
Colin McCabe
f79b3e6b17 HDFS-5326. add modifyDirective to cacheAdmin (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539839 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-07 22:07:16 +00:00
Colin McCabe
77afc605fd HDFS-5468. CacheAdmin help command does not recognize commands (Stephen Chu via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539786 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-07 19:58:52 +00:00
Jing Zhao
9964a94386 HDFS-5443. Delete 0-sized block when deleting an under-construction file that is included in snapshot. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539754 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-07 18:37:38 +00:00
Arpit Agarwal
a604e3b73b HDFS-5472. Fix TestDatanodeManager, TestSafeMode and TestNNThroughputBenchmark. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1539736 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-07 17:57:30 +00:00
Tsz-wo Sze
75777f1626 HDFS-5470. Add back trunk's reportDiff algorithm to the branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1539504 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-07 00:13:05 +00:00
Arpit Agarwal
fa5ba6d977 HDFS-5439. Fix TestPendingReplication. (Contributed by Junping Du, Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1539247 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-06 06:50:33 +00:00
Arpit Agarwal
103fd6c6d8 Merging r1538408 through r1539244 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1539245 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-06 06:14:13 +00:00
Arpit Agarwal
3e3fea7cd4 HDFS-5466. Update storage IDs when the pipeline is updated. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1539203 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-06 00:25:10 +00:00
Andrew Wang
8639ecb26d HDFS-5458. Datanode failed volume threshold ignored if exception is thrown in getDataDirsFromURIs. Contributed by Mike Mellenthin. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539091 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 18:27:00 +00:00
Jing Zhao
a6250a4943 HDFS-5427. Not able to read deleted files from snapshot directly under snapshottable dir after checkpoint and NN restart. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538875 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 06:38:50 +00:00
Chris Nauroth
c47b3d7502 HDFS-5456. NameNode startup progress creates new steps if caller attempts to create a counter for a step that doesn't already exist. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538872 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 06:04:07 +00:00
Arpit Agarwal
d2b7b6589d HDFS-5457. Fix TestDatanodeRegistration, TestFsck and TestAddBlockRetry. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1538794 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-04 22:49:00 +00:00
Arpit Agarwal
1d357365b7 HDFS-5455. NN should update storageMap on first heartbeat. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1538787 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-04 21:47:51 +00:00
Arpit Agarwal
0091606e2d HDFS-5448. Fix break caused by previous checkin for HDFS-5448. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1538655 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-04 16:00:12 +00:00
Arpit Agarwal
9660bfa84c HDFS-5448. Datanode should generate its ID on first registration 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1538496 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-04 02:13:04 +00:00
Tsz-wo Sze
7e46025f91 Merge r1537584 through r1538407 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1538408 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-03 17:51:52 +00:00
Tsz-wo Sze
26a1fda51e HDFS-5452. Fix TestReplicationPolicy and TestBlocksScheduledCounter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1538407 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-03 17:49:08 +00:00
Colin McCabe
7abf7e8151 HADOOP-9478. Fix race conditions during the initialization of Configuration related to deprecatedKeyMap (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538248 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-02 18:42:27 +00:00
Uma Maheswara Rao G
833da65d86 HDFS-5344. Make LsSnapshottableDir as Tool interface implementation. Contributed by Sathish. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538156 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-02 09:02:35 +00:00
Arpit Agarwal
cbdeba29bd HDFS-5447. Fix TestJspHelper in branch HDFS-2832. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1538145 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-02 05:21:48 +00:00
Andrew Wang
3b91b7dece HDFS-5037. Active NN should trigger its own edit log rolls. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538059 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-01 20:49:54 +00:00
Arpit Agarwal
2af5083a35 HDFS-5437. Fix TestBlockReport and TestBPOfferService failures. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1537365 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-31 02:32:25 +00:00
Arpit Agarwal
4eebf56ccf Merging r1536890 through r1537251 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1537252 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-30 19:28:44 +00:00
Andrew Wang
75a162ff92 HDFS-5433. When reloading fsimage during checkpointing, we should clear existing snapshottable directories. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537192 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-30 17:27:51 +00:00
Arpit Agarwal
68a79b0d3f HDFS-5436. Move HsFtpFileSystem and HFtpFileSystem into org.apache.hdfs.web. (Contributed by Haohui Mai) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536921 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 22:44:34 +00:00
Arpit Agarwal
c2495a7bff Merging r1536572 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1536889 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 21:04:31 +00:00
Colin McCabe
1b48f44feb Merge HDFS-4949 branch back into trunk 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536572 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 00:49:20 +00:00
Arpit Agarwal
dc0b44a884 HDFS-5435. File append fails to initialize storageIDs. (Junping Du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1536434 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-28 17:29:06 +00:00
Arpit Agarwal
67b93d6e25 Merging r1535792 through r1536181 from trunk to HDFS-2832. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1536182 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-27 18:43:58 +00:00
Arpit Agarwal
b9d561c548 HDFS-5214. Fix NPEs in BlockManager and DirectoryScanner. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1536179 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-27 18:29:49 +00:00
Jing Zhao
5829029154 HDFS-5257. addBlock() retry should return LocatedBlock with locations else client will get AIOBE. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535811 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-25 18:43:06 +00:00
Tsz-wo Sze
ec01db59ce HDFS-5417. Fix storage IDs in PBHelper and UpgradeUtilities. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1535791 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-25 17:21:49 +00:00
Andrew Wang
dc2ee20aec HDFS-5419. Fixup test-patch.sh warnings on HDFS-4949 branch. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1535607 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-25 01:56:05 +00:00
Andrew Wang
e87b2a3684 merge trunk into HDFS-4949 branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1535563 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-24 21:38:25 +00:00
Arpit Agarwal
1fcc24d56a Merging r1535122 through r1535532 from trunk to HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1535534 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-24 20:33:04 +00:00
Andrew Wang
222dbfe2ce HDFS-5405. Fix possible RetryCache hang for caching RPC handlers in FSNamesystem. (Contributed by Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1535282 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-24 05:40:28 +00:00
Kihwal Lee
bf3271bd2b HDFS-5341. Reduce fsdataset lock duration during directory scanning. Contributed by Qus-Jiawei. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535188 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 21:29:51 +00:00
Jing Zhao
1387fb1394 HDFS-5363. Refactor WebHdfsFileSystem: move SPENGO-authenticated connection creation to URLConnectionFactory. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535169 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 20:48:25 +00:00
Arpit Agarwal
f39f8c5734 HDFS-5401. Fix NPE in Directory Scanner. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1535158 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 20:28:54 +00:00
Tsz-wo Sze
b442fe92fb Merge r1534894 through r1535121 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1535122 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 19:00:35 +00:00
Aaron Myers
82ed72d1d4 HDFS-5403. WebHdfs client cannot communicate with older WebHdfs servers post HDFS-5306. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535056 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 15:15:29 +00:00
Arpit Agarwal
01f37e42f0 HDFS-5390. Send one incremental block report per storage directory. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1534891 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 01:28:48 +00:00
Arpit Agarwal
b884af72c5 HDFS-5398. NameNode changes to process storage reports per storage directory. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1534847 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 23:29:40 +00:00
Arpit Agarwal
ef17685f79 Merging r1534279 through r1534706 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1534707 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 17:24:52 +00:00
Colin McCabe
d45ca49e3d HDFS-5400. DFS_CLIENT_MMAP_CACHE_THREAD_RUNS_PER_TIMEOUT constant is set to the wrong value (Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534655 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 15:10:17 +00:00
Arpit Agarwal
0ebab3a88a HDFS-5377. Heartbeats from Datandode should include one storage report per storage directory 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1534464 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 02:22:17 +00:00
Tsz-wo Sze
df87ed34f2 HDFS-4885. Improve the verifyBlockPlacement() API in BlockPlacementPolicy. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534426 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 00:20:39 +00:00
Tsz-wo Sze
40fec97bef svn merge -c -1534417 . for reverting HDFS-4885, which should be committed to trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1534424 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-22 00:19:04 +00:00
Tsz-wo Sze
12ed24af66 HDFS-4885. Improve the verifyBlockPlacement() API in BlockPlacementPolicy. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1534417 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 23:48:08 +00:00
Jing Zhao
fc99f3e8a3 HDFS-5382. Implement the UI of browsing filesystems in HTML 5 page. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534368 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 20:42:13 +00:00
Colin McCabe
69e5f90e9f HDFS-5385. Caching RPCs are AtMostOnce, but do not persist client ID and call ID to edit log. (Chris Nauroth via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1534345 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 19:42:35 +00:00
Colin McCabe
f9c08d02eb HDFS-5378. In CacheReport, don't send genstamp and length on the wire (Contributed by Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1534334 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 19:29:46 +00:00
Arpit Agarwal
62f95b003b Merging r1533208 through r1534278 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1534279 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-21 17:11:34 +00:00
Uma Maheswara Rao G
7fd9148800 HDFS-5331 make SnapshotDiff.java to a o.a.h.util.Tool interface implementation. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533917 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-20 15:23:10 +00:00
Colin McCabe
d61af97810 HDFS-5203. Concurrent clients that add a cache directive on the same path may prematurely uncache each other. (Chris Nauroth via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1533651 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-18 22:15:26 +00:00
Colin McCabe
1d96e36013 HDFS-5388. Loading fsimage fails to find cache pools during namenode startup (Chris Nauroth via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1533616 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-18 20:20:20 +00:00
Jonathan Turner Eagles
a19b956975 HDFS-4511. Cover package org.apache.hadoop.hdfs.tools with unit test (Andrey Klochkov via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533270 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-17 21:49:30 +00:00
Suresh Srinivas
631ccbdd20 HDFS-5374. Remove deadcode in DFSOutputStream. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533258 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-17 20:47:31 +00:00
Andrew Wang
dcb0b85333 HDFS-5383. fix broken caching unit tests (Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1533253 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-17 20:31:13 +00:00
Arpit Agarwal
f8d5755a69 Merging r1532911 through r1533207 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1533208 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-17 18:54:53 +00:00
Chris Nauroth
1fde8bcc85 HDFS-5336. DataNode should not output StartupProgress metrics. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533183 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-17 18:06:01 +00:00
Andrew Wang
34f08944b7 merge trunk to branch HDFS-4949 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1532952 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-17 02:14:33 +00:00
Tsz-wo Sze
1ba58e2e2c HDFS-4376. Fix race conditions in Balancer. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532932 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 23:06:00 +00:00
Colin McCabe
40c97caf19 add missing file for HDFS-5096 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1532925 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 22:15:47 +00:00
Colin McCabe
3cc7a38a53 HDFS-5096. Automatically cache new data added to a cached path (contributed by Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1532924 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 22:15:33 +00:00
Kihwal Lee
a0fc90ea92 HDFS-5346. Avoid unnecessary call to getNumLiveDataNodes() for each block during IBR processing. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532915 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 21:14:28 +00:00
Arpit Agarwal
a6076adbef Merging r1532877 through r1532910 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1532913 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 21:12:17 +00:00
Arpit Agarwal
168b133337 Merging r1527684 through r1532876 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1532910 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 21:07:28 +00:00
Suresh Srinivas
dea1b2e84a HADOOP-10005. No need to check INFO severity level is enabled or not. Contributed by Jackie Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532907 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 21:00:07 +00:00
Suresh Srinivas
f28f5ed628 HDFS-5370. Typo in Error Message: different between range in condition and range in error message. Contributed by Kousuke Saruta. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532899 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 20:45:03 +00:00
Chris Nauroth
8da82eba1c HDFS-5373. hdfs cacheadmin -addDirective short usage does not mention -replication parameter. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1532888 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 20:23:14 +00:00
Tsz-wo Sze
ffe347151e HDFS-5283. Under construction blocks only inside snapshots should not be counted in safemode threshhold. Contributed by Vinay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532857 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 18:38:31 +00:00
Tsz-wo Sze
0a5f0efcb4 HDFS-5338. Add a conf to disable hostname check in datanode registration. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532468 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-15 18:21:22 +00:00
Andrew Wang
efe545b0c2 HDFS-5358. Add replication field to PathBasedCacheDirective. (Contributed by Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1532124 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-14 22:56:11 +00:00
Colin McCabe
15d08c4778 HDFS-5349. DNA_CACHE and DNA_UNCACHE should be by blockId only (cmccabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1532116 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-14 22:19:10 +00:00
Jing Zhao
cf0cf0a691 HDFS-5342. Provide more information in the FSNamesystem JMX interfaces. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532090 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-14 20:56:37 +00:00
Brandon Li
7cd044b1aa HDFS-5329. Update FSNamesystem#getListing() to handle inode path in startAfter token. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531617 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-13 04:28:49 +00:00
Andrew Wang
09e9e57a0b HDFS-5348. Fix error message when dfs.datanode.max.locked.memory is improperly configured. (Contributed by Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1531460 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-11 23:13:13 +00:00
Jing Zhao
f7eaacc103 HDFS-5322. HDFS delegation token not found in cache errors seen on secure HA clusters. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531436 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-11 21:49:01 +00:00
Chris Nauroth
8111c3af6b HDFS-5224. Refactor PathBasedCache* methods to use a Path rather than a String. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1531406 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-11 19:44:20 +00:00
Luke Lu
d0aaa5ba9f HDFS-5276. Remove volatile from LightWeightHashSet. (Junping Du via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531225 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-11 08:29:06 +00:00
Suresh Srinivas
f2f5cdb555 HDFS-5335. Hive query failed with possible race in dfs output stream. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531152 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-10 23:58:08 +00:00
Colin McCabe
b8ae4e2a8e HDFS-5323. Remove some deadcode in BlockManager (Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530814 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 22:27:27 +00:00
Andrew Wang
3fc8792b5c HDFS-5304. Expose if a block replica is cached in getFileBlockLocations. (Contributed by Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1530802 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 21:30:08 +00:00
Brandon Li
09998741ac HDFS-5316. Namenode ignores the default https port. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530150 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-08 04:26:01 +00:00
Brandon Li
61c1dd96c3 HDFS-5317. Go back to DFS Home link does not work on datanode webUI. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530114 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-08 00:07:12 +00:00
Jing Zhao
1fe1942328 HDFS-5291. Standby namenode after transition to active goes into safemode. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530112 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-07 23:58:44 +00:00
Colin McCabe
b60e18db74 HDFS-5314. Do not expose CachePool type in AddCachePoolOp (Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1530073 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-07 21:26:01 +00:00
Brandon Li
267a606831 HDFS-5307. Support both HTTP and HTTPS in jsp pages. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530027 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-07 19:02:55 +00:00
Jing Zhao
7317e97bd7 HDFS-5299. DFS client hangs in updatePipeline RPC when failover happened. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529660 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 18:39:03 +00:00
Jing Zhao
cb5a51565a HADOOP-10017. Fix NPE in DFSClient#getDelegationToken when doing Distcp from a secured cluster to an insecured cluster. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529571 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 06:15:31 +00:00
Brandon Li
8e08046661 HDFS-5306. Datanode https port is not available at the namenode. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529562 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 03:22:57 +00:00
Jing Zhao
cae1ed9f6c HDFS-5308. Replace HttpConfig#getSchemePrefix with implicit schemes in HDFS JSP. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529512 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-05 20:40:48 +00:00
Andrew Wang
eb2175db1a HDFS-5190. Move cache pool related CLI commands to CacheAdmin. (Contributed by Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1529334 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 22:28:23 +00:00
Jing Zhao
3c1f18de43 HDFS-5300. FSNameSystem#deleteSnapshot() should not check owner in case of permissions disabled. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529294 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 20:54:06 +00:00
Andrew Wang
af1ac9a5e8 HDFS-5119. Persist CacheManager state in the edit log. (Contributed by Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1529238 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 17:46:18 +00:00
Chris Nauroth
fc5bd930df HDFS-5279. Guard against NullPointerException in NameNode JSP pages before initialization of FSNamesystem. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528308 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 05:25:18 +00:00
Suresh Srinivas
703838c598 HADOOP-10012. Secure Oozie jobs fail with delegation token renewal exception in Namenode HA setup. Contributed by Daryn Sharp and Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528301 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 04:00:06 +00:00
Arpit Agarwal
b99d160437 HDFS-5255. Distcp job fails with hsftp when https is enabled in insecure cluster. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528279 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 00:03:15 +00:00
Arpit Agarwal
6331ff024c Merging r1526971 through r1527683 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1527697 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-30 18:28:07 +00:00
Chris Nauroth
eccdb9aa8b HDFS-5260. Merge zero-copy memory-mapped HDFS client reads to trunk and branch-2. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527113 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-27 22:51:12 +00:00