big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,658 Commits 2 Branches 2 Tags 264 MiB
d088d2ddea
Commit Graph

823 Commits

Author SHA1 Message Date
Todd Lipcon
1cea56d6ef Merge trunk into QJM branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383030 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-10 18:45:45 +00:00
Aaron Myers
16234cbd5f HDFS-3897. QJM: TestBlockToken fails after HDFS-3893. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1381842 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-07 02:01:21 +00:00
Aaron Myers
df801074c9 HDFS-3893. QJM: Make QJM work with security enabled. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1381770 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 21:42:14 +00:00
Daryn Sharp
82b981cca4 HDFS-3890. filecontext mkdirs doesn't apply umask as expected (Tom Graves via daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381606 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 14:06:58 +00:00
Todd Lipcon
cae8116a14 HDFS-3726. If a logger misses an RPC, don't retry that logger until next segment. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1381482 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 07:03:57 +00:00
Todd Lipcon
437948ea1c HDFS-3891. Make selectInputStreams throw IOE instead of RTE. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1381481 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 06:57:18 +00:00
Eli Collins
01db4d2bb2 HDFS-3828. Block Scanner rescans blocks too frequently. Contributed by Andy Isaacson 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381472 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 06:19:30 +00:00
Eli Collins
91e832e7b4 HADOOP-8770. NN should not RPC to self to find trash defaults. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381319 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 19:42:09 +00:00
Todd Lipcon
99ec5bd8d3 Merge trunk into QJM branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380990 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:57:47 +00:00
Todd Lipcon
31142aa892 HDFS-1490. TransferFSImage should timeout. Contributed by Dmytro Molkov and Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380988 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:53:15 +00:00
Todd Lipcon
c334cc89a8 HDFS-2793. Add an admin command to trigger an edit log roll. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380982 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:40:23 +00:00
Todd Lipcon
13daca1ef6 HDFS-3870. Add metrics to JournalNode. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380980 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:30:51 +00:00
Todd Lipcon
f6b7f067c3 HDFS-3884. Journal format() should reset cached values. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380979 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:27:49 +00:00
Todd Lipcon
72485f3112 HDFS-3869. Expose non-file journal manager details in web UI. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380978 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:16:02 +00:00
Todd Lipcon
8021d9199f HDFS-3863. Track last "committed" txid in QJM. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380976 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:13:19 +00:00
Tsz-wo Sze
f3927595cc HDFS-3887. Remove redundant chooseTarget methods in BlockPlacementPolicy. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380934 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-04 23:48:01 +00:00
Suresh Srinivas
da3bd67138 HDFS-3880. Use Builder to build RPC server in HDFS. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379917 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-02 06:20:41 +00:00
Daryn Sharp
94c69f2c82 HDFS-3852. TestHftpDelegationToken is broken after HADOOP-8225 (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379623 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 21:31:55 +00:00
Daryn Sharp
3819c964fa HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379615 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 21:24:48 +00:00
Suresh Srinivas
2fd49f5a57 HDFS-3833. TestDFSShell fails on windows due to concurrent file read/write. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379525 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 17:43:35 +00:00
Eli Collins
7d1c8d92f9 HDFS-3733. Audit logs should include WebHDFS access. Contributed by Andy Isaacson 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379278 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 04:08:23 +00:00
Aaron Myers
b2d186d865 HDFS-3864. NN does not update internal file mtime for OP_CLOSE when reading from the edit log. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378413 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-29 01:19:07 +00:00
Aaron Myers
d4d2bf73a9 HDFS-3849. When re-loading the FSImage, we should clear the existing genStamp and leases. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378364 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-28 22:09:27 +00:00
Todd Lipcon
1e68d4726b HDFS-3845. Fixes for edge cases in QJM recovery protocol. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1377809 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-27 19:55:01 +00:00
Suresh Srinivas
deead78e35 HDFS-3844. Add @Override and remove {@inheritdoc} and unnecessary imports. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377168 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-25 01:03:22 +00:00
Eli Collins
a11042365f HDFS-3731. 2.0 release upgrade must handle blocks being written from 1.0. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377137 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-24 22:10:38 +00:00
Aaron Myers
a8ff292669 Merge trunk into HDFS-3077 branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1377092 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-24 20:38:08 +00:00
Aaron Myers
92cb6b093c HDFS-3678. Edit log files are never being purged from 2NN. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377046 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-24 18:52:59 +00:00
Tsz-wo Sze
c46de830da HDFS-3177. Update DFSClient and DataXceiver to handle different checkum types in file checksum computation. Contributed by Kihwal Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376928 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-24 14:15:14 +00:00
Eli Collins
26c259a638 Ammend previous commit to remove TestFileCreationNamenodeRestart. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376698 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-23 21:02:57 +00:00
Eli Collins
6885d03501 HDFS-3715. Fix TestFileCreation#testFileCreationNamenodeRestart. Contributed by Andrew Whang 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376689 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-23 20:49:55 +00:00
Aaron Myers
de8b34a70e HDFS-3835. Long-lived 2NN cannot perform a checkpoint if security is enabled and the NN restarts with outstanding delegation tokens. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376188 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-22 18:47:07 +00:00
Jitendra Nath Pandey
901b1739c6 HDFS-3819. Should check whether invalidate work percentage default value is not greater than 1.0f. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376185 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-22 18:44:44 +00:00
Suresh Srinivas
557b937f47 HDFS-3827. TestHASafeMode#assertSafemode method should be made static. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375828 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-21 22:25:49 +00:00
Suresh Srinivas
6c0ccb5989 HDFS-2686. Remove DistributedUpgrade related code. Contributed by Suresh Srinivas 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375800 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-21 21:18:40 +00:00
Eli Collins
97bb7b180e HDFS-3707. TestFSInputChecker: improper use of skip. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375336 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-21 00:00:33 +00:00
Tsz-wo Sze
b0ea77303b HADOOP-8240. Add a new API to allow users to specify a checksum type on FileSystem.create(..). Contributed by Kihwal Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374696 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-19 05:46:57 +00:00
Suresh Srinivas
086c855198 HDFS-3816. Invalidate work percentage default value should be 0.32f. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374645 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-18 19:50:15 +00:00
Eli Collins
18c5bc86ca HADOOP-8689. Make trash a server side configuration option. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374472 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 23:22:17 +00:00
Aaron Myers
fccace6116 HDFS-3672. Expose disk-location information for blocks to enable better scheduling. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374355 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 16:52:07 +00:00
Daryn Sharp
8fa10b184e HADOOP-7967. Need generalized multi-token filesystem support (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374271 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 14:05:11 +00:00
Eli Collins
cf93dfba4e HDFS-2963. Console Output is confusing while executing metasave (dfsadmin command). Contributed by Andrew Wang 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374040 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-16 20:35:50 +00:00
Tsz-wo Sze
3cab01ba6e HADOOP-8700. Use enum to define the checksum constants in DataChecksum. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373683 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-16 01:32:10 +00:00
Todd Lipcon
5789de83a7 HDFS-3800. improvements to QJM fault testing. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373587 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 19:18:57 +00:00
Thomas White
4e09b481e7 HADOOP-8278. Make sure components declare correct set of dependencies. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373574 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 19:10:52 +00:00
Todd Lipcon
d5abe22844 Merge trunk into branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373573 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 19:04:24 +00:00
Todd Lipcon
42cdc1b083 HDFS-3797. QJM: add segment txid as a parameter to journal() RPC. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373571 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 18:58:51 +00:00
Todd Lipcon
89153a9c68 HDFS-3796. Speed up edit log tests by avoiding fsync(). Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373567 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 18:53:13 +00:00
Todd Lipcon
c95a1674b6 HDFS-3799. QJM: handle empty log segments during recovery. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373183 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 00:57:24 +00:00
Todd Lipcon
4a9b3c693d HDFS-3798. Avoid throwing NPE when finalizeSegment() is called on invalid segment. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373179 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 00:54:49 +00:00
Todd Lipcon
160bfcd6c2 HDFS-3795. QJM: validate journal dir at startup. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373178 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 00:52:24 +00:00
Todd Lipcon
f765fdb657 HDFS-3793. Implement genericized format() in QJM. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373177 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 00:48:11 +00:00
Suresh Srinivas
28e87740c5 HDFS-3723. Add support -h, -help to all the commands. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373173 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 00:28:41 +00:00
Suresh Srinivas
231a52a7df Reverting previous incomplete change r1373170 for HDFS-3723 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373172 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 00:25:07 +00:00
Suresh Srinivas
f026d8bb1b HDFS-3723. Add support -h, -help to all the commands. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373170 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 00:03:19 +00:00
Eli Collins
f98d8eb291 HDFS-3150. Add option for clients to contact DNs via hostname. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373094 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-14 20:59:54 +00:00
Todd Lipcon
b38bd555e8 HDFS-3765. namenode -initializeSharedEdits should be able to initialize all shared storages. Contributed by Vinay and Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373061 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-14 20:11:40 +00:00
Tsz-wo Sze
0c27c77080 HDFS-3658. Fix bugs in TestDFSClientRetries and add more tests. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372707 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-14 03:02:43 +00:00
Todd Lipcon
4b67401565 Merge trunk into branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1372630 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-13 21:31:44 +00:00
Todd Lipcon
7aacfd5a08 HDFS-3789. JournalManager#format() should be able to throw IOException. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372566 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-13 19:36:31 +00:00
Aaron Myers
bbe0e823db Revert HDFS-3719. See discussion there and HDFS-3770 for more info. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372544 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-13 18:42:51 +00:00
Todd Lipcon
302f1e96ce Merge trunk into branch. 
Branch will not build after this commit: need to implement new JournalManager
interfaces in QuorumJournalManager in a follow-up.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1371518 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-09 22:29:36 +00:00
Todd Lipcon
9d0f8792a9 HDFS-3695. Genericize format() to non-file JournalManagers. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371513 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-09 22:13:54 +00:00
Todd Lipcon
6dcf2e4815 HDFS-3755. Creating an already-open-for-write file with overwrite=true fails. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370937 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-08 20:15:40 +00:00
Todd Lipcon
3a53ef4a80 HDFS-3741. Exhaustive failure injection test for skipped RPCs. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1370497 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-07 20:24:01 +00:00
Aaron Myers
9b4a7900c7 HDFS-3637. Add support for encrypting the DataTransferProtocol. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370354 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-07 16:40:03 +00:00
Aaron Myers
294b514596 HDFS-3719. Re-enable append-related tests in TestFileConcurrentReader. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1369848 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-06 13:59:00 +00:00
Aaron Myers
8ea7112595 HDFS-3738. TestDFSClientRetries#testFailuresArePerOperation sets incorrect timeout config. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367844 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-01 02:16:57 +00:00
Tsz-wo Sze
cb787968c5 HDFS-3667. Add retry support to WebHdfsFileSystem. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367841 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-01 01:41:23 +00:00
Tsz-wo Sze
556be2af92 svn merge -c -1366601 for reverting HDFS-3667. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367407 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 04:33:59 +00:00
Todd Lipcon
e08604907c HDFS-3725. Fix QJM startup when individual JNs have gaps. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1367366 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-30 23:35:22 +00:00
Todd Lipcon
e1dff3df99 Merge trunk into QJM branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1367365 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-30 23:31:42 +00:00
Robert Joseph Evans
48a2e2b4e2 HADOOP-8550. hadoop fs -touchz automatically created parent directories (John George via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367202 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-30 18:11:32 +00:00
Tsz-wo Sze
e4eec269d9 HDFS-3667. Add retry support to WebHdfsFileSystem. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366601 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-28 05:57:47 +00:00
Daryn Sharp
cbd59c1c50 HDFS-3553. Hftp proxy tokens are broken (daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366471 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-27 18:02:56 +00:00
Aaron Myers
c1ea9b4490 HDFS-3650. Use MutableQuantiles to provide latency histograms for various operations. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366246 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-27 00:26:21 +00:00
Tsz-wo Sze
8f395c2f78 HDFS-3696. Set chunked streaming mode in WebHdfsFileSystem write operations to get around a Java library bug causing OutOfMemoryError. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365839 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-25 23:37:25 +00:00
Todd Lipcon
972953bd77 HDFS-3626. Creating file with invalid path can corrupt edit log. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365801 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-25 21:52:38 +00:00
Todd Lipcon
b17018e4b8 HDFS-3693. JNStorage should read its storage info even before a writer becomes active. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1365794 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-25 21:47:19 +00:00
Todd Lipcon
d2d0736de4 HDFS-3692. Support purgeEditLogs() call to remotely purge logs on JNs. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1365792 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-25 21:44:26 +00:00
Todd Lipcon
939f4a9f92 HDFS-3694. Fix getEditLogManifest to fetch httpPort if necessary. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1365788 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-25 21:40:17 +00:00
Robert Joseph Evans
0bfa7d79d0 HADOOP-8551. fs -mkdir creates parent directories without the -p option (John George via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365588 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-25 13:56:29 +00:00
Aaron Myers
d8f3913856 HDFS-3711. Manually convert remaining tests to JUnit4. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365119 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-24 15:19:08 +00:00
Eli Collins
1fd21078d8 HDFS-3709. TestStartup tests still binding to the ephemeral port. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1364865 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-24 01:53:29 +00:00
Todd Lipcon
9833468302 HDFS-3597. SNN fails to start after DFS upgrade. Contributed by Andy Isaacson. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363899 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-20 18:58:58 +00:00
Todd Lipcon
74d4573a23 HDFS-3077. Quorum-based protocol for reading and writing edit logs. Contributed by Todd Lipcon based on initial work from Brandon Li and Hari Mankude. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1363596 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-20 00:25:50 +00:00
Robert Joseph Evans
4c51dacd52 Reverting HADOOP-8551. fs -mkdir creates parent directories without the -p option 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363547 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-19 21:15:42 +00:00
Robert Joseph Evans
52c8271bf1 HADOOP-8551. fs -mkdir creates parent directories without the -p option (Daryn Sharp and John George via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363435 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-19 17:46:47 +00:00
Todd Lipcon
23b6ed973e HDFS-3605. Block mistakenly marked corrupt during edit log catchup phase of failover. Contributed by Todd Lipcon and Brahma Reddy Battula. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363175 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-18 23:42:13 +00:00
Daryn Sharp
0a6806ce8c HDFS-3646. LeaseRenewer can hold reference to inactive DFSClient instances forever (Kihwal Lee via daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363170 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-18 23:21:30 +00:00
Tsz-wo Sze
6e56376023 HDFS-3577. In DatanodeWebHdfsMethods, use MessageBodyWriter instead of StreamingOutput, otherwise, it will fail to transfer large files. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362976 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-18 15:12:47 +00:00
Eli Collins
6981b14003 Revert HDFS-3654. TestJspHelper#testGetUgi fails with NPE. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362759 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-18 05:05:50 +00:00
Aaron Myers
e2253b539e HDFS-3583. Convert remaining tests to Junit4. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362753 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-18 03:46:28 +00:00
Eli Collins
527933f4f3 HDFS-3666. Plumb more exception messages to terminate. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362270 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-16 21:26:58 +00:00
Eli Collins
18f1feb64b MAPREDUCE-4447. Remove aop from cruft from the ant build. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362204 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-16 19:09:33 +00:00
Eli Collins
572823c82c HDFS-3665. Add a test for renaming across file systems via a symlink. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362177 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-16 18:29:25 +00:00
Harsh J
0e8e499ff4 HDFS-3659. Add missing @Override to methods across the hadoop-hdfs project. Contributed by Brandon Li. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361894 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-16 02:58:03 +00:00
Eli Collins
64fc2cb03d HDFS-3663. MiniDFSCluster should capture the code path that led to the first ExitException. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361775 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-15 19:43:39 +00:00
Eli Collins
32d4c148df HDFS-3654. TestJspHelper#testGetUgi fails with NPE. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361463 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-14 01:07:57 +00:00
Eli Collins
06d635cd2c HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361449 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-14 00:18:56 +00:00
Sanjay Radia
6a59f08a15 HDFS-3630 Modify TestPersistBlocks to use both flush and hflush (sanjay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360991 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-12 22:46:53 +00:00
Eli Collins
4a5ba3b7bd HDFS-3641. Move server Util time methods to common and use now instead of System#currentTimeMillis. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360858 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-12 19:01:49 +00:00
Eli Collins
cdae6953e8 HDFS-3582. Hook daemon process exit for testing. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360329 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-11 17:58:02 +00:00
Aaron Myers
fb4a6ac9e9 HDFS-3568. fuse_dfs: add support for security. Contributed by Colin McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359824 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-10 18:28:13 +00:00
Eli Collins
5952c7bb62 HDFS-3614. Revert unused MiniDFSCluster constructor from HDFS-3049. Contributed by Arun Murthy 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358825 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-08 19:53:14 +00:00
Eli Collins
21fdf16b0d HDFS-3548. NamenodeFsck.copyBlock fails to create a Block Reader. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358822 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-08 19:41:55 +00:00
Uma Maheswara Rao G
07295260b1 HDFS-3482. hdfs balancer throws ArrayIndexOutOfBoundsException if option is specified without values. Contributed by Madhukara Phatak. 
Submitted by:	Madhukara Phatak.
Reviewed by:	Uma Maheswara Rao G.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358812 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-08 18:39:09 +00:00
Eli Collins
6dcf42262d HDFS-3603. Decouple TestHDFSTrash from TestTrash. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358804 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-08 18:15:30 +00:00
Uma Maheswara Rao G
d4fb882163 HDFS-3541. Deadlock between recovery, xceiver and packet responder. Contributed by Vinay. 
Submitted by:	Vinay
Reviewed by:	Uma Maheswara Rao G


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358794 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-08 17:42:19 +00:00
Harsh J
e5b00f1bdc HDFS-2988. Improve error message when storage directory lock fails. Contributed by Miomir Boljanovic. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358683 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-08 00:36:38 +00:00
Aaron Myers
fb95fce240 Fix issue with NN/DN re-registration. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358347 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-06 18:59:55 +00:00
Todd Lipcon
e0ef844280 HDFS-3170. Add more useful metrics for write latency. Contributed by Matthew Jacobs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1357970 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-05 22:18:30 +00:00
Tsz-wo Sze
4d0cab2729 HDFS-3601. Add BlockPlacementPolicyWithNodeGroup to support block placement with 4-layer network topology. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1357442 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-05 01:31:57 +00:00
Eli Collins
359b4efd19 HDFS-3581. FSPermissionChecker#checkPermission sticky bit check missing range check. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356971 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-03 21:43:25 +00:00
Todd Lipcon
b8389e4c73 HDFS-3343. Improve metrics for DN read latency. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356928 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-03 20:45:21 +00:00
Eli Collins
161ed29087 HADOOP-8463. hadoop.security.auth_to_local needs a key definition and doc. Contributed by Madhukara Phatak 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356513 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-02 22:46:55 +00:00
Todd Lipcon
7accbabdee HDFS-3573. Supply NamespaceInfo when instantiating JournalManagers. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356388 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-02 18:21:49 +00:00
Tsz-wo Sze
28e8151ad3 HDFS-3157. Fix a bug in the case that the generation stamps of the stored block in a namenode and the reported block from a datanode do not match. Contributed by Ashish Singhi 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356086 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-02 07:26:03 +00:00
Harsh J
06d0109314 HADOOP-8449. hadoop fs -text fails with compressed sequence files with the codec file extension. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355636 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-30 05:04:23 +00:00
Todd Lipcon
802f885a55 HDFS-3446. HostsFileReader silently ignores bad includes/excludes. Contributed by Matthew Jacobs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355584 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-29 22:24:29 +00:00
Suresh Srinivas
19dd66a3f6 HDFS-3510. Editlog pre-allocation is performed prior to writing edits to avoid partial edits case disk out of space. Contributed by Collin McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355189 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-28 23:00:10 +00:00
Todd Lipcon
1d54e2b331 HDFS-3571. Allow EditLogFileInputStream to read from a remote URL. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355174 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-28 21:46:12 +00:00
Aaron Myers
9255cb83c8 HDFS-3559. DFSTestUtil: use Builder class to construct DFSTestUtil instances. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355141 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-28 20:04:49 +00:00
Tsz-wo Sze
f105784d6a svn merge -c -1354790 for reverting HDFS-3576 since it requires more changes. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354792 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-28 01:25:32 +00:00
Tsz-wo Sze
44389399d4 HDFS-3576. Move the definition of the constant NameNode.DEFAULT_PORT to DFSConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354790 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-28 01:11:22 +00:00
Tsz-wo Sze
bf1649d5fd HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354316 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-27 02:51:39 +00:00
Eli Collins
0270889b4e HDFS-3535. Audit logging should log denied accesses. Contributed by Andy Isaacson 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354144 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-26 18:14:09 +00:00
Tsz-wo Sze
bbab35e6d8 HDFS-3498. Support replica removal in BlockPlacementPolicy and make BlockPlacementPolicyDefault extensible for reusing code in subclasses. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1353807 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-26 03:25:47 +00:00
Tsz-wo Sze
361ea9a62c HDFS-3516. Check content-type in WebHdfsFileSystem. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1353800 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-26 01:58:13 +00:00
Tsz-wo Sze
ed7040f07b HADOOP-8470. Add NetworkTopologyWithNodeGroup, a 4-layer implementation of NetworkTopology. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1351445 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-18 18:23:10 +00:00
Tsz-wo Sze
9ab9bc9f37 Revert r1351163 for fixing the JIRA number; it should be HADOOP-8470 but not HADOOP-8468. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1351444 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-18 18:20:41 +00:00
Tsz-wo Sze
19ac5c4e8b HADOOP-8468. Add NetworkTopologyWithNodeGroup, a 4-layer implementation of NetworkTopology. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1351163 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-17 21:12:25 +00:00
Tsz-wo Sze
5572f73d45 HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1350825 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-15 23:00:36 +00:00
Tsz-wo Sze
9d2b1b2ee8 HDFS-3524. Update TestFileLengthOnClusterRestart for HDFS-3522. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1350384 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-14 20:17:54 +00:00
Aaron Myers
9cabf93e7a HDFS-766. Error message not clear for set space quota out of boundary values. Contributed by Jon Zuanich. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349639 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-13 05:29:57 +00:00
Eli Collins
fda95a6a3d HDFS-3478. Test quotas with Long.Max_Value. Contributed by Sujay Rau 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349634 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-13 05:04:20 +00:00
Eli Collins
6702d5dbd4 HDFS-3372. offlineEditsViewer should be able to read a binary edits file with recovery mode. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349628 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-13 03:31:47 +00:00
Tsz-wo Sze
45fafc2b8f HDFS-3504. Support configurable retry policy in DFSClient for RPC connections and RPC calls, and add MultipleLinearRandomRetry, a new retry policy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349124 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-12 05:27:15 +00:00
Todd Lipcon
9947d8054c HDFS-3049. During the normal NN startup process, fall back on a different edit log if we see one that is corrupt. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349114 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-12 04:15:18 +00:00
Tsz-wo Sze
543f86631b HDFS-3522. If a namenode is in safemode, it should throw SafeModeException when getBlockLocations has zero locations. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349088 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-12 01:55:51 +00:00
Eli Collins
9e7334a8fe HDFS-3517. TestStartup should bind ephemeral ports. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349004 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-11 20:52:45 +00:00
Tsz-wo Sze
ca39e780c4 HDFS-3052. Change INodeFile and INodeFileUnderConstruction to package private. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348998 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-11 20:38:46 +00:00
Eli Collins
56d2ef6f5e HDFS-2797. Fix misuses of InputStream#skip in the edit log code. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348945 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-11 18:01:38 +00:00
Tsz-wo Sze
5991ed9cbd HDFS-3490. DatanodeWebHdfsMethods throws NullPointerException if NamenodeRpcAddressParam is not set. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348287 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-09 00:54:10 +00:00
Aaron Myers
bc367be4ef HDFS-3514. Add missing TestParallelLocalRead. Contributed by Henry Robinson. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348207 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-08 19:50:24 +00:00
Aaron Myers
b98e26af58 HDFS-2914. HA: Standby should not enter safemode when resources are low. Contributed by Vinay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347895 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-08 06:57:52 +00:00
Aaron Myers
072ae08a00 HDFS-3040. TestMulitipleNNDataBlockScanner is misspelled. Contributed by Madhukara Phatak. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347891 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-08 06:38:14 +00:00
Todd Lipcon
05a73a3a1e Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347796 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-07 21:06:13 +00:00
Todd Lipcon
6d8efb7378 HDFS-3485. DataTransferThrottler will over-throttle when currentTimeMillis jumps. Contributed by Andy Isaacson. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347751 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-07 18:41:24 +00:00
Todd Lipcon
bef65d0601 HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347192 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-06 22:48:48 +00:00
Aaron Myers
a4b1793df0 HDFS-3266. DFSTestUtil#waitCorruptReplicas doesn't sleep between checks. Contributed by Madhukara Phatak. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1346192 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-04 23:06:19 +00:00
Eli Collins
30807fec82 HADOOP-8450. Remove src/test/system. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1345794 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-03 22:14:16 +00:00
Uma Maheswara Rao G
28ca281cd6 HDFS-2025. Go Back to File View link is not working in tail.jsp. Contributed by Ashish and Sravan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1345563 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-02 19:41:33 +00:00
Aaron Myers
d1207cbd88 HDFS-3442. Incorrect count for Missing Replicas in FSCK report. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1345408 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-02 00:39:34 +00:00
Eli Collins
4eb6d65eef HDFS-3486. offlineimageviewer can't read fsimage files that contain persistent delegation tokens. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344970 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-01 02:31:29 +00:00
Eli Collins
675a7e4acb HDFS-3484. hdfs fsck doesn't work if NN HTTP address is set to 0.0.0.0 even if NN RPC address is configured. Contributed by Aaron T. Myers 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344908 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-31 21:02:08 +00:00
Tsz-wo Sze
83cf475050 HDFS-744. Support hsync in HDFS. Contributed by Lars Hofhans 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344419 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-30 19:10:09 +00:00
Harsh J
f2c3d3ac52 HDFS-3476. Correct the default used in TestDFSClientRetries.busyTest() after HDFS-3462. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344394 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-30 18:33:41 +00:00
Harsh J
5cf21e4383 HDFS-3462. TestDFSClientRetries.busyTest() should restore default xceiver count in the config. Contributed by Madhukara Phatak. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1343798 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-29 16:01:44 +00:00
Uma Maheswara Rao G
e767845c36 HDFS-2717. BookKeeper Journal output stream doesn't check addComplete rc. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342534 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-25 08:56:59 +00:00
Konstantin Shvachko
2e0a15c488 HDFS-3368. Missing blocks due to bad DataNodes coming up and down. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342512 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-25 06:29:17 +00:00
Eli Collins
22cb0ec82a HADOOP-8422. Deprecate FileSystem#getDefault* and getServerDefault methods that don't take a Path argument. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342495 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-25 04:10:15 +00:00
Eli Collins
39661c06f7 Revert HDFS-3058 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342493 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-25 03:56:12 +00:00
Uma Maheswara Rao G
64ec449010 HDFS-3058. HA: Bring BookKeeperJournalManager up to date with HA changes. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342483 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-25 01:31:23 +00:00
Todd Lipcon
6c544adeb3 Merge HDFS-3042 (automatic failover) to trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342112 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-24 00:56:20 +00:00
Todd Lipcon
74dfa8f1f2 HDFS-2982. Startup performance suffers when there are many edit log segments. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342042 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-23 20:42:48 +00:00
Tsz-wo Sze
ade42bce10 HDFS-3436. In DataNode.transferReplicaForPipelineRecovery(..), it should use the stored generation stamp to check if the block is valid. Contributed by Vinay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1341961 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-23 17:59:30 +00:00
Eli Collins
260dfa367e HDFS-2885. Remove "federation" from the nameservice config options. Contributed by Tsz Wo (Nicholas), SZE 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1340841 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-20 21:13:17 +00:00
Todd Lipcon
1a3bd5d42f Merge trunk into auto-HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1340622 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-20 06:12:14 +00:00
Todd Lipcon
d1a54b872d HDFS-3438. BootstrapStandby should not require a rollEdits on active node. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1340343 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-19 05:13:38 +00:00
Todd Lipcon
7d1f97b821 HDFS-3440. More effectively limit stream memory consumption when reading corrupt edit logs. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339978 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-18 05:28:18 +00:00
Todd Lipcon
5258d6bf3f HDFS-3391. Fix InvalidateBlocks to compare blocks including their generation stamps. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339897 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-17 22:30:46 +00:00
Todd Lipcon
15ddb6634f HDFS-2800. Fix cancellation of checkpoints in the standby node to be more reliable. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339745 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-17 17:49:48 +00:00
Eli Collins
5ef725eb53 HDFS-3434. InvalidProtocolBufferException when visiting DN browseDirectory.jsp. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339712 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-17 17:01:13 +00:00
Aaron Myers
1377709b4c HDFS-3433. GetImageServlet should allow administrative requestors when security is enabled. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339540 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-17 10:23:18 +00:00
Todd Lipcon
ba1e4a823a HDFS-3422. TestStandbyIsHot timeouts too aggressive. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339452 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-17 02:45:47 +00:00
Todd Lipcon
0600e834be HDFS-3432. TestDFSZKFailoverController tries to fail over too early. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1339434 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-17 00:36:13 +00:00
Todd Lipcon
e14e10fb86 Merge trunk into auto-HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1339410 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-16 22:47:37 +00:00
Aaron Myers
7286e8a9da HDFS-3413. TestFailureToReadEdits timing out. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339250 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-16 16:24:38 +00:00
Eli Collins
dac828de0f HDFS-3197. Incorrect class comments in a few tests. Contributed by Andy Isaacson 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338880 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-15 20:37:12 +00:00
Eli Collins
7428aeca86 HDFS-3418. Rename BlockWithLocationsProto datanodeIDs field to storageIDs. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338830 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-15 19:03:14 +00:00
Todd Lipcon
52ecdb751e HDFS-3412. Fix findbugs warnings in auto-HA branch. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1338817 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-15 18:16:49 +00:00
Eli Collins
e9a7648f62 HDFS-3416. Cleanup DatanodeID and DatanodeRegistration constructors used by testing. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338784 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-15 16:23:52 +00:00
Todd Lipcon
95710c15b7 HDFS-3335. check for edit log corruption at the end of the log. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338492 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-15 00:41:37 +00:00
Aaron Myers
a8eef98f41 HDFS-3163. TestHDFSCLI.testAll fails if the user name is not all lowercase. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338486 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-15 00:14:51 +00:00
Todd Lipcon
bcdb125643 HDFS-3031. Fix complete() and getAdditionalBlock() RPCs to be idempotent. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338466 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-14 22:35:25 +00:00
Aaron Myers
05c5fcfb42 HDFS-3414. Balancer does not find NameNode if rpc-address or servicerpc-address are not set in client configs. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338275 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-14 16:25:17 +00:00
Todd Lipcon
4cd70e87be Merge trunk into auto-HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1337645 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-12 20:52:34 +00:00
Aaron Myers
5dbd09ee76 HDFS-3026. HA: Handle failure during HA state transition. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1337030 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-11 05:57:37 +00:00
Tsz-wo Sze
ba6360e493 HDFS-3385. The last block of INodeFileUnderConstruction is not necessarily a BlockInfoUnderConstruction, so do not cast it in FSNamesystem.recoverLeaseInternal(..). 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336976 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-11 00:25:48 +00:00
Eli Collins
fcb4b1995b HDFS-3401. Cleanup DatanodeDescriptor creation in the tests. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336972 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-11 00:03:37 +00:00
Eli Collins
98b00d7cc0 HDFS-3134. harden edit log loader against malformed or malicious input. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336943 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-10 23:13:02 +00:00
Tsz-wo Sze
7e8e983620 HDFS-3369. Rename {get|set|add}INode(..) methods in BlockManager and BlocksMap to {get|set|add}BlockCollection(..). Contributed by John George 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336909 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-10 21:41:25 +00:00
Eli Collins
6a0865440e HDFS-3230. Cleanup DatanodeID creation in the tests. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336815 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-10 17:58:55 +00:00
Uma Maheswara Rao G
f1ff05bf47 Reverting (Need to re-do the patch. new BlockInfo does not set iNode ) HDFS-3157. Error in deleting block is keep on coming from DN even after the block report and directory scanning has happened. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336572 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-10 09:59:39 +00:00
Uma Maheswara Rao G
408e558d8d HDFS-3157. Error in deleting block is keep on coming from DN even after the block report and directory scanning has happened. Contributed by Ashish Singhi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335719 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-08 19:45:38 +00:00
Tsz-wo Sze
f0f9a3631f HDFS-3363. Define BlockCollection and MutableBlockCollection interfaces so that INodeFile and INodeFileUnderConstruction do not have to be used in block management. Contributed by John George 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335304 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-08 00:06:09 +00:00
Todd Lipcon
567aed4f2c HDFS-3365. Enable users to disable socket caching in DFS client configuration. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335222 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-07 19:56:23 +00:00
Todd Lipcon
eca2c850a5 HDFS-3376. DFSClient fails to make connection to DN if there are many unusable cached sockets. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335115 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-07 16:42:19 +00:00
Aaron Myers
51e520c68a HADOOP-8349. ViewFS doesn't work when the root of a file system is mounted. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1334231 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-04 22:14:10 +00:00
Todd Lipcon
a701c792f8 HDFS-3357. DataXceiver reads from client socket with incorrect/no timeout. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1334116 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-04 18:50:54 +00:00
Todd Lipcon
25882b199b HDFS-3359. DFSClient.close should close cached sockets. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333624 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-03 21:13:43 +00:00
Aaron Myers
470b1197be HDFS-3351. NameNode#initializeGenericKeys should always set fs.defaultFS regardless of whether HA or Federation is enabled. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333299 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-03 03:00:10 +00:00
Todd Lipcon
f6c01e2f8e Merge trunk into auto-HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1333291 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-03 02:14:01 +00:00
Todd Lipcon
9d5799553f HADOOP-8279. Allow manual failover to be invoked when auto-failover is enabled. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1333288 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-03 01:56:33 +00:00
Todd Lipcon
cbc2424290 HDFS-3330. If GetImageServlet throws an Error or RTE, response should not have HTTP "OK" status. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333286 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-03 01:44:22 +00:00
Uma Maheswara Rao G
d37ec9d09e HDFS-3286. When the threshold value for balancer is zero, unexpected output is displayed. Contributed by Ashish Singhi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1332531 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-01 06:44:12 +00:00
Uma Maheswara Rao G
086fa860c0 HDFS-3275. Skip format for non-file based directories. Contributed by Amith D K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1332527 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-01 05:47:27 +00:00
Tsz-wo Sze
920b8fac18 HDFS-3334. Fix ByteRangeInputStream stream leakage. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331570 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-27 20:13:17 +00:00
Tsz-wo Sze
3e835c47d1 HDFS-3181. Fix a test case in TestLeaseRecovery2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331138 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-26 22:44:34 +00:00
Tsz-wo Sze
1a76c82a31 HDFS-3322. Use HdfsDataInputStream and HdfsDataOutputStream in Hdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331114 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-26 21:50:50 +00:00
Uma Maheswara Rao G
d28b982428 HDFS-3222. DFSInputStream#openInfo should not silently get the length as 0 when locations length is zero for last partial block. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331061 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-26 20:18:01 +00:00
Eli Collins
2b254dad2e HDFS-3258. Test for HADOOP-8144 (pseudoSortByDistance in NetworkTopology for first rack local node). Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330613 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-26 00:47:16 +00:00
Tsz-wo Sze
3b773da036 HDFS-3319. Change DFSOutputStream to not to start a thread in constructors. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330535 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-25 20:17:20 +00:00
Tsz-wo Sze
2584779166 HDFS-3321. Fix safe mode turn off tip message. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330506 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-25 19:34:36 +00:00
Tsz-wo Sze
f86352c2df HDFS-3318. Use BoundedInputStream in ByteRangeInputStream, otherwise, it hangs on transfers >2 GB. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330500 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-25 19:24:00 +00:00
Tsz-wo Sze
ea32198db4 HDFS-3298. Add HdfsDataOutputStream as a public API. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330064 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-24 23:39:43 +00:00
Tsz-wo Sze
93c5585d31 HDFS-3312. In HftpFileSystem, the namenode URI is non-secure but the delegation tokens have to use secure URI. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329462 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-23 21:27:01 +00:00
Sanjay Radia
589c68ae09 HADOOP-8285 Use ProtoBuf for RpcPayLoadHeader (sanjay radia) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329319 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-23 16:34:21 +00:00
Tsz-wo Sze
cdbe15ccdc HDFS-3308. Uses canonical URI to select delegation tokens in HftpFileSystem and WebHdfsFileSystem. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328541 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-20 23:59:19 +00:00
Tsz-wo Sze
201af20406 Revert r1328482 for HDFS-3308. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328487 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-20 19:32:23 +00:00
Tsz-wo Sze
ab8f458742 HDFS-3308. Uses canonical URI to select delegation tokens in HftpFileSystem and WebHdfsFileSystem. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328482 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-20 19:16:58 +00:00
Aaron Myers
49ed783a3a HDFS-3305. GetImageServlet should consider SBN a valid requestor in a secure HA setup. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328115 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-19 20:43:03 +00:00
Eli Collins
4550627cab HDFS-3169. TestFsck should test multiple -move operations in a row. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327776 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-19 01:51:13 +00:00
Eli Collins
54e61ffdc0 HDFS-3206. Miscellaneous xml cleanups for OEV. Contributed by Colin Patrick McCabe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327768 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-19 01:31:53 +00:00
Todd Lipcon
ca49126e3d Merge trunk into auto-HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1327724 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-18 23:35:30 +00:00
Tsz-wo Sze
c80dbe5e09 HDFS-2652. Add support for host-based delegation tokens. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327309 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-17 22:21:33 +00:00
Aaron Myers
c21bd72a2f HADOOP-8280. Move VersionUtil/TestVersionUtil and GenericTestUtils from HDFS into Common. Contributed by Ahmed Radwan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327182 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-17 17:06:38 +00:00
Aaron Myers
1675c18ded Revert HADOOP-8280 so it can be recommitted using `svn mv' 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327179 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-17 16:49:04 +00:00
Tsz-wo Sze
f1667dee14 HDFS-3279. Move the FSEditLog constructor with @VisibleForTesting to TestEditLog. Contributed by Arpit Gupta 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326762 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-16 19:31:34 +00:00
Todd Lipcon
258da66cc7 HDFS-3268. FileContext API mishandles token service and incompatible with HA. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326747 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-16 19:08:29 +00:00
Aaron Myers
551468385c HADOOP-8280. Move VersionUtil/TestVersionUtil and GenericTestUtils from HDFS into Common. Contributed by Ahmed Radwan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326727 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-16 18:28:36 +00:00
Aaron Myers
15fe3ae61b HDFS-2708. Stats for the # of blocks per DN. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326039 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-14 01:51:03 +00:00
Tsz-wo Sze
841fdc5628 HDFS-3273. Refactor BackupImage and FSEditLog, and rename JournalListener.rollLogs(..) to startLogSegment(..). 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326016 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-13 23:41:01 +00:00
Eli Collins
5a20d446cf HDFS-2799. Trim fs.checkpoint.dir values. Contributed by Amith D K 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325963 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-13 21:31:27 +00:00
Aaron Myers
4f230adc13 HDFS-3256. HDFS considers blocks under-replicated if topology script is configured with only 1 rack. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325531 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 21:28:44 +00:00
Aaron Myers
07a4367445 HDFS-3259. NameNode#initializeSharedEdits should populate shared edits dir with edit log segments. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325518 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 21:11:32 +00:00
Todd Lipcon
72406e940a HDFS-3255. HA DFS returns wrong token service. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325414 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 18:08:00 +00:00
Todd Lipcon
989d65ab97 HDFS-3261. TestHASafeMode fails on HDFS-3042 branch. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1325140 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 07:05:36 +00:00
Aaron Myers
4ea042666c HDFS-3260. TestDatanodeRegistration should set minimum DN version in addition to minimum NN version. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325119 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 05:27:03 +00:00
Aaron Myers
7f427646df HDFS-2983. Relax the build version check to permit rolling upgrades within a release. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325110 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 03:36:13 +00:00
Tsz-wo Sze
ed678e52ce HDFS-3179. Improve the exception message thrown by DataStreamer when it failed to add a datanode. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1324892 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-11 17:49:28 +00:00
Todd Lipcon
2bf19979b3 Merge trunk into auto-failover branch. 
Needs a few tweaks to fix compilation - will do in followup commit. This is just a straight merge


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1324567 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-11 05:47:40 +00:00
Todd Lipcon
41a014c31b HADOOP-8247. Add a config to enable auto-HA, which disables manual FailoverController. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1324566 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-11 05:40:26 +00:00
Todd Lipcon
21824d8232 HDFS-3247. Improve bootstrapStandby behavior when original NN is not active. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1324558 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-11 05:16:06 +00:00
Eli Collins
80447bd35a HDFS-3244. Remove dead writable code from hdfs/protocol. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1312061 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-11 01:10:38 +00:00
Todd Lipcon
58228e4e7b HDFS-3243. TestParallelRead timing out on jenkins. Contributed by Henry Robinson. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1312029 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-10 22:31:18 +00:00
Todd Lipcon
3ffdb9152e HDFS-3094. add -nonInteractive and -force option to namenode -format command. Contributed by Arpit Gupta. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1312025 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-10 22:22:38 +00:00
Aaron Myers
53dc8546d4 HDFS-3235. MiniDFSClusterManager doesn't correctly support -format option. Contributed by Henry Robinson. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311556 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-10 02:25:40 +00:00
Aaron Myers
9597c81f35 HDFS-3236. NameNode does not initialize generic conf keys when started with -initializeSharedEditsDir. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311554 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-10 02:17:17 +00:00