Tsz-wo Sze
ba2ee1d7fb
HDFS-4107. Add utility methods for casting INode to INodeFile and INodeFileUnderConstruction.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1402265 13f79535-47bb-0310-9956-ffa450edef68
2012-10-25 18:44:46 +00:00
Suresh Srinivas
611684e2e5
HDFS-4110. Refine a log printed in JNStorage. Contributed by Liang Xie.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1402203 13f79535-47bb-0310-9956-ffa450edef68
2012-10-25 16:15:59 +00:00
Suresh Srinivas
2d5334931e
HDFS-4097. Provide CLI support for createSnapshot. Contributed by Brandon Li.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1401971 13f79535-47bb-0310-9956-ffa450edef68
2012-10-25 04:08:36 +00:00
Tsz-wo Sze
14d8d0a670
Merge r1400738 through r1401868 from trunk.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1401869 13f79535-47bb-0310-9956-ffa450edef68
2012-10-24 20:42:03 +00:00
Tsz-wo Sze
b897bb8992
HDFS-4091. Add snapshot quota to limit the number of snapshots allowed.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1401868 13f79535-47bb-0310-9956-ffa450edef68
2012-10-24 20:39:26 +00:00
Suresh Srinivas
4fde5c0190
HDFS-4084. Provide CLI support to allow and disallow snapshot on a directory. Contributed by Brondon Li.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1401444 13f79535-47bb-0310-9956-ffa450edef68
2012-10-23 20:26:08 +00:00
Daryn Sharp
a92313f7bd
HDFS-4090. getFileChecksum() result incompatible when called against zero-byte files. (Kihwal Lee via daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401309 13f79535-47bb-0310-9956-ffa450edef68
2012-10-23 15:03:53 +00:00
Tsz-wo Sze
75cdb5bb49
HDFS-4099. Clean up replication code and add more javadoc.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400986 13f79535-47bb-0310-9956-ffa450edef68
2012-10-22 18:30:16 +00:00
Tsz-wo Sze
9a0651b4b8
HDFS-4078. Handle replication in snapshots.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400743 13f79535-47bb-0310-9956-ffa450edef68
2012-10-22 00:11:25 +00:00
Suresh Srinivas
f29724956a
Merging trunk changes
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400740 13f79535-47bb-0310-9956-ffa450edef68
2012-10-21 23:10:11 +00:00
Tsz-wo Sze
820b5495ca
HDFS-4079. Add SnapshotManager which maintains a list for all the snapshottable directories and supports snapshot methods such as setting a directory to snapshottable and creating a snapshot.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400728 13f79535-47bb-0310-9956-ffa450edef68
2012-10-21 21:35:13 +00:00
Suresh Srinivas
625d7cf20b
HDFS-4087. Protocol changes for listSnapshots functionality. Contributed by Brandon Li.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400698 13f79535-47bb-0310-9956-ffa450edef68
2012-10-21 19:05:49 +00:00
Uma Maheswara Rao G
35ec20b7c6
HDFS-4022. Replication not happening for appended block. Contributed by Vinay.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400578 13f79535-47bb-0310-9956-ffa450edef68
2012-10-21 06:31:31 +00:00
Tsz-wo Sze
d9a9daeb2d
Merge r1399946 through r1400349 from trunk.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400351 13f79535-47bb-0310-9956-ffa450edef68
2012-10-20 00:33:57 +00:00
Suresh Srinivas
a2d837b0e9
HDFS-4077. Adding a file missed in the commit r1400318.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400350 13f79535-47bb-0310-9956-ffa450edef68
2012-10-20 00:31:57 +00:00
Tsz-wo Sze
661a8d176a
HDFS-4088. Remove "throws QuotaExceededException" from an INodeDirectoryWithQuota constructor.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400345 13f79535-47bb-0310-9956-ffa450edef68
2012-10-20 00:06:12 +00:00
Suresh Srinivas
af130d4baf
HDFS-4077. Add support for Snapshottable Directory. Contributed by Nicholas.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400318 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 22:21:50 +00:00
Suresh Srinivas
f5c4defcb3
HDFS-4083. Protocol changes for snapshots. Contributed by Suresh Srinivas.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400316 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 22:15:27 +00:00
Suresh Srinivas
ecffab63af
HDFS-4086. Add editlog opcodes to allow and disallow snapshots on a directory. Contributed by Brandon Li.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400298 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 21:15:11 +00:00
Suresh Srinivas
40f8b7dcfb
Reverting the change r1400285 since it included unrelated changes
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400290 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 21:04:29 +00:00
Suresh Srinivas
4cccdc2a96
Moving HDFS-2802 related changes to a separate CHANGES.txt
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400285 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 21:01:26 +00:00
Tsz-wo Sze
e75424544f
HDFS-4082. Add editlog opcodes for snapshot create and delete operations. Contributed by suresh
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400247 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 19:38:48 +00:00
Tsz-wo Sze
5c1a7b9d5d
HDFS-4076. Support snapshot of single files. (szetszwo)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802@1400245 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 19:35:19 +00:00
Suresh Srinivas
0603447f2a
HDFS-4072. On file deletion remove corresponding blocks pending replications. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399965 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 04:55:11 +00:00
Eli Collins
7da5d03c35
HDFS-4053. Increase the default block size. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399908 13f79535-47bb-0310-9956-ffa450edef68
2012-10-18 22:56:30 +00:00
Eli Collins
57ee22e589
HADOOP-8925. Remove the packaging. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399889 13f79535-47bb-0310-9956-ffa450edef68
2012-10-18 22:30:11 +00:00
Tsz-wo Sze
4af8761e4c
HDFS-4074. Remove the unused default constructor from INode. Contributed by Brandon Li
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399866 13f79535-47bb-0310-9956-ffa450edef68
2012-10-18 21:29:12 +00:00
Tsz-wo Sze
c1bd54daa3
HDFS-4073. Two minor improvements to FSDirectory. Contributed by Jing Zhao
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399861 13f79535-47bb-0310-9956-ffa450edef68
2012-10-18 21:18:25 +00:00
Eli Collins
4d5600f6c7
HDFS-4068. DatanodeID and DatanodeInfo member should be private. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399443 13f79535-47bb-0310-9956-ffa450edef68
2012-10-17 21:34:48 +00:00
Eli Collins
a258263ecf
HDFS-4029. GenerationStamp should use an AtomicLong. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399096 13f79535-47bb-0310-9956-ffa450edef68
2012-10-17 04:58:24 +00:00
Aaron Myers
402836479f
HDFS-3997. OfflineImageViewer incorrectly passes value of imageVersion when visiting IS_COMPRESSED element. Contributed by Mithun Radhakrishnan.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399062 13f79535-47bb-0310-9956-ffa450edef68
2012-10-17 01:10:55 +00:00
Suresh Srinivas
0ef9c6f71a
HDFS-4059. Add number of stale DataNodes to metrics. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1398949 13f79535-47bb-0310-9956-ffa450edef68
2012-10-16 19:39:50 +00:00
Suresh Srinivas
b7887f31fb
HDFS-4052. BlockManager#invalidateWork should print log outside the lock. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1398631 13f79535-47bb-0310-9956-ffa450edef68
2012-10-16 03:37:49 +00:00
Eli Collins
a75673cbd8
HDFS-4058. DirectoryScanner may fail with IOOB if the directory scanning threads return out of volume order. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1398612 13f79535-47bb-0310-9956-ffa450edef68
2012-10-16 02:12:59 +00:00
Aaron Myers
557ffe2101
HDFS-2946. HA: Put a cap on the number of completed edits files retained by the NN. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1398609 13f79535-47bb-0310-9956-ffa450edef68
2012-10-16 01:59:46 +00:00
Todd Lipcon
ded304e6a6
HDFS-4049. Fix hflush performance regression due to nagling delays. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1398591 13f79535-47bb-0310-9956-ffa450edef68
2012-10-16 00:55:29 +00:00
Tsz-wo Sze
698fa5077a
HDFS-4036. Remove "throw UnresolvedLinkException" from FSDirectory.unprotectedAddFile(..). Contributed by Jing Zhao
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1398293 13f79535-47bb-0310-9956-ffa450edef68
2012-10-15 14:05:31 +00:00
Tsz-wo Sze
ad06a08713
HDFS-4037. Rename the getReplication() method in BlockCollection to getBlockReplication().
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1398288 13f79535-47bb-0310-9956-ffa450edef68
2012-10-15 13:48:56 +00:00
Suresh Srinivas
89b66da42c
HDFS-4044. Duplicate ChecksumType definition in HDFS .proto files. Contributed by Binglin Chang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397580 13f79535-47bb-0310-9956-ffa450edef68
2012-10-12 14:32:16 +00:00
Suresh Srinivas
94ed14c48c
HADOOP-8911. CRLF characters in source and text files. Contributed Raja Aluri.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397432 13f79535-47bb-0310-9956-ffa450edef68
2012-10-12 04:35:42 +00:00
Suresh Srinivas
2887bbb33c
HDFS-3912. Detect and avoid stale datanodes for writes. Contributed by Jing Zhao
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397211 13f79535-47bb-0310-9956-ffa450edef68
2012-10-11 18:08:27 +00:00
Todd Lipcon
ff8f84ccf6
Merge trunk into branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1396918 13f79535-47bb-0310-9956-ffa450edef68
2012-10-11 06:14:26 +00:00
Suresh Srinivas
33c16f2a24
HADOOP-8913. hadoop-metrics2.properties should give units in comment for sampling period. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396904 13f79535-47bb-0310-9956-ffa450edef68
2012-10-11 05:17:03 +00:00
Aaron Myers
8078e16015
HDFS-4021. Misleading error message when resources are low on the NameNode. Contributed by Christopher Conner.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396881 13f79535-47bb-0310-9956-ffa450edef68
2012-10-11 00:32:46 +00:00
Jason Darrell Lowe
08f35a04c6
HDFS-3224. Bug in check for DN re-registration with different storage ID. Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396798 13f79535-47bb-0310-9956-ffa450edef68
2012-10-10 21:15:06 +00:00
Todd Lipcon
70fc5cf01c
HDFS-4017. Unclosed FileInputStream in GetJournalEditServlet. Contributed by Chao Shi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1395727 13f79535-47bb-0310-9956-ffa450edef68
2012-10-08 19:11:48 +00:00
Eli Collins
5a5473b29f
HDFS-4006. TestCheckpoint#testSecondaryHasVeryOutOfDateImage occasionally fails due to unexpected exit. Contributed by Todd Lipcon
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395387 13f79535-47bb-0310-9956-ffa450edef68
2012-10-07 21:35:51 +00:00
Aaron Myers
353958dccc
HDFS-3483. Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1394864 13f79535-47bb-0310-9956-ffa450edef68
2012-10-06 00:13:28 +00:00
Suresh Srinivas
dfb8369c29
HDFS-3735. NameNode WebUI should allow sorting live datanode list by fields Block Pool Used, Block Pool Used(%) and Failed Volumes. Contributed by Brahma Reddy Battula.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1394385 13f79535-47bb-0310-9956-ffa450edef68
2012-10-05 07:02:52 +00:00
Aaron Myers
6582da034b
HDFS-3813. Log error message if security and WebHDFS are enabled but principal/keytab are not configured. Contributed by Stephen Chu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1394340 13f79535-47bb-0310-9956-ffa450edef68
2012-10-05 00:54:05 +00:00
Eli Collins
ee13fb4f69
HDFS-3916. libwebhdfs (C client) code cleanups. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1393890 13f79535-47bb-0310-9956-ffa450edef68
2012-10-04 02:53:28 +00:00
Eli Collins
43ae84761f
HDFS-3996. Add debug log removed in HDFS-3873 back. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1393777 13f79535-47bb-0310-9956-ffa450edef68
2012-10-03 21:30:45 +00:00
Harsh J
34becbb019
HADOOP-8386. hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu). Contributed by Christopher Berner and Andy Isaacson. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391780 13f79535-47bb-0310-9956-ffa450edef68
2012-09-29 10:57:53 +00:00
Aaron Myers
e8dc3b3a8a
HDFS-3896. Add descriptions for dfs.namenode.rpc-address and dfs.namenode.servicerpc-address to hdfs-default.xml. Contributed by Jeff Lord.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391708 13f79535-47bb-0310-9956-ffa450edef68
2012-09-29 00:51:48 +00:00
Todd Lipcon
802ccafcd9
Fix some spurious whitespace changes in branch (per Suresh's review)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1391247 13f79535-47bb-0310-9956-ffa450edef68
2012-09-27 22:37:58 +00:00
Eli Collins
a33ae4020d
HADOOP-8855. SSL-based image transfer does not work when Kerberos is disabled. Contributed by Todd Lipcon
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390841 13f79535-47bb-0310-9956-ffa450edef68
2012-09-27 05:03:42 +00:00
Eli Collins
47dfcf45af
HDFS-3972. Trash emptier fails in secure HA cluster. Contributed by Todd Lipcon
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390729 13f79535-47bb-0310-9956-ffa450edef68
2012-09-26 21:25:04 +00:00
Tsz-wo Sze
a7bcdcc051
HDFS-3373. Change DFSClient input stream socket cache to global static and add a thread to cleanup expired cache entries. Contributed by John George
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390466 13f79535-47bb-0310-9956-ffa450edef68
2012-09-26 13:23:21 +00:00
Todd Lipcon
b1ceaac3e6
Merge trunk into branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1390199 13f79535-47bb-0310-9956-ffa450edef68
2012-09-25 22:43:04 +00:00
Harsh J
9c9b3f09ed
HADOOP-8158. Interrupting hadoop fs -put from the command line causes a LeaseExpiredException. Contributed by Daryn Sharp. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389010 13f79535-47bb-0310-9956-ffa450edef68
2012-09-23 10:50:44 +00:00
Eli Collins
fa6d519657
HDFS-3964. Make NN log of fs.defaultFS debug rather than info. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1388335 13f79535-47bb-0310-9956-ffa450edef68
2012-09-21 06:17:29 +00:00
Eli Collins
7dee7f11f6
HDFS-3932. NameNode Web UI broken if the rpc-address is set to the wildcard. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1388321 13f79535-47bb-0310-9956-ffa450edef68
2012-09-21 04:51:43 +00:00
Eli Collins
d26f05b7f5
HDFS-3949. NameNodeRpcServer#join should join on both client and server RPC servers. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1387838 13f79535-47bb-0310-9956-ffa450edef68
2012-09-20 01:33:13 +00:00
Todd Lipcon
8a3e64c77f
HDFS-3956. QJM: purge temporary files when no longer within retention period. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1387817 13f79535-47bb-0310-9956-ffa450edef68
2012-09-19 23:40:59 +00:00
Todd Lipcon
f26c63df21
Amend HFDS-3955. Forgot an import of ImmutableList in previous patch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1387740 13f79535-47bb-0310-9956-ffa450edef68
2012-09-19 20:24:12 +00:00
Todd Lipcon
3ccd905d8a
HDFS-3955. QJM: Make acceptRecovery() atomic. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1387706 13f79535-47bb-0310-9956-ffa450edef68
2012-09-19 18:57:11 +00:00
Todd Lipcon
663e7484c0
HDFS-3950. QJM: misc TODO cleanup, improved log messages, etc. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1387704 13f79535-47bb-0310-9956-ffa450edef68
2012-09-19 18:52:15 +00:00
Alejandro Abdelnur
8bf6b80f2b
Reverting HADOOP-8805
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1387697 13f79535-47bb-0310-9956-ffa450edef68
2012-09-19 18:06:42 +00:00
Alejandro Abdelnur
7fb42f8f4b
HDFS-3951. datanode web ui does not work over HTTPS when datanode is started in secure mode. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1387688 13f79535-47bb-0310-9956-ffa450edef68
2012-09-19 17:39:46 +00:00
Todd Lipcon
e9f4de5ced
Merge trunk into branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1387449 13f79535-47bb-0310-9956-ffa450edef68
2012-09-19 04:34:55 +00:00
Alejandro Abdelnur
9f6cfe32e4
HADOOP-8805. Move protocol buffer implementation of GetUserMappingProtocol from HDFS to Common. (bowang via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1387298 13f79535-47bb-0310-9956-ffa450edef68
2012-09-18 18:15:37 +00:00
Eli Collins
a8e4bb81b2
HDFS-3939. NN RPC address cleanup. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1387278 13f79535-47bb-0310-9956-ffa450edef68
2012-09-18 17:42:09 +00:00
Eli Collins
7e3776ca6e
HDFS-3936. MiniDFSCluster shutdown races with BlocksMap usage. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1387255 13f79535-47bb-0310-9956-ffa450edef68
2012-09-18 16:59:47 +00:00
Todd Lipcon
83c14fbd24
HDFS-3943. QJM: remove currently-unused md5sum field. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1386863 13f79535-47bb-0310-9956-ffa450edef68
2012-09-17 21:51:40 +00:00
Thomas White
d89828573d
HADOOP-8780. Update DeprecatedProperties apt file. Contributed by Ahmed Radwan
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384833 13f79535-47bb-0310-9956-ffa450edef68
2012-09-14 16:05:30 +00:00
Aaron Myers
86635168fe
HDFS-3924. Multi-byte id in HdfsVolumeId. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384602 13f79535-47bb-0310-9956-ffa450edef68
2012-09-13 23:48:25 +00:00
Suresh Srinivas
d543140089
HDFS-3703. Datanodes are marked stale if heartbeat is not received in configured timeout and are selected as the last location to read from. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384209 13f79535-47bb-0310-9956-ffa450edef68
2012-09-13 05:46:18 +00:00
Eli Collins
2ba149f85c
HDFS-3902. TestDatanodeBlockScanner#testBlockCorruptionPolicy is broken. Contributed by Andy Isaacson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384081 13f79535-47bb-0310-9956-ffa450edef68
2012-09-12 18:36:01 +00:00
Todd Lipcon
80a8d57ffc
HDFS-3925. Prettify PipelineAck#toString() for printing to a log. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384078 13f79535-47bb-0310-9956-ffa450edef68
2012-09-12 18:32:57 +00:00
Eli Collins
414abe6918
HDFS-3664. BlockManager race when stopping active services. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383753 13f79535-47bb-0310-9956-ffa450edef68
2012-09-12 04:10:58 +00:00
Todd Lipcon
a93ba1648a
HDFS-3840. JournalNodes log JournalNotFormattedException backtrace error before being formatted. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383252 13f79535-47bb-0310-9956-ffa450edef68
2012-09-11 06:33:47 +00:00
Todd Lipcon
8a8c9c18d3
HDFS-3906. QJM: quorum timeout on failover with large log segment. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383251 13f79535-47bb-0310-9956-ffa450edef68
2012-09-11 06:31:42 +00:00
Todd Lipcon
c859e87d1e
HDFS-3915. QJM: Failover fails with auth error in secure cluster. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383242 13f79535-47bb-0310-9956-ffa450edef68
2012-09-11 04:53:08 +00:00
Todd Lipcon
c5199cace6
HDFS-3914. QJM: acceptRecovery should abort current segment. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383148 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 23:20:49 +00:00
Todd Lipcon
729ec86907
HDFS-3899. Add client side metrics for QJM. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383139 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 22:39:46 +00:00
Todd Lipcon
959afc0fd3
HDFS-3901. QJM: send 'heartbeat' messages to JNs even when they are out-of-sync. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383137 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 22:30:52 +00:00
Todd Lipcon
60c20e559b
HDFS-3900. QJM: avoid validating log segments on log rolls. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383041 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 18:53:41 +00:00
Todd Lipcon
ca4582222e
HDFS-3885. QJM: optimize log sync when JN is lagging behind. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383039 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 18:51:15 +00:00
Todd Lipcon
aa65777ef0
HDFS-3898. QJM: enable TCP_NODELAY for IPC. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383033 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 18:46:58 +00:00
Eli Collins
e1ffefa860
HDFS-3907. Allow multiple users for local block readers. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383031 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 18:46:13 +00:00
Todd Lipcon
1cea56d6ef
Merge trunk into QJM branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383030 13f79535-47bb-0310-9956-ffa450edef68
2012-09-10 18:45:45 +00:00
Eli Collins
3a4dcfba57
HDFS-2757. Cannot read a local block that's being written to when using the local read short circuit. Contributed by Jean-Daniel Cryans
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382409 13f79535-47bb-0310-9956-ffa450edef68
2012-09-09 06:37:31 +00:00
Aaron Myers
df801074c9
HDFS-3893. QJM: Make QJM work with security enabled. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1381770 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 21:42:14 +00:00
Todd Lipcon
8b88542f60
Amend HDFS-3726. Add previously missing file JournalOutOfSyncException.java
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1381681 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 16:59:51 +00:00
Daryn Sharp
82b981cca4
HDFS-3890. filecontext mkdirs doesn't apply umask as expected (Tom Graves via daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381606 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 14:06:58 +00:00
Todd Lipcon
cae8116a14
HDFS-3726. If a logger misses an RPC, don't retry that logger until next segment. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1381482 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 07:03:57 +00:00
Todd Lipcon
437948ea1c
HDFS-3891. Make selectInputStreams throw IOE instead of RTE. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1381481 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 06:57:18 +00:00
Eli Collins
01db4d2bb2
HDFS-3828. Block Scanner rescans blocks too frequently. Contributed by Andy Isaacson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381472 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 06:19:30 +00:00
Todd Lipcon
99ec5bd8d3
Merge trunk into QJM branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380990 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 04:57:47 +00:00
Todd Lipcon
31142aa892
HDFS-1490. TransferFSImage should timeout. Contributed by Dmytro Molkov and Vinay.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380988 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 04:53:15 +00:00
Todd Lipcon
c334cc89a8
HDFS-2793. Add an admin command to trigger an edit log roll. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380982 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 04:40:23 +00:00
Todd Lipcon
13daca1ef6
HDFS-3870. Add metrics to JournalNode. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380980 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 04:30:51 +00:00
Todd Lipcon
f6b7f067c3
HDFS-3884. Journal format() should reset cached values. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380979 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 04:27:49 +00:00
Todd Lipcon
72485f3112
HDFS-3869. Expose non-file journal manager details in web UI. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380978 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 04:16:02 +00:00
Todd Lipcon
8021d9199f
HDFS-3863. Track last "committed" txid in QJM. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380976 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 04:13:19 +00:00
Tsz-wo Sze
658e7b5818
HDFS-3888. Clean up BlockPlacementPolicyDefault. Contributed by Jing Zhao
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380939 13f79535-47bb-0310-9956-ffa450edef68
2012-09-04 23:56:10 +00:00
Tsz-wo Sze
f3927595cc
HDFS-3887. Remove redundant chooseTarget methods in BlockPlacementPolicy. Contributed by Jing Zhao
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380934 13f79535-47bb-0310-9956-ffa450edef68
2012-09-04 23:48:01 +00:00
Uma Maheswara Rao G
3969bcb7c9
HDFS-3469. start-dfs.sh will start zkfc, but stop-dfs.sh will not stop zkfc similarly. Contributed by Vinay
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380003 13f79535-47bb-0310-9956-ffa450edef68
2012-09-02 16:20:13 +00:00
Suresh Srinivas
da3bd67138
HDFS-3880. Use Builder to build RPC server in HDFS. Contributed by Brandon Li.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379917 13f79535-47bb-0310-9956-ffa450edef68
2012-09-02 06:20:41 +00:00
Harsh J
54e612bfb9
HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379828 13f79535-47bb-0310-9956-ffa450edef68
2012-09-01 19:22:11 +00:00
Tsz-wo Sze
50222ff529
HDFS-3871. Change NameNodeProxies to use RetryUtils. Contributed by Arun C Murthy
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379743 13f79535-47bb-0310-9956-ffa450edef68
2012-09-01 11:41:15 +00:00
Eli Collins
6f5ae49063
HDFS-3877. QJM: Provide defaults for dfs.journalnode.*address. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1379681 13f79535-47bb-0310-9956-ffa450edef68
2012-09-01 00:35:19 +00:00
Owen O'Malley
c76a7893f9
HDFS-3466. Get HTTP kerberos principal from the web authentication keytab.
...
(omalley)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379646 13f79535-47bb-0310-9956-ffa450edef68
2012-08-31 22:31:03 +00:00
Daryn Sharp
3819c964fa
HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379615 13f79535-47bb-0310-9956-ffa450edef68
2012-08-31 21:24:48 +00:00
Eli Collins
7d1c8d92f9
HDFS-3733. Audit logs should include WebHDFS access. Contributed by Andy Isaacson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379278 13f79535-47bb-0310-9956-ffa450edef68
2012-08-31 04:08:23 +00:00
Daryn Sharp
bb64f860eb
HDFS-3861. Deadlock in DFSClient (Kihwal Lee via daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379093 13f79535-47bb-0310-9956-ffa450edef68
2012-08-30 19:46:14 +00:00
Aaron Myers
b2d186d865
HDFS-3864. NN does not update internal file mtime for OP_CLOSE when reading from the edit log. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378413 13f79535-47bb-0310-9956-ffa450edef68
2012-08-29 01:19:07 +00:00
Aaron Myers
d4d2bf73a9
HDFS-3849. When re-loading the FSImage, we should clear the existing genStamp and leases. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378364 13f79535-47bb-0310-9956-ffa450edef68
2012-08-28 22:09:27 +00:00
Aaron Myers
8d724dd8b9
HDFS-3860. HeartbeatManager#Monitor may wrongly hold the writelock of namesystem. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378228 13f79535-47bb-0310-9956-ffa450edef68
2012-08-28 17:01:38 +00:00
Eli Collins
b29cb2d997
HDFS-3856. TestHDFSServerPorts failure is causing surefire fork failure. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377934 13f79535-47bb-0310-9956-ffa450edef68
2012-08-28 01:41:13 +00:00
Todd Lipcon
1e68d4726b
HDFS-3845. Fixes for edge cases in QJM recovery protocol. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1377809 13f79535-47bb-0310-9956-ffa450edef68
2012-08-27 19:55:01 +00:00
Suresh Srinivas
735046ebec
HDFS-3851. DFSOutputStream class code cleanup. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377372 13f79535-47bb-0310-9956-ffa450edef68
2012-08-26 04:00:26 +00:00
Aaron Myers
56285e1bee
HDFS-3683. Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377367 13f79535-47bb-0310-9956-ffa450edef68
2012-08-26 00:31:12 +00:00
Suresh Srinivas
deead78e35
HDFS-3844. Add @Override and remove {@inheritdoc} and unnecessary imports. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377168 13f79535-47bb-0310-9956-ffa450edef68
2012-08-25 01:03:22 +00:00
Eli Collins
a11042365f
HDFS-3731. 2.0 release upgrade must handle blocks being written from 1.0. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377137 13f79535-47bb-0310-9956-ffa450edef68
2012-08-24 22:10:38 +00:00
Aaron Myers
a8ff292669
Merge trunk into HDFS-3077 branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1377092 13f79535-47bb-0310-9956-ffa450edef68
2012-08-24 20:38:08 +00:00
Aaron Myers
92cb6b093c
HDFS-3678. Edit log files are never being purged from 2NN. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377046 13f79535-47bb-0310-9956-ffa450edef68
2012-08-24 18:52:59 +00:00
Tsz-wo Sze
c46de830da
HDFS-3177. Update DFSClient and DataXceiver to handle different checkum types in file checksum computation. Contributed by Kihwal Lee
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376928 13f79535-47bb-0310-9956-ffa450edef68
2012-08-24 14:15:14 +00:00
Eli Collins
c10a73350e
Revert HDFS-3837. Fix DataNode.recoverBlock findbugs warning.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376701 13f79535-47bb-0310-9956-ffa450edef68
2012-08-23 21:08:19 +00:00
Aaron Myers
de8b34a70e
HDFS-3835. Long-lived 2NN cannot perform a checkpoint if security is enabled and the NN restarts with outstanding delegation tokens. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376188 13f79535-47bb-0310-9956-ffa450edef68
2012-08-22 18:47:07 +00:00
Jitendra Nath Pandey
901b1739c6
HDFS-3819. Should check whether invalidate work percentage default value is not greater than 1.0f. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376185 13f79535-47bb-0310-9956-ffa450edef68
2012-08-22 18:44:44 +00:00
Suresh Srinivas
380870d544
HDFS-3832. Remove protocol methods related to DistributedUpgrade. Contributed by Suresh Srinivas.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376139 13f79535-47bb-0310-9956-ffa450edef68
2012-08-22 16:59:50 +00:00
Eli Collins
89e16be81b
HDFS-3837. Fix DataNode.recoverBlock findbugs warning. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376129 13f79535-47bb-0310-9956-ffa450edef68
2012-08-22 16:26:50 +00:00
Eli Collins
2b030c6a4a
HDFS-3830. test_libhdfs_threaded: use forceNewInstance. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376121 13f79535-47bb-0310-9956-ffa450edef68
2012-08-22 16:19:27 +00:00
Suresh Srinivas
a99cfe3ca0
HDFS-3817. Avoid printing SafeModeException stack trace in RPC server. Contributed by Brandon Li.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376054 13f79535-47bb-0310-9956-ffa450edef68
2012-08-22 13:59:40 +00:00
Suresh Srinivas
6c0ccb5989
HDFS-2686. Remove DistributedUpgrade related code. Contributed by Suresh Srinivas
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375800 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 21:18:40 +00:00
Tsz-wo Sze
f2dd818201
HADOOP-8239. Add subclasses of MD5MD5CRC32FileChecksum to support file checksum with CRC32C. Contributed by Kihwal Lee
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375450 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 09:44:46 +00:00
Eli Collins
a6262430ff
HDFS-2727. libhdfs should get the default block size from the server. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375383 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 03:48:00 +00:00
Aaron Myers
f9a0d78ef5
HDFS-3826. QJM: Some trivial logging / exception text improvements. Contributed by Todd Lipcon and Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1375356 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 00:48:06 +00:00
Aaron Myers
643608ea57
HDFS-3823. QJM: TestQJMWithFaults fails occasionally because of missed setting of HTTP port. Contributed by Todd Lipcon and Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1375323 13f79535-47bb-0310-9956-ffa450edef68
2012-08-20 23:44:14 +00:00
Tsz-wo Sze
b0ea77303b
HADOOP-8240. Add a new API to allow users to specify a checksum type on FileSystem.create(..). Contributed by Kihwal Lee
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374696 13f79535-47bb-0310-9956-ffa450edef68
2012-08-19 05:46:57 +00:00
Suresh Srinivas
086c855198
HDFS-3816. Invalidate work percentage default value should be 0.32f. Contributed by Jing Zhao
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374645 13f79535-47bb-0310-9956-ffa450edef68
2012-08-18 19:50:15 +00:00
Eli Collins
18c5bc86ca
HADOOP-8689. Make trash a server side configuration option. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374472 13f79535-47bb-0310-9956-ffa450edef68
2012-08-17 23:22:17 +00:00
Aaron Myers
fccace6116
HDFS-3672. Expose disk-location information for blocks to enable better scheduling. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374355 13f79535-47bb-0310-9956-ffa450edef68
2012-08-17 16:52:07 +00:00
Daryn Sharp
8fa10b184e
HADOOP-7967. Need generalized multi-token filesystem support (daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374271 13f79535-47bb-0310-9956-ffa450edef68
2012-08-17 14:05:11 +00:00
Tsz-wo Sze
ffbadbdd36
HDFS-2421. Improve the concurrency of SerialNumberMap in NameNode. Contributed by Jing Zhao and Weiyan Wang
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374127 13f79535-47bb-0310-9956-ffa450edef68
2012-08-17 01:54:16 +00:00
Tsz-wo Sze
87d87b04e6
HDFS-3788. ByteRangeInputStream should not expect HTTP Content-Length header when chunked transfer-encoding is used.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374122 13f79535-47bb-0310-9956-ffa450edef68
2012-08-17 01:42:35 +00:00
Aaron Myers
bbe4584ebf
HDFS-3808. fuse_dfs: postpone libhdfs intialization until after fork. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374106 13f79535-47bb-0310-9956-ffa450edef68
2012-08-16 23:47:33 +00:00
Eli Collins
cf93dfba4e
HDFS-2963. Console Output is confusing while executing metasave (dfsadmin command). Contributed by Andrew Wang
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374040 13f79535-47bb-0310-9956-ffa450edef68
2012-08-16 20:35:50 +00:00
Eli Collins
4524655afd
HDFS-3194. DataNode block scanner is running too frequently. Contributed by Andy Isaacson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373928 13f79535-47bb-0310-9956-ffa450edef68
2012-08-16 17:13:13 +00:00
Tsz-wo Sze
3cab01ba6e
HADOOP-8700. Use enum to define the checksum constants in DataChecksum.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373683 13f79535-47bb-0310-9956-ffa450edef68
2012-08-16 01:32:10 +00:00
Eli Collins
f75deca0f9
HDFS-3048. Small race in BlockManager#close. Contributed by Andy Isaacson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373664 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 22:43:01 +00:00
Todd Lipcon
d5abe22844
Merge trunk into branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373573 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 19:04:24 +00:00
Todd Lipcon
42cdc1b083
HDFS-3797. QJM: add segment txid as a parameter to journal() RPC. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373571 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 18:58:51 +00:00
Todd Lipcon
89153a9c68
HDFS-3796. Speed up edit log tests by avoiding fsync(). Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373567 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 18:53:13 +00:00
Tsz-wo Sze
9dff7876d9
HDFS-3802. StartupOption.name in HdfsServerConstants should be final. Contributed by Jing Zhao
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373253 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 08:07:29 +00:00
Tsz-wo Sze
f41497c141
HDFS-3794. WebHDFS OPEN returns the incorrect Content-Length in the HTTP header when offset is specified and length is omitted. Contributed by Ravi Prakash
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373193 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 02:23:21 +00:00
Todd Lipcon
c95a1674b6
HDFS-3799. QJM: handle empty log segments during recovery. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373183 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 00:57:24 +00:00
Todd Lipcon
4a9b3c693d
HDFS-3798. Avoid throwing NPE when finalizeSegment() is called on invalid segment. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373179 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 00:54:49 +00:00
Todd Lipcon
160bfcd6c2
HDFS-3795. QJM: validate journal dir at startup. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373178 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 00:52:24 +00:00
Todd Lipcon
f765fdb657
HDFS-3793. Implement genericized format() in QJM. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373177 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 00:48:11 +00:00
Suresh Srinivas
6c96039693
HDFS-3803. Change BlockPoolSliceScanner chatty INFO log to DEBUG. Contributed by Andrew Purtell
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373174 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 00:33:53 +00:00
Suresh Srinivas
28e87740c5
HDFS-3723. Add support -h, -help to all the commands. Contributed by Jing Zhao
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373173 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 00:28:41 +00:00
Suresh Srinivas
231a52a7df
Reverting previous incomplete change r1373170 for HDFS-3723
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373172 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 00:25:07 +00:00
Suresh Srinivas
f026d8bb1b
HDFS-3723. Add support -h, -help to all the commands. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373170 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 00:03:19 +00:00
Eli Collins
f98d8eb291
HDFS-3150. Add option for clients to contact DNs via hostname. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373094 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 20:59:54 +00:00
Daryn Sharp
f46b6ea55c
HDFS-3718. Datanode won't shutdown because of runaway DataBlockScanner thread (Kihwal Lee via daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373090 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 20:56:49 +00:00
Todd Lipcon
b38bd555e8
HDFS-3765. namenode -initializeSharedEdits should be able to initialize all shared storages. Contributed by Vinay and Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373061 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 20:11:40 +00:00
Tsz-wo Sze
0c27c77080
HDFS-3658. Fix bugs in TestDFSClientRetries and add more tests.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372707 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 03:02:43 +00:00
Todd Lipcon
ab3ab17a43
HDFS-3792. Fix two findbugs introduced by HDFS-3695. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372690 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 00:52:10 +00:00
Aaron Myers
4139298b61
HDFS-3790. test_fuse_dfs.c doesn't compile on centos 5. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372676 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 00:04:55 +00:00
Todd Lipcon
4b67401565
Merge trunk into branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1372630 13f79535-47bb-0310-9956-ffa450edef68
2012-08-13 21:31:44 +00:00
Todd Lipcon
c93185df66
HDFS-3276. initializeSharedEdits should have a -nonInteractive flag. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372628 13f79535-47bb-0310-9956-ffa450edef68
2012-08-13 21:26:39 +00:00
Todd Lipcon
7aacfd5a08
HDFS-3789. JournalManager#format() should be able to throw IOException. Contributed by Ivan Kelly.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1372566 13f79535-47bb-0310-9956-ffa450edef68
2012-08-13 19:36:31 +00:00
Eli Collins
bcb715459c
HDFS-3758. TestFuseDFS test failing. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371555 13f79535-47bb-0310-9956-ffa450edef68
2012-08-10 03:42:36 +00:00
Alejandro Abdelnur
4bca22005f
HADOOP-8681. add support for HTTPS to the web UIs. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371525 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 22:52:56 +00:00
Todd Lipcon
302f1e96ce
Merge trunk into branch.
...
Branch will not build after this commit: need to implement new JournalManager
interfaces in QuorumJournalManager in a follow-up.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1371518 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 22:29:36 +00:00
Todd Lipcon
9d0f8792a9
HDFS-3695. Genericize format() to non-file JournalManagers. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371513 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 22:13:54 +00:00
Aaron Myers
9ea7c06468
HDFS-3721. hsync support broke wire compatibility. Contributed by Todd Lipcon and Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371495 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 21:31:12 +00:00
Aaron Myers
fb970f3df7
HDFS-3634. Add self-contained, mavenized fuse_dfs test. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371231 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 14:44:05 +00:00
Todd Lipcon
6dcf2e4815
HDFS-3755. Creating an already-open-for-write file with overwrite=true fails. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370937 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 20:15:40 +00:00
Aaron Myers
b587af4631
HDFS-3710. libhdfs misuses O_RDONLY/WRONLY/RDWR. Contributed by Andy Isaacson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370898 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 19:11:30 +00:00
Aaron Myers
bc88c7b043
HDFS-3773. TestNNWithQJM fails after HDFS-3741. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1370846 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 17:21:27 +00:00
Aaron Myers
34554d1e11
HDFS-3760. primitiveCreate is a write, not a read. Contributed by Andy Isaacson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370649 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 02:53:56 +00:00
Todd Lipcon
3a53ef4a80
HDFS-3741. Exhaustive failure injection test for skipped RPCs. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1370497 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 20:24:01 +00:00
Eli Collins
c12e994eda
HDFS-3754. BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370495 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 20:17:59 +00:00
Aaron Myers
9b4a7900c7
HDFS-3637. Add support for encrypting the DataTransferProtocol. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370354 13f79535-47bb-0310-9956-ffa450edef68
2012-08-07 16:40:03 +00:00
Aaron Myers
b4545bf5aa
Add two new files missed by last commit of HDFS-3579.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370017 13f79535-47bb-0310-9956-ffa450edef68
2012-08-06 21:04:33 +00:00
Aaron Myers
f7b37c009b
HDFS-3579. libhdfs: fix exception handling. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370015 13f79535-47bb-0310-9956-ffa450edef68
2012-08-06 21:02:30 +00:00
Alejandro Abdelnur
42d071ab6f
HDFS-3756. DelegationTokenFetcher creates 2 HTTP connections, the second one not properly configured. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1368733 13f79535-47bb-0310-9956-ffa450edef68
2012-08-02 22:10:51 +00:00
Eli Collins
1264b8e707
HADOOP-8620. Add -Drequire.fuse and -Drequire.snappy. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1368251 13f79535-47bb-0310-9956-ffa450edef68
2012-08-01 21:23:10 +00:00
Tsz-wo Sze
cb787968c5
HDFS-3667. Add retry support to WebHdfsFileSystem.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367841 13f79535-47bb-0310-9956-ffa450edef68
2012-08-01 01:41:23 +00:00
Tsz-wo Sze
556be2af92
svn merge -c -1366601 for reverting HDFS-3667.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367407 13f79535-47bb-0310-9956-ffa450edef68
2012-07-31 04:33:59 +00:00
Todd Lipcon
e08604907c
HDFS-3725. Fix QJM startup when individual JNs have gaps. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1367366 13f79535-47bb-0310-9956-ffa450edef68
2012-07-30 23:35:22 +00:00
Todd Lipcon
e1dff3df99
Merge trunk into QJM branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1367365 13f79535-47bb-0310-9956-ffa450edef68
2012-07-30 23:31:42 +00:00
Aaron Myers
aeade62b95
HDFS-3732. fuse_dfs: incorrect configuration value checked for connection expiry timer period. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367199 13f79535-47bb-0310-9956-ffa450edef68
2012-07-30 18:07:07 +00:00
Tsz-wo Sze
e4eec269d9
HDFS-3667. Add retry support to WebHdfsFileSystem.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366601 13f79535-47bb-0310-9956-ffa450edef68
2012-07-28 05:57:47 +00:00
Suresh Srinivas
0c4e670eff
HDFS-3679. fuse_dfs notrash option sets usetrash. Contributed by Conrad Meyer.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366545 13f79535-47bb-0310-9956-ffa450edef68
2012-07-27 21:51:07 +00:00
Daryn Sharp
cbd59c1c50
HDFS-3553. Hftp proxy tokens are broken (daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366471 13f79535-47bb-0310-9956-ffa450edef68
2012-07-27 18:02:56 +00:00
Daryn Sharp
b3b72482e4
HADOOP-8613. AbstractDelegationTokenIdentifier#getUser() should set token auth type. (daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366440 13f79535-47bb-0310-9956-ffa450edef68
2012-07-27 16:39:51 +00:00
Aaron Myers
c1ea9b4490
HDFS-3650. Use MutableQuantiles to provide latency histograms for various operations. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1366246 13f79535-47bb-0310-9956-ffa450edef68
2012-07-27 00:26:21 +00:00
Tsz-wo Sze
8f395c2f78
HDFS-3696. Set chunked streaming mode in WebHdfsFileSystem write operations to get around a Java library bug causing OutOfMemoryError.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365839 13f79535-47bb-0310-9956-ffa450edef68
2012-07-25 23:37:25 +00:00
Todd Lipcon
972953bd77
HDFS-3626. Creating file with invalid path can corrupt edit log. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365801 13f79535-47bb-0310-9956-ffa450edef68
2012-07-25 21:52:38 +00:00
Todd Lipcon
b17018e4b8
HDFS-3693. JNStorage should read its storage info even before a writer becomes active. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1365794 13f79535-47bb-0310-9956-ffa450edef68
2012-07-25 21:47:19 +00:00
Todd Lipcon
d2d0736de4
HDFS-3692. Support purgeEditLogs() call to remotely purge logs on JNs. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1365792 13f79535-47bb-0310-9956-ffa450edef68
2012-07-25 21:44:26 +00:00
Todd Lipcon
939f4a9f92
HDFS-3694. Fix getEditLogManifest to fetch httpPort if necessary. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1365788 13f79535-47bb-0310-9956-ffa450edef68
2012-07-25 21:40:17 +00:00
Aaron Myers
d8f3913856
HDFS-3711. Manually convert remaining tests to JUnit4. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1365119 13f79535-47bb-0310-9956-ffa450edef68
2012-07-24 15:19:08 +00:00
Todd Lipcon
b7b8db51de
HDFS-3697. Enable fadvise readahead by default. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1364698 13f79535-47bb-0310-9956-ffa450edef68
2012-07-23 16:27:59 +00:00
Aaron Myers
02a5551ba3
HDFS-3608. fuse_dfs: detect changes in UID ticket cache. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363904 13f79535-47bb-0310-9956-ffa450edef68
2012-07-20 19:15:52 +00:00
Todd Lipcon
9833468302
HDFS-3597. SNN fails to start after DFS upgrade. Contributed by Andy Isaacson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363899 13f79535-47bb-0310-9956-ffa450edef68
2012-07-20 18:58:58 +00:00
Todd Lipcon
74d4573a23
HDFS-3077. Quorum-based protocol for reading and writing edit logs. Contributed by Todd Lipcon based on initial work from Brandon Li and Hari Mankude.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1363596 13f79535-47bb-0310-9956-ffa450edef68
2012-07-20 00:25:50 +00:00
Eli Collins
6383679872
HDFS-3690. BlockPlacementPolicyDefault incorrectly casts LOG. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363576 13f79535-47bb-0310-9956-ffa450edef68
2012-07-19 23:04:01 +00:00
Eli Collins
5d5a83877a
HDFS-1249. With fuse-dfs, chown which only has owner (or only group) argument fails with Input/output error. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363466 13f79535-47bb-0310-9956-ffa450edef68
2012-07-19 18:37:47 +00:00
Eli Collins
7d5d63c13e
HDFS-3675. libhdfs: follow documented return codes. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363459 13f79535-47bb-0310-9956-ffa450edef68
2012-07-19 18:26:04 +00:00
Eli Collins
15eec42316
HDFS-3673. libhdfs: fix some compiler warnings. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363457 13f79535-47bb-0310-9956-ffa450edef68
2012-07-19 18:22:56 +00:00
Todd Lipcon
23b6ed973e
HDFS-3605. Block mistakenly marked corrupt during edit log catchup phase of failover. Contributed by Todd Lipcon and Brahma Reddy Battula.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363175 13f79535-47bb-0310-9956-ffa450edef68
2012-07-18 23:42:13 +00:00
Daryn Sharp
0a6806ce8c
HDFS-3646. LeaseRenewer can hold reference to inactive DFSClient instances forever (Kihwal Lee via daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1363170 13f79535-47bb-0310-9956-ffa450edef68
2012-07-18 23:21:30 +00:00
Tsz-wo Sze
6e56376023
HDFS-3577. In DatanodeWebHdfsMethods, use MessageBodyWriter instead of StreamingOutput, otherwise, it will fail to transfer large files.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362976 13f79535-47bb-0310-9956-ffa450edef68
2012-07-18 15:12:47 +00:00
Eli Collins
d61c5bcb0b
Revert HDFS-3639. JspHelper#getUGI should always verify the token if security is enabled.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362760 13f79535-47bb-0310-9956-ffa450edef68
2012-07-18 05:11:01 +00:00
Eli Collins
6981b14003
Revert HDFS-3654. TestJspHelper#testGetUgi fails with NPE.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362759 13f79535-47bb-0310-9956-ffa450edef68
2012-07-18 05:05:50 +00:00
Eli Collins
527933f4f3
HDFS-3666. Plumb more exception messages to terminate. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362270 13f79535-47bb-0310-9956-ffa450edef68
2012-07-16 21:26:58 +00:00
Todd Lipcon
6df36b58e1
HDFS-3625. Fix TestBackupNode by properly initializing edit log during startup. Contributed by Junping Du.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362189 13f79535-47bb-0310-9956-ffa450edef68
2012-07-16 18:54:46 +00:00
Eli Collins
69f7a329ba
HDFS-3537. Move libhdfs and fuse-dfs source to native subdirectories. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362119 13f79535-47bb-0310-9956-ffa450edef68
2012-07-16 16:42:53 +00:00
Harsh J
0e8e499ff4
HDFS-3659. Add missing @Override to methods across the hadoop-hdfs project. Contributed by Brandon Li. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361894 13f79535-47bb-0310-9956-ffa450edef68
2012-07-16 02:58:03 +00:00
Eli Collins
3e5e1008db
Amend HDFS-799. Remove unnecessary tls check. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361561 13f79535-47bb-0310-9956-ffa450edef68
2012-07-14 17:02:05 +00:00
Eli Collins
32d4c148df
HDFS-3654. TestJspHelper#testGetUgi fails with NPE. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361463 13f79535-47bb-0310-9956-ffa450edef68
2012-07-14 01:07:57 +00:00
Eli Collins
f5dd4583bb
HDFS-3539. libhdfs code cleanups. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361459 13f79535-47bb-0310-9956-ffa450edef68
2012-07-14 00:47:43 +00:00
Eli Collins
cfd692b866
HDFS-3609. libhdfs: don't force the URI to look like hdfs://hostname:port. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361454 13f79535-47bb-0310-9956-ffa450edef68
2012-07-14 00:34:28 +00:00
Eli Collins
06d635cd2c
HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361449 13f79535-47bb-0310-9956-ffa450edef68
2012-07-14 00:18:56 +00:00
Eli Collins
8c2a2f909c
HDFS-470. libhdfs should handle 0-length reads from FSInputStream correctly. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361446 13f79535-47bb-0310-9956-ffa450edef68
2012-07-14 00:03:58 +00:00
Eli Collins
dcf0ac7a8a
HDFS-3606. libhdfs: create self-contained unit test. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361440 13f79535-47bb-0310-9956-ffa450edef68
2012-07-13 23:28:03 +00:00
Harsh J
d9911e923d
HDFS-3612. Single namenode image directory config warning can be improved. Contributed by Andy Isaacson. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361059 13f79535-47bb-0310-9956-ffa450edef68
2012-07-13 07:20:27 +00:00
Eli Collins
0f3ba4870b
HDFS-799. libhdfs must call DetachCurrentThread when a thread is destroyed. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361008 13f79535-47bb-0310-9956-ffa450edef68
2012-07-13 00:10:04 +00:00
Eli Collins
e2318c2895
HDFS-3633. libhdfs: hdfsDelete should pass JNI_FALSE or JNI_TRUE. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1361005 13f79535-47bb-0310-9956-ffa450edef68
2012-07-12 23:52:48 +00:00
Eli Collins
4a5ba3b7bd
HDFS-3641. Move server Util time methods to common and use now instead of System#currentTimeMillis. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360858 13f79535-47bb-0310-9956-ffa450edef68
2012-07-12 19:01:49 +00:00
Eli Collins
8ced7381a4
HDFS-3639. JspHelper#getUGI should always verify the token if security is enabled. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360485 13f79535-47bb-0310-9956-ffa450edef68
2012-07-12 00:12:15 +00:00
Eli Collins
cdae6953e8
HDFS-3582. Hook daemon process exit for testing. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360329 13f79535-47bb-0310-9956-ffa450edef68
2012-07-11 17:58:02 +00:00
Aaron Myers
664ca6b75e
HDFS-3615. Two BlockTokenSecretManager findbugs warnings. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360255 13f79535-47bb-0310-9956-ffa450edef68
2012-07-11 16:09:18 +00:00
Harsh J
2c27b5826d
HDFS-3611. NameNode prints unnecessary WARNs about edit log normally skipping a few bytes. Contributed by Colin Patrick McCabe. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360001 13f79535-47bb-0310-9956-ffa450edef68
2012-07-11 03:37:07 +00:00
Harsh J
c81aed0271
HDFS-3613. GSet prints some INFO level values, which aren't really very useful to all. Contributed by Andrew Wang. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359999 13f79535-47bb-0310-9956-ffa450edef68
2012-07-11 03:15:03 +00:00
Harsh J
3b5ea87502
HDFS-3629. Fix the typo in the error message about inconsistent storage layout version. Contributed by Brandon Li. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359905 13f79535-47bb-0310-9956-ffa450edef68
2012-07-10 21:14:27 +00:00
Aaron Myers
fb4a6ac9e9
HDFS-3568. fuse_dfs: add support for security. Contributed by Colin McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359824 13f79535-47bb-0310-9956-ffa450edef68
2012-07-10 18:28:13 +00:00
Harsh J
e0f96aa249
HDFS-3555. idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359619 13f79535-47bb-0310-9956-ffa450edef68
2012-07-10 11:52:13 +00:00
Eli Collins
21fdf16b0d
HDFS-3548. NamenodeFsck.copyBlock fails to create a Block Reader. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358822 13f79535-47bb-0310-9956-ffa450edef68
2012-07-08 19:41:55 +00:00
Uma Maheswara Rao G
07295260b1
HDFS-3482. hdfs balancer throws ArrayIndexOutOfBoundsException if option is specified without values. Contributed by Madhukara Phatak.
...
Submitted by: Madhukara Phatak.
Reviewed by: Uma Maheswara Rao G.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358812 13f79535-47bb-0310-9956-ffa450edef68
2012-07-08 18:39:09 +00:00
Eli Collins
52f5c70330
HDFS-711. hdfsUtime does not handle atime = 0 or mtime = 0 correctly. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358810 13f79535-47bb-0310-9956-ffa450edef68
2012-07-08 18:29:00 +00:00
Uma Maheswara Rao G
d4fb882163
HDFS-3541. Deadlock between recovery, xceiver and packet responder. Contributed by Vinay.
...
Submitted by: Vinay
Reviewed by: Uma Maheswara Rao G
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358794 13f79535-47bb-0310-9956-ffa450edef68
2012-07-08 17:42:19 +00:00
Harsh J
e5b00f1bdc
HDFS-2988. Improve error message when storage directory lock fails. Contributed by Miomir Boljanovic. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358683 13f79535-47bb-0310-9956-ffa450edef68
2012-07-08 00:36:38 +00:00
Eli Collins
535b039581
HDFS-3604. Add dfs.webhdfs.enabled to hdfs-default.xml. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358659 13f79535-47bb-0310-9956-ffa450edef68
2012-07-07 22:35:14 +00:00
Aaron Myers
fb95fce240
Fix issue with NN/DN re-registration.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1358347 13f79535-47bb-0310-9956-ffa450edef68
2012-07-06 18:59:55 +00:00
Todd Lipcon
e0ef844280
HDFS-3170. Add more useful metrics for write latency. Contributed by Matthew Jacobs.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1357970 13f79535-47bb-0310-9956-ffa450edef68
2012-07-05 22:18:30 +00:00
Tsz-wo Sze
4d0cab2729
HDFS-3601. Add BlockPlacementPolicyWithNodeGroup to support block placement with 4-layer network topology. Contributed by Junping Du
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1357442 13f79535-47bb-0310-9956-ffa450edef68
2012-07-05 01:31:57 +00:00
Eli Collins
359b4efd19
HDFS-3581. FSPermissionChecker#checkPermission sticky bit check missing range check. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356971 13f79535-47bb-0310-9956-ffa450edef68
2012-07-03 21:43:25 +00:00
Todd Lipcon
3728d16160
HDFS-3574. Fix small race and do some cleanup in GetImageServlet. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356939 13f79535-47bb-0310-9956-ffa450edef68
2012-07-03 20:55:29 +00:00
Todd Lipcon
b8389e4c73
HDFS-3343. Improve metrics for DN read latency. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356928 13f79535-47bb-0310-9956-ffa450edef68
2012-07-03 20:45:21 +00:00
Todd Lipcon
8dd3148e73
HDFS-3190. Simple refactors in existing NN code to assist QuorumJournalManager extension. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356525 13f79535-47bb-0310-9956-ffa450edef68
2012-07-02 23:59:41 +00:00
Todd Lipcon
7accbabdee
HDFS-3573. Supply NamespaceInfo when instantiating JournalManagers. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356388 13f79535-47bb-0310-9956-ffa450edef68
2012-07-02 18:21:49 +00:00
Tsz-wo Sze
28e8151ad3
HDFS-3157. Fix a bug in the case that the generation stamps of the stored block in a namenode and the reported block from a datanode do not match. Contributed by Ashish Singhi
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356086 13f79535-47bb-0310-9956-ffa450edef68
2012-07-02 07:26:03 +00:00
Suresh Srinivas
19dd66a3f6
HDFS-3510. Editlog pre-allocation is performed prior to writing edits to avoid partial edits case disk out of space. Contributed by Collin McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355189 13f79535-47bb-0310-9956-ffa450edef68
2012-06-28 23:00:10 +00:00
Todd Lipcon
1d54e2b331
HDFS-3571. Allow EditLogFileInputStream to read from a remote URL. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355174 13f79535-47bb-0310-9956-ffa450edef68
2012-06-28 21:46:12 +00:00
Eli Collins
550853203b
HDFS-3475. Make the replication monitor multipliers configurable. Contributed by Harsh J Chouraria
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355089 13f79535-47bb-0310-9956-ffa450edef68
2012-06-28 17:54:52 +00:00
Tsz-wo Sze
f105784d6a
svn merge -c -1354790 for reverting HDFS-3576 since it requires more changes.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354792 13f79535-47bb-0310-9956-ffa450edef68
2012-06-28 01:25:32 +00:00
Tsz-wo Sze
44389399d4
HDFS-3576. Move the definition of the constant NameNode.DEFAULT_PORT to DFSConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT. Contributed by Brandon Li
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354790 13f79535-47bb-0310-9956-ffa450edef68
2012-06-28 01:11:22 +00:00
Todd Lipcon
5770a453f3
HDFS-3572. Cleanup code which inits SPNEGO in HttpServer. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354767 13f79535-47bb-0310-9956-ffa450edef68
2012-06-27 22:41:41 +00:00
Tsz-wo Sze
bf1649d5fd
HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354316 13f79535-47bb-0310-9956-ffa450edef68
2012-06-27 02:51:39 +00:00
Eli Collins
0270889b4e
HDFS-3535. Audit logging should log denied accesses. Contributed by Andy Isaacson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1354144 13f79535-47bb-0310-9956-ffa450edef68
2012-06-26 18:14:09 +00:00
Tsz-wo Sze
bbab35e6d8
HDFS-3498. Support replica removal in BlockPlacementPolicy and make BlockPlacementPolicyDefault extensible for reusing code in subclasses. Contributed by Junping Du
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1353807 13f79535-47bb-0310-9956-ffa450edef68
2012-06-26 03:25:47 +00:00
Tsz-wo Sze
361ea9a62c
HDFS-3516. Check content-type in WebHdfsFileSystem.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1353800 13f79535-47bb-0310-9956-ffa450edef68
2012-06-26 01:58:13 +00:00
Scott Chun-Yang Chen
277b3dd736
MAPREDUCE-3868. Make Raid Compile. (Weiyan Wang via schen)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1351548 13f79535-47bb-0310-9956-ffa450edef68
2012-06-19 00:55:28 +00:00
Tsz-wo Sze
5572f73d45
HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1350825 13f79535-47bb-0310-9956-ffa450edef68
2012-06-15 23:00:36 +00:00
Aaron Myers
9cabf93e7a
HDFS-766. Error message not clear for set space quota out of boundary values. Contributed by Jon Zuanich.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349639 13f79535-47bb-0310-9956-ffa450edef68
2012-06-13 05:29:57 +00:00
Eli Collins
6702d5dbd4
HDFS-3372. offlineEditsViewer should be able to read a binary edits file with recovery mode. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349628 13f79535-47bb-0310-9956-ffa450edef68
2012-06-13 03:31:47 +00:00
Eli Collins
07b8584431
HDFS-3531. EditLogFileOutputStream#preallocate should check for incomplete writes. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349616 13f79535-47bb-0310-9956-ffa450edef68
2012-06-13 02:42:52 +00:00
Alejandro Abdelnur
4d4661802e
HDFS-3228. Move DelegationTokenRenewer to common (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349555 13f79535-47bb-0310-9956-ffa450edef68
2012-06-12 21:52:46 +00:00
Eli Collins
797c8548d0
Revert HADOOP-8491
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349133 13f79535-47bb-0310-9956-ffa450edef68
2012-06-12 06:22:06 +00:00
Tsz-wo Sze
45fafc2b8f
HDFS-3504. Support configurable retry policy in DFSClient for RPC connections and RPC calls, and add MultipleLinearRandomRetry, a new retry policy.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349124 13f79535-47bb-0310-9956-ffa450edef68
2012-06-12 05:27:15 +00:00
Eli Collins
8879653ab4
HDFS-3520. Add transfer rate logging to TransferFsImage. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349117 13f79535-47bb-0310-9956-ffa450edef68
2012-06-12 05:00:07 +00:00
Todd Lipcon
9947d8054c
HDFS-3049. During the normal NN startup process, fall back on a different edit log if we see one that is corrupt. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349114 13f79535-47bb-0310-9956-ffa450edef68
2012-06-12 04:15:18 +00:00
Tsz-wo Sze
543f86631b
HDFS-3522. If a namenode is in safemode, it should throw SafeModeException when getBlockLocations has zero locations. Contributed by Brandon Li
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349088 13f79535-47bb-0310-9956-ffa450edef68
2012-06-12 01:55:51 +00:00
Eli Collins
6e3ebf4d73
HADOOP-8491. Check for short writes when using FileChannel#write and related methods. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349019 13f79535-47bb-0310-9956-ffa450edef68
2012-06-11 21:20:28 +00:00
Tsz-wo Sze
ca39e780c4
HDFS-3052. Change INodeFile and INodeFileUnderConstruction to package private.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348998 13f79535-47bb-0310-9956-ffa450edef68
2012-06-11 20:38:46 +00:00
Alejandro Abdelnur
8a9e63e468
HADOOP-8368. Use CMake rather than autotools to build native code (ccccabe via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348957 13f79535-47bb-0310-9956-ffa450edef68
2012-06-11 18:34:40 +00:00
Eli Collins
56d2ef6f5e
HDFS-2797. Fix misuses of InputStream#skip in the edit log code. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348945 13f79535-47bb-0310-9956-ffa450edef68
2012-06-11 18:01:38 +00:00
Tsz-wo Sze
5991ed9cbd
HDFS-3490. DatanodeWebHdfsMethods throws NullPointerException if NamenodeRpcAddressParam is not set.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1348287 13f79535-47bb-0310-9956-ffa450edef68
2012-06-09 00:54:10 +00:00
Aaron Myers
b98e26af58
HDFS-2914. HA: Standby should not enter safemode when resources are low. Contributed by Vinay.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347895 13f79535-47bb-0310-9956-ffa450edef68
2012-06-08 06:57:52 +00:00
Tsz-wo Sze
415ce38b82
HADOOP-8469. Make NetworkTopology class pluggable. Contributed by Junping Du
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347867 13f79535-47bb-0310-9956-ffa450edef68
2012-06-08 02:29:40 +00:00
Todd Lipcon
05a73a3a1e
Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347796 13f79535-47bb-0310-9956-ffa450edef68
2012-06-07 21:06:13 +00:00
Todd Lipcon
6d8efb7378
HDFS-3485. DataTransferThrottler will over-throttle when currentTimeMillis jumps. Contributed by Andy Isaacson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347751 13f79535-47bb-0310-9956-ffa450edef68
2012-06-07 18:41:24 +00:00
Tsz-wo Sze
743cbf6ef3
svn merge -c -1347092 for reverting HADOOP-8368 again.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347738 13f79535-47bb-0310-9956-ffa450edef68
2012-06-07 17:56:04 +00:00
Todd Lipcon
bef65d0601
HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347192 13f79535-47bb-0310-9956-ffa450edef68
2012-06-06 22:48:48 +00:00
Eli Collins
8813f42175
HDFS-3505. DirectoryScanner does not join all threads in shutdown. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347117 13f79535-47bb-0310-9956-ffa450edef68
2012-06-06 20:56:26 +00:00
Alejandro Abdelnur
8ad1fcb3c3
svn merge -c -1346491 for re-committing HADOOP-8368. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1347092 13f79535-47bb-0310-9956-ffa450edef68
2012-06-06 20:17:23 +00:00
Tsz-wo Sze
d44513c376
svn merge -c -1345421 for reverting HADOOP-8368.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1346491 13f79535-47bb-0310-9956-ffa450edef68
2012-06-05 17:56:20 +00:00
Eli Collins
8bab8b7752
HDFS-3501. Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1346106 13f79535-47bb-0310-9956-ffa450edef68
2012-06-04 19:37:56 +00:00
Uma Maheswara Rao G
28ca281cd6
HDFS-2025. Go Back to File View link is not working in tail.jsp. Contributed by Ashish and Sravan.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1345563 13f79535-47bb-0310-9956-ffa450edef68
2012-06-02 19:41:33 +00:00
Alejandro Abdelnur
2db30031d2
HADOOP-8368. Use CMake rather than autotools to build native code (ccccabe via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1345421 13f79535-47bb-0310-9956-ffa450edef68
2012-06-02 02:34:38 +00:00
Aaron Myers
d1207cbd88
HDFS-3442. Incorrect count for Missing Replicas in FSCK report. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1345408 13f79535-47bb-0310-9956-ffa450edef68
2012-06-02 00:39:34 +00:00
Eli Collins
f343419055
HDFS-3487. offlineimageviewer should give byte offset information when it encounters an exception. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344972 13f79535-47bb-0310-9956-ffa450edef68
2012-06-01 02:37:09 +00:00
Eli Collins
4eb6d65eef
HDFS-3486. offlineimageviewer can't read fsimage files that contain persistent delegation tokens. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344970 13f79535-47bb-0310-9956-ffa450edef68
2012-06-01 02:31:29 +00:00
Eli Collins
675a7e4acb
HDFS-3484. hdfs fsck doesn't work if NN HTTP address is set to 0.0.0.0 even if NN RPC address is configured. Contributed by Aaron T. Myers
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344908 13f79535-47bb-0310-9956-ffa450edef68
2012-05-31 21:02:08 +00:00
Tsz-wo Sze
83cf475050
HDFS-744. Support hsync in HDFS. Contributed by Lars Hofhans
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344419 13f79535-47bb-0310-9956-ffa450edef68
2012-05-30 19:10:09 +00:00
Uma Maheswara Rao G
47a29c6329
HDFS-3398. Client will not retry when primaryDN is down once it's just got pipeline. Contributed by Amith D K.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1343944 13f79535-47bb-0310-9956-ffa450edef68
2012-05-29 19:37:03 +00:00
Harsh J
d2b57e00bc
HADOOP-8358. Config-related WARN for dfs.web.ugi can be avoided. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1343290 13f79535-47bb-0310-9956-ffa450edef68
2012-05-28 15:36:00 +00:00
Uma Maheswara Rao G
e767845c36
HDFS-2717. BookKeeper Journal output stream doesn't check addComplete rc. Contributed by Ivan Kelly.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342534 13f79535-47bb-0310-9956-ffa450edef68
2012-05-25 08:56:59 +00:00
Harsh J
befd45fcb1
HDFS-2391. Newly set BalancerBandwidth value is not displayed anywhere. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342520 13f79535-47bb-0310-9956-ffa450edef68
2012-05-25 07:04:35 +00:00
Konstantin Shvachko
2e0a15c488
HDFS-3368. Missing blocks due to bad DataNodes coming up and down.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342512 13f79535-47bb-0310-9956-ffa450edef68
2012-05-25 06:29:17 +00:00
Eli Collins
39661c06f7
Revert HDFS-3058
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342493 13f79535-47bb-0310-9956-ffa450edef68
2012-05-25 03:56:12 +00:00
Uma Maheswara Rao G
64ec449010
HDFS-3058. HA: Bring BookKeeperJournalManager up to date with HA changes. Contributed by Ivan Kelly.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342483 13f79535-47bb-0310-9956-ffa450edef68
2012-05-25 01:31:23 +00:00
Todd Lipcon
6c544adeb3
Merge HDFS-3042 (automatic failover) to trunk.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342112 13f79535-47bb-0310-9956-ffa450edef68
2012-05-24 00:56:20 +00:00
Todd Lipcon
14bf2d4357
HDFS-1013. Miscellaneous improvements to HTML markup for web UIs. Contributed by Eugene Koontz.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342102 13f79535-47bb-0310-9956-ffa450edef68
2012-05-24 00:27:51 +00:00
Todd Lipcon
74dfa8f1f2
HDFS-2982. Startup performance suffers when there are many edit log segments. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1342042 13f79535-47bb-0310-9956-ffa450edef68
2012-05-23 20:42:48 +00:00
Tsz-wo Sze
ade42bce10
HDFS-3436. In DataNode.transferReplicaForPipelineRecovery(..), it should use the stored generation stamp to check if the block is valid. Contributed by Vinay
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1341961 13f79535-47bb-0310-9956-ffa450edef68
2012-05-23 17:59:30 +00:00
Eli Collins
445018cea8
HDFS-3454. Balancer unconditionally logs InterruptedException at INFO level on shutdown if security is enabled. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1341916 13f79535-47bb-0310-9956-ffa450edef68
2012-05-23 15:56:59 +00:00
Tsz-wo Sze
aa80e6895d
HDFS-3415. Make sure all layout versions are the same for all storage directories in the Namenode. Contributed by Brandon Li
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1341676 13f79535-47bb-0310-9956-ffa450edef68
2012-05-22 23:18:04 +00:00
Tsz-wo Sze
d69510a7f8
HDFS-3394. Do not use generic in INodeFile.getLastBlock(): the run-time ClassCastException check is useless since generic type information is only available in compile-time.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1341141 13f79535-47bb-0310-9956-ffa450edef68
2012-05-21 18:51:15 +00:00
Eli Collins
260dfa367e
HDFS-2885. Remove "federation" from the nameservice config options. Contributed by Tsz Wo (Nicholas), SZE
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1340841 13f79535-47bb-0310-9956-ffa450edef68
2012-05-20 21:13:17 +00:00
Todd Lipcon
1a3bd5d42f
Merge trunk into auto-HA branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1340622 13f79535-47bb-0310-9956-ffa450edef68
2012-05-20 06:12:14 +00:00
Todd Lipcon
d1a54b872d
HDFS-3438. BootstrapStandby should not require a rollEdits on active node. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1340343 13f79535-47bb-0310-9956-ffa450edef68
2012-05-19 05:13:38 +00:00
Aaron Myers
ea0baa3609
HDFS-3444. hdfs groups command doesn't work with security enabled. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1340260 13f79535-47bb-0310-9956-ffa450edef68
2012-05-18 21:25:21 +00:00
Todd Lipcon
7d1f97b821
HDFS-3440. More effectively limit stream memory consumption when reading corrupt edit logs. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339978 13f79535-47bb-0310-9956-ffa450edef68
2012-05-18 05:28:18 +00:00
Todd Lipcon
5258d6bf3f
HDFS-3391. Fix InvalidateBlocks to compare blocks including their generation stamps. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339897 13f79535-47bb-0310-9956-ffa450edef68
2012-05-17 22:30:46 +00:00
Todd Lipcon
15ddb6634f
HDFS-2800. Fix cancellation of checkpoints in the standby node to be more reliable. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339745 13f79535-47bb-0310-9956-ffa450edef68
2012-05-17 17:49:48 +00:00
Eli Collins
5ef725eb53
HDFS-3434. InvalidProtocolBufferException when visiting DN browseDirectory.jsp. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339712 13f79535-47bb-0310-9956-ffa450edef68
2012-05-17 17:01:13 +00:00
Eli Collins
6d71407582
HDFS-1153. dfsnodelist.jsp should handle invalid input parameters. Contributed by Ravi Phulari
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339706 13f79535-47bb-0310-9956-ffa450edef68
2012-05-17 16:55:49 +00:00
Aaron Myers
1377709b4c
HDFS-3433. GetImageServlet should allow administrative requestors when security is enabled. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339540 13f79535-47bb-0310-9956-ffa450edef68
2012-05-17 10:23:18 +00:00
Todd Lipcon
e14e10fb86
Merge trunk into auto-HA branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1339410 13f79535-47bb-0310-9956-ffa450edef68
2012-05-16 22:47:37 +00:00
Eli Collins
7428aeca86
HDFS-3418. Rename BlockWithLocationsProto datanodeIDs field to storageIDs. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338830 13f79535-47bb-0310-9956-ffa450edef68
2012-05-15 19:03:14 +00:00
Todd Lipcon
52ecdb751e
HDFS-3412. Fix findbugs warnings in auto-HA branch. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1338817 13f79535-47bb-0310-9956-ffa450edef68
2012-05-15 18:16:49 +00:00
Eli Collins
c4e4e56160
HDFS-3419. Cleanup LocatedBlock. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338802 13f79535-47bb-0310-9956-ffa450edef68
2012-05-15 17:23:50 +00:00
Eli Collins
e9a7648f62
HDFS-3416. Cleanup DatanodeID and DatanodeRegistration constructors used by testing. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338784 13f79535-47bb-0310-9956-ffa450edef68
2012-05-15 16:23:52 +00:00
Eli Collins
e4df14f8f1
HDFS-3417. Rename BalancerDatanode#getName to getDisplayName to be consistent with Datanode. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338767 13f79535-47bb-0310-9956-ffa450edef68
2012-05-15 16:01:44 +00:00
Todd Lipcon
95710c15b7
HDFS-3335. check for edit log corruption at the end of the log. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338492 13f79535-47bb-0310-9956-ffa450edef68
2012-05-15 00:41:37 +00:00
Todd Lipcon
bcdb125643
HDFS-3031. Fix complete() and getAdditionalBlock() RPCs to be idempotent. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338466 13f79535-47bb-0310-9956-ffa450edef68
2012-05-14 22:35:25 +00:00
Aaron Myers
05c5fcfb42
HDFS-3414. Balancer does not find NameNode if rpc-address or servicerpc-address are not set in client configs. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338275 13f79535-47bb-0310-9956-ffa450edef68
2012-05-14 16:25:17 +00:00
Harsh J
f1560d379d
HDFS-2312. FSNamesystem javadoc incorrectly says its for DNs. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1337761 13f79535-47bb-0310-9956-ffa450edef68
2012-05-13 03:33:19 +00:00
Aaron Myers
f22341ab1a
HDFS-3404. Make putImage in GetImageServlet infer remote address to fetch from request. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1337755 13f79535-47bb-0310-9956-ffa450edef68
2012-05-13 02:51:33 +00:00
Todd Lipcon
4cd70e87be
Merge trunk into auto-HA branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1337645 13f79535-47bb-0310-9956-ffa450edef68
2012-05-12 20:52:34 +00:00
Thomas White
aea890f7d2
MAPREDUCE-4148. MapReduce should not have a compile-time dependency on HDFS.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1337199 13f79535-47bb-0310-9956-ffa450edef68
2012-05-11 15:00:48 +00:00
Aaron Myers
5dbd09ee76
HDFS-3026. HA: Handle failure during HA state transition. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1337030 13f79535-47bb-0310-9956-ffa450edef68
2012-05-11 05:57:37 +00:00
Eli Collins
8def56e51e
HDFS-3400. DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1337017 13f79535-47bb-0310-9956-ffa450edef68
2012-05-11 03:15:54 +00:00
Tsz-wo Sze
ba6360e493
HDFS-3385. The last block of INodeFileUnderConstruction is not necessarily a BlockInfoUnderConstruction, so do not cast it in FSNamesystem.recoverLeaseInternal(..).
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336976 13f79535-47bb-0310-9956-ffa450edef68
2012-05-11 00:25:48 +00:00
Eli Collins
98b00d7cc0
HDFS-3134. harden edit log loader against malformed or malicious input. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336943 13f79535-47bb-0310-9956-ffa450edef68
2012-05-10 23:13:02 +00:00
Tsz-wo Sze
7e8e983620
HDFS-3369. Rename {get|set|add}INode(..) methods in BlockManager and BlocksMap to {get|set|add}BlockCollection(..). Contributed by John George
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336909 13f79535-47bb-0310-9956-ffa450edef68
2012-05-10 21:41:25 +00:00
Eli Collins
6a0865440e
HDFS-3230. Cleanup DatanodeID creation in the tests. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336815 13f79535-47bb-0310-9956-ffa450edef68
2012-05-10 17:58:55 +00:00
Aaron Myers
f5960f71d9
HDFS-3395. NN doesn't start with HA+security enabled and HTTP address set to 0.0.0.0. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336690 13f79535-47bb-0310-9956-ffa450edef68
2012-05-10 13:49:06 +00:00
Uma Maheswara Rao G
f1ff05bf47
Reverting (Need to re-do the patch. new BlockInfo does not set iNode ) HDFS-3157. Error in deleting block is keep on coming from DN even after the block report and directory scanning has happened.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336572 13f79535-47bb-0310-9956-ffa450edef68
2012-05-10 09:59:39 +00:00
Eli Collins
a67054d7c3
HDFS-3328. NPE in DataNode.getIpcPort. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336480 13f79535-47bb-0310-9956-ffa450edef68
2012-05-10 04:52:46 +00:00
Todd Lipcon
b75f0187c8
HDFS-3341, HADOOP-8340. SNAPSHOT build versions should compare as less than their eventual release. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336459 13f79535-47bb-0310-9956-ffa450edef68
2012-05-10 00:30:53 +00:00
Aaron Myers
e3fbc52686
HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1336324 13f79535-47bb-0310-9956-ffa450edef68
2012-05-09 18:16:40 +00:00
Tsz-wo Sze
27ce5a6fb0
Remove the empty file FSInodeInfo.java for HDFS-3363.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335788 13f79535-47bb-0310-9956-ffa450edef68
2012-05-08 21:53:11 +00:00
Uma Maheswara Rao G
408e558d8d
HDFS-3157. Error in deleting block is keep on coming from DN even after the block report and directory scanning has happened. Contributed by Ashish Singhi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335719 13f79535-47bb-0310-9956-ffa450edef68
2012-05-08 19:45:38 +00:00
Eli Collins
1d7b980592
HDFS-3378. Remove DFS_NAMENODE_SECONDARY_HTTPS_PORT_KEY and DEFAULT. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335309 13f79535-47bb-0310-9956-ffa450edef68
2012-05-08 00:25:34 +00:00
Tsz-wo Sze
f0f9a3631f
HDFS-3363. Define BlockCollection and MutableBlockCollection interfaces so that INodeFile and INodeFileUnderConstruction do not have to be used in block management. Contributed by John George
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335304 13f79535-47bb-0310-9956-ffa450edef68
2012-05-08 00:06:09 +00:00
Todd Lipcon
7aa2889f82
HDFS-3375. Put client name in DataXceiver thread name for readBlock and keepalive. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335270 13f79535-47bb-0310-9956-ffa450edef68
2012-05-07 21:34:30 +00:00
Todd Lipcon
567aed4f2c
HDFS-3365. Enable users to disable socket caching in DFS client configuration. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335222 13f79535-47bb-0310-9956-ffa450edef68
2012-05-07 19:56:23 +00:00
Todd Lipcon
eca2c850a5
HDFS-3376. DFSClient fails to make connection to DN if there are many unusable cached sockets. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335115 13f79535-47bb-0310-9956-ffa450edef68
2012-05-07 16:42:19 +00:00
Aaron Myers
51e520c68a
HADOOP-8349. ViewFS doesn't work when the root of a file system is mounted. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1334231 13f79535-47bb-0310-9956-ffa450edef68
2012-05-04 22:14:10 +00:00
Eli Collins
5dbbe0e0a5
HDFS-2617. Replaced Kerberized SSL for image transfer and fsck with SPNEGO-based solution. Contributed by Jakob Homan, Alejandro Abdelnur, and Aaron T. Myers
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1334216 13f79535-47bb-0310-9956-ffa450edef68
2012-05-04 21:58:44 +00:00
Tsz-wo Sze
0bf8f11860
HDFS-3303. Remove Writable implementation from RemoteEditLogManifest. Contributed by Brandon Li
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1334156 13f79535-47bb-0310-9956-ffa450edef68
2012-05-04 20:16:23 +00:00
Todd Lipcon
a701c792f8
HDFS-3357. DataXceiver reads from client socket with incorrect/no timeout. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1334116 13f79535-47bb-0310-9956-ffa450edef68
2012-05-04 18:50:54 +00:00
Tsz-wo Sze
dd8b7ae9d8
HDFS-3350. In INode, add final to compareTo(..), equals(..) and hashCode(), and remove synchronized from updatePermissionStatus(..).
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333679 13f79535-47bb-0310-9956-ffa450edef68
2012-05-03 23:33:44 +00:00
Todd Lipcon
03181022ab
HADOOP-8350. Improve NetUtils.getInputStream to return a stream which has a tunable timeout. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333649 13f79535-47bb-0310-9956-ffa450edef68
2012-05-03 21:57:10 +00:00
Todd Lipcon
25882b199b
HDFS-3359. DFSClient.close should close cached sockets. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333624 13f79535-47bb-0310-9956-ffa450edef68
2012-05-03 21:13:43 +00:00
Uma Maheswara Rao G
9aae7c22ac
HDFS-3332. NullPointerException in DN when directoryscanner is trying to report bad blocks. Contributed by Amith D K.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333587 13f79535-47bb-0310-9956-ffa450edef68
2012-05-03 19:05:38 +00:00
Aaron Myers
470b1197be
HDFS-3351. NameNode#initializeGenericKeys should always set fs.defaultFS regardless of whether HA or Federation is enabled. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333299 13f79535-47bb-0310-9956-ffa450edef68
2012-05-03 03:00:10 +00:00
Todd Lipcon
f6c01e2f8e
Merge trunk into auto-HA branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1333291 13f79535-47bb-0310-9956-ffa450edef68
2012-05-03 02:14:01 +00:00
Todd Lipcon
9d5799553f
HADOOP-8279. Allow manual failover to be invoked when auto-failover is enabled. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1333288 13f79535-47bb-0310-9956-ffa450edef68
2012-05-03 01:56:33 +00:00
Todd Lipcon
cbc2424290
HDFS-3330. If GetImageServlet throws an Error or RTE, response should not have HTTP "OK" status. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333286 13f79535-47bb-0310-9956-ffa450edef68
2012-05-03 01:44:22 +00:00
Alejandro Abdelnur
e1a961d1bc
HDFS-3336. hdfs launcher script will be better off not special casing namenode command with regards to hadoop.security.logger (rvs via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333236 13f79535-47bb-0310-9956-ffa450edef68
2012-05-02 22:47:04 +00:00
Tsz-wo Sze
8620a99d1e
HDFS-3339. Change INode to package private. Contributed by John George
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1332876 13f79535-47bb-0310-9956-ffa450edef68
2012-05-01 23:02:53 +00:00
Uma Maheswara Rao G
d37ec9d09e
HDFS-3286. When the threshold value for balancer is zero, unexpected output is displayed. Contributed by Ashish Singhi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1332531 13f79535-47bb-0310-9956-ffa450edef68
2012-05-01 06:44:12 +00:00
Uma Maheswara Rao G
086fa860c0
HDFS-3275. Skip format for non-file based directories. Contributed by Amith D K.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1332527 13f79535-47bb-0310-9956-ffa450edef68
2012-05-01 05:47:27 +00:00
Tsz-wo Sze
2af70cb517
HDFS-3293. Add toString(), equals(..) and hashCode() to JournalInfo. Contributed by Hari Mankude
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1332458 13f79535-47bb-0310-9956-ffa450edef68
2012-04-30 22:57:22 +00:00
Matthew Foley
c203c7628c
HDFS-3265. PowerPc Build error. Contributed by Kumar Ravi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1332107 13f79535-47bb-0310-9956-ffa450edef68
2012-04-30 08:27:34 +00:00
Eli Collins
a22c138291
HDFS-3326. Append enabled log message uses the wrong variable. Contributed by Matthew Jacobs
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331626 13f79535-47bb-0310-9956-ffa450edef68
2012-04-27 23:03:45 +00:00
Tsz-wo Sze
cd271773ac
HDFS-3331. In namenode, check superuser privilege for setBalancerBandwidth and acquire the write lock for finalizeUpgrade.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331598 13f79535-47bb-0310-9956-ffa450edef68
2012-04-27 21:17:53 +00:00
Tsz-wo Sze
920b8fac18
HDFS-3334. Fix ByteRangeInputStream stream leakage. Contributed by Daryn Sharp
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331570 13f79535-47bb-0310-9956-ffa450edef68
2012-04-27 20:13:17 +00:00
Tsz-wo Sze
1a76c82a31
HDFS-3322. Use HdfsDataInputStream and HdfsDataOutputStream in Hdfs.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331114 13f79535-47bb-0310-9956-ffa450edef68
2012-04-26 21:50:50 +00:00
Uma Maheswara Rao G
d28b982428
HDFS-3222. DFSInputStream#openInfo should not silently get the length as 0 when locations length is zero for last partial block. Contributed by Uma Maheswara Rao G.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1331061 13f79535-47bb-0310-9956-ffa450edef68
2012-04-26 20:18:01 +00:00
Tsz-wo Sze
3b773da036
HDFS-3319. Change DFSOutputStream to not to start a thread in constructors.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330535 13f79535-47bb-0310-9956-ffa450edef68
2012-04-25 20:17:20 +00:00
Tsz-wo Sze
2584779166
HDFS-3321. Fix safe mode turn off tip message. Contributed by Ravi Prakash
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330506 13f79535-47bb-0310-9956-ffa450edef68
2012-04-25 19:34:36 +00:00
Tsz-wo Sze
f86352c2df
HDFS-3318. Use BoundedInputStream in ByteRangeInputStream, otherwise, it hangs on transfers >2 GB. Contributed by Daryn Sharp
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330500 13f79535-47bb-0310-9956-ffa450edef68
2012-04-25 19:24:00 +00:00
Tsz-wo Sze
ea32198db4
HDFS-3298. Add HdfsDataOutputStream as a public API.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330064 13f79535-47bb-0310-9956-ffa450edef68
2012-04-24 23:39:43 +00:00
Alejandro Abdelnur
706e861a85
HADOOP-7549. Use JDK ServiceLoader mechanism to find FileSystem implementations. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329994 13f79535-47bb-0310-9956-ffa450edef68
2012-04-24 20:33:05 +00:00
Tsz-wo Sze
93c5585d31
HDFS-3312. In HftpFileSystem, the namenode URI is non-secure but the delegation tokens have to use secure URI. Contributed by Daryn Sharp
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329462 13f79535-47bb-0310-9956-ffa450edef68
2012-04-23 21:27:01 +00:00
Sanjay Radia
589c68ae09
HADOOP-8285 Use ProtoBuf for RpcPayLoadHeader (sanjay radia)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329319 13f79535-47bb-0310-9956-ffa450edef68
2012-04-23 16:34:21 +00:00
Tsz-wo Sze
cdbe15ccdc
HDFS-3308. Uses canonical URI to select delegation tokens in HftpFileSystem and WebHdfsFileSystem. Contributed by Daryn Sharp
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328541 13f79535-47bb-0310-9956-ffa450edef68
2012-04-20 23:59:19 +00:00
Tsz-wo Sze
201af20406
Revert r1328482 for HDFS-3308.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328487 13f79535-47bb-0310-9956-ffa450edef68
2012-04-20 19:32:23 +00:00
Tsz-wo Sze
ab8f458742
HDFS-3308. Uses canonical URI to select delegation tokens in HftpFileSystem and WebHdfsFileSystem. Contributed by Daryn Sharp
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328482 13f79535-47bb-0310-9956-ffa450edef68
2012-04-20 19:16:58 +00:00
Aaron Myers
49ed783a3a
HDFS-3305. GetImageServlet should consider SBN a valid requestor in a secure HA setup. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328115 13f79535-47bb-0310-9956-ffa450edef68
2012-04-19 20:43:03 +00:00
Uma Maheswara Rao G
ba4ec5f0aa
iHDFS-3282. Expose getFileLength API. Contributed by Uma Maheswara Rao G.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327790 13f79535-47bb-0310-9956-ffa450edef68
2012-04-19 02:41:32 +00:00
Uma Maheswara Rao G
c6d3537d33
HDFS-3282. Expose getFileLength API. Contributed by Uma Maheswara Rao G.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327788 13f79535-47bb-0310-9956-ffa450edef68
2012-04-19 02:34:44 +00:00
Eli Collins
54e61ffdc0
HDFS-3206. Miscellaneous xml cleanups for OEV. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327768 13f79535-47bb-0310-9956-ffa450edef68
2012-04-19 01:31:53 +00:00
Eli Collins
0c16ec4d6e
HDFS-891. DataNode no longer needs to check for dfs.network.script. Contributed by Harsh J
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327762 13f79535-47bb-0310-9956-ffa450edef68
2012-04-19 01:09:12 +00:00
Todd Lipcon
ca49126e3d
Merge trunk into auto-HA branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1327724 13f79535-47bb-0310-9956-ffa450edef68
2012-04-18 23:35:30 +00:00
Tsz-wo Sze
2bf8be3c72
HDFS-3292. Remove the deprecated DiskStatus, getDiskStatus(), getRawCapacity() and getRawUsed() from DistributedFileSystem. Contributed by Arpit Gupta
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327664 13f79535-47bb-0310-9956-ffa450edef68
2012-04-18 20:21:08 +00:00
Tsz-wo Sze
eeec4dc72a
HDFS-3294. Fix code indentation in NamenodeWebHdfsMethods and DatanodeWebHdfsMethods.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327365 13f79535-47bb-0310-9956-ffa450edef68
2012-04-18 03:25:08 +00:00
Tsz-wo Sze
c80dbe5e09
HDFS-2652. Add support for host-based delegation tokens. Contributed by Daryn Sharp
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327309 13f79535-47bb-0310-9956-ffa450edef68
2012-04-17 22:21:33 +00:00
Aaron Myers
c21bd72a2f
HADOOP-8280. Move VersionUtil/TestVersionUtil and GenericTestUtils from HDFS into Common. Contributed by Ahmed Radwan.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327182 13f79535-47bb-0310-9956-ffa450edef68
2012-04-17 17:06:38 +00:00
Aaron Myers
1675c18ded
Revert HADOOP-8280 so it can be recommitted using `svn mv'
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327179 13f79535-47bb-0310-9956-ffa450edef68
2012-04-17 16:49:04 +00:00
Eli Collins
1d5861a8c4
HDFS-3165. HDFS Balancer scripts are refering to wrong path of hadoop-daemon.sh. Contributed by Amith D K
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326848 13f79535-47bb-0310-9956-ffa450edef68
2012-04-17 00:50:40 +00:00
Todd Lipcon
574f99bd6b
HDFS-3284. bootstrapStandby fails in secure cluster. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326813 13f79535-47bb-0310-9956-ffa450edef68
2012-04-16 21:51:52 +00:00
Tsz-wo Sze
f1667dee14
HDFS-3279. Move the FSEditLog constructor with @VisibleForTesting to TestEditLog. Contributed by Arpit Gupta
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326762 13f79535-47bb-0310-9956-ffa450edef68
2012-04-16 19:31:34 +00:00
Todd Lipcon
258da66cc7
HDFS-3268. FileContext API mishandles token service and incompatible with HA. Contributed by Daryn Sharp.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326747 13f79535-47bb-0310-9956-ffa450edef68
2012-04-16 19:08:29 +00:00
Aaron Myers
551468385c
HADOOP-8280. Move VersionUtil/TestVersionUtil and GenericTestUtils from HDFS into Common. Contributed by Ahmed Radwan.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326727 13f79535-47bb-0310-9956-ffa450edef68
2012-04-16 18:28:36 +00:00
Aaron Myers
15fe3ae61b
HDFS-2708. Stats for the # of blocks per DN. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326039 13f79535-47bb-0310-9956-ffa450edef68
2012-04-14 01:51:03 +00:00
Tsz-wo Sze
841fdc5628
HDFS-3273. Refactor BackupImage and FSEditLog, and rename JournalListener.rollLogs(..) to startLogSegment(..).
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1326016 13f79535-47bb-0310-9956-ffa450edef68
2012-04-13 23:41:01 +00:00
Eli Collins
5a20d446cf
HDFS-2799. Trim fs.checkpoint.dir values. Contributed by Amith D K
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325963 13f79535-47bb-0310-9956-ffa450edef68
2012-04-13 21:31:27 +00:00
Todd Lipcon
2ed825e66c
HDFS-3159. Document NN auto-failover setup and configuration. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1325539 13f79535-47bb-0310-9956-ffa450edef68
2012-04-12 21:53:42 +00:00
Aaron Myers
4f230adc13
HDFS-3256. HDFS considers blocks under-replicated if topology script is configured with only 1 rack. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325531 13f79535-47bb-0310-9956-ffa450edef68
2012-04-12 21:28:44 +00:00
Aaron Myers
07a4367445
HDFS-3259. NameNode#initializeSharedEdits should populate shared edits dir with edit log segments. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325518 13f79535-47bb-0310-9956-ffa450edef68
2012-04-12 21:11:32 +00:00
Todd Lipcon
72406e940a
HDFS-3255. HA DFS returns wrong token service. Contributed by Daryn Sharp.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325414 13f79535-47bb-0310-9956-ffa450edef68
2012-04-12 18:08:00 +00:00
Aaron Myers
7f427646df
HDFS-2983. Relax the build version check to permit rolling upgrades within a release. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325110 13f79535-47bb-0310-9956-ffa450edef68
2012-04-12 03:36:13 +00:00
Tsz-wo Sze
ed678e52ce
HDFS-3179. Improve the exception message thrown by DataStreamer when it failed to add a datanode.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1324892 13f79535-47bb-0310-9956-ffa450edef68
2012-04-11 17:49:28 +00:00
Todd Lipcon
6f2aae046d
Resolve trivial compilation error after previous merge
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1324570 13f79535-47bb-0310-9956-ffa450edef68
2012-04-11 05:52:31 +00:00
Todd Lipcon
2bf19979b3
Merge trunk into auto-failover branch.
...
Needs a few tweaks to fix compilation - will do in followup commit. This is just a straight merge
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1324567 13f79535-47bb-0310-9956-ffa450edef68
2012-04-11 05:47:40 +00:00
Todd Lipcon
41a014c31b
HADOOP-8247. Add a config to enable auto-HA, which disables manual FailoverController. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1324566 13f79535-47bb-0310-9956-ffa450edef68
2012-04-11 05:40:26 +00:00
Todd Lipcon
dab66bee55
HDFS-3249. Use ToolRunner.confirmPrompt in NameNode. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1324564 13f79535-47bb-0310-9956-ffa450edef68
2012-04-11 05:28:58 +00:00
Todd Lipcon
21824d8232
HDFS-3247. Improve bootstrapStandby behavior when original NN is not active. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1324558 13f79535-47bb-0310-9956-ffa450edef68
2012-04-11 05:16:06 +00:00
Eli Collins
80447bd35a
HDFS-3244. Remove dead writable code from hdfs/protocol. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1312061 13f79535-47bb-0310-9956-ffa450edef68
2012-04-11 01:10:38 +00:00
Todd Lipcon
7c2e10e0cd
HDFS-3248. bootstrapStandby repeated twice in hdfs namenode usage message. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1312028 13f79535-47bb-0310-9956-ffa450edef68
2012-04-10 22:28:25 +00:00
Todd Lipcon
3ffdb9152e
HDFS-3094. add -nonInteractive and -force option to namenode -format command. Contributed by Arpit Gupta.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1312025 13f79535-47bb-0310-9956-ffa450edef68
2012-04-10 22:22:38 +00:00
Eli Collins
926f0a5ae0
HDFS-3238. ServerCommand and friends don't need to be writables. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311774 13f79535-47bb-0310-9956-ffa450edef68
2012-04-10 15:03:17 +00:00
Todd Lipcon
df3e1a3158
HDFS-3240. Drop log level of "heartbeat: ..." in BPServiceActor to DEBUG. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311577 13f79535-47bb-0310-9956-ffa450edef68
2012-04-10 05:12:00 +00:00
Aaron Myers
9597c81f35
HDFS-3236. NameNode does not initialize generic conf keys when started with -initializeSharedEditsDir. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311554 13f79535-47bb-0310-9956-ffa450edef68
2012-04-10 02:17:17 +00:00
Todd Lipcon
91528ce9ed
HDFS-3234. Accidentally left log message in GetConf after HDFS-3226. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311541 13f79535-47bb-0310-9956-ffa450edef68
2012-04-10 00:59:38 +00:00
Eli Collins
706394d039
HDFS-3004. Implement Recovery Mode. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311394 13f79535-47bb-0310-9956-ffa450edef68
2012-04-09 19:39:58 +00:00
Uma Maheswara Rao G
3ced5ea066
HDFS-3119. Overreplicated block is not deleted even after the replication factor is reduced after sync follwed by closing that file. Contributed by Ashish Singhi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311380 13f79535-47bb-0310-9956-ffa450edef68
2012-04-09 19:11:58 +00:00
Todd Lipcon
ee5daee1b7
HDFS-3214. InterDatanodeProtocolServerSideTranslatorPB doesn't handle null response from initReplicaRecovery. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311125 13f79535-47bb-0310-9956-ffa450edef68
2012-04-09 03:22:41 +00:00
Todd Lipcon
693ec453d2
HADOOP-8246. Auto-HA: automatically scope znode by nameservice ID. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1310919 13f79535-47bb-0310-9956-ffa450edef68
2012-04-07 23:24:57 +00:00
Todd Lipcon
ca6f0940fd
Merge trunk into auto-failover branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1310905 13f79535-47bb-0310-9956-ffa450edef68
2012-04-07 22:53:55 +00:00
Tsz-wo Sze
74b4b45651
HDFS-3211. Add fence(..) and replace NamenodeRegistration with JournalInfo and epoch in JournalProtocol. Contributed by suresh
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310649 13f79535-47bb-0310-9956-ffa450edef68
2012-04-07 00:52:12 +00:00
Todd Lipcon
2ab67dce11
HDFS-3226. Allow GetConf tool to print arbitrary keys. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310648 13f79535-47bb-0310-9956-ffa450edef68
2012-04-07 00:15:30 +00:00
Todd Lipcon
d166bd184d
HDFS-3110. Use directRead API to reduce the number of buffer copies in libhdfs. Contributed by Henry Robinson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310185 13f79535-47bb-0310-9956-ffa450edef68
2012-04-06 06:31:10 +00:00
Todd Lipcon
30068a1d83
Merge trunk into auto-failover branch.
...
Addressed two semantic conflicts after the commit of HADOOP-8077 -- test code was referencing the old constant for the fencing methods config key.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1310174 13f79535-47bb-0310-9956-ffa450edef68
2012-04-06 05:07:33 +00:00
Todd Lipcon
d483b6f3fc
HADOOP-8077. HA: fencing method should be able to be configured on a per-NN or per-NS basis. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310173 13f79535-47bb-0310-9956-ffa450edef68
2012-04-06 04:56:26 +00:00
Todd Lipcon
844faefd08
HDFS-3200. Scope all ZKFC configurations by nameservice. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1310165 13f79535-47bb-0310-9956-ffa450edef68
2012-04-06 04:27:44 +00:00
Eli Collins
e505b7e704
HDFS-3208. Bogus entries in hosts files are incorrectly displayed in the report. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310138 13f79535-47bb-0310-9956-ffa450edef68
2012-04-06 00:20:55 +00:00
Eli Collins
112c324151
HDFS-3210. JsonUtil#toJsonMap for for a DatanodeInfo should use "ipAddr" instead of "name". Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310135 13f79535-47bb-0310-9956-ffa450edef68
2012-04-06 00:10:54 +00:00
Suresh Srinivas
861c872541
HDFS-3204. Minor modification to JournalProtocol.proto to make it generic. Contributed by Suresh Srinivas.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310134 13f79535-47bb-0310-9956-ffa450edef68
2012-04-06 00:08:19 +00:00
Tsz-wo Sze
80bbefafaf
HDFS-3178. Add states and state handler for journal synchronization in JournalService.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310115 13f79535-47bb-0310-9956-ffa450edef68
2012-04-05 22:35:53 +00:00
Aaron Myers
8e739ff01e
HDFS-3202. NamespaceInfo PB translation drops build version. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309955 13f79535-47bb-0310-9956-ffa450edef68
2012-04-05 17:18:26 +00:00
Eli Collins
4f6e0a5a65
HDFS-3050. rework OEV to share more code with the NameNode. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309629 13f79535-47bb-0310-9956-ffa450edef68
2012-04-05 00:02:11 +00:00
Aaron Myers
ba688e11c1
HDFS-3102. Add CLI tool to initialize the shared-edits dir. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309580 13f79535-47bb-0310-9956-ffa450edef68
2012-04-04 20:15:08 +00:00
Todd Lipcon
00a1548005
Merge trunk into auto-HA branch
...
Resolved some trivial conflicts in NNHAServiceTarget
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1309576 13f79535-47bb-0310-9956-ffa450edef68
2012-04-04 20:00:15 +00:00
Todd Lipcon
a7d6bdc63b
HDFS-3084. FenceMethod.tryFence() and ShellCommandFencer should pass namenodeId as well as host:port. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309526 13f79535-47bb-0310-9956-ffa450edef68
2012-04-04 18:40:43 +00:00
Aaron Myers
7525ee5b67
HDFS-3000. Add a public API for setting quotas. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309227 13f79535-47bb-0310-9956-ffa450edef68
2012-04-04 03:18:49 +00:00
Tsz-wo Sze
8c0366bf10
HDFS-3168. Remove unnecessary "throw IOException" and change fields to final in FSNamesystem and BlockManager.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309218 13f79535-47bb-0310-9956-ffa450edef68
2012-04-04 02:51:34 +00:00
Todd Lipcon
30e1b3bba8
HADOOP-8215. Security support for ZK Failover controller. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1309185 13f79535-47bb-0310-9956-ffa450edef68
2012-04-03 23:37:15 +00:00
Todd Lipcon
39a5fd5c71
Merge trunk into auto-failover branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1309164 13f79535-47bb-0310-9956-ffa450edef68
2012-04-03 21:48:23 +00:00
Tsz-wo Sze
e561cb9e78
HDFS-3176. Use MD5MD5CRC32FileChecksum.readFields() in JsonUtil . Contributed by Kihwal Lee
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309114 13f79535-47bb-0310-9956-ffa450edef68
2012-04-03 19:44:53 +00:00
Tsz-wo Sze
0db8003e2e
HDFS-3166. Add timeout to Hftp connections. Contributed by Daryn Sharp
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1309103 13f79535-47bb-0310-9956-ffa450edef68
2012-04-03 19:26:57 +00:00
Todd Lipcon
789ed57e71
HDFS-2185. HDFS portion of ZK-based FailoverController. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1308637 13f79535-47bb-0310-9956-ffa450edef68
2012-04-03 00:59:02 +00:00
Suresh Srinivas
e449de0526
HDFS-3126. Journal stream from Namenode to BackupNode needs to have timeout. Contributed by Hari Mankude.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308636 13f79535-47bb-0310-9956-ffa450edef68
2012-04-03 00:55:08 +00:00
Eli Collins
4f15b9dfed
HDFS-3148. The client should be able to use multiple local interfaces for data transfer. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308617 13f79535-47bb-0310-9956-ffa450edef68
2012-04-02 23:20:34 +00:00
Eli Collins
72b3f302dc
HDFS-3148. The client should be able to use multiple local interfaces for data transfer. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308614 13f79535-47bb-0310-9956-ffa450edef68
2012-04-02 23:13:06 +00:00
Tsz-wo Sze
bc13dfb142
HDFS-3130. Move fsdataset implementation to a package.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308437 13f79535-47bb-0310-9956-ffa450edef68
2012-04-02 17:38:56 +00:00
Eli Collins
be7dd8333a
HDFS-3144. Refactor DatanodeID#getName by use. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308205 13f79535-47bb-0310-9956-ffa450edef68
2012-04-01 22:12:12 +00:00
Eli Collins
43afa4ee51
HDFS-2995. start-dfs.sh should only start the 2NN for namenodes with dfs.namenode.secondary.http-address configured. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308174 13f79535-47bb-0310-9956-ffa450edef68
2012-04-01 18:41:58 +00:00
Eli Collins
0663dbaac0
HDFS-3171. The DatanodeID "name" field is overloaded. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308014 13f79535-47bb-0310-9956-ffa450edef68
2012-04-01 03:41:41 +00:00
Eli Collins
8bd825bb6f
HDFS-3164. Move DatanodeInfo#hostName to DatanodeID. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307890 13f79535-47bb-0310-9956-ffa450edef68
2012-03-31 19:58:51 +00:00
Eli Collins
eeb687daa7
HDFS-3172. dfs.upgrade.permission is dead code. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307888 13f79535-47bb-0310-9956-ffa450edef68
2012-03-31 19:56:23 +00:00
Aaron Myers
ff897e75c9
HDFS-3070. HDFS balancer doesn't ensure that hdfs-site.xml is loaded. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307841 13f79535-47bb-0310-9956-ffa450edef68
2012-03-31 16:20:54 +00:00
Eli Collins
a8cbf19548
HDFS-3138. Move DatanodeInfo#ipcPort to DatanodeID. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307553 13f79535-47bb-0310-9956-ffa450edef68
2012-03-30 17:50:05 +00:00
Eli Collins
64641c28b5
HDFS-3137. Bump LAST_UPGRADABLE_LAYOUT_VERSION to -16. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307173 13f79535-47bb-0310-9956-ffa450edef68
2012-03-30 00:11:14 +00:00
Eli Collins
da58921cf5
HDFS-3066. Cap space usage of default log4j rolling policy. Contributed by Patrick Hunt
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307100 13f79535-47bb-0310-9956-ffa450edef68
2012-03-29 20:51:29 +00:00
Aaron Myers
17d22e6dc4
HDFS-3158. LiveNodes member of NameNodeMXBean should list non-DFS used space and capacity per DN. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1306635 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 22:59:52 +00:00
Tsz-wo Sze
99a68a1423
HDFS-3155. Clean up FSDataset implemenation related code.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1306582 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 20:37:34 +00:00
Eli Collins
0475795066
HDFS-3139. Minor Datanode logging improvement. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1306549 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 19:33:22 +00:00
Suresh Srinivas
940dac52a7
HDFS-3125. Add JournalService to enable Journal Daemon. Contributed by Suresh Srinivas.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305726 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 04:02:03 +00:00
Tsz-wo Sze
525ceb99ad
HDFS-2413. Add an API DistributedFileSystem.isInSafeMode() and change DistributedFileSystem to @InterfaceAudience.LimitedPrivate. Contributed by harsh
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305632 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 22:18:04 +00:00
Tsz-wo Sze
67bdbd60cd
(recommit) HDFS-3089. Move FSDatasetInterface and the related classes to a package.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305603 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 21:14:12 +00:00
Tsz-wo Sze
5e38a9acea
Revert 1305590 for HDFS-3089.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305598 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 21:09:14 +00:00
Tsz-wo Sze
1ebcc378af
HDFS-3089. Move FSDatasetInterface and the related classes to a package.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305590 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 21:01:47 +00:00
Aaron Myers
ce1a7ec975
HDFS-2941. Add an administrative command to download a copy of the fsimage from the NN. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305447 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 17:02:40 +00:00
Todd Lipcon
840df19cee
HADOOP-8193. Refactor FailoverController/HAAdmin code to add an abstract class for "target" services. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305195 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 01:54:14 +00:00
Todd Lipcon
39775dca68
Revert HADOOP-8193 from r1304967. Patch introduced some NPEs in a test case.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1305152 13f79535-47bb-0310-9956-ffa450edef68
2012-03-25 21:38:31 +00:00
Aaron Myers
ea868d3d8b
HDFS-3116. Typo in fetchdt error message. Contributed by AOE Takashi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1304996 13f79535-47bb-0310-9956-ffa450edef68
2012-03-25 08:45:04 +00:00
Todd Lipcon
90ea037d21
HADOOP-8193. Refactor FailoverController/HAAdmin code to add an abstract class for "target" services. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1304967 13f79535-47bb-0310-9956-ffa450edef68
2012-03-24 23:48:07 +00:00
Todd Lipcon
4cb809bf04
HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1304681 13f79535-47bb-0310-9956-ffa450edef68
2012-03-24 00:21:52 +00:00
Todd Lipcon
b795c65f49
HDFS-3071. haadmin failover command does not provide enough detail when target NN is not ready to be active. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1304203 13f79535-47bb-0310-9956-ffa450edef68
2012-03-23 06:29:40 +00:00
Eli Collins
4feef86372
HDFS-3044. fsck move should be non-destructive by default. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1304063 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 21:11:18 +00:00
Tsz-wo Sze
0c4acdc176
HDFS-3100. In BlockSender, throw an exception when it needs to verify checksum but the meta data does not exist. Contributed by Brandon Li
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1303628 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 00:10:20 +00:00
Todd Lipcon
2f820dde17
HDFS-309. FSEditLog should log progress during replay. Contributed by Sho Shimauchi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1303485 13f79535-47bb-0310-9956-ffa450edef68
2012-03-21 18:14:09 +00:00
Todd Lipcon
f55a1c0876
HDFS-2834. Add a ByteBuffer-based read API to DFSInputStream. Contributed by Henry Robinson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1303474 13f79535-47bb-0310-9956-ffa450edef68
2012-03-21 17:30:37 +00:00
Tsz-wo Sze
9e108e61fb
HDFS-3086. Change Datanode not to send storage list in registration.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1303318 13f79535-47bb-0310-9956-ffa450edef68
2012-03-21 09:07:33 +00:00
Aaron Myers
4ceca97606
HDFS-3083. Cannot run an MR job with HA and security enabled when second-listed NN active. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1303098 13f79535-47bb-0310-9956-ffa450edef68
2012-03-20 19:00:56 +00:00
Aaron Myers
21426e6e42
HADOOP-8121. Active Directory Group Mapping Service. Contributed by Jonathan Natkins.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1302740 13f79535-47bb-0310-9956-ffa450edef68
2012-03-20 01:00:14 +00:00
Tsz-wo Sze
6326605acb
HDFS-3105. Add DatanodeStorage information to block recovery.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1302683 13f79535-47bb-0310-9956-ffa450edef68
2012-03-19 22:09:14 +00:00
Uma Maheswara Rao G
f77b1a63ec
HDFS-3091. Update the usage limitations of ReplaceDatanodeOnFailure policy in the config description for the smaller clusters. Contributed by Nicholas.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1302624 13f79535-47bb-0310-9956-ffa450edef68
2012-03-19 19:54:13 +00:00
Uma Maheswara Rao G
3573d504e4
HDFS-3111. Missing license headers in trunk. (Contributed by Uma Maheswara Rao G).
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1301871 13f79535-47bb-0310-9956-ffa450edef68
2012-03-17 06:50:36 +00:00
Tsz-wo Sze
662b1887af
HDFS-3088. Move FSDatasetInterface inner classes to a package.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1301661 13f79535-47bb-0310-9956-ffa450edef68
2012-03-16 17:32:57 +00:00
Tsz-wo Sze
7e5b60116e
HDFS-3101. Cannot read empty file using WebHDFS.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1301287 13f79535-47bb-0310-9956-ffa450edef68
2012-03-16 00:01:23 +00:00
Todd Lipcon
49619e8997
HDFS-3062. Fix bug which prevented MR job submission from creating delegation tokens on an HA cluster. Contributed by Mingjie Lai.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1301285 13f79535-47bb-0310-9956-ffa450edef68
2012-03-16 00:00:29 +00:00
Aaron Myers
65425b0961
HDFS-3099. SecondaryNameNode does not properly initialize metrics system. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1301222 13f79535-47bb-0310-9956-ffa450edef68
2012-03-15 21:51:23 +00:00
Aaron Myers
6d96a28a08
HDFS-3067. NPE in DFSInputStream.readBuffer if read is repeated on corrupted block. Contributed by Henry Robinson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1301182 13f79535-47bb-0310-9956-ffa450edef68
2012-03-15 20:26:29 +00:00
Tsz-wo Sze
1177d4edc2
HDFS-3005. FSVolume.decDfsUsed(..) should be synchronized.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1301127 13f79535-47bb-0310-9956-ffa450edef68
2012-03-15 18:24:40 +00:00
Todd Lipcon
59eb544744
HDFS-3093. Fix bug where namenode -format interpreted the -force flag in reverse. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1300814 13f79535-47bb-0310-9956-ffa450edef68
2012-03-15 05:36:03 +00:00
Tsz-wo Sze
3e582c690c
HDFS-3082. Clean up FSDatasetInterface and change DataNode.data to package private.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1300392 13f79535-47bb-0310-9956-ffa450edef68
2012-03-13 22:52:17 +00:00
Eli Collins
ab20dae519
HDFS-2303. Unbundle jsvc. Contributed by Roman Shaposhnik and Mingjie Lai.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1299963 13f79535-47bb-0310-9956-ffa450edef68
2012-03-13 03:38:57 +00:00
Todd Lipcon
1a75ec8288
HDFS-2731. Add command to bootstrap the Standby Node's name directories from the Active NameNode. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1299807 13f79535-47bb-0310-9956-ffa450edef68
2012-03-12 19:41:21 +00:00
Tsz-wo Sze
c8f2889bde
HDFS-3056: add the new file for the previous commit.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1299144 13f79535-47bb-0310-9956-ffa450edef68
2012-03-10 01:58:29 +00:00
Tsz-wo Sze
501a283986
HDFS-3056. Add a new interface RollingLogs for DataBlockScanner logging.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1299139 13f79535-47bb-0310-9956-ffa450edef68
2012-03-10 01:52:17 +00:00
Uma Maheswara Rao G
069fb57f4e
HDFS-2976 removed the unused imports that were missed in previous commit.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1298508 13f79535-47bb-0310-9956-ffa450edef68
2012-03-08 18:56:43 +00:00
Uma Maheswara Rao G
6c0c140350
HDFS-2976 corrected the previous wrong commit for this issue.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1298507 13f79535-47bb-0310-9956-ffa450edef68
2012-03-08 18:51:46 +00:00
Uma Maheswara Rao G
b19d1186ab
HDFS-2976. Remove unnecessary method (tokenRefetchNeeded) in DFSClient.
...
(Contributed by Uma Maheswara Rao G)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1298495 13f79535-47bb-0310-9956-ffa450edef68
2012-03-08 18:31:33 +00:00
Tsz-wo Sze
0ee71adeb4
HDFS-3032. Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1297328 13f79535-47bb-0310-9956-ffa450edef68
2012-03-06 02:42:46 +00:00
Todd Lipcon
87d1c67944
Merge trunk into HA branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1296025 13f79535-47bb-0310-9956-ffa450edef68
2012-03-02 01:48:32 +00:00
Aaron Myers
7be4e5bd22
HDFS-3039. Address findbugs and javadoc warnings on branch. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1296017 13f79535-47bb-0310-9956-ffa450edef68
2012-03-02 01:32:49 +00:00
Tsz-wo Sze
b2f67b4704
HADOOP-8124, HDFS-3034, MAPREDUCE-3956. Remove the deprecated Syncable.sync().
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295999 13f79535-47bb-0310-9956-ffa450edef68
2012-03-01 23:51:19 +00:00
Aaron Myers
f22677ef8e
HDFS-3036. Remove unused method DFSUtil#isDefaultNamenodeAddress. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295961 13f79535-47bb-0310-9956-ffa450edef68
2012-03-01 22:52:05 +00:00
Tsz-wo Sze
9e31bf675d
HDFS-3021. Use generic type to declare FSDatasetInterface.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295929 13f79535-47bb-0310-9956-ffa450edef68
2012-03-01 21:58:39 +00:00
Todd Lipcon
6e5e1e54bc
Amend HDFS-3023. Add new layout version number to ImageLoaderCurrent to fix TestOfflineImageViewer.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1295721 13f79535-47bb-0310-9956-ffa450edef68
2012-03-01 18:23:19 +00:00
Aaron Myers
0d7de507a5
HDFS-2979. Balancer should use logical uri for creating failover proxy with HA enabled. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1295473 13f79535-47bb-0310-9956-ffa450edef68
2012-03-01 08:50:10 +00:00
Aaron Myers
9318ff4250
Revert commit of HDFS-2979.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1295435 13f79535-47bb-0310-9956-ffa450edef68
2012-03-01 07:03:40 +00:00
Todd Lipcon
30cffeb388
HDFS-3023. Optimize entries in edits log for persistBlocks call. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1295356 13f79535-47bb-0310-9956-ffa450edef68
2012-03-01 00:37:09 +00:00
Todd Lipcon
1ba357553a
Merge trunk into HA branch.
...
Resolved some semantic conflicts in TestFileAppendRestart - we now log more OP_ADDs in the HA branch than we did in trunk.
Resolved some conflicts around removal of VersionedProtocol, etc.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1295342 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 23:47:49 +00:00
Aaron Myers
01b17c40cf
HDFS-2979. Balancer should use logical uri for creating failover proxy with HA enabled. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1295340 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 23:42:52 +00:00
Aaron Myers
7b6b204924
HDFS-3027. Implement a simple NN health check. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1295300 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 21:57:56 +00:00
Todd Lipcon
28de3628f5
HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295239 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 19:57:18 +00:00
Jitendra Nath Pandey
ca9b4de772
HDFS-3012. Exception while renewing delegation token. Contributed by Bobby Evans.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295232 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 19:36:17 +00:00
Eli Collins
fbf3a9ee3d
HDFS-2992. Edit log failure trace should include transaction ID of error. Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295227 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 19:31:36 +00:00
Todd Lipcon
34c73db5e1
HDFS-2991. Fix case where OP_ADD would not be logged in append(). Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295214 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 18:52:00 +00:00
Jitendra Nath Pandey
e57fd85fd4
HDFS-3030. Remove getProtocolVersion and getProtocolSignature from translators.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295017 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 08:52:45 +00:00
Aaron Myers
e58b7e202b
HDFS-3014. FSEditLogOp and its subclasses should have toString() method. Contributed by Sho Shimauchi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1294926 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 01:17:37 +00:00
Aaron Myers
978a8050e2
HDFS-2920. fix remaining TODO items. Contributed by Aaron T. Myers and Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1294923 13f79535-47bb-0310-9956-ffa450edef68
2012-02-29 01:09:07 +00:00
Aaron Myers
c69dfdd5e1
HDFS-2958. Sweep for remaining proxy construction which doesn't go through failover path.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1294811 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 20:09:18 +00:00
Todd Lipcon
1ab31b1715
HDFS-3019. Fix silent failure of TestEditLogJournalFailures. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1294772 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 18:38:40 +00:00
Aaron Myers
375c6b8f2e
Merge trunk into HA branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1294762 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 18:33:41 +00:00
Todd Lipcon
7decf112c0
HDFS-3024. Improve performance of stringification in addStoredBlock. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1294754 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 18:04:13 +00:00
Todd Lipcon
da9aa34bec
HDFS-3013. HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configuration. Contributed by Mingjie Lai.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1294425 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 00:43:54 +00:00
Aaron Myers
ae9014ef6a
Merge trunk into HA branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1293742 13f79535-47bb-0310-9956-ffa450edef68
2012-02-26 04:57:17 +00:00
Aaron Myers
b57260f848
HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293707 13f79535-47bb-0310-9956-ffa450edef68
2012-02-25 22:14:53 +00:00
Tsz-wo Sze
dacbeb5f67
HDFS-3006. In WebHDFS, when the return body is empty, set the Content-Type to application/octet-stream instead of application/json.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293487 13f79535-47bb-0310-9956-ffa450edef68
2012-02-25 00:15:37 +00:00
Todd Lipcon
481f84597b
HDFS-2904. Client support for getting delegation tokens. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1293486 13f79535-47bb-0310-9956-ffa450edef68
2012-02-25 00:03:26 +00:00
Eli Collins
78f22bc525
HDFS-3008. Negative caching of local addrs doesn't work. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293419 13f79535-47bb-0310-9956-ffa450edef68
2012-02-24 21:14:04 +00:00
Suresh Srinivas
0e79131981
HDFS-3009. Remove duplicate code in DFSClient#isLocalAddress by using NetUtils. Contributed by Hari Mankude.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293390 13f79535-47bb-0310-9956-ffa450edef68
2012-02-24 19:29:41 +00:00
Aaron Myers
62c7e2edfc
HDFS-3003. Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString(). Contributed by Brandon Li.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293338 13f79535-47bb-0310-9956-ffa450edef68
2012-02-24 17:03:17 +00:00
Aaron Myers
f3026e1085
Merge trunk into HA branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1292838 13f79535-47bb-0310-9956-ffa450edef68
2012-02-23 16:06:53 +00:00
Eli Collins
2e9b8df963
HDFS-2993. HA: BackupNode#checkOperation should permit CHECKPOINT operations. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1292688 13f79535-47bb-0310-9956-ffa450edef68
2012-02-23 07:58:33 +00:00
Tsz-wo Sze
f11b0788df
HDFS-2981. In hdfs-default.xml, the default value of dfs.client.block.write.replace-datanode-on-failure.enable should be true.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1292626 13f79535-47bb-0310-9956-ffa450edef68
2012-02-23 02:27:07 +00:00
Eli Collins
8db31c5972
HDFS-2922. HA: close out operation categories. Contributed by Eli Collins
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1292620 13f79535-47bb-0310-9956-ffa450edef68
2012-02-23 02:08:54 +00:00
Todd Lipcon
90a14f89e1
HDFS-2973. Re-enable NO_ACK optimization for block deletion. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1292611 13f79535-47bb-0310-9956-ffa450edef68
2012-02-23 01:25:14 +00:00
Todd Lipcon
475db83b87
HDFS-2985. Improve logging when replicas are marked as corrupt. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1292609 13f79535-47bb-0310-9956-ffa450edef68
2012-02-23 01:16:33 +00:00
Todd Lipcon
7527e943e6
HDFS-2972. Small optimization building incremental block report. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1292497 13f79535-47bb-0310-9956-ffa450edef68
2012-02-22 20:37:28 +00:00
Todd Lipcon
c14912785d
HDFS-2929. Stress test and fixes for block synchronization. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1292494 13f79535-47bb-0310-9956-ffa450edef68
2012-02-22 20:31:52 +00:00
Tsz-wo Sze
efbc58f30c
HDFS-2907. Add a conf property dfs.datanode.fsdataset.factory to make FSDataset in Datanode pluggable.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1292419 13f79535-47bb-0310-9956-ffa450edef68
2012-02-22 17:47:39 +00:00
Aaron Myers
f0db867292
Merge trunk into HA branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1291400 13f79535-47bb-0310-9956-ffa450edef68
2012-02-20 18:33:37 +00:00
Steve Loughran
a57b9b8335
HDFS-2725 script to mention dfs command
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1245943 13f79535-47bb-0310-9956-ffa450edef68
2012-02-18 15:25:11 +00:00
Aaron Myers
41e56dfece
HDFS-2952. NN should not start with upgrade option or with a pending an unfinalized upgrade. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1245875 13f79535-47bb-0310-9956-ffa450edef68
2012-02-18 07:12:27 +00:00