big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,846 Commits 2 Branches 2 Tags 264 MiB
7a2443e9f8
Commit Graph

1371 Commits

Author SHA1 Message Date
Jing Zhao
7a2443e9f8 HDFS-5122. Support failover and retry in WebHdfsFileSystem for NN HA. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524562 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-18 20:47:16 +00:00
Arpit Agarwal
22b401284b HDFS-5031. BlockScanner scans the block multiple times. (Vinay via Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524553 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-18 20:08:50 +00:00
Jing Zhao
f278a491dc HDFS-5219. Add configuration keys for retry policy in WebHDFSFileSystem. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524498 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-18 17:29:27 +00:00
Tsz-wo Sze
f98c343c7f HDFS-5207. In BlockPlacementPolicy.chooseTarget(..), change the writer and the excludedNodes parameter types respectively to Node and Set. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523875 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-17 02:38:47 +00:00
Tsz-wo Sze
d01caeee77 HDFS-5188. In BlockPlacementPolicy, reduce the number of chooseTarget(..) methods; replace HashMap with Map in parameter declarations and cleanup some related code. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523400 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-15 04:15:57 +00:00
Jing Zhao
1260d598d3 HDFS-4096. Add snapshot information to namenode WebUI. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523091 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-13 20:41:11 +00:00
Jing Zhao
965ce2041a HDFS-5192. NameNode may fail to start when dfs.client.test.drop.namenode.response.number is set. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1522775 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-13 04:29:50 +00:00
Andrew Wang
e4374d8036 HDFS-4680. Audit logging of delegation tokens for MR tracing. (Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1522012 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-11 19:57:37 +00:00
Andrew Wang
4f2bf68b73 HDFS-5170. BlockPlacementPolicyDefault uses the wrong classname when alerting to enable debug logging. (Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520961 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-09 04:54:57 +00:00
Colin McCabe
6431192c0e HDFS-4879. Add BlockedArrayList collection to avoid CMS full GCs (Contributed by Todd Lipcon) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520667 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-06 19:05:26 +00:00
Jing Zhao
36d0b822ef HDS-5118. Provide testing support for DFSClient to drop RPC responses. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520637 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-06 17:17:30 +00:00
Colin McCabe
a628391955 HDFS-5164. DeleteSnapshot should check if OperationCategory.WRITE is possible before taking write lock (contributed by Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520492 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-06 05:23:54 +00:00
Chris Nauroth
e10dbf41bc HDFS-4491. Add/delete files missed in prior commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520482 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-06 03:14:20 +00:00
Chris Nauroth
5eb618ee1f HDFS-4491. Parallel testing HDFS. Contributed by Andrey Klochkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520479 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-06 03:06:41 +00:00
Aaron Myers
8aea748ec3 HDFS-5159. Secondary NameNode fails to checkpoint if error occurs downloading edits on first checkpoint. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520363 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-05 16:21:26 +00:00
Kihwal Lee
c28c964612 HDFS-5150. Allow per NN SPN for internal SPNEGO. Contributed By Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519681 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-03 13:49:25 +00:00
Jing Zhao
25d4c2fd53 HDFS-5140. Too many safemode monitor threads being created in the standby namenode causing it to fail with out of memory error. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518899 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-30 07:36:45 +00:00
Suresh Srinivas
daa0713eb0 HDFS-5144. Document time unit to NameNodeMetrics. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518895 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-30 07:01:47 +00:00
Konstantin Shvachko
eb484bb562 HDFS-5077. NPE in FSNamesystem.commitBlockSynchronization(). Contributed by Plamen Jeliazkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518851 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-29 23:08:02 +00:00
Kihwal Lee
1bfcab9689 HDFS-5132. Deadlock in NameNode between SafeModeMonitor#run and DatanodeManager#handleHeartbeat. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517989 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-27 21:04:24 +00:00
Kihwal Lee
10a62366a5 HDFS-5128. Allow multiple net interfaces to be used with HA namenode RPC server. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517981 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-27 20:53:31 +00:00
Kihwal Lee
39252995c4 HDFS-3245. Add metrics and web UI for cluster version summary. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517937 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-27 19:21:15 +00:00
Jing Zhao
7cfbfe52f1 HDFS-5124. DelegationTokenSecretManager#retrievePassword can cause deadlock in NameNode. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516671 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-23 00:55:53 +00:00
Jing Zhao
6fd8766a51 HDFS-5045. Add more unit tests for retry cache to cover all AtMostOnce methods. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516348 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-22 01:06:57 +00:00
Kihwal Lee
2499a86664 HDFS-4994. Audit log getContentSummary() calls. Contributed by Robert Parker. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516237 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-21 18:15:26 +00:00
Luke Lu
782191f1ba HADOOP-9784. Add a builder for HttpServer. (Junping Du via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516128 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-21 10:12:13 +00:00
Chris Nauroth
5e68bc4cd6 HDFS-4594. WebHDFS open sets Content-Length header to what is specified by length parameter rather than how much data is actually returned. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1515989 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-20 22:15:22 +00:00
Arpit Agarwal
3015429368 HDFS-2933. Improve DataNode Web UI Index Page. (Vivek Ganesan via Arpit Agarwal) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1515890 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-20 17:18:53 +00:00
Brandon Li
214d4377fc HDFS-5104 Support dotdot name in NFS LOOKUP operation. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1515042 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-17 21:16:50 +00:00
Jing Zhao
52f0259502 HADOOP-9880. SASL changes from HADOOP-9421 breaks Secure HA NN. Contributed by Daryn Sharp. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514913 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-16 22:35:19 +00:00
Jing Zhao
8d21926c26 HDFS-5102. Snapshot names should not be allowed to contain slash characters. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514797 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-16 17:14:34 +00:00
Konstantin Shvachko
10ec8a248e HDFS-2994. If lease soft limit is recovered successfully the append can fail. Contributed by Tao Luo. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514500 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 21:21:10 +00:00
Chris Nauroth
2fc7e14e39 HDFS-5099. Namenode#copyEditLogSegmentsToSharedDir should close EditLogInputStreams upon finishing. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514481 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 20:43:46 +00:00
Jing Zhao
02b19e0738 HDFS-5076. Add MXBean methods to query NN's transaction information and JournalNode's journal status. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514422 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 18:22:52 +00:00
Jing Zhao
8172215e56 HDFS-5080. BootstrapStandby not working with QJM when the existing NN is active. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514386 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 17:19:52 +00:00
Tsz-wo Sze
0182ea16d3 HDFS-4898. BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to properly fallback to local rack. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514156 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 04:52:52 +00:00
Konstantin Shvachko
4c8db60092 HDFS-5079. Cleaning up NNHAStatusHeartbeat.State from DatanodeProtocolProtos. Contributed by Tao Luo. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514118 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 01:29:16 +00:00
Suresh Srinivas
b32ace11f1 HDFS-5051. nn fails to download checkpointed image from snn in some setups. Contributed by Vinay and Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514110 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 00:45:48 +00:00
Andrew Wang
49a892056d HDFS-4816. transitionToActive blocks if the SBN is doing checkpoint image transfer. (Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514095 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-14 23:32:29 +00:00
Jing Zhao
2bea314456 HDFS-5091. Support for spnego keytab separate from the JournalNode keytab for secure HA. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1513700 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-14 00:19:35 +00:00
Kihwal Lee
2f988135e3 HDFS-4993. Fsck can fail if a file is renamed or deleted. Contributed by Robert Parker. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512451 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-09 19:13:22 +00:00
Kihwal Lee
81f9786ae6 HDFS-5047. Supress logging of full stack trace of quota and lease exceptions. Contributed by Robert Parker. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512057 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-08 21:46:34 +00:00
Jing Zhao
54e144059e HDFS-4926. namenode webserver's page has a tooltip that is inconsistent with the datanode HTML link. Contributed by Vivek Ganesan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1511477 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-07 20:34:39 +00:00
Kihwal Lee
ccdb978603 HDFS-4366. Block Replication Policy Implementation May Skip Higher-Priority Blocks for Lower-Priority Blocks. Contributed by Derek Dagit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1510724 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-05 20:48:59 +00:00
Suresh Srinivas
390d689816 HDFS-5061. Make FSNameSystem#auditLoggers an unmodifiable list. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1510611 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-05 17:08:09 +00:00
Colin McCabe
f216f22915 HADOOP-9761. ViewFileSystem#rename fails when using DistributedFileSystem (Andrew Wang via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509874 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-02 21:42:38 +00:00
Tsz-wo Sze
c04a7d974a HDFS-5028. LeaseRenewer throws ConcurrentModificationException when timeout. Contributed by zhaoyunjiong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509565 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-02 07:08:35 +00:00
Brandon Li
4f68aa0600 HDFS-5043. For HdfsFileStatus, set default value of childrenNum to -1 instead of 0 to avoid confusing applications. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508694 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-31 00:49:47 +00:00
Suresh Srinivas
8c7a7e6196 HDFS-5025. Record ClientId and CallId in EditLog to enable rebuilding retry cache in case of HA failover. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508332 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-30 07:51:38 +00:00
Aaron Myers
215d6331d6 HDFS-5027. On startup, DN should scan volumes in parallel. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508165 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-29 18:28:07 +00:00