big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,266 Commits 2 Branches 2 Tags 264 MiB
a626fa04f9
Commit Graph

468 Commits

Author SHA1 Message Date
Todd Lipcon
737df8b67b HDFS-2716. Configuration needs to allow different dfs.http.addresses for each HA NN. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1226020 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-31 01:53:23 +00:00
Todd Lipcon
20a6560bdf HDFS-2692. Fix bugs related to failover from/into safe mode. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1225709 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-30 00:30:27 +00:00
Todd Lipcon
15ebdd5664 HDFS-2714. Fix test cases which use standalone FSNamesystems. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1225708 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-30 00:25:21 +00:00
Aaron Myers
f216303598 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1225612 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-29 18:25:48 +00:00
Harsh J
6c349f9420 HDFS-2729. Update BlockManager's comments regarding the invalid block set (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1225591 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-29 16:51:59 +00:00
Harsh J
41737432c0 HDFS-2726. Fix a logging issue under DFSClient's createBlockOutputStream method (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1225456 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-29 07:22:54 +00:00
Aaron Myers
a500ba50c2 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1224965 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-27 18:18:36 +00:00
Alejandro Abdelnur
ae0d48854d HDFS-2707. HttpFS should read the hadoop-auth secret from a file instead inline from the configuration. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1224794 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-26 19:37:25 +00:00
Alejandro Abdelnur
6b4f40cbf9 HttpFS server should check that upload requests have correct content-type. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221616 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 05:26:20 +00:00
Todd Lipcon
31c91706f7 HDFS-1972. Fencing mechanism for block invalidations and replications. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1221608 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 04:32:40 +00:00
Todd Lipcon
36d1c49486 HDFS-2693. Fix synchronization issues around state transition. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1221582 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 03:03:23 +00:00
Alejandro Abdelnur
8fe3dd3fea HDFS-2657. TestHttpFSServer and TestServerWebApp are failing on trunk. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221580 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 03:03:16 +00:00
Alejandro Abdelnur
45620eee68 HDFS-2646. Hadoop HttpFS introduced 4 findbug warnings. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221572 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 02:46:04 +00:00
Tsz-wo Sze
3503d9a4a8 HDFS-2706. Use configuration for blockInvalidateLimit if it is set. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221106 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-20 05:49:24 +00:00
Todd Lipcon
57ef902bbc HDFS-2682. When a FailoverProxyProvider is used, Client should not retry for 45 times if it is timing out to connect to server. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1220965 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 21:49:39 +00:00
Todd Lipcon
4bb0456c66 HDFS-2678. When a FailoverProxyProvider is used, DFSClient should not retry connection ten times before failing over. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1220942 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 20:57:13 +00:00
Todd Lipcon
08bde10e24 Merge trunk into HA branch to pick up fix for bkjournal findbugs 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1220941 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 20:46:58 +00:00
Todd Lipcon
dfba847eb6 Amend HDFS-234 (bkjournal contrib). Fix findbugs build 
- Fixes pom parent to point to hadoop-project instead of hadoop-project-dist
- Fix relative path in pom to have the correct number of ../s


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1220940 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 20:42:25 +00:00
Alejandro Abdelnur
de928a4aa0 HDFS-2658. HttpFS introduced 70 javadoc warnings. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1220810 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 15:53:53 +00:00
Todd Lipcon
8ff28f4549 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1220616 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 05:34:51 +00:00
Todd Lipcon
625d1b380a HDFS-2677. Web UI should indicate the NN state. Contributed by Eli Collins. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1220613 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 05:26:07 +00:00
Todd Lipcon
329717264f HDFS-2679. Add interface to query current state to HAServiceProtocol. Contributed by Eli Collins. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1220612 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 05:24:32 +00:00
Eli Collins
5e185702ea HDFS-2335. DataNodeCluster and NNStorage always pull fresh entropy. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1220510 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-18 18:12:48 +00:00
Todd Lipcon
a89baed833 HDFS-2553. Fix BlockPoolSliceScanner spinning in a tight loop. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1220317 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-18 01:13:00 +00:00
Todd Lipcon
fa1edc9a1e HDFS-2700. Fix failing TestDataNodeMultipleRegistrations in trunk. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1220315 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-18 01:10:49 +00:00
Aaron Myers
08c9dc3780 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1215367 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-17 00:31:46 +00:00
Suresh Srinivas
89553be90e HDFS-2687. Tests failing with ClassCastException post protobuf RPC changes. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1215366 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-17 00:26:46 +00:00
Aaron Myers
52c1491155 HDFS-2694. Removal of Avro broke non-PB NN services. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1215364 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-17 00:19:01 +00:00
Thomas White
9b7617a909 HDFS-2640. Javadoc generation hangs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1215354 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-17 00:09:04 +00:00
Todd Lipcon
371f4228e8 HDFS-2684. Fix up some failing unit tests on HA branch. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1215241 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-16 18:36:24 +00:00
Todd Lipcon
cdb9f01ad4 HDFS-2667. Fix transition from active to standby. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1215037 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-16 04:25:13 +00:00
Aaron Myers
71071b904d HDFS-2602. NN should log newly-allocated blocks without losing BlockInfo. Contributed by Aaron T. Myers 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1215036 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-16 04:18:58 +00:00
Aaron Myers
353bc3799d HDFS-2689. HA: BookKeeperEditLogInputStream doesn't implement isInProgress(). Contributed by Aaron T. Myers 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214965 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-15 21:54:00 +00:00
Aaron Myers
d937abd434 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214944 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-15 20:05:38 +00:00
Alejandro Abdelnur
6cf3a15c52 HADOOP_7917. compilation of protobuf files fails in windows/cygwin. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214790 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-15 14:41:01 +00:00
Todd Lipcon
afd0333196 HDFS-2683. Authority-based lookup of proxy provider fails if path becomes canonicalized. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214579 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-15 00:42:50 +00:00
Todd Lipcon
b77c107ccb Committing rest of merge from trunk (accidentally only committed the HDFS portion before) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214546 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 23:34:04 +00:00
Todd Lipcon
ecdf9da770 Fix treatment of NNHAStatusHeartbeat in protobuffer. 
Committing without pre-commit review since it's a pretty trivial merge fix


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214543 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 23:28:43 +00:00
Todd Lipcon
8134b1c870 Merge trunk into HA branch. 
Several conflicts around introduction of protobuf translator for DatanodeProtocol - mostly trivial resolutions.

NB: this does not successfully pass any tests since the HAStatus field needs
to be integrated into the HeartbeatResponse Protobuf implementation.
That will be a separate commit for clearer history.



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214518 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 22:47:41 +00:00
Todd Lipcon
6c2da4bc0f HDFS-2680. DFSClient should construct failover proxy with exponential backoff. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214487 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 21:41:31 +00:00
Sanjay Radia
d8dfcdcbc2 HDFS-2669 Enable protobuf rpc for ClientNamenodeProtocol 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214128 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 09:27:51 +00:00
Todd Lipcon
7e8accd68e HDFS-2671. NN should throw StandbyException in response to RPCs in STANDBY state. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214117 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 08:56:21 +00:00
Suresh Srinivas
95bdb29aaf r1214102 should have removed the file. Committing this to remove empty file. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214103 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 08:25:47 +00:00
Suresh Srinivas
60dd76c6ab HDFS-2676. Remove Avro RPC. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214102 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 08:20:23 +00:00
Todd Lipcon
b9238e3bad HDFS-2675. Reduce warning verbosity when double-closing edit logs. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214098 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 08:13:55 +00:00
Eli Collins
bdc2a77d6b HDFS-2649. eclipse:eclipse build fails for hadoop-hdfs-httpfs. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214046 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 03:42:24 +00:00
Jitendra Nath Pandey
3cffe34177 HDFS-2661. Enable protobuf RPC for DatanodeProtocol. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214033 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 02:15:37 +00:00
Tsz-wo Sze
433a6e78f6 HDFS-2545. Change WebHDFS to support multiple namenodes in federation. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214027 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 01:47:57 +00:00
Suresh Srinivas
3954a2fb1c HDFS-2663. Optional protobuf parameters are not handled correctly. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213985 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:31:44 +00:00
Suresh Srinivas
6a609cb471 Reverting the patch r1213981 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213984 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:27:58 +00:00
Jitendra Nath Pandey
1f2f4e105b HDFS-234. Integration with BookKeeper logging system. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213983 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:27:13 +00:00
Suresh Srinivas
b5229fd19b HDFS-2663. Optional protobuf parameters are not handled correctly. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213981 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:22:28 +00:00
Suresh Srinivas
3001a172c8 Reverting r1213512 because it committed changes that were not part of the patch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213980 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:17:02 +00:00
Aaron Myers
a0fe4f476a Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1213867 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 19:02:37 +00:00
Suresh Srinivas
f2f4e93413 HDFS-2650. Replace @inheritDoc with @Override. Contributed by Hari Mankude. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213813 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 18:07:29 +00:00
Suresh Srinivas
4ec8424e5d HDFS-2666. Fix TestBackupNode failure. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213808 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 17:59:37 +00:00
Eli Collins
c3f7c1f3e5 Add missing file to previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213593 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 08:10:29 +00:00
Eli Collins
5f39d6c239 HDFS-2654. Make BlockReaderLocal not extend RemoteBlockReader2. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213592 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 08:09:50 +00:00
Eli Collins
197634f2f7 HDFS-2653. DFSClient should cache whether addrs are non-local when short-circuiting is enabled. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213586 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 07:51:23 +00:00
Eli Collins
2892b058e0 Put HDFS-1765 in the right place in the changelog. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213541 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 02:23:07 +00:00
Eli Collins
43100e9c0e HDFS-1765. Block Replication should respect under-replication block priority. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213537 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 01:58:25 +00:00
Suresh Srinivas
13345f3a85 HDFS-2663. Handle protobuf optional parameters correctly. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213512 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 00:21:48 +00:00
Aaron Myers
89379bc1a3 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1213389 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 19:41:20 +00:00
Konstantin Shvachko
3128c78636 JDiff output for release 0.22.0 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213146 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 05:45:08 +00:00
Sanjay Radia
48da033901 HDFS-2651 ClientNameNodeProtocol Translators for Protocol Buffers (sanjay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213143 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 05:36:35 +00:00
Konstantin Shvachko
2c1ee1d342 Preparing for release 0.22.0 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213121 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 00:26:01 +00:00
Suresh Srinivas
2740112bb6 HDFS-2647. Used protobuf based RPC for InterDatanodeProtocol, ClientDatanodeProtocol, JournalProtocol, NamenodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213040 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-11 18:53:21 +00:00
Alejandro Abdelnur
e7682e1496 HDFS-2511. Add dev script to generate HDFS protobufs. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1212609 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-09 20:21:19 +00:00
Jitendra Nath Pandey
38a19bc293 HDFS-2642. Protobuf translators for DatanodeProtocol. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1212606 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-09 20:02:33 +00:00
Tsz-wo Sze
9b1f47226b HDFS-2594. Support getDelegationTokens and createSymlink in WebHDFS. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1212299 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-09 09:26:33 +00:00
Todd Lipcon
2481474bd9 HDFS-2634. Standby needs to ingest latest edit logs before transitioning to active. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1212187 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-08 23:55:40 +00:00
Alejandro Abdelnur
3334306512 HDFS-2178. Contributing Hoop to HDFS, replacement for HDFS proxy with read/write capabilities. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1212060 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-08 19:25:28 +00:00
Sanjay Radia
9f69ae5a9e HDFS-2597 ClientNameNodeProtocol in Protocol Buffers (sanjay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1211769 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-08 06:48:45 +00:00
Aaron Myers
d9ea5bb489 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1211749 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-08 02:57:47 +00:00
Todd Lipcon
6016e95fee HDFS-2627. Determine DN's view of which NN is active based on heartbeat responses. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1211735 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-08 02:00:20 +00:00
Aaron Myers
a27adf3de4 HDFS-2430. The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1211650 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-07 21:47:13 +00:00
Suresh Srinivas
c17bb83644 HDFS-2636. Implement protobuf service for ClientDatanodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1211249 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-07 01:05:56 +00:00
Suresh Srinivas
7a59150bff HDFS-2629. Implement protobuf service for InterDatanodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1211206 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-06 22:19:10 +00:00
Jitendra Nath Pandey
b981daa07f HDFS-1580. Committing the test case, it was missed in earlier commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1210749 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-06 02:15:56 +00:00
Jitendra Nath Pandey
8c0f2a0d59 HDFS-2618. Fix for the build failure caused by previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1210746 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-06 02:08:48 +00:00
Suresh Srinivas
0a713035f2 HDFS-2618. Implement protobuf service for NamenodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1210719 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-06 00:25:25 +00:00
Todd Lipcon
f39aac60e0 Merge trunk into branch. 
Resolved conflicts generated by commit of HDFS-1580 in trunk:
- made EditLogInputStream.isInProgress public
- fixed trivial conflict in DFSConfigKeys


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1210666 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-05 22:10:35 +00:00
Suresh Srinivas
c256f8266d HDFS-2581. Implement protobuf service for JournalProtocol. Contributed Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1210657 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-05 21:52:23 +00:00
Todd Lipcon
6491444357 Fix to previous trunk merge - RPC's addProtocol method now needs an RpcKind 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1210638 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-05 21:13:07 +00:00
Jitendra Nath Pandey
d18e5b3844 HDFS-1580. Add interface for generic Write Ahead Logging mechanisms. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1210602 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-05 20:10:27 +00:00
Aaron Myers
8a4db4d331 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1210547 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-05 18:00:44 +00:00
Todd Lipcon
ad7fe4e21e HDFS-2625. TestDfsOverAvroRpc failing after introduction of HeartbeatResponse type. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1210342 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-05 06:38:52 +00:00
Todd Lipcon
c7f5167845 HDFS-2624. ConfiguredFailoverProxyProvider doesn't correctly stop ProtocolTranslators. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1210341 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-05 06:37:46 +00:00
Todd Lipcon
b3f28dbb3d HDFS-2626. BPOfferService.verifyAndSetNamespaceInfo needs to be synchronized. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1210340 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-05 06:36:00 +00:00
Sanjay Radia
65200998c0 HADOOP-7862 Move the support for multiple protocols to lower layer so that Writable, PB and Avro can all use it (includes HDFS and MR changes to match) (Sanjay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1210208 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-04 20:44:36 +00:00
Todd Lipcon
74b1f069cc Amend HDFS-2616 to fix mocking in TestBPOfferService 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1209315 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-02 00:03:17 +00:00
Todd Lipcon
28dbd56de0 HDFS-2623. Add test case for hot standby capability. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1209256 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 21:37:08 +00:00
Todd Lipcon
f4fa76719e HDFS-2612. Handle refreshNameNodes in federated HA clusters. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1209249 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 21:26:08 +00:00
Todd Lipcon
4cbead8484 HDFS-2622. Fix TestDFSUpgrade in HA branch. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1209230 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 20:38:46 +00:00
Suresh Srinivas
8dbb523768 HDFS-2616. Change DatanodeProtocol#sendHeartbeat() to return HeartbeatResponse. (suresh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1208987 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 08:03:41 +00:00
Suresh Srinivas
1e346aa829 HDFS-1971. Send block report from datanode to both active and standby namenodes. (sanjay, todd via suresh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1208925 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 01:10:28 +00:00
Todd Lipcon
f87a4b40bc HDFS-1975. Support for sharing the namenode state from active to standby. Contributed by Jitendra Nath Pandey, Aaron T Myers, and Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1208813 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-30 21:46:22 +00:00
Aaron Myers
0eec2218a1 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1208644 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-30 18:27:04 +00:00
Eli Collins
68ff73c802 Ammend HDFS-2570 to shorten description. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1208313 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-30 07:49:15 +00:00
Todd Lipcon
bb91ebb53c HDFS-2591. MiniDFSCluster support to mix and match federation with HA. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1208297 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-30 06:49:30 +00:00
Alejandro Abdelnur
afbc3aa30c HDFS-2606. webhdfs client filesystem impl must set the content-type header for create/append. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1208157 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-30 00:25:54 +00:00
Tsz-wo Sze
0398a9e88d HDFS-2604. Add a log message to show if WebHDFS is enabled and a configuration section in the forrest doc. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1208140 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-29 23:38:59 +00:00
Aaron Myers
ee97365f58 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1208002 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-29 17:55:28 +00:00
Todd Lipcon
9146ad23f3 HDFS-2582. Scope dfs.ha.namenodes config by nameservice. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1207738 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-29 02:27:45 +00:00
Konstantin Shvachko
68f56ca906 HDFS-2287. Fix second assert. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1207710 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-29 00:03:49 +00:00
Konstantin Shvachko
3083d7d7db Fix CHANGES.txt. Moving changes from 0.21.1 section to 0.22.0. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1207709 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-29 00:00:20 +00:00
Konstantin Shvachko
82da8fb490 Fix CHANGES.txt. Some changes have been committed under 0.21.1 section rather than 0.22.0. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1207706 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 23:53:49 +00:00
Konstantin Shvachko
32935d3e67 Fix CHANGES.txt for HDFS-2346 in trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1207700 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 23:40:24 +00:00
Eli Collins
b32543d9ae HDFS-2596. TestDirectoryScanner doesn't test parallel scans. (eli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1207694 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 23:20:16 +00:00
Konstantin Shvachko
49c44117f3 Fix typo in HDFS-993 commit. Was -933. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1207676 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 22:51:45 +00:00
Tsz-wo Sze
91a75ad34f Fix WebHDFS capitalization in CHANGES.txt. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1207585 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 20:47:38 +00:00
Aaron Myers
73b3de6204 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1207490 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 18:21:46 +00:00
Matthew Foley
e9c2307838 HDFS-2316. Record completion of umbrella jira. Contributed by Tsz Wo (Nicholas), Sze. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1206990 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 06:30:49 +00:00
Tsz-wo Sze
4b65d10cfe HDFS-2590. Fix the missing links in the WebHDFS forrest doc. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1206178 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-25 13:58:29 +00:00
Konstantin Shvachko
080a34143d HADOOP-7861. changes2html.pl generates links to HADOOP, HDFS, and MAPREDUCE jiras. Contributed by Konstantin Shvachko. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1205988 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-24 21:44:36 +00:00
Todd Lipcon
5f5b3bbdcd HDFS-2577. NN fails to start since it tries to start secret manager in safemode. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1205689 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-24 00:55:37 +00:00
Tsz-wo Sze
d3a51478e3 HDFS-2587. Add apt doc for WebHDFS REST API. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1205626 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 22:02:19 +00:00
Matthew Foley
db092f5048 HADOOP-7827 HDFS-2588 MAPREDUCE-3464. jsp pages missing DOCTYPE. Contributed by Dave Vronay. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1205519 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 18:15:37 +00:00
Tsz-wo Sze
7381884e85 HDFS-2552. Add Forrest doc for WebHDFS REST API. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1205146 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-22 20:04:58 +00:00
Suresh Srinivas
49417594e8 HDFS-2519. Add protobuf service for DatanodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204825 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-22 06:25:13 +00:00
Todd Lipcon
6a0671977b Merge trunk into HDFS-1623 branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1204794 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-22 03:18:47 +00:00
Tsz-wo Sze
2ab10e29d9 HDFS-2246. Enable reading a block directly from local file system for a client on the same node as the block file. Contributed by Andrew Purtell, Suresh Srinivas and Jitendra Nath Pandey 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204792 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-22 02:57:04 +00:00
Konstantin Boudnik
a2e66776a9 HDFS-2573. TestFiDataXceiverServer is failing, not testing OOME. Contributed by Konstantin Boudnik. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204781 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-22 02:15:49 +00:00
Suresh Srinivas
5293760973 HDFS-2410. Further cleanup of hardcoded configuration keys and values. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204753 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 23:49:44 +00:00
Suresh Srinivas
b76b85986e HDFS-2520. Add protobuf service for InterDatanodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204709 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 21:11:43 +00:00
Suresh Srinivas
4754b80862 HDFS-2518. Add protobuf service for NamenodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204707 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 21:10:04 +00:00
Todd Lipcon
39ce694d05 HDFS-2566. Move BPOfferService to be a non-inner class. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204659 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 19:27:00 +00:00
Todd Lipcon
68173af69d HDFS-2575. DFSTestUtil may create empty files. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204652 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 19:03:31 +00:00
Suresh Srinivas
54ac7e5413 HDFS-2517. Add protobuf service for JounralProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204544 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 15:39:33 +00:00
Eli Collins
9c2f4f634d HDFS-2514. Link resolution bug for intermediate symlinks with relative targets. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204370 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 07:01:29 +00:00
Todd Lipcon
5bc66c4c86 HDFS-2532. TestDfsOverAvroRpc timing out in trunk. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204366 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 06:41:56 +00:00
Harsh J
67f25b0a1b HDFS-2572. Remove Unnecessary double-check in DN#getHostName (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204177 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 15:18:14 +00:00
Eli Collins
5573b691d5 HDFS-2567. When 0 DNs are available, show a proper error when trying to browse DFS via web UI. Contributed by Harsh J 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204131 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 06:28:37 +00:00
Eli Collins
8686a00c7a HDFS-2570. Add descriptions for dfs.*.https.address in hdfs-default.xml. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204129 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 06:22:44 +00:00
Eli Collins
8e8bb50afd HDFS-2454. Move maxXceiverCount check to before starting the thread in dataXceiver. Contributed by Harsh J 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204124 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 04:27:59 +00:00
Eli Collins
513718f92d HDFS-2568. Use a set to manage child sockets in XceiverServer. Contributed by Harsh J 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204122 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 04:13:32 +00:00
Eli Collins
b2313021fd HDFS-2536. Remove unused imports. Contributed by Harsh J 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204120 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 03:58:50 +00:00
Eli Collins
0e5c1a45bf HDFS-208. name node should warn if only one dir is listed in dfs.name.dir. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204119 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 03:38:51 +00:00
Eli Collins
84f0c0f180 HDFS-2502. hdfs-default.xml should include dfs.name.dir.restore. Contributed by Harsh J 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204117 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 03:29:20 +00:00
Eli Collins
7a8015c0e2 HDFS-2541. For a sufficiently large value of blocks, the DN Scanner may request a random number with a negative seed value. Contributed by Harsh J 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204114 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-20 02:51:50 +00:00
Eli Collins
b7cd8c0f86 HDFS-2564. Cleanup unnecessary exceptions thrown and unnecessary casts. Contributed by Hari Mankude 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203950 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-19 02:34:25 +00:00
Todd Lipcon
1f92266516 HDFS-2563. Some cleanup in BPOfferService. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203943 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-19 01:31:59 +00:00
Aaron Myers
67c372991c Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1203781 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-18 18:29:26 +00:00
Todd Lipcon
905a127850 HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203543 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-18 09:04:20 +00:00
Thomas White
6a01c22755 HDFS-2543. HADOOP_PREFIX cannot be overridden. Contributed by Bruno Mahé. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203486 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-18 05:10:40 +00:00
Thomas White
dc5a0ab245 HDFS-2544. Hadoop scripts unconditionally source "$bin"/../libexec/hadoop-config.sh. Contributed by Bruno Mahé 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203452 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-18 00:53:46 +00:00
Todd Lipcon
0864ef1908 HDFS-2560. Refactor BPOfferService to be a static inner class. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203444 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-18 00:45:31 +00:00
Aaron Myers
0d171f6513 Merge trunk into HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1202013 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-15 02:39:13 +00:00
Todd Lipcon
9a3f147fdd HDFS-2476. More CPU efficient data structure for under-replicated, over-replicated, and invalidated blocks. Contributed by Tomasz Nykiel. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1201991 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-15 01:13:58 +00:00
Todd Lipcon
51ce383112 HDFS-2532. Add timeout to TestDfsOverAvroRpc 
This test is timing out on trunk, causing tests below it to
fail spuriously. This patch doesn't fix the issue -- just adds
a JUnit timeout so that the failure is properly attributed
to this test.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1201963 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-14 23:34:21 +00:00
Tsz-wo Sze
09a156fcce HDFS-2539. Support doAs and GETHOMEDIRECTORY in webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200731 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-11 04:19:45 +00:00
Tsz-wo Sze
94c631af1f HDFS-2540. Webhdfs: change "Expect: 100-continue" to two-step write; change "HdfsFileStatus" and "localName" respectively to "FileStatus" and "pathSuffix" in JSON response. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1199396 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-08 19:25:57 +00:00
Hairong Kuang
6dff9329d5 HDFS-2495. Increase granularity of write operations in ReplicationMonitor thus reducing contention for write lock. Contributed by Tomasz Nykiel. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1199024 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-08 00:16:23 +00:00
Tsz-wo Sze
a590b498ac HDFS-2528. Webhdfs: set delegation kind to WEBHDFS and add a HDFS token when http requests are redirected to datanode. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1198903 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-07 20:05:16 +00:00
Hairong Kuang
db13c99940 Add the missing test file to HDFS-2477. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1197801 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-04 21:49:15 +00:00
Tsz-wo Sze
6afe3e0d22 HDFS-2527. WebHdfs: remove the use of "Range" header in Open; use ugi username if renewer parameter is null in GetDelegationToken; response OK when setting replication for non-files; rename GETFILEBLOCKLOCATIONS to GET_BLOCK_LOCATIONS and state that it is a private unstable API; replace isDirectory and isSymlink with enum {FILE, DIRECTORY, SYMLINK} in HdfsFileStatus JSON object. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1197329 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 22:34:47 +00:00
Todd Lipcon
40fe96546f HDFS-2129. Simplify BlockReader to not inherit from FSInputChecker. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196976 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 06:54:59 +00:00
Todd Lipcon
fd8c757b9f Add new section to HDFS CHANGES.txt for 0.23.1. Moved two things that were recently committed post-0.23.0 to this new section. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196904 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 01:02:28 +00:00
Todd Lipcon
11cf658d0a HDFS-2533. Remove needless synchronization on some FSDataSet methods. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196902 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 01:00:25 +00:00
Todd Lipcon
f84552ac35 HDFS-2130. Switch default checksum to CRC32C. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196889 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 00:35:37 +00:00
Hairong Kuang
a4baabe4a4 HDFS-2477. Optimize computing the diff between a block report and the namenode state. Contributed by Tomasz Nykiel. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196676 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 16:31:21 +00:00
Aaron Myers
571d7990f4 Merge trunk into HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1196458 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 05:34:31 +00:00
Jitendra Nath Pandey
bd21ddcb78 HDFS-2416. distcp with a webhdfs uri on a secure cluster fails. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196434 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 02:08:20 +00:00
Aaron Myers
abe45f59ac HDFS-2526. (Client)NamenodeProtocolTranslatorR23 do not need to keep a reference to rpcProxyWithoutRetry (atm) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196171 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 17:59:53 +00:00
Eli Collins
4961441584 HDFS-2308. NamenodeProtocol.endCheckpoint is vestigial and can be removed. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196113 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 16:44:28 +00:00
Arun Murthy
4e42044b1c Preparing for release 0.23.0. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195872 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 08:46:22 +00:00
Konstantin Shvachko
22000ce96d HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195869 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 08:37:11 +00:00
Arun Murthy
a3a4ccfed7 Preparing for release 0.23.0. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195838 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 06:02:53 +00:00
Todd Lipcon
1c940637b1 HDFS-2521. Remove custom checksum headers from data transfer protocol. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195829 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 05:17:08 +00:00
Tsz-wo Sze
bb8fd6a267 TestHDFSCLI is failing due to HADOOP-7360. HDFS-2038 is going to fix it. Disable it for the moment. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195754 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 01:31:06 +00:00
Todd Lipcon
3ab295994a HDFS-2523. Small NN fixes to include HAServiceProtocol and prevent NPE on shutdown. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1195753 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 01:29:59 +00:00
Tsz-wo Sze
8d1a09b741 HDFS-2065. Add null checks in DFSClient.getFileChecksum(..). Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195731 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 00:38:57 +00:00
Todd Lipcon
c46876982e HDFS-2512. Add textual error message to data transfer protocol responses. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195693 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 21:53:58 +00:00
Tsz-wo Sze
32cad9affe HDFS-2385. Support renew and cancel delegation tokens in webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195656 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 20:37:16 +00:00
Jitendra Nath Pandey
7cb77a3b1b HDFS-2334. Add Closeable to JournalManager. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195620 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 19:03:55 +00:00
Konstantin Shvachko
64c019cccc HDFS-2285. BackupNode should reject requests to modify namespace. Contributed by Konstantin Shvachko and Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195013 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 20:11:35 +00:00
Tsz-wo Sze
2eca60199a Revert 1190677 for HDFS-2509. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1194986 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 19:31:06 +00:00
Tsz-wo Sze
2dcd43c7b1 HDFS-2436. Change FSNamesystem.setTimes(..) for allowing setting times on directories. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190708 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 00:19:21 +00:00
Thomas White
cd77a1668f HADOOP-7763. Add top-level navigation to APT docs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190703 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 00:16:16 +00:00
Tsz-wo Sze
c8ac31e9a0 HDFS-2509. Add a test for DistributedFileSystem.getFileChecksum(..) on directories or non existing files. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190677 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 23:42:27 +00:00
Todd Lipcon
6b0963c53b HDFS-2465. Add HDFS support for fadvise readahead and drop-behind. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190626 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 22:18:42 +00:00
Suresh Srinivas
c0c938f242 HDFS-2499. RPC client is created incorrectly introduced in HDFS-2459. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190620 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 22:06:04 +00:00
Tsz-wo Sze
9692cfc9ae HDFS-2493. Remove reference to FSNamesystem in blockmanagement classes. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190491 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 18:28:26 +00:00
Eric Yang
da1db28e93 HADOOP-7740. Fixed security audit logger configuration. (Arpit Gupta via Eric Yang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190452 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 17:08:09 +00:00
Sanjay Radia
b49c95a5d9 HDFS-2479 HDFS Client Data Types in Protocol Buffers (sanjay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190127 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 01:52:00 +00:00
Tsz-wo Sze
4a98639a55 Move HDFS-2355, HDFS-2371 and HDFS-2298 entries from trunk to 0.23 in CHANGES.txt. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190126 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 01:48:12 +00:00
Aaron Myers
efb2d93f77 HDFS-2393. Mark appropriate methods of ClientProtocol with the idempotent annotation. (atm) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1190088 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 23:26:44 +00:00
Aaron Myers
180646dea3 HDFS-2418. Change ConfiguredFailoverProxyProvider to take advantage of HDFS-2231. (atm) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1190078 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 23:15:07 +00:00
Tsz-wo Sze
8cb0d4b380 HDFS-2432. Webhdfs: response FORBIDDEN when setReplication on non-files; clear umask before creating a flie; throw IllegalArgumentException if setOwner with both owner and group empty; throw FileNotFoundException if getFileStatus on non-existing files; fix bugs in getBlockLocations; and changed getFileChecksum json response root to "FileChecksum". 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190077 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 23:13:26 +00:00
Todd Lipcon
221aadbc5b HDFS-2500. Avoid file system operations in BPOfferService thread while processing deletes. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190071 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 22:47:19 +00:00
Todd Lipcon
646e855f6e HDFS-2507. Allow saveNamespace operations to be canceled. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190060 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 22:11:10 +00:00
Todd Lipcon
ebb6cc60c4 HDFS-2507. Allow saveNamespace operations to be canceled. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1190059 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 22:10:58 +00:00
Suresh Srinivas
456ca74b96 HDFS-2496. Separate datatypes for DatanodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189901 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 18:09:42 +00:00
Tsz-wo Sze
10dc6b0927 HDFS-1869. mkdirs should use the supplied permission for all of the created directories. Contributed by Daryn Sharp 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189546 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 00:02:19 +00:00
Tsz-wo Sze
753c14cf4e Move webhdfs entries from trunk to 0.23 in CHANGES.txt. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189541 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 23:49:45 +00:00
Tsz-wo Sze
758194382c HDFS-2494. Close the streams and DFSClient in DatanodeWebHdfsMethods. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189360 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 18:24:29 +00:00
Jitendra Nath Pandey
8534e70044 HDFS-2411. The auth to local mappings are not respected with webhdfs and security enabled. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189355 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 18:15:58 +00:00
Tsz-wo Sze
8335995630 HDFS-2501. Add version prefix and root methods to webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189028 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 05:16:57 +00:00
Mahadev Konar
d19cfe0164 MAPREDUCE-2746. Yarn servers can't communicate with each other with hadoop.security.authorization set to true (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188522 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:07:13 +00:00
Suresh Srinivas
a268a3c0e4 HDFS-2488. Separate datatypes for InterDatanodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188487 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:32:49 +00:00