big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,509 Commits 2 Branches 2 Tags 264 MiB
c1314eb2a3
Commit Graph

114 Commits

Author SHA1 Message Date
Todd Lipcon
b1ceaac3e6 Merge trunk into branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1390199 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-25 22:43:04 +00:00
Alejandro Abdelnur
8bf6b80f2b Reverting HADOOP-8805 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1387697 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-19 18:06:42 +00:00
Todd Lipcon
e9f4de5ced Merge trunk into branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1387449 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-19 04:34:55 +00:00
Alejandro Abdelnur
9f6cfe32e4 HADOOP-8805. Move protocol buffer implementation of GetUserMappingProtocol from HDFS to Common. (bowang via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1387298 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-18 18:15:37 +00:00
Todd Lipcon
83c14fbd24 HDFS-3943. QJM: remove currently-unused md5sum field. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1386863 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-17 21:51:40 +00:00
Todd Lipcon
a93ba1648a HDFS-3840. JournalNodes log JournalNotFormattedException backtrace error before being formatted. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383252 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-11 06:33:47 +00:00
Todd Lipcon
959afc0fd3 HDFS-3901. QJM: send 'heartbeat' messages to JNs even when they are out-of-sync. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1383137 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-10 22:30:52 +00:00
Todd Lipcon
99ec5bd8d3 Merge trunk into QJM branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380990 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:57:47 +00:00
Todd Lipcon
c334cc89a8 HDFS-2793. Add an admin command to trigger an edit log roll. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380982 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:40:23 +00:00
Todd Lipcon
8021d9199f HDFS-3863. Track last "committed" txid in QJM. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1380976 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 04:13:19 +00:00
Todd Lipcon
1e68d4726b HDFS-3845. Fixes for edge cases in QJM recovery protocol. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1377809 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-27 19:55:01 +00:00
Aaron Myers
a8ff292669 Merge trunk into HDFS-3077 branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1377092 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-24 20:38:08 +00:00
Tsz-wo Sze
c46de830da HDFS-3177. Update DFSClient and DataXceiver to handle different checkum types in file checksum computation. Contributed by Kihwal Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376928 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-24 14:15:14 +00:00
Suresh Srinivas
380870d544 HDFS-3832. Remove protocol methods related to DistributedUpgrade. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376139 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-22 16:59:50 +00:00
Suresh Srinivas
6c0ccb5989 HDFS-2686. Remove DistributedUpgrade related code. Contributed by Suresh Srinivas 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375800 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-21 21:18:40 +00:00
Tsz-wo Sze
b0ea77303b HADOOP-8240. Add a new API to allow users to specify a checksum type on FileSystem.create(..). Contributed by Kihwal Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374696 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-19 05:46:57 +00:00
Eli Collins
18c5bc86ca HADOOP-8689. Make trash a server side configuration option. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374472 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 23:22:17 +00:00
Aaron Myers
fccace6116 HDFS-3672. Expose disk-location information for blocks to enable better scheduling. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374355 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 16:52:07 +00:00
Todd Lipcon
42cdc1b083 HDFS-3797. QJM: add segment txid as a parameter to journal() RPC. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373571 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 18:58:51 +00:00
Todd Lipcon
f765fdb657 HDFS-3793. Implement genericized format() in QJM. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1373177 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 00:48:11 +00:00
Todd Lipcon
302f1e96ce Merge trunk into branch. 
Branch will not build after this commit: need to implement new JournalManager
interfaces in QuorumJournalManager in a follow-up.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1371518 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-09 22:29:36 +00:00
Aaron Myers
9b4a7900c7 HDFS-3637. Add support for encrypting the DataTransferProtocol. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370354 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-07 16:40:03 +00:00
Todd Lipcon
d2d0736de4 HDFS-3692. Support purgeEditLogs() call to remotely purge logs on JNs. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1365792 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-25 21:44:26 +00:00
Todd Lipcon
74d4573a23 HDFS-3077. Quorum-based protocol for reading and writing edit logs. Contributed by Todd Lipcon based on initial work from Brandon Li and Hari Mankude. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1363596 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-20 00:25:50 +00:00
Todd Lipcon
e0ef844280 HDFS-3170. Add more useful metrics for write latency. Contributed by Matthew Jacobs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1357970 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-05 22:18:30 +00:00
Tsz-wo Sze
83cf475050 HDFS-744. Support hsync in HDFS. Contributed by Lars Hofhans 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344419 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-30 19:10:09 +00:00
Todd Lipcon
1a3bd5d42f Merge trunk into auto-HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1340622 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-20 06:12:14 +00:00
Todd Lipcon
d1a54b872d HDFS-3438. BootstrapStandby should not require a rollEdits on active node. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1340343 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-19 05:13:38 +00:00
Todd Lipcon
e14e10fb86 Merge trunk into auto-HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1339410 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-16 22:47:37 +00:00
Eli Collins
7428aeca86 HDFS-3418. Rename BlockWithLocationsProto datanodeIDs field to storageIDs. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1338830 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-15 19:03:14 +00:00
Todd Lipcon
9d5799553f HADOOP-8279. Allow manual failover to be invoked when auto-failover is enabled. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1333288 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-03 01:56:33 +00:00
Todd Lipcon
ca49126e3d Merge trunk into auto-HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1327724 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-18 23:35:30 +00:00
Aaron Myers
7f427646df HDFS-2983. Relax the build version check to permit rolling upgrades within a release. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325110 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 03:36:13 +00:00
Todd Lipcon
2bf19979b3 Merge trunk into auto-failover branch. 
Needs a few tweaks to fix compilation - will do in followup commit. This is just a straight merge


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1324567 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-11 05:47:40 +00:00
Todd Lipcon
ee5daee1b7 HDFS-3214. InterDatanodeProtocolServerSideTranslatorPB doesn't handle null response from initReplicaRecovery. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311125 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-09 03:22:41 +00:00
Todd Lipcon
ca6f0940fd Merge trunk into auto-failover branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1310905 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-07 22:53:55 +00:00
Tsz-wo Sze
74b4b45651 HDFS-3211. Add fence(..) and replace NamenodeRegistration with JournalInfo and epoch in JournalProtocol. Contributed by suresh 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310649 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-07 00:52:12 +00:00
Todd Lipcon
30068a1d83 Merge trunk into auto-failover branch. 
Addressed two semantic conflicts after the commit of HADOOP-8077 -- test code was referencing the old constant for the fencing methods config key.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1310174 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-06 05:07:33 +00:00
Suresh Srinivas
861c872541 HDFS-3204. Minor modification to JournalProtocol.proto to make it generic. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1310134 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-06 00:08:19 +00:00
Todd Lipcon
789ed57e71 HDFS-2185. HDFS portion of ZK-based FailoverController. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3042@1308637 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-03 00:59:02 +00:00
Eli Collins
be7dd8333a HDFS-3144. Refactor DatanodeID#getName by use. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308205 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-01 22:12:12 +00:00
Eli Collins
8bd825bb6f HDFS-3164. Move DatanodeInfo#hostName to DatanodeID. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1307890 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-31 19:58:51 +00:00
Tsz-wo Sze
9e108e61fb HDFS-3086. Change Datanode not to send storage list in registration. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1303318 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-21 09:07:33 +00:00
Tsz-wo Sze
6326605acb HDFS-3105. Add DatanodeStorage information to block recovery. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1302683 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-19 22:09:14 +00:00
Todd Lipcon
c14bb61a82 Merge trunk into HA branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1242635 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-10 01:49:08 +00:00
Suresh Srinivas
f88574acde HDFS-2880. Protobuf chagnes in DatanodeProtocol to add multiple storages. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240653 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-05 01:39:30 +00:00
Todd Lipcon
846f97312c HDFS-2691. Fixes for pipeline recovery in an HA cluster: report RBW replicas immediately upon pipeline creation. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1237935 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-30 19:16:15 +00:00
Jitendra Nath Pandey
98302971c2 HDFS-2801. Provide a method in client side translators to check for a methods supported in underlying protocol. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1237192 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-29 02:01:24 +00:00
Aaron Myers
ede7f21278 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1230248 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 20:53:50 +00:00
Jitendra Nath Pandey
8faf7e8fb6 HDFS-2739. SecondaryNameNode doesn't start up. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229877 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 02:54:48 +00:00
Aaron Myers
d680080da0 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1228339 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-06 18:47:05 +00:00
Jitendra Nath Pandey
4efd3699a6 HDFS-2697. Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings protocol to protocol buffers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227887 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 23:03:31 +00:00
Todd Lipcon
8134b1c870 Merge trunk into HA branch. 
Several conflicts around introduction of protobuf translator for DatanodeProtocol - mostly trivial resolutions.

NB: this does not successfully pass any tests since the HAStatus field needs
to be integrated into the HeartbeatResponse Protobuf implementation.
That will be a separate commit for clearer history.



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1214518 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 22:47:41 +00:00
Sanjay Radia
d8dfcdcbc2 HDFS-2669 Enable protobuf rpc for ClientNamenodeProtocol 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214128 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 09:27:51 +00:00
Jitendra Nath Pandey
3cffe34177 HDFS-2661. Enable protobuf RPC for DatanodeProtocol. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214033 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 02:15:37 +00:00
Suresh Srinivas
3954a2fb1c HDFS-2663. Optional protobuf parameters are not handled correctly. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213985 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:31:44 +00:00
Suresh Srinivas
6a609cb471 Reverting the patch r1213981 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213984 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:27:58 +00:00
Suresh Srinivas
b5229fd19b HDFS-2663. Optional protobuf parameters are not handled correctly. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213981 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:22:28 +00:00
Suresh Srinivas
3001a172c8 Reverting r1213512 because it committed changes that were not part of the patch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213980 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:17:02 +00:00
Aaron Myers
a0fe4f476a Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1213867 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 19:02:37 +00:00
Suresh Srinivas
13345f3a85 HDFS-2663. Handle protobuf optional parameters correctly. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213512 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 00:21:48 +00:00
Aaron Myers
89379bc1a3 Merge trunk into HA branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1213389 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 19:41:20 +00:00
Sanjay Radia
48da033901 HDFS-2651 ClientNameNodeProtocol Translators for Protocol Buffers (sanjay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213143 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 05:36:35 +00:00
Alejandro Abdelnur
e7682e1496 HDFS-2511. Add dev script to generate HDFS protobufs. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1212609 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-09 20:21:19 +00:00