big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,081 Commits 2 Branches 2 Tags 264 MiB
52004aa8e9
Commit Graph

222 Commits

Author SHA1 Message Date
Mahadev Konar
d19cfe0164 MAPREDUCE-2746. Yarn servers can't communicate with each other with hadoop.security.authorization set to true (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188522 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:07:13 +00:00
Suresh Srinivas
a268a3c0e4 HDFS-2488. Separate datatypes for InterDatanodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188487 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:32:49 +00:00
Suresh Srinivas
4846ca7863 HDFS-2497 Fix TestBackupNode failure. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188436 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 23:07:13 +00:00
Sanjay Radia
0920056f04 HDFS-2481 Unknown protocol: org.apache.hadoop.hdfs.protocol.ClientProtocol (sanjay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188300 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 19:08:03 +00:00
Suresh Srinivas
8fe95d276d Added missing files for HDFS-2489 commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188286 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 18:46:59 +00:00
Suresh Srinivas
2b35e8aa4f HDFS-2489. Move Finalize and Register to separate file out of DatanodeCommand.java. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188282 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 18:40:13 +00:00
Konstantin Shvachko
6e0991704f HDFS-2452. OutOfMemoryError in DataXceiverServer takes down the DataNode. Contributed by Uma Maheswara Rao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187965 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-23 20:17:04 +00:00
Steve Loughran
2690d0db0e HDFS-2485 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187887 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-23 14:13:58 +00:00
Konstantin Shvachko
7dd869c2a9 HDFS-2491. TestBalancer can fail when datanode utilization and avgUtilization is exactly same. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187837 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-23 00:35:55 +00:00
Suresh Srinivas
d2e1a71918 HDFS-2480. Separate datatypes for NamenodeProtocol. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187505 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 19:03:18 +00:00
Tsz-wo Sze
7c48130d41 HDFS-2427. Change the default permission in webhdfs to 755 and add range check/validation for all parameters. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187140 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 00:27:08 +00:00
Suresh Srinivas
520d2502c5 HDFS-2459. Separate datatypes for Journal Protocol. (suresh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186896 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 16:31:57 +00:00
Tsz-wo Sze
0bd8f0bd40 HDFS-2453. Fix http response code for partial content in webhdfs, added getDefaultBlockSize() and getDefaultReplication() in WebHdfsFileSystem and cleared content type in ExceptionHandler. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186508 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 21:38:58 +00:00
Owen O'Malley
073a13aeb1 HDFS-2467. HftpFileSystem uses incorrect compare for finding delegation 
tokens. (omalley)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185842 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 20:33:16 +00:00
Jitendra Nath Pandey
8d4842383e HDFS-2188. Make FSEditLog create its journals from a list of URIs rather than NNStorage. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185354 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 20:23:57 +00:00
Tsz-wo Sze
50cb2771e9 HDFS-2439. Fix NullPointerException in webhdfs when opening a non-existing file or creating a file without specifying the replication parameter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183554 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 23:25:09 +00:00
Konstantin Shvachko
4cb7a4854d HDFS-2012. Balancer incorrectly treats nodes whose utilization equals avgUtilization. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183457 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 18:31:55 +00:00
Jitendra Nath Pandey
002dd6968b MAPREDUCE-2764. Fix renewal of dfs delegation tokens. Contributed by Owen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183187 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 01:24:20 +00:00
Tsz-wo Sze
c46dbedaf9 HDFS-2424. Added a root element "HdfsFileStatuses" for the response of webhdfs listStatus. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183175 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 00:30:30 +00:00
Tsz-wo Sze
5904e00b4f HDFS-2428. Convert com.sun.jersey.api.ParamException$QueryParamException to IllegalArgumentException and response it http BAD_REQUEST in webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183098 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-13 21:06:41 +00:00
Tsz-wo Sze
7d1a5078b7 HDFS-2441. Remove the Content-Type set by HttpServer.QuotingInputFilter in webhdfs responses. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183081 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-13 20:41:00 +00:00
Harsh J
532872bf74 HDFS-2349. Corruption detected during block transfers should log WARNs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182205 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 04:48:52 +00:00
Aaron Myers
f527f989af HDFS-2422. The NN should tolerate the same number of low-resource volumes as failed volumes (atm) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181316 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 00:37:29 +00:00
Suresh Srinivas
676f488eff HDFS-2404. webhdfs liststatus json response is not correct. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180757 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 01:49:37 +00:00
Steve Loughran
3ab2e79ad7 HDFS-2209 datanode connection failure logging 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180353 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-08 10:41:56 +00:00
Todd Lipcon
54e81464a3 HDFS-2412. Add backwards-compatibility layer for renamed FSConstants class. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180202 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 20:28:39 +00:00
Sanjay Radia
e4db38bdbe HDFS-2181 Separate HDFS Client wire protocol data types (sanjay) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179877 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-06 21:58:22 +00:00
Jitendra Nath Pandey
6f1d18bd47 HDFS-2409. _HOST in dfs.web.authentication.kerberos.principal. Incorporates HDFS-2405 as well. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179861 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-06 21:30:06 +00:00
Tsz-wo Sze
a2d7287873 HDFS-2403. NamenodeWebHdfsMethods.generateDelegationToken(..) does not use the renewer parameter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179856 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-06 21:17:19 +00:00
Tsz-wo Sze
1b1016beeb HDFS-2395. Add a root element in the JSON responses of webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179169 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 11:29:05 +00:00
Eric Yang
3edf3579c8 HADOOP-7684. Added init.d script for jobhistory server and 
secondary namenode. (Eric Yang)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178653 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:44:08 +00:00
Eric Yang
abc2b45dfa HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178644 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:13:58 +00:00
Tsz-wo Sze
dc8464f943 HDFS-2348. Support getContentSummary and getFileChecksum in webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177905 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-01 04:49:00 +00:00
Doug Cutting
f48280ac56 HDFS-2298. Fix TestDfsOverAvroRpc by changing ClientProtocol to not include multiple methods of the same name. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177757 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 18:01:27 +00:00
Jitendra Nath Pandey
1ae5b5e338 HDFS-2158. Add JournalSet to manage the set of journals. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177473 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 01:14:37 +00:00
Suresh Srinivas
e90a5b4043 HDFS-2371. Refactor BlockSender.java for better readability. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177161 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-29 04:40:15 +00:00
Suresh Srinivas
e9dd78d9fe HDFS-2355. Federation: enable using the same configuration file across all the nodes in the cluster. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177100 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 22:58:10 +00:00
Tsz-wo Sze
96f9fc9199 HDFS-2363. Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176733 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 05:49:17 +00:00
Jitendra Nath Pandey
825f9c80a4 HDFS-2361. hftp is broken, fixed username checks in JspHelper. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176729 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 05:29:09 +00:00
Tsz-wo Sze
59265d6ed8 HDFS-2368. Move SPNEGO conf properties from hdfs-default.xml to hdfs-site.xml. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176719 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 04:51:29 +00:00
Suresh Srinivas
201b7879ba HDFS-2373. Commands using webhdfs and hftp print unnecessary debug info on the console with security enabled. Contributed by Arpit Gupta. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176654 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 23:16:17 +00:00
Tsz-wo Sze
bf78f15ffb HDFS-2366. Initialize WebHdfsFileSystem.ugi in object construction. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176178 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 02:43:37 +00:00
Konstantin Shvachko
5a6f8e38c0 HDFS-2290. Block with corrupt replica is not getting replicated. Contributed by Benoy Antony. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175175 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-24 15:04:34 +00:00
Tsz-wo Sze
83a83d3b73 HDFS-2356. Support case insensitive query parameter names in webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175113 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-24 06:15:50 +00:00
Tsz-wo Sze
d773bf0fb5 HDFS-46. Change default namespace quota of root directory from Integer.MAX_VALUE to Long.MAX_VALUE. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173990 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-22 08:25:20 +00:00
Sanjay Radia
7d8e9d6960 HADOOP-7524 Change RPC to allow multiple protocols including multuple versions of the same protocol (sanjay Radia) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173488 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 04:25:55 +00:00
Tsz-wo Sze
4dc4e9e63f HDFS-2340. Support getFileBlockLocations and getDelegationToken in webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173468 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 02:56:08 +00:00
Matthew Foley
9d14f100c8 HDFS-2347. Fix checkpointTxnCount's comment about editlog size. Contributed by Uma Maheswara Rao G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172916 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-20 00:26:32 +00:00
Tsz-wo Sze
e34d2c075d HDFS-2330. In NNStorage and FSImagePreTransactionalStorageInspector, IOExceptions of stream closures can mask root exceptions. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172219 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-18 08:30:47 +00:00
Tsz-wo Sze
dafa8f7a77 HDFS-362. FSEditLog should not writes long and short as UTF8, and should not use ArrayWritable for writing non-array items. Contributed by Uma Maheswara Rao G 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171945 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-17 10:53:44 +00:00
Aaron Myers
a89d4609b2 HDFS-2337. DFSClient shouldn't keep multiple RPC proxy references (atm) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171711 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-16 18:38:32 +00:00
Tsz-wo Sze
78e3821b81 HDFS-2318. Provide authentication to webhdfs using SPNEGO and delegation tokens. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171611 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-16 15:02:30 +00:00
Tsz-wo Sze
cde0e484e7 HDFS-2338. Add configuration option to enable/disable webhdfs. Contributed by jitendra 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171379 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-16 04:01:51 +00:00
Tsz-wo Sze
376a1a2511 HDFS-2333. Change DFSOutputStream back to package private, otherwise, there are two SC_START_IN_CTOR findbugs warnings. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171136 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-15 15:28:23 +00:00
Tsz-wo Sze
be2b0921fa HDFS-2331. Fix WebHdfsFileSystem compilation problems for a bug in JDK version < 1.6.0_26. Contributed by Abhijit Suresh Shingate 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170996 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-15 08:04:46 +00:00
Thomas White
e4dc2e1f56 HDFS-2323. start-dfs.sh script fails for tarball install 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170233 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-13 16:53:57 +00:00
Tsz-wo Sze
6c3b59505b HDFS-2317. Support read access to HDFS in webhdfs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170085 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-13 08:34:27 +00:00
Tsz-wo Sze
61d0b7530c HDFS-2284. Add a new FileSystem, webhdfs://, for supporting write Http access to HDFS. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1167662 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-11 01:41:42 +00:00
Todd Lipcon
06e84a1bca HDFS-2223. Untangle depencencies between NN components. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166466 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-07 23:23:24 +00:00
Jitendra Nath Pandey
bdc3720d5b HDFS-2018. Move all journal stream management code into one place. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165826 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 20:27:38 +00:00
Todd Lipcon
b0632df93a HDFS-2197. Refactor RPC call implementations out of NameNode class. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165463 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 00:41:47 +00:00
Todd Lipcon
d1438b501d HDFS-2310. TestBackupNode fails since HADOOP-7524 went in. Contributed by Ivan Kelly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165461 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 00:34:57 +00:00
Aaron Myers
458c34a2ab Follow-up commit for HDFS-1620 since I missed some files. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165103 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-04 19:38:50 +00:00
Aaron Myers
8ae98a9d1c HDFS-1620. Rename HdfsConstants -> HdfsServerConstants, FSConstants -> HdfsConstants. (Harsh J Chouraria via atm) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165096 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-04 19:30:12 +00:00
Tsz-wo Sze
aba0e225fc HDFS-1217. Change some NameNode methods from public to package private. Constributed by Laxman 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1163081 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-30 04:02:22 +00:00
Arun Murthy
8b56463315 HDFS-2289. Ensure jsvc is bundled with the HDFS distribution artifact. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1162421 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-27 19:07:04 +00:00
Todd Lipcon
7cf49f5d9c HDFS-2286. DataXceiverServer logs AsynchronousCloseException at shutdown. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1162003 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-26 06:44:13 +00:00
Hairong Kuang
73451ed2d9 HDFS-395. DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161992 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-26 04:46:42 +00:00
Hairong Kuang
e680023f8b Revert 1161976 since the log message was incorrectly marked the issue as HDFS-349. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161991 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-26 04:41:52 +00:00
Hairong Kuang
e6339be188 HDFS-349. DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161976 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-26 03:10:13 +00:00
Arun Murthy
a44ffcc0be Preparing for hadoop-0.24.0 development. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161736 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 20:32:04 +00:00
Arun Murthy
cd7157784e HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161332 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 00:14:24 +00:00