big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,883 Commits 2 Branches 2 Tags 264 MiB
f19465f72f
Commit Graph

4883 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Suresh Srinivas
3704e091a8 HDFS-4610. Use common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute. Contrbitued by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477427 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-30 00:35:53 +00:00
Suresh Srinivas
27d4592771 YARN-506. Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477408 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 23:09:28 +00:00
Suresh Srinivas
8f7ce62085 MAPREDUCE-5177. Use common utils FileUtil#setReadable/Writable/Executable & FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477403 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 23:00:39 +00:00
Suresh Srinivas
e2091275dc HDFS-4610. Reverting the patch Jenkins build is not run. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477396 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 22:40:03 +00:00
Suresh Srinivas
fb30240b87 HDFS-4610. Use common utils FileUtil#setReadable/Writable/Executable & FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477385 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 22:26:53 +00:00
Suresh Srinivas
30b7a9c49f HADOOP-9413. Add common utils for File#setReadable/Writable/Executable & File#canRead/Write/Execute that work cross-platform. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477376 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 22:12:57 +00:00
Aaron Myers
ce7e5565f4 HDFS-4305. Add a configurable limit on number of blocks per file, and min block size. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477354 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 21:28:48 +00:00
Aaron Myers
ce73b3b01f HDFS-4687. TestDelegationTokenForProxyUser#testWebHdfsDoAs is flaky with JDK7. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477344 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 21:10:50 +00:00
Aaron Myers
c06bbc25e8 HDFS-4733. Make HttpFS username pattern configurable. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477237 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 18:05:04 +00:00
Suresh Srinivas
e17d5a9683 HDFS-4734. HDFS Tests that use ShellCommandFencer are broken on Windows. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476877 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 02:07:58 +00:00
Suresh Srinivas
293fb1ec1e HADOOP-9524. Fix ShellCommandFencer to work on Windows. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476856 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 23:16:47 +00:00
Vinod Kumar Vavilapalli
ae393ab26e MAPREDUCE-5179. Fix unit test in TestHSWebServices which fails when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476835 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 21:02:25 +00:00
Vinod Kumar Vavilapalli
d9ce42479e YARN-576. Modified ResourceManager to reject NodeManagers that don't satisy minimum resource requirements. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476824 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 20:03:37 +00:00
Suresh Srinivas
3e00835dad HDFS-4705. Address HDFS test failures on Windows because of invalid dfs.namenode.name.dir. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476610 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 16:16:19 +00:00
Suresh Srinivas
e2d901f4da HADOOP-9490. LocalFileSystem#reportChecksumFailure not closing the checksum file handle before rename. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476609 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 16:10:31 +00:00
Suresh Srinivas
f5ef2db500 HADOOP-9500. TestUserGroupInformation#testGetServerSideGroups fails on Windows due to failure to find winutils.exe. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476606 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 15:44:33 +00:00
Suresh Srinivas
f55fd818e0 HDFS-4722. TestGetConf#testFederation times out on Windows. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476597 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:53:37 +00:00
Suresh Srinivas
fbf10b2604 HDFS-4740. Fixes for a few test failures on Windows. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476596 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:47:15 +00:00
Suresh Srinivas
0181ce03af HDFS-4743. TestNNStorageRetentionManager fails on Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476591 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:38:02 +00:00
Suresh Srinivas
90612f6423 HDFS-4748. MiniJournalCluster#restartJournalNode leaks resources, which causes sporadic test failures. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476587 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:29:41 +00:00
Suresh Srinivas
fe3853484e HADOOP-9290. Some tests cannot load native library on windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476586 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:24:02 +00:00
Suresh Srinivas
b7afde6d52 HDFS-4741. TestStorageRestore#testStorageRestoreFailure fails on Windows. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476585 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:16:32 +00:00
Suresh Srinivas
4e21c347d2 HDFS-4768. File handle leak in datanode when a block pool is removed. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476579 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:08:34 +00:00
Jason Darrell Lowe
fc45cbf8df HADOOP-9504. MetricsDynamicMBeanBase has concurrency issues in createMBeanInfo. Contributed by Liang Xie 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476487 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 04:11:23 +00:00
Tsz-wo Sze
75b4231a8c HDFS-4761. When resetting FSDirectory, the inodeMap should also be reset. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476452 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 23:47:57 +00:00
Tsz-wo Sze
d2495d553f HDFS-4721. Speed up lease recovery by avoiding stale datanodes and choosing the datanode with the most recent heartbeat as the primary. Contributed by Varun Sharma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476399 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 20:51:45 +00:00
Devaraj Das
5d2ffde68e HDFS-2576. Enhances the DistributedFileSystem's create API so that clients can specify favored datanodes for a file's blocks. Contributed by Devaraj Das and Pritam Damania. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476395 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 20:39:51 +00:00
Vinod Kumar Vavilapalli
95fa3e82e4 YARN-363. Add webapps/proxy directory without which YARN proxy-server fails when started in stand-alone mode. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476330 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 18:24:30 +00:00
Vinod Kumar Vavilapalli
f0ac5a849c YARN-591. Moved RM recovery related records out of public API as they do not belong there. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476324 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 18:02:45 +00:00
Vinod Kumar Vavilapalli
635f96e74e YARN-562. Missed files from previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476038 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 04:42:34 +00:00
Vinod Kumar Vavilapalli
fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Tsz-wo Sze
06fb184d4d HDFS-4757. Update FSDirectory#inodeMap when replacing an INodeDirectory while setting quota. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476005 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 00:50:09 +00:00
Tsz-wo Sze
e9c83b20f6 HADOOP-9503. Remove sleep between IPC client connect timeouts. Contributed by Varun Sharma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475959 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 21:07:38 +00:00
Suresh Srinivas
a89f8e4d95 Cleanup CHANGES.txt and move the jira to appropriate 2.x release section to make it consistent with branch-2 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475779 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 14:03:22 +00:00
Thomas White
d10428cab2 YARN-289. Fair scheduler allows reservations that won't fit on node. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475681 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 09:15:51 +00:00
Thomas White
a43c344eee YARN-595. Refactor fair scheduler to use common Resources. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475670 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 08:33:43 +00:00
Vinod Kumar Vavilapalli
448df18646 YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah. 
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 05:42:22 +00:00
Suresh Srinivas
16ef8dd3a5 HDFS-4745. TestDataTransferKeepalive#testSlowReader has race condition that causes sporadic failure. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475623 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 04:25:58 +00:00
Siddharth Seth
e96077aaf4 MAPREDUCE-5181. RMCommunicator should not use AMToken from the env. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471816 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 02:50:16 +00:00
Siddharth Seth
be256a1819 YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471814 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 02:46:47 +00:00
Arun Murthy
daaa73b657 MAPREDUCE-5166. Fix ConcurrentModificationException due to insufficient synchronization on updates to task Counters. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471796 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 00:54:04 +00:00
Aaron Myers
e6b5b01561 HDFS-4739. NN can miscalculate the number of extra edit log segments to retain. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471769 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 23:29:30 +00:00
Vinod Kumar Vavilapalli
a2f97fd9cb YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471608 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 19:42:47 +00:00
Suresh Srinivas
a09c29b8c0 HDFS-4339. Add the missed entry CHANGES.txt from r1465835 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471595 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 18:41:12 +00:00
Arun Murthy
40e78c2ca2 MAPREDUCE-4737. Ensure that mapreduce APIs are semantically consistent with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that cleanup is now called even if there is an error. The old mapred API already ensures that Mapper.close and Reducer.close are invoked during error handling. Note that it is an incompatible change, however end-users can override Mapper.run and Reducer.run to get the old (inconsistent) behaviour. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471556 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 17:38:43 +00:00
Robert Joseph Evans
0777474cc2 MAPREDUCE-5069. add concrete common implementations of CombineFileInputFormat (Sangjin Lee via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471424 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 14:11:50 +00:00
Aaron Myers
4c662fd3fe HDFS-4737. JVM path embedded in fuse binaries. Contributed by Sean Mackrory. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471215 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 00:47:13 +00:00
Vinod Kumar Vavilapalli
cd975d4b64 YARN-581. Added a test to verify that app delegation tokens are restored after RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471187 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 22:49:44 +00:00
Vinod Kumar Vavilapalli
27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00
Tsz-wo Sze
631be95398 HDFS-4732. Fix TestDFSUpgradeFromImage which fails on Windows due to failure to unpack old image tarball that contains hard links. Chris Nauroth 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471090 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 18:47:47 +00:00