big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,016 Commits 2 Branches 2 Tags 264 MiB
c7007a8a35
Commit Graph

91 Commits

Author SHA1 Message Date
Mahadev Konar
f3c35222e6 MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240603 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 20:20:00 +00:00
Vinod Kumar Vavilapalli
94242c9385 MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240413 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 00:04:53 +00:00
Mahadev Konar
947ede4c4e MAPREDUCE-3696. MR job via oozie does not work on hadoop 23. (John George via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1239310 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-01 20:43:10 +00:00
Alejandro Abdelnur
e3d555a656 MAPREDUCE-3499. New MiniMR does not setup proxyuser configuration correctly, thus tests using doAs do not work. (johnvijoe via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1239207 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-01 17:03:27 +00:00
Arun Murthy
a8b6ee0491 MAPREDUCE-3756. Made single shuffle limit configurable. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238721 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-31 18:08:11 +00:00
Vinod Kumar Vavilapalli
76b653a367 MAPREDUCE-3716. Fixing YARN+MR to allow MR jobs to be able to use java.io.File.createTempFile to create temporary files as part of their tasks. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238136 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-31 02:23:38 +00:00
Mahadev Konar
7e4725db41 MAPREDUCE-3703. ResourceManager should provide node lists in JMX output. (Eric Payne via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1237920 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-30 18:47:55 +00:00
Arun Murthy
0cb3dd913b MAPREDUCE-3720. Changed bin/mapred job -list to not print job-specific information not available at RM. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236588 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-27 08:44:19 +00:00
Mahadev Konar
c43a5a992e MAPREDUCE-3717. JobClient test jar has missing files to run all the test programs. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235639 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-25 06:24:46 +00:00
Vinod Kumar Vavilapalli
dc615c312b MAPREDUCE-3710. Improved FileInputFormat to return better locality for the last split. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235510 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 21:29:39 +00:00
Alejandro Abdelnur
f73daf6af1 MAPREDUCE-3505. yarn APPLICATION_CLASSPATH needs to be overridable. (ahmed via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235391 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 18:21:27 +00:00
Mahadev Konar
27c56b02b3 MAPREDUCE-3646. Remove redundant URL info from "mapred job" output. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235034 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-23 22:22:53 +00:00
Alejandro Abdelnur
10325d9732 MAPREDUCE-3582. Move successfully passing MR1 tests to MR2 maven tree.(ahmed via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1233090 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-18 22:10:12 +00:00
Arun Murthy
d737c323a6 MAPREDUCE-3669. Allow clients to talk to MR HistoryServer using both delegation tokens and kerberos. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1232580 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-17 21:25:08 +00:00
Vinod Kumar Vavilapalli
b62c1b8563 MAPREDUCE-3404. Corrected MR AM to honor speculative configuration and enable speculating either maps or reduces. Contributed by Eric Payne. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1231395 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-13 23:41:23 +00:00
Mahadev Konar
bc4b1f48d3 MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229855 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 01:50:10 +00:00
Vinod Kumar Vavilapalli
7712e70090 MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR AM. Improved the earlier patch to not to JobHistoryServer repeatedly. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229787 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 23:07:54 +00:00
Alejandro Abdelnur
c27de4bd0b MAPREDUCE-1744. DistributedCache creates its own FileSytem instance when adding a file/archive to the path. (Dick King via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227238 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-04 17:51:17 +00:00
Arun Murthy
96247ead03 MAPREDUCE-3376. Fixed Task to ensure it passes reporter to combiners using old MR api. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221501 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-20 22:01:13 +00:00
Mahadev Konar
beec374542 MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR ApplicationMaster (Anupam Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214662 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-15 08:58:51 +00:00
Vinod Kumar Vavilapalli
b7ae5a6cb7 MAPREDUCE-3426. Fixed MR AM in uber mode to write map intermediate outputs in the correct directory to work properly in secure mode. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213987 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:35:11 +00:00
Mahadev Konar
9b75b05163 MAPREDUCE-3328. mapred queue -list output inconsistent and missing child queues. (Ravi Prakash via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213504 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 23:56:26 +00:00
Thomas White
714ae6e62f MAPREDUCE-3369. Migrate MR1 tests to run on MR2 using the new interfaces introduced in MAPREDUCE-3169. Contributed by Ahmed Radwan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1209281 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 22:29:42 +00:00
Siddharth Seth
449554f782 MAPREDUCE-3169 amendment. Deprecate MiniMRCluster. Contributed by Ahmed Radwan) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1209235 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 21:00:28 +00:00
Siddharth Seth
102541b12b MAPREDUCE-3443. JobClient and Job should function in the context of the UGI which created them. (Contributed by Mahadev Konar) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1209231 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 20:39:37 +00:00
Mahadev Konar
ea17da82f7 MAPREDUCE-3121. NodeManager should handle disk-failures (Ravi Gummadi via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1208131 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-29 23:17:54 +00:00
Arun Murthy
f17ed541c7 MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to cut down noise. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1205628 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 22:04:33 +00:00
Mahadev Konar
63edb9d724 MAPREDUCE-3434. Nightly build broken (Hitesh Shah via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1204638 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-21 18:25:51 +00:00
Alejandro Abdelnur
26447229ba HADOOP-7590. Mavenize streaming and MR examples. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203941 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-19 01:24:32 +00:00
Alejandro Abdelnur
ed45ad0fcc MAPREDUCE-3415. improve MiniMRYarnCluster & DistributedShell JAR resolution (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203411 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-17 22:58:44 +00:00
Alejandro Abdelnur
302e3dfe2c MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides client APIs cross MR1 and MR2. (Ahmed via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1203371 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-17 21:13:29 +00:00
Vinod Kumar Vavilapalli
9d7402e0af MAPREDUCE-3407. Fixed pom files to refer to the correct MR app-jar needed by the integration tests. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1202766 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-16 16:30:05 +00:00
Vinod Kumar Vavilapalli
27c9b8833b MAPREDUCE-3370. More changes accidentally missed from the previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200775 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-11 08:51:21 +00:00
Vinod Kumar Vavilapalli
9c17410c50 MAPREDUCE-3370. Fixed MiniMRYarnCluster and related tests to not use a hard-coded path for the mr-app jar. Contributed by Ahmed Radwan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200768 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-11 08:44:52 +00:00
Vinod Kumar Vavilapalli
200bb27557 MAPREDUCE-3392. Fixed Cluster's getDelegationToken's API to return null when there isn't a supported token. Contributed by John George. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200484 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 19:19:25 +00:00
Vinod Kumar Vavilapalli
15150cd997 MAPREDUCE-3280. Removed the unnecessary job user-name configuration in mapred-site.xml. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200441 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 17:28:16 +00:00
Mahadev Konar
6733a1ca5e MAPREDUCE-3291. App fail to launch due to delegation token not found in cache (Robert Evans via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1198583 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-06 22:58:47 +00:00
Vinod Kumar Vavilapalli
47a381e306 MAPREDUCE-3146. Added a MR specific command line to dump logs for a given TaskAttemptID. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195349 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 06:42:06 +00:00
Vinod Kumar Vavilapalli
670fa24b48 MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the JobHistoryServer. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:45:04 +00:00
Vinod Kumar Vavilapalli
f114dad5ef MAPREDUCE-2986. Fixed MiniYARNCluster to support multiple NodeManagers. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189721 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 12:42:12 +00:00
Arun Murthy
8aabd3d4e6 MAPREDUCE-2858. Added a WebApp Proxy for applications. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189036 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 06:29:21 +00:00
Mahadev Konar
2adc51c618 MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189023 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 04:31:28 +00:00
Mahadev Konar
d19cfe0164 MAPREDUCE-2746. Yarn servers can't communicate with each other with hadoop.security.authorization set to true (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188522 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:07:13 +00:00
Arun Murthy
c70ee2488c MAPREDUCE-3248. Fixed log4j properties. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188476 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:03:08 +00:00
Vinod Kumar Vavilapalli
5795fcfd99 MAPREDUCE-3058. Fixed MR YarnChild to report failure when task throws an error and thus prevent a hanging task and job. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187654 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-22 06:14:46 +00:00
Mahadev Konar
4186121c08 MAPREDUCE-3239. Use new createSocketAddr API in MRv2 to give better error messages on misconfig (Todd Lipcon via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187556 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 21:36:58 +00:00
Mahadev Konar
45b2bce871 MAPREDUCE-3163. JobClient spews errors when killing MR2 job. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187491 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 18:35:35 +00:00
Vinod Kumar Vavilapalli
df2991c0cb MAPREDUCE-3104. Implemented Application-acls. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186748 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 11:45:38 +00:00
Vinod Kumar Vavilapalli
3e3abcb769 MAPREDUCE-3199. Fixed pom files to include correct log4j configuration for tests. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185967 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 04:30:27 +00:00
Arun Murthy
ab787f44aa MAPREDUCE-3165. Ensure logging options are set correctly for MR AM and tasks. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185887 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 21:45:38 +00:00