big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,838 Commits 2 Branches 2 Tags 264 MiB
6d45b9e31d
Commit Graph

443 Commits

Author SHA1 Message Date
Arun Murthy
b3f90c7060 MAPREDUCE-3275. Added documentation for AM WebApp Proxy. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195579 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 17:33:29 +00:00
Arun Murthy
9db078212f MAPREDUCE-3035. Fixed MR JobHistory to ensure rack information is present. Contributed by chakravarthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195575 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 17:27:42 +00:00
Amar Kamat
7e056015ad MAPREDUCE-3241. [Rumen] Fix Rumen to ignore the AMStartedEvent. (amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195535 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 15:51:32 +00:00
Vinod Kumar Vavilapalli
21b1e1da49 MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195416 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 11:29:23 +00:00
Vinod Kumar Vavilapalli
a75c4cf4e4 MAPREDUCE-2696. Fixed NodeManager to cleanup logs in a thread when logs' aggregation is not enabled. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195383 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 09:55:29 +00:00
Vinod Kumar Vavilapalli
47a381e306 MAPREDUCE-3146. Added a MR specific command line to dump logs for a given TaskAttemptID. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195349 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 06:42:06 +00:00
Vinod Kumar Vavilapalli
5f9e67e226 MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195340 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 05:51:33 +00:00
Arun Murthy
85fce9017c Adding executable svn prop on mock-container-executor. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195336 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 05:29:13 +00:00
Vinod Kumar Vavilapalli
273e092c93 MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing TestResourceTrackerService sometimes. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195319 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 03:30:56 +00:00
Vinod Kumar Vavilapalli
1c8d64f38a MAPREDUCE-3274. Fixed a race condition in MRAppMaster that was causing a task-scheduling deadlock. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195145 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-30 11:29:41 +00:00
Arun Murthy
7f4dc27757 MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1194850 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 09:35:36 +00:00
Thomas White
cd77a1668f HADOOP-7763. Add top-level navigation to APT docs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190703 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 00:16:16 +00:00
Alejandro Abdelnur
1bcff6860c Adding missing CHANGES.txt entry for MAPREDUCE-3014 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190585 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 21:27:08 +00:00
Vinod Kumar Vavilapalli
c1e333f32f MAPREDUCE-3306. Fixed a bug in NodeManager ApplicationImpl that was causing NodeManager to crash. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190523 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 19:45:57 +00:00
Arun Murthy
b93abdcd66 MAPREDUCE-3304. Fixed intermittent test failure due to a race in TestRMContainerAllocator#testBlackListedNodes. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190495 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 18:45:48 +00:00
Vinod Kumar Vavilapalli
cbdb07f4ca MAPREDUCE-2775. Fixed ResourceManager and NodeManager to force a decommissioned node to shutdown. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190467 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 17:31:06 +00:00
Eric Yang
da1db28e93 HADOOP-7740. Fixed security audit logger configuration. (Arpit Gupta via Eric Yang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190452 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 17:08:09 +00:00
Vinod Kumar Vavilapalli
a39992ea63 MAPREDUCE-3296. Fixed the remaining nine FindBugs warnings. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190187 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 07:44:35 +00:00
Vinod Kumar Vavilapalli
670fa24b48 MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the JobHistoryServer. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:45:04 +00:00
Mahadev Konar
47c28ce14d MAPREDUCE-3292. In secure mode job submission fails with Provider org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier not found. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190171 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:13:18 +00:00
Mahadev Konar
062876bed7 MAPREDUCE-3185. RM Web UI does not sort the columns in some cases. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190165 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:04:51 +00:00
Arun Murthy
69dd284452 MAPREDUCE-3290. Fixed a NPE in ClientRMService. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190162 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 05:45:59 +00:00
Arun Murthy
34e6de8f85 MAPREDUCE-3258. Fixed AM & JobHistory web-ui to display counters properly. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190160 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 05:08:04 +00:00
Mahadev Konar
9b34900c89 MAPREDUCE-3209. Jenkins reports 160 FindBugs warnings (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190140 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 02:42:11 +00:00
Arun Murthy
3cccbc9e16 MAPREDUCE-2736. Removing bin/(start,stop)-mapred.sh 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190129 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 02:00:03 +00:00
Arun Murthy
caf07897a7 MAPREDUCE-2736. Resurrecting bin/mapred and bin/mapred-config.sh. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190128 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 01:56:57 +00:00
Mahadev Konar
b304062f1f MAPREDUCE-3186. User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed. (Eric Payne via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190122 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 01:40:24 +00:00
Mahadev Konar
35b62e6e72 MAPREDUCE-3282. bin/mapred job -list throws exception. (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190110 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 01:16:29 +00:00
Arun Murthy
739acd6e14 MAPREDUCE-3284. Moved JobQueueClient to hadoop-mapreduce-client-core. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190084 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 23:21:09 +00:00
Vinod Kumar Vavilapalli
b9f6890db9 MAPREDUCE-3281. Fixed one more bug due to which TestLinuxContainerExecutorWithMocks was failing. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190007 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 20:38:40 +00:00
Eli Collins
1ad8415b72 MAPREDUCE-2736. Remove unused contrib components dependent on MR1. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189982 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 20:06:26 +00:00
Arun Murthy
724f217343 MAPREDUCE-3228. Fixed MR AM to timeout RPCs to bad NodeManagers. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189879 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 17:32:02 +00:00
Vinod Kumar Vavilapalli
1b8511ef89 MAPREDUCE-3281. Fixed a bug in TestLinuxContainerExecutorWithMocks. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189818 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 15:55:41 +00:00
Vinod Kumar Vavilapalli
f114dad5ef MAPREDUCE-2986. Fixed MiniYARNCluster to support multiple NodeManagers. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189721 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 12:42:12 +00:00
Vinod Kumar Vavilapalli
2fe343f963 MAPREDUCE-3240. Fixed NodeManager to be able to forcefully cleanup its containers (process-trees) irrespective of whether the container succeeded, or killed. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189711 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 12:03:17 +00:00
Arun Murthy
f799e61269 MAPREDUCE-3279. Fixed TestJobHistoryParsing which assumed user name to be mapred all the time. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189670 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 08:31:44 +00:00
Arun Murthy
fc74b5a25c MAPREDUCE-3259. Added java.library.path of NodeManager to ContainerLocalizer in LinuxContainerExecutor. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189667 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 08:28:34 +00:00
Arun Murthy
db8ac0ec3c MAPREDUCE-3257. Added authorization checks for the protocol between ResourceManager and ApplicatoinMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189630 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 06:24:22 +00:00
Arun Murthy
5fadd4de2a MAPREDUCE-3175. Add authorization to admin web-pages such as /stacks, /jmx etc. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189619 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 06:11:08 +00:00
Mahadev Konar
33dc6578de MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189616 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 06:04:17 +00:00
Arun Murthy
3e4efbb609 MAPREDUCE-3254. Fixed streaming to set the job.jar by using the right JobConf ctor. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189587 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 02:47:07 +00:00
Arun Murthy
237154982b MAPREDUCE-3205. Fix memory specifications to be physical rather than virtual, allowing for a ratio between the two to be configurable. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189542 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 23:58:31 +00:00
Arun Murthy
8aabd3d4e6 MAPREDUCE-2858. Added a WebApp Proxy for applications. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189036 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 06:29:21 +00:00
Mahadev Konar
2adc51c618 MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189023 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 04:31:28 +00:00
Arun Murthy
29c6c3ed32 MAPREDUCE-2977. Fix ResourceManager to renew HDFS delegation tokens for applications. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189012 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 02:53:22 +00:00
Arun Murthy
03659985a0 MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188999 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 02:16:19 +00:00
Arun Murthy
7dcef364a9 MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188997 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 02:12:50 +00:00
Arun Murthy
2486e4eb09 MAPREDUCE-3253. Fixed ContextFactory to clone JobContext correctly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188842 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 18:14:12 +00:00
Arun Murthy
fffdf661e3 MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce) to JobSummary logs. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188528 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:26:14 +00:00
Mahadev Konar
d19cfe0164 MAPREDUCE-2746. Yarn servers can't communicate with each other with hadoop.security.authorization set to true (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188522 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:07:13 +00:00
Todd Lipcon
bccfd6a09f Amend MAPREDUCE-3159. Fix JavaDoc warnings in previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188479 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:14:17 +00:00
Arun Murthy
c70ee2488c MAPREDUCE-3248. Fixed log4j properties. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188476 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:03:08 +00:00
Arun Murthy
6217e54718 MAPREDUCE-3159. Ensure DefaultContainerExecutor doesn't delete application directories during app-init. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188466 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 00:36:19 +00:00
Todd Lipcon
43553df4e3 MAPREDUCE-3252. Fix map tasks to not rewrite data an extra time when map output fits in spill buffer. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188424 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 22:33:02 +00:00
Arun Murthy
a503755990 MAPREDUCE-3249. Ensure shuffle-port is correctly used duringMR AM recovery. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188388 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 21:19:31 +00:00
Arun Murthy
6288dfa873 MAPREDUCE-3028. Added job-end notification support. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188377 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 21:02:43 +00:00
Vinod Kumar Vavilapalli
7ce1c4ab35 MAPREDUCE-2747. Cleaned up LinuxContainerExecutor binary sources and changed the configuration to use yarn names. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188236 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 17:09:37 +00:00
Vinod Kumar Vavilapalli
4086566144 MAPREDUCE-2708. Designed and implemented MR Application Master recovery to make MR AMs resume their progress after restart. Contributed by Sharad Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188043 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 08:41:48 +00:00
Vinod Kumar Vavilapalli
977d7cc5b9 MAPREDUCE-3233. Fixed a bug in MR Job so as to be able to restart the application on AM crash. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187669 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-22 08:11:47 +00:00
Vinod Kumar Vavilapalli
665a90cbf3 MAPREDUCE-3087. Fixed the mapreduce classpath to correctly include the generated-classpath file needed for tests. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187658 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-22 06:46:12 +00:00
Vinod Kumar Vavilapalli
5795fcfd99 MAPREDUCE-3058. Fixed MR YarnChild to report failure when task throws an error and thus prevent a hanging task and job. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187654 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-22 06:14:46 +00:00
Mahadev Konar
e175574635 MAPREDUCE-3242. Trunk compilation broken with bad interaction from MAPREDUCE-3070 and MAPREDUCE-3239. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187587 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 22:44:39 +00:00
Mahadev Konar
4186121c08 MAPREDUCE-3239. Use new createSocketAddr API in MRv2 to give better error messages on misconfig (Todd Lipcon via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187556 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 21:36:58 +00:00
Arun Murthy
89e7ca6a52 MAPREDUCE-3070. Fix NodeManager to use ephemeral ports by default. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187496 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 18:47:39 +00:00
Mahadev Konar
45b2bce871 MAPREDUCE-3163. JobClient spews errors when killing MR2 job. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187491 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 18:35:35 +00:00
Todd Lipcon
5945a05eee MAPREDUCE-2629. Workaround a JVM class loading quirk which prevents JIT compilation of inner classes methods in ReduceContextImpl. Contributed by Eric Caspole. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187183 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 05:20:27 +00:00
Arun Murthy
02d5fa3e62 MAPREDUCE-3226. Fix shutdown of fetcher threads. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187116 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 22:44:38 +00:00
Alejandro Abdelnur
49baada6cc MAPREDUCE-3204. mvn site:site fails on MapReduce. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187062 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 21:38:58 +00:00
Vinod Kumar Vavilapalli
df2991c0cb MAPREDUCE-3104. Implemented Application-acls. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186748 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 11:45:38 +00:00
Suresh Srinivas
37137845c8 Merging change r1186579 for HDFS-2471 from 0.23 branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186584 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 00:38:42 +00:00
Arun Murthy
aeae1ba497 MAPREDUCE-3188. Ensure correct shutdown in services. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186554 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:59:30 +00:00
Arun Murthy
fc362b1caf MAPREDUCE-3179. Ensure failed tests exit with right error code. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186552 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:54:23 +00:00
Arun Murthy
7a2744c458 MAPREDUCE-3212. Fix usage/help message for bin/yarn. Contributed by Bhallamudi Venkata Siva Kamesh. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186546 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:43:01 +00:00
Arun Murthy
7ca7fdadc4 MAPREDUCE-3208. Fix NPE task/container log appenders. Contributed by liangzhwa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186542 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:38:17 +00:00
Arun Murthy
74748ec625 MAPREDUCE-2693. Fix NPE in job-blacklisting. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186529 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:02:21 +00:00
Jitendra Nath Pandey
ba66ca6856 MAPREDUCE-3149. Add a test to verify that TokenCache handles file system uri with no authority. Contributed by John George. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186516 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 21:52:50 +00:00
Arun Murthy
e549ac9369 MAPREDUCE-2788. Normalize resource requests in FifoScheduler appropriately. Contributed by Ahmed Radwan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186467 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 20:37:01 +00:00
Arun Murthy
9473b2e43f MAPREDUCE-3181. Fixed MapReduce runtime to load yarn-default.xml and yarn-site.xml. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186458 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 20:28:11 +00:00
Vinod Kumar Vavilapalli
94e1703b72 MAPREDUCE-3176. Fixed ant mapreduce tests that are timing out because of wrong framework name. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186368 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 17:55:08 +00:00
Vinod Kumar Vavilapalli
e3bb120e9f MAPREDUCE-3162. Separated application-init and container-init event types in NodeManager's Application state machine. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185988 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 06:40:03 +00:00
Vinod Kumar Vavilapalli
13e4562924 MAPREDUCE-3144. Augmented JobHistory with the information needed for serving aggregated logs. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185976 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 05:21:18 +00:00
Vinod Kumar Vavilapalli
3e3abcb769 MAPREDUCE-3199. Fixed pom files to include correct log4j configuration for tests. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185967 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 04:30:27 +00:00
Mahadev Konar
b4d20c707a MAPREDUCE-3203. Fix some javac warnings in MRAppMaster. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185909 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 22:46:42 +00:00
Arun Murthy
ab787f44aa MAPREDUCE-3165. Ensure logging options are set correctly for MR AM and tasks. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185887 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 21:45:38 +00:00
Arun Murthy
51a667bef8 MAPREDUCE-2762. Cleanup MR staging directory on completion. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185880 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 21:37:31 +00:00
Arun Murthy
bb00ab68c3 MAPREDUCE-279. Adding a changelog to branch-0.23. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185488 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 05:30:32 +00:00
Mahadev Konar
9ff7c1b1de MAPREDUCE-3197. TestMRClientService failing on building clean checkout of branch 0.23 (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185486 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 05:08:32 +00:00
Arun Murthy
c1d90772b6 MAPREDUCE-3068. Added a whitelist of environment variables for containers from the NodeManager and set MALLOC_ARENA_MAX for all daemons and containers. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185447 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 01:22:14 +00:00
Mahadev Konar
955d8eca21 MAPREDUCE-3196. TestLinuxContainerExecutorWithMocks fails on Mac OSX. (Arun Murthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185417 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 22:41:26 +00:00
Alejandro Abdelnur
15a1d9af19 MAPREDUCE-3003. Publish MR JARs to Maven snapshot repository. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185286 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 17:06:42 +00:00
Vinod Kumar Vavilapalli
a26b1672a8 MAPREDUCE-3032. Fixed TaskAttemptImpl so that JobHistory can have error information about failed tasks. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185247 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 15:17:06 +00:00
Arun Murthy
9f36bdd731 MAPREDUCE-3127. Fixed CHANGES.txt to reflect correct jira. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184992 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 04:22:28 +00:00
Arun Murthy
6b2f3dd5ce MAPREDUCE-3101. Changed default value of yarn.resourcemanager.acl.enable to true and added some more documentation. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184989 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 04:08:36 +00:00
Arun Murthy
bc6220c538 MAPREDUCE-3136. Added the missing link to index.html. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184981 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 02:35:20 +00:00
Mahadev Konar
29f65be3e2 MAPREDUCE-3189. Add link decoration back to MR2's CSS. (Todd Lipcon via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184979 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 02:22:16 +00:00
Arun Murthy
86e1c9536f MAPREDUCE-3136. Added documentation for setting up Hadoop clusters in both non-secure and secure mode for both HDFS & YARN. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184977 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 02:18:06 +00:00
Arun Murthy
4872615441 MAPREDUCE-3190. Ensure bin/yarn fails early with a clear error message when HADOOP_COMMON_HOME or HADOOP_HDFS_HOME are not set. Contributed by todd & acmurthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184975 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 02:14:10 +00:00
Mahadev Konar
94b3bf9f75 MAPREDUCE-2840. mr279 TestUberAM.testSleepJob test fails. (jonathan eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184971 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 01:19:51 +00:00
Mahadev Konar
68328ae926 MAPREDUCE-3187. Add names for various unnamed threads in MR2. (Todd Lipcon and Siddharth Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184904 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-16 19:27:02 +00:00
Mahadev Konar
b5e21e7e63 MAPREDUCE-3057. Job History Server goes of OutOfMemory with 1200 Jobs and Heap Size set to 10 GB. (Eric Payne via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183545 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 22:49:19 +00:00
Mahadev Konar
b3284c4607 MAPREDUCE-3059. QueueMetrics do not have metrics for aggregate containers-allocated and aggregate containers-released. (Devaraj K via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183540 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 22:38:52 +00:00
Jitendra Nath Pandey
57f99c2b02 MAPREDUCE-3192. Fix Javadoc warning in JobClient.java and Cluster.java. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183513 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 21:49:35 +00:00
Arun Murthy
09eb725c06 MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for containers. Contributed by John George. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183188 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 01:27:10 +00:00
Jitendra Nath Pandey
002dd6968b MAPREDUCE-2764. Fix renewal of dfs delegation tokens. Contributed by Owen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183187 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 01:24:20 +00:00
Arun Murthy
c7fb49b3c5 MAPREDUCE-3170. Fixed job output commit for deep hierarchies. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183185 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 01:16:30 +00:00
Alejandro Abdelnur
59df2d48c3 MAPREDUCE-3183. hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml missing license header. (Hitesh Shah via tucu). 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183160 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-13 23:16:07 +00:00
Alejandro Abdelnur
e9544ef3d2 HADOOP-7737. normalize hadoop-mapreduce & hadoop-dist dist/tar build with common/hdfs (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183068 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-13 20:23:59 +00:00
Mahadev Konar
6b6f02c51d MAPREDUCE-3133. Running a set of methods in a Single Test Class. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182688 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-13 07:07:01 +00:00
Arun Murthy
26d63ecdae MAPREDUCE-2789. Complete schedulingInfo on CLI. Contributed by Eric Payne. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182616 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 23:29:01 +00:00
Arun Murthy
277e520579 MAPREDUCE-2666. Retrieve shuffle port number from JobHistory on MR AM restart. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182613 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 23:12:01 +00:00
Alejandro Abdelnur
327e02fdf4 MAPREDUCE-3171. normalize nodemanager native code compilation with common/hdfs native. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182575 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 20:38:51 +00:00
Ravi Gummadi
f143895bc2 MAPREDUCE-3166. [Rumen] Make Rumen use job history api instead of relying on current history file name format. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182330 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 12:19:02 +00:00
Ravi Gummadi
efb3cd64a2 MAPREDUCE-3157. [Rumen] Fix TraceBuilder to handle 0.20 history file names also. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182293 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 09:58:14 +00:00
Vinod Kumar Vavilapalli
d6546fc0a4 MAPREDUCE-3125. Modified TaskImpl to consider only non-failed, non-killed task-attempts for obtaining task's progress. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182230 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 06:51:32 +00:00
Vinod Kumar Vavilapalli
85588e5ce9 MAPREDUCE-3140. Fixed the invalid JobHistory URL for failed applications. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182216 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 06:23:20 +00:00
Alejandro Abdelnur
3f998db918 HADOOP-7642. create hadoop-dist module where TAR stitching would happen. Contributed by Thomas White. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182151 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 00:27:32 +00:00
Arun Murthy
ac64df85a9 MAPREDUCE-3148. Ported MAPREDUCE-2702 to old mapred api for aiding task recovery. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182008 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 18:27:51 +00:00
Arun Murthy
b8f0836f94 MAPREDUCE-3126. Fixed a corner case in CapacityScheduler where headroom wasn't updated on changes to cluster size. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182000 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 18:24:32 +00:00
Vinod Kumar Vavilapalli
1c358b0796 MAPREDUCE-2668. Fixed AuxServices to send a signal on application-finish to all the services. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181803 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 14:26:17 +00:00
Vinod Kumar Vavilapalli
7d5434c6b3 MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181793 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 14:09:32 +00:00
Mahadev Konar
5a249b76d3 MAPREDUCE-3167. container-executor is not being packaged with the assembly target. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181634 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 06:58:20 +00:00
Vinod Kumar Vavilapalli
11b9dd4e84 MAPREDUCE-3161. Improved some javadocs and fixed some typos in YARN. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181622 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 04:45:28 +00:00
Arun Murthy
42e93829e5 MAPREDUCE-3158. Fix test failures in MRv1 due to default framework being set to yarn. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181310 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 00:26:41 +00:00
Vinod Kumar Vavilapalli
a8190ce5c5 MAPREDUCE-2988. Reenabled TestLinuxContainerExecutor reflecting the current NodeManager code. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180833 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 09:18:44 +00:00
Arun Murthy
d0016c6120 MAPREDUCE-3154. Fix JobSubmitter to check for output specs before copying job submission files to fail fast. Contributed by Abhijit Suresh Shingate. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180774 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:29:38 +00:00
Arun Murthy
811a6a9e30 MAPREDUCE-3123. Fix NM to quote symlink names to escape special characters. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180772 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:27:01 +00:00
Arun Murthy
4f1cdab6d8 MAPREDUCE-3153. Fix TestFileOutputCommitter which was broken by MAPREDUCE-2702. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180769 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:10:45 +00:00
Arun Murthy
3ddb52cfc0 MAPREDUCE-2876. Use a different config for ContainerAllocationExpirer. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180767 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:06:47 +00:00
Konstantin Shvachko
a09a2b48cd Move MAPREDUCE-3138 changes to Release 0.22 section. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180751 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 00:36:04 +00:00
Alejandro Abdelnur
4972959173 Reverted MAPREDUCE-3003 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180283 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-08 00:38:41 +00:00
Arun Murthy
047b175aac MAPREDUCE-2802. Ensure JobHistory filenames have jobId. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180220 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 20:44:37 +00:00
Arun Murthy
6b343adfe8 MAPREDUCE-3033. Ensure Master interface pays attention to classic v/s yarn frameworks. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180214 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 20:39:33 +00:00
Alejandro Abdelnur
68d1162f7a MAPREDUCE-3003. Publish Yarn and MapReduce artifacts to Maven snapshot repository. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180155 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 19:01:01 +00:00
Thomas White
02ce6b3d07 MAPREDUCE-3014. Rename and invert logic of '-cbuild' profile to 'native' and off by default. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180116 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 17:33:10 +00:00
Vinod Kumar Vavilapalli
02a81203bd MAPREDUCE-2751. Modified NodeManager to stop leaving around local files after application finishes. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180071 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 15:23:20 +00:00
Vinod Kumar Vavilapalli
fa2529c931 MAPREDUCE-3141. Fix the broken MRAppMaster to work over YARN in security mode.(vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180007 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 11:35:17 +00:00
Vinod Kumar Vavilapalli
a56f3931d1 MAPREDUCE-2783. Fixing RM web-UI to show no tracking-URL when AM crashes. Contributed by Eric Payne. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179975 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 09:17:57 +00:00
Mahadev Konar
b8102dbdf8 MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179936 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 05:27:36 +00:00
Amar Kamat
c1c0e8c9ea MAPREDUCE-3008. Improvements to cumulative CPU emulation for short running tasks in Gridmix. (amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179933 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 04:59:58 +00:00
Vinod Kumar Vavilapalli
282e2e910c MAPREDUCE-2913. Fixed TestMRJobs.testFailingMapper to assert the correct TaskCompletionEventStatus. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179319 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 17:07:20 +00:00
Vinod Kumar Vavilapalli
9a4e890f4a MAPREDUCE-2738. Added the missing cluster level statisticss on the RM web UI. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179229 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 14:01:32 +00:00
Vinod Kumar Vavilapalli
f29bfa0e0e MAPREDUCE-2702. Added a new API in OutputCommitter for recovering the outputs of tasks from a crashed job so as to support MR Application Master recovery. Contributed by Sharad Agarwal and Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179188 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 12:15:44 +00:00
Vinod Kumar Vavilapalli
f24dcb3449 MAPREDUCE-2907. Changed log level for various messages in ResourceManager from INFO to DEBUG. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179178 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 11:56:54 +00:00
Vinod Kumar Vavilapalli
66137cf17c MAPREDUCE-3013. Removed YarnConfiguration.YARN_SECURITY_INFO and its usage as it doesn't affect security any more. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 11:43:05 +00:00
Amar Kamat
45e6fb00af MAPREDUCE-2950. [Gridmix] TestUserResolve fails in trunk. (Ravi Gummadi via amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179090 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 08:03:49 +00:00
Konstantin Shvachko
ff9d7de705 Move MAPREDUCE-2531 changes to Release 0.22 section. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179078 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 07:39:35 +00:00
Vinod Kumar Vavilapalli
e979a3ddb1 MAPREDUCE-3056. Changed the default staging directory to not include user.name to prevent issues with non-secure mode. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178740 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 09:37:56 +00:00
Eric Yang
a48e6f2519 MAPREDUCE-3112. Fixed recursive sourcing of HADOOP_OPTS environment 
variable. (Eric Yang)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178657 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:51:13 +00:00
Eric Yang
3edf3579c8 HADOOP-7684. Added init.d script for jobhistory server and 
secondary namenode. (Eric Yang)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178653 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:44:08 +00:00
Arun Murthy
4f97e37648 MAPREDUCE-3138. Add a utility to help applications bridge changes in Context Objects APIs due to MAPREDUCE-954. Contributed by omalley. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178650 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:42:37 +00:00
Eric Yang
abc2b45dfa HADOOP-7603. Set hdfs, mapred uid, and hadoop uid to fixed numbers. (Eric Yang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178644 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:13:58 +00:00
Arun Murthy
12743d2169 MAPREDUCE-2792. Replace usage of node ip-addresses with hostnames. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178631 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-03 23:21:20 +00:00
Arun Murthy
43b59b894b MAPREDUCE-3134. Added documentation the CapacityScheduler. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178530 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-03 19:35:41 +00:00
Arun Murthy
7aa8a3c9f8 MAPREDUCE-2889. Added documentation for writing new YARN applications. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177920 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-01 05:22:50 +00:00
Arun Murthy
f3b98a26c4 MAPREDUCE-3113. Ensure bin/yarn and bin/yarn-daemon.sh identify the root of the install properly. Contributed by Xie Xianshan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177874 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 23:58:58 +00:00
Arun Murthy
fad230a49d MAPREDUCE-2719. Add a simple, DistributedShell, application to illustrate alternate frameworks on YARN. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177864 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 22:25:32 +00:00
Arun Murthy
dcf9d475e0 MAPREDUCE-3050. Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177859 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 22:19:17 +00:00
Konstantin Shvachko
960d7643fb MAPREDUCE-2779. JobSplitWriter.java can't handle large job.split file. Contributed by Ming Ma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177779 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 19:04:56 +00:00
Vinod Kumar Vavilapalli
063e33a862 MAPREDUCE-3098. Fixed RM and MR AM to report YarnApplicationState and application's FinalStatus separately. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177633 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 12:46:32 +00:00
Arun Murthy
964d4a1666 MAPREDUCE-2996. Add uber-ness information to JobHistory. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177531 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 06:47:53 +00:00
Arun Murthy
0b3c654d83 MAPREDUCE-2791. Added missing info on 'job -status' output. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177487 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 03:08:35 +00:00
Vinod Kumar Vavilapalli
71886657f8 MAPREDUCE-3001. Added task-specific counters to AppMaster and JobHistory web-UIs. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177203 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-29 08:08:49 +00:00
Vinod Kumar Vavilapalli
44090734c9 MAPREDUCE-3114. Fixed invalid ApplicationURL on RM WebUI. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177187 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-29 07:28:29 +00:00
Arun Murthy
312a7e7100 MAPREDUCE-3078. Ensure MapReduce AM reports progress correctly for displaying on the RM Web-UI. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176762 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 07:31:03 +00:00
Vinod Kumar Vavilapalli
f4a425e10d MAPREDUCE-3110. Fixed TestRPC failure. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176739 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 06:03:18 +00:00
Vinod Kumar Vavilapalli
cc42ccf02a MAPREDUCE-2843. Fixed the node-table to be completely displayed and making node entries on RM UI to be sortable. Contributed by Abhijit Suresh Shingate. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176730 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 05:35:43 +00:00
Arun Murthy
eda0ad4fd0 MAPREDUCE-3041. Fixed ClientRMProtocol to provide min/max resource capabilities along-with new ApplicationId for application submission. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176715 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 04:20:16 +00:00
Arun Murthy
d06916f033 MAPREDUCE-3021. Adding additional files needed for new web-ui. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176708 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 04:02:34 +00:00
Arun Murthy
5f6adcf396 MAPREDUCE-3021. Change base urls for RM web-ui. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176681 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 01:23:04 +00:00
Mahadev Konar
8e4c70fb20 MAPREDUCE-3099. Add docs for setting up a single node MRv2 cluster. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176607 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 20:41:56 +00:00
Mahadev Konar
0d2bb06236 MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176600 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 20:30:24 +00:00
Mahadev Konar
359c17071b MAPREDUCE-3095. fairscheduler ivy including wrong version for hdfs. (John George via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176566 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 19:44:56 +00:00
Giridharan Kesavan
98cc200700 MAPREDUCE-3081. Fix vaidya startup script. Contributed by Suhas 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176550 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 19:35:29 +00:00
Vinod Kumar Vavilapalli
87b969c835 MAPREDUCE-2999. Fix YARN webapp framework to properly filter servlet paths. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176469 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 17:03:19 +00:00
Vinod Kumar Vavilapalli
6507a0bc35 MAPREDUCE-3092. Removed a special comparator for JobIDs in JobHistory as JobIDs are already comparable. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176453 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 16:11:54 +00:00
Arun Murthy
17da0bdb27 MAPREDUCE-3067. Ensure exit-code is set correctly for containers. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176235 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 06:56:50 +00:00
Mahadev Konar
70388cffe0 MAPREDUCE-3071. app master configuration web UI link under the Job menu opens up application menu. (thomas graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176203 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 05:30:01 +00:00
Arun Murthy
81926396f6 MAPREDUCE-2984. Better error message for displaying completed containers. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176045 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 20:45:18 +00:00
Vinod Kumar Vavilapalli
eff931a1b1 MAPREDUCE-3031. Proper handling of killed containers to prevent stuck containers/AMs on an external kill signal. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175960 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 17:27:15 +00:00
Vinod Kumar Vavilapalli
1e6dfa7472 MAPREDUCE-2646. Fixed AMRMProtocol to return containers based on priority. Contributed by Sharad Agarwal and Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175859 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 13:25:27 +00:00
Arun Murthy
c9a7d3dbf9 MAPREDUCE-3090. Fix MR AM to use ApplicationAttemptId rather than (ApplicationId, startCount) consistently. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175718 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 08:44:41 +00:00
Mahadev Konar
5a3040cad4 MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175449 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 18:02:51 +00:00
Vinod Kumar Vavilapalli
a5c9ede143 MAPREDUCE-2952. Fixed ResourceManager/MR-client to consume diagnostics for AM failures in a couple of corner cases. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175403 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 14:46:59 +00:00
Arun Murthy
c285cf3114 MAPREDUCE-3053. Better diagnostic message for unknown methods in ProtoBuf RPCs. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175357 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 09:50:02 +00:00
Arun Murthy
5ace0cabe5 MAPREDUCE-2990. Fixed display of NodeHealthStatus. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175351 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 09:36:12 +00:00
Arun Murthy
d09ceac1f7 MAPREDUCE-2691. Increase threadpool size for launching containers in MapReduce ApplicationMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175294 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 04:43:51 +00:00
Vinod Kumar Vavilapalli
b549c10782 MAPREDUCE-3055. Simplified ApplicationAttemptId passing to ApplicationMaster via environment variable. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1174785 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-23 14:07:42 +00:00
Arun Murthy
1eb7008753 MAPREDUCE-3073. Fixed build issues in MR1. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1174294 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-22 18:05:30 +00:00
Vinod Kumar Vavilapalli
4806d7ba74 MAPREDUCE-2754. Fixed MR AM stdout, stderr and syslog to redirect to correct log-files. Contributed by Ravi Teja Ch N V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1174194 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-22 15:14:31 +00:00
Mahadev Konar
3557202a92 MAPREDUCE-3044. Pipes jobs stuck without making progress. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173909 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 23:52:50 +00:00
Arun Murthy
ed7eda00cb MAPREDUCE-3066. Fixed default ResourceTracker address for the NodeManager. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173904 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 23:32:31 +00:00
Arun Murthy
32f28e6994 MAPREDUCE-3062. Fixed default RMAdmin address. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173838 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 20:37:08 +00:00
Vinod Kumar Vavilapalli
d00b3c49f6 MAPREDUCE-2880. Improved classpath-construction for mapreduce AM and containers. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173783 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 18:28:23 +00:00
Vinod Kumar Vavilapalli
af61f4ae15 MAPREDUCE-2970. Fixed NPEs in corner cases with different configurations for mapreduce.framework.name. Contributed by Venu Gopala Rao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173534 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 09:13:34 +00:00
Vinod Kumar Vavilapalli
01fbb0fb45 MAPREDUCE-2930. Added the ability to be able to generate graphs from the state-machine definitions. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173524 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 08:37:00 +00:00
Arun Murthy
0e870d7d18 MAPREDUCE-3023. Fixed clients to display queue state correctly. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173458 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 01:32:14 +00:00
Arun Murthy
1d067c6e2b MAPREDUCE-2998. Fixed a bug in TaskAttemptImpl which caused it to fork bin/mapred too many times. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173456 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 01:26:48 +00:00
Arun Murthy
339b85b88e MAPREDUCE-3036. Fixed metrics for reserved resources in CS. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173453 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 01:14:20 +00:00
Arun Murthy
b8e8b8da75 MAPREDUCE-3018. Fixed -file option for streaming. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173451 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 01:10:49 +00:00
Arun Murthy
f0fedda8ef MAPREDUCE-3040. Fixed extra copy of Configuration in YarnClientProtocolProvider and ensured MiniMRYarnCluster sets JobHistory configuration for tests. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172929 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-20 00:46:22 +00:00