big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,816 Commits 2 Branches 2 Tags 264 MiB
a8ad1e8089
Commit Graph

1569 Commits

Author SHA1 Message Date
Arun Murthy
b451dd7aaa MAPREDUCE-3220. Fixed TestCombineOutputCollector. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195745 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 01:04:15 +00:00
Arun Murthy
5f7e991659 MAPREDUCE-3321. Disabled a few MR tests for 0.23. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195743 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 00:57:22 +00:00
Arun Murthy
b3f90c7060 MAPREDUCE-3275. Added documentation for AM WebApp Proxy. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195579 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 17:33:29 +00:00
Arun Murthy
9db078212f MAPREDUCE-3035. Fixed MR JobHistory to ensure rack information is present. Contributed by chakravarthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195575 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 17:27:42 +00:00
Amar Kamat
7e056015ad MAPREDUCE-3241. [Rumen] Fix Rumen to ignore the AMStartedEvent. (amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195535 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 15:51:32 +00:00
Vinod Kumar Vavilapalli
21b1e1da49 MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195416 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 11:29:23 +00:00
Vinod Kumar Vavilapalli
a75c4cf4e4 MAPREDUCE-2696. Fixed NodeManager to cleanup logs in a thread when logs' aggregation is not enabled. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195383 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 09:55:29 +00:00
Vinod Kumar Vavilapalli
47a381e306 MAPREDUCE-3146. Added a MR specific command line to dump logs for a given TaskAttemptID. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195349 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 06:42:06 +00:00
Vinod Kumar Vavilapalli
5f9e67e226 MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195340 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 05:51:33 +00:00
Vinod Kumar Vavilapalli
273e092c93 MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing TestResourceTrackerService sometimes. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195319 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 03:30:56 +00:00
Vinod Kumar Vavilapalli
1c8d64f38a MAPREDUCE-3274. Fixed a race condition in MRAppMaster that was causing a task-scheduling deadlock. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195145 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-30 11:29:41 +00:00
Arun Murthy
7f4dc27757 MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1194850 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 09:35:36 +00:00
Alejandro Abdelnur
1bcff6860c Adding missing CHANGES.txt entry for MAPREDUCE-3014 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190585 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 21:27:08 +00:00
Vinod Kumar Vavilapalli
c1e333f32f MAPREDUCE-3306. Fixed a bug in NodeManager ApplicationImpl that was causing NodeManager to crash. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190523 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 19:45:57 +00:00
Arun Murthy
b93abdcd66 MAPREDUCE-3304. Fixed intermittent test failure due to a race in TestRMContainerAllocator#testBlackListedNodes. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190495 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 18:45:48 +00:00
Vinod Kumar Vavilapalli
cbdb07f4ca MAPREDUCE-2775. Fixed ResourceManager and NodeManager to force a decommissioned node to shutdown. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190467 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 17:31:06 +00:00
Vinod Kumar Vavilapalli
a39992ea63 MAPREDUCE-3296. Fixed the remaining nine FindBugs warnings. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190187 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 07:44:35 +00:00
Vinod Kumar Vavilapalli
670fa24b48 MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the JobHistoryServer. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:45:04 +00:00
Mahadev Konar
47c28ce14d MAPREDUCE-3292. In secure mode job submission fails with Provider org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier not found. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190171 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:13:18 +00:00
Mahadev Konar
062876bed7 MAPREDUCE-3185. RM Web UI does not sort the columns in some cases. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190165 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:04:51 +00:00
Arun Murthy
69dd284452 MAPREDUCE-3290. Fixed a NPE in ClientRMService. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190162 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 05:45:59 +00:00
Arun Murthy
34e6de8f85 MAPREDUCE-3258. Fixed AM & JobHistory web-ui to display counters properly. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190160 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 05:08:04 +00:00
Mahadev Konar
9b34900c89 MAPREDUCE-3209. Jenkins reports 160 FindBugs warnings (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190140 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 02:42:11 +00:00
Mahadev Konar
b304062f1f MAPREDUCE-3186. User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed. (Eric Payne via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190122 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 01:40:24 +00:00
Mahadev Konar
35b62e6e72 MAPREDUCE-3282. bin/mapred job -list throws exception. (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190110 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 01:16:29 +00:00
Arun Murthy
739acd6e14 MAPREDUCE-3284. Moved JobQueueClient to hadoop-mapreduce-client-core. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190084 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 23:21:09 +00:00
Eli Collins
1ad8415b72 MAPREDUCE-2736. Remove unused contrib components dependent on MR1. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189982 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 20:06:26 +00:00
Arun Murthy
724f217343 MAPREDUCE-3228. Fixed MR AM to timeout RPCs to bad NodeManagers. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189879 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 17:32:02 +00:00
Vinod Kumar Vavilapalli
1b8511ef89 MAPREDUCE-3281. Fixed a bug in TestLinuxContainerExecutorWithMocks. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189818 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 15:55:41 +00:00
Vinod Kumar Vavilapalli
f114dad5ef MAPREDUCE-2986. Fixed MiniYARNCluster to support multiple NodeManagers. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189721 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 12:42:12 +00:00
Vinod Kumar Vavilapalli
2fe343f963 MAPREDUCE-3240. Fixed NodeManager to be able to forcefully cleanup its containers (process-trees) irrespective of whether the container succeeded, or killed. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189711 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 12:03:17 +00:00
Arun Murthy
f799e61269 MAPREDUCE-3279. Fixed TestJobHistoryParsing which assumed user name to be mapred all the time. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189670 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 08:31:44 +00:00
Arun Murthy
fc74b5a25c MAPREDUCE-3259. Added java.library.path of NodeManager to ContainerLocalizer in LinuxContainerExecutor. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189667 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 08:28:34 +00:00
Arun Murthy
db8ac0ec3c MAPREDUCE-3257. Added authorization checks for the protocol between ResourceManager and ApplicatoinMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189630 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 06:24:22 +00:00
Arun Murthy
5fadd4de2a MAPREDUCE-3175. Add authorization to admin web-pages such as /stacks, /jmx etc. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189619 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 06:11:08 +00:00
Mahadev Konar
33dc6578de MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically. (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189616 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 06:04:17 +00:00
Arun Murthy
3e4efbb609 MAPREDUCE-3254. Fixed streaming to set the job.jar by using the right JobConf ctor. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189587 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 02:47:07 +00:00
Arun Murthy
237154982b MAPREDUCE-3205. Fix memory specifications to be physical rather than virtual, allowing for a ratio between the two to be configurable. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189542 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 23:58:31 +00:00
Arun Murthy
8aabd3d4e6 MAPREDUCE-2858. Added a WebApp Proxy for applications. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189036 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 06:29:21 +00:00
Mahadev Konar
2adc51c618 MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189023 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 04:31:28 +00:00
Arun Murthy
29c6c3ed32 MAPREDUCE-2977. Fix ResourceManager to renew HDFS delegation tokens for applications. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189012 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 02:53:22 +00:00
Arun Murthy
03659985a0 MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188999 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 02:16:19 +00:00
Arun Murthy
7dcef364a9 MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188997 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 02:12:50 +00:00
Arun Murthy
2486e4eb09 MAPREDUCE-3253. Fixed ContextFactory to clone JobContext correctly. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188842 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 18:14:12 +00:00
Arun Murthy
fffdf661e3 MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce) to JobSummary logs. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188528 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:26:14 +00:00
Mahadev Konar
d19cfe0164 MAPREDUCE-2746. Yarn servers can't communicate with each other with hadoop.security.authorization set to true (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188522 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:07:13 +00:00
Arun Murthy
c70ee2488c MAPREDUCE-3248. Fixed log4j properties. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188476 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:03:08 +00:00
Arun Murthy
6217e54718 MAPREDUCE-3159. Ensure DefaultContainerExecutor doesn't delete application directories during app-init. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188466 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 00:36:19 +00:00
Todd Lipcon
43553df4e3 MAPREDUCE-3252. Fix map tasks to not rewrite data an extra time when map output fits in spill buffer. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188424 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 22:33:02 +00:00
Arun Murthy
a503755990 MAPREDUCE-3249. Ensure shuffle-port is correctly used duringMR AM recovery. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188388 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 21:19:31 +00:00
Arun Murthy
6288dfa873 MAPREDUCE-3028. Added job-end notification support. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188377 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 21:02:43 +00:00
Vinod Kumar Vavilapalli
7ce1c4ab35 MAPREDUCE-2747. Cleaned up LinuxContainerExecutor binary sources and changed the configuration to use yarn names. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188236 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 17:09:37 +00:00
Vinod Kumar Vavilapalli
4086566144 MAPREDUCE-2708. Designed and implemented MR Application Master recovery to make MR AMs resume their progress after restart. Contributed by Sharad Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188043 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 08:41:48 +00:00
Vinod Kumar Vavilapalli
977d7cc5b9 MAPREDUCE-3233. Fixed a bug in MR Job so as to be able to restart the application on AM crash. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187669 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-22 08:11:47 +00:00
Vinod Kumar Vavilapalli
665a90cbf3 MAPREDUCE-3087. Fixed the mapreduce classpath to correctly include the generated-classpath file needed for tests. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187658 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-22 06:46:12 +00:00
Vinod Kumar Vavilapalli
5795fcfd99 MAPREDUCE-3058. Fixed MR YarnChild to report failure when task throws an error and thus prevent a hanging task and job. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187654 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-22 06:14:46 +00:00
Mahadev Konar
e175574635 MAPREDUCE-3242. Trunk compilation broken with bad interaction from MAPREDUCE-3070 and MAPREDUCE-3239. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187587 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 22:44:39 +00:00
Mahadev Konar
4186121c08 MAPREDUCE-3239. Use new createSocketAddr API in MRv2 to give better error messages on misconfig (Todd Lipcon via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187556 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 21:36:58 +00:00
Arun Murthy
89e7ca6a52 MAPREDUCE-3070. Fix NodeManager to use ephemeral ports by default. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187496 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 18:47:39 +00:00
Mahadev Konar
45b2bce871 MAPREDUCE-3163. JobClient spews errors when killing MR2 job. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187491 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 18:35:35 +00:00
Todd Lipcon
5945a05eee MAPREDUCE-2629. Workaround a JVM class loading quirk which prevents JIT compilation of inner classes methods in ReduceContextImpl. Contributed by Eric Caspole. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187183 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 05:20:27 +00:00
Arun Murthy
02d5fa3e62 MAPREDUCE-3226. Fix shutdown of fetcher threads. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187116 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 22:44:38 +00:00
Alejandro Abdelnur
49baada6cc MAPREDUCE-3204. mvn site:site fails on MapReduce. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187062 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 21:38:58 +00:00
Vinod Kumar Vavilapalli
df2991c0cb MAPREDUCE-3104. Implemented Application-acls. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186748 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 11:45:38 +00:00
Arun Murthy
aeae1ba497 MAPREDUCE-3188. Ensure correct shutdown in services. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186554 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:59:30 +00:00
Arun Murthy
fc362b1caf MAPREDUCE-3179. Ensure failed tests exit with right error code. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186552 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:54:23 +00:00
Arun Murthy
7a2744c458 MAPREDUCE-3212. Fix usage/help message for bin/yarn. Contributed by Bhallamudi Venkata Siva Kamesh. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186546 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:43:01 +00:00
Arun Murthy
7ca7fdadc4 MAPREDUCE-3208. Fix NPE task/container log appenders. Contributed by liangzhwa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186542 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:38:17 +00:00
Arun Murthy
74748ec625 MAPREDUCE-2693. Fix NPE in job-blacklisting. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186529 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 22:02:21 +00:00
Jitendra Nath Pandey
ba66ca6856 MAPREDUCE-3149. Add a test to verify that TokenCache handles file system uri with no authority. Contributed by John George. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186516 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 21:52:50 +00:00
Arun Murthy
e549ac9369 MAPREDUCE-2788. Normalize resource requests in FifoScheduler appropriately. Contributed by Ahmed Radwan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186467 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 20:37:01 +00:00
Arun Murthy
9473b2e43f MAPREDUCE-3181. Fixed MapReduce runtime to load yarn-default.xml and yarn-site.xml. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186458 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 20:28:11 +00:00
Vinod Kumar Vavilapalli
94e1703b72 MAPREDUCE-3176. Fixed ant mapreduce tests that are timing out because of wrong framework name. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186368 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 17:55:08 +00:00
Vinod Kumar Vavilapalli
e3bb120e9f MAPREDUCE-3162. Separated application-init and container-init event types in NodeManager's Application state machine. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185988 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 06:40:03 +00:00
Vinod Kumar Vavilapalli
13e4562924 MAPREDUCE-3144. Augmented JobHistory with the information needed for serving aggregated logs. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185976 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 05:21:18 +00:00
Vinod Kumar Vavilapalli
3e3abcb769 MAPREDUCE-3199. Fixed pom files to include correct log4j configuration for tests. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185967 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 04:30:27 +00:00
Mahadev Konar
b4d20c707a MAPREDUCE-3203. Fix some javac warnings in MRAppMaster. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185909 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 22:46:42 +00:00
Arun Murthy
ab787f44aa MAPREDUCE-3165. Ensure logging options are set correctly for MR AM and tasks. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185887 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 21:45:38 +00:00
Arun Murthy
51a667bef8 MAPREDUCE-2762. Cleanup MR staging directory on completion. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185880 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 21:37:31 +00:00
Arun Murthy
bb00ab68c3 MAPREDUCE-279. Adding a changelog to branch-0.23. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185488 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 05:30:32 +00:00
Mahadev Konar
9ff7c1b1de MAPREDUCE-3197. TestMRClientService failing on building clean checkout of branch 0.23 (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185486 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 05:08:32 +00:00
Arun Murthy
c1d90772b6 MAPREDUCE-3068. Added a whitelist of environment variables for containers from the NodeManager and set MALLOC_ARENA_MAX for all daemons and containers. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185447 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 01:22:14 +00:00
Mahadev Konar
955d8eca21 MAPREDUCE-3196. TestLinuxContainerExecutorWithMocks fails on Mac OSX. (Arun Murthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185417 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 22:41:26 +00:00
Alejandro Abdelnur
15a1d9af19 MAPREDUCE-3003. Publish MR JARs to Maven snapshot repository. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185286 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 17:06:42 +00:00
Vinod Kumar Vavilapalli
a26b1672a8 MAPREDUCE-3032. Fixed TaskAttemptImpl so that JobHistory can have error information about failed tasks. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185247 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 15:17:06 +00:00
Arun Murthy
9f36bdd731 MAPREDUCE-3127. Fixed CHANGES.txt to reflect correct jira. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184992 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 04:22:28 +00:00
Arun Murthy
6b2f3dd5ce MAPREDUCE-3101. Changed default value of yarn.resourcemanager.acl.enable to true and added some more documentation. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184989 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 04:08:36 +00:00
Mahadev Konar
29f65be3e2 MAPREDUCE-3189. Add link decoration back to MR2's CSS. (Todd Lipcon via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184979 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 02:22:16 +00:00
Arun Murthy
86e1c9536f MAPREDUCE-3136. Added documentation for setting up Hadoop clusters in both non-secure and secure mode for both HDFS & YARN. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184977 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 02:18:06 +00:00
Arun Murthy
4872615441 MAPREDUCE-3190. Ensure bin/yarn fails early with a clear error message when HADOOP_COMMON_HOME or HADOOP_HDFS_HOME are not set. Contributed by todd & acmurthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184975 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 02:14:10 +00:00
Mahadev Konar
94b3bf9f75 MAPREDUCE-2840. mr279 TestUberAM.testSleepJob test fails. (jonathan eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184971 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 01:19:51 +00:00
Mahadev Konar
68328ae926 MAPREDUCE-3187. Add names for various unnamed threads in MR2. (Todd Lipcon and Siddharth Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184904 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-16 19:27:02 +00:00
Mahadev Konar
b5e21e7e63 MAPREDUCE-3057. Job History Server goes of OutOfMemory with 1200 Jobs and Heap Size set to 10 GB. (Eric Payne via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183545 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 22:49:19 +00:00
Mahadev Konar
b3284c4607 MAPREDUCE-3059. QueueMetrics do not have metrics for aggregate containers-allocated and aggregate containers-released. (Devaraj K via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183540 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 22:38:52 +00:00
Jitendra Nath Pandey
57f99c2b02 MAPREDUCE-3192. Fix Javadoc warning in JobClient.java and Cluster.java. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183513 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 21:49:35 +00:00
Arun Murthy
09eb725c06 MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for containers. Contributed by John George. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183188 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 01:27:10 +00:00
Jitendra Nath Pandey
002dd6968b MAPREDUCE-2764. Fix renewal of dfs delegation tokens. Contributed by Owen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183187 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 01:24:20 +00:00
Arun Murthy
c7fb49b3c5 MAPREDUCE-3170. Fixed job output commit for deep hierarchies. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183185 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 01:16:30 +00:00
Alejandro Abdelnur
59df2d48c3 MAPREDUCE-3183. hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml missing license header. (Hitesh Shah via tucu). 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183160 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-13 23:16:07 +00:00
Mahadev Konar
6b6f02c51d MAPREDUCE-3133. Running a set of methods in a Single Test Class. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182688 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-13 07:07:01 +00:00
Arun Murthy
26d63ecdae MAPREDUCE-2789. Complete schedulingInfo on CLI. Contributed by Eric Payne. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182616 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 23:29:01 +00:00
Arun Murthy
277e520579 MAPREDUCE-2666. Retrieve shuffle port number from JobHistory on MR AM restart. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182613 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 23:12:01 +00:00
Alejandro Abdelnur
327e02fdf4 MAPREDUCE-3171. normalize nodemanager native code compilation with common/hdfs native. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182575 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 20:38:51 +00:00
Ravi Gummadi
f143895bc2 MAPREDUCE-3166. [Rumen] Make Rumen use job history api instead of relying on current history file name format. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182330 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 12:19:02 +00:00
Ravi Gummadi
efb3cd64a2 MAPREDUCE-3157. [Rumen] Fix TraceBuilder to handle 0.20 history file names also. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182293 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 09:58:14 +00:00
Vinod Kumar Vavilapalli
d6546fc0a4 MAPREDUCE-3125. Modified TaskImpl to consider only non-failed, non-killed task-attempts for obtaining task's progress. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182230 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 06:51:32 +00:00
Vinod Kumar Vavilapalli
85588e5ce9 MAPREDUCE-3140. Fixed the invalid JobHistory URL for failed applications. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182216 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 06:23:20 +00:00
Arun Murthy
ac64df85a9 MAPREDUCE-3148. Ported MAPREDUCE-2702 to old mapred api for aiding task recovery. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182008 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 18:27:51 +00:00
Arun Murthy
b8f0836f94 MAPREDUCE-3126. Fixed a corner case in CapacityScheduler where headroom wasn't updated on changes to cluster size. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182000 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 18:24:32 +00:00
Vinod Kumar Vavilapalli
1c358b0796 MAPREDUCE-2668. Fixed AuxServices to send a signal on application-finish to all the services. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181803 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 14:26:17 +00:00
Vinod Kumar Vavilapalli
7d5434c6b3 MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181793 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 14:09:32 +00:00
Mahadev Konar
5a249b76d3 MAPREDUCE-3167. container-executor is not being packaged with the assembly target. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181634 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 06:58:20 +00:00
Vinod Kumar Vavilapalli
11b9dd4e84 MAPREDUCE-3161. Improved some javadocs and fixed some typos in YARN. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181622 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 04:45:28 +00:00
Arun Murthy
42e93829e5 MAPREDUCE-3158. Fix test failures in MRv1 due to default framework being set to yarn. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181310 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 00:26:41 +00:00
Vinod Kumar Vavilapalli
a8190ce5c5 MAPREDUCE-2988. Reenabled TestLinuxContainerExecutor reflecting the current NodeManager code. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180833 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 09:18:44 +00:00
Arun Murthy
d0016c6120 MAPREDUCE-3154. Fix JobSubmitter to check for output specs before copying job submission files to fail fast. Contributed by Abhijit Suresh Shingate. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180774 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:29:38 +00:00
Arun Murthy
811a6a9e30 MAPREDUCE-3123. Fix NM to quote symlink names to escape special characters. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180772 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:27:01 +00:00
Arun Murthy
4f1cdab6d8 MAPREDUCE-3153. Fix TestFileOutputCommitter which was broken by MAPREDUCE-2702. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180769 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:10:45 +00:00
Arun Murthy
3ddb52cfc0 MAPREDUCE-2876. Use a different config for ContainerAllocationExpirer. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180767 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:06:47 +00:00
Konstantin Shvachko
a09a2b48cd Move MAPREDUCE-3138 changes to Release 0.22 section. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180751 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 00:36:04 +00:00
Alejandro Abdelnur
4972959173 Reverted MAPREDUCE-3003 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180283 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-08 00:38:41 +00:00
Arun Murthy
047b175aac MAPREDUCE-2802. Ensure JobHistory filenames have jobId. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180220 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 20:44:37 +00:00
Arun Murthy
6b343adfe8 MAPREDUCE-3033. Ensure Master interface pays attention to classic v/s yarn frameworks. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180214 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 20:39:33 +00:00
Alejandro Abdelnur
68d1162f7a MAPREDUCE-3003. Publish Yarn and MapReduce artifacts to Maven snapshot repository. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180155 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 19:01:01 +00:00
Vinod Kumar Vavilapalli
02a81203bd MAPREDUCE-2751. Modified NodeManager to stop leaving around local files after application finishes. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180071 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 15:23:20 +00:00
Vinod Kumar Vavilapalli
fa2529c931 MAPREDUCE-3141. Fix the broken MRAppMaster to work over YARN in security mode.(vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180007 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 11:35:17 +00:00
Vinod Kumar Vavilapalli
a56f3931d1 MAPREDUCE-2783. Fixing RM web-UI to show no tracking-URL when AM crashes. Contributed by Eric Payne. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179975 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 09:17:57 +00:00
Mahadev Konar
b8102dbdf8 MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179936 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 05:27:36 +00:00
Amar Kamat
c1c0e8c9ea MAPREDUCE-3008. Improvements to cumulative CPU emulation for short running tasks in Gridmix. (amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179933 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 04:59:58 +00:00
Vinod Kumar Vavilapalli
282e2e910c MAPREDUCE-2913. Fixed TestMRJobs.testFailingMapper to assert the correct TaskCompletionEventStatus. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179319 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 17:07:20 +00:00
Vinod Kumar Vavilapalli
9a4e890f4a MAPREDUCE-2738. Added the missing cluster level statisticss on the RM web UI. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179229 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 14:01:32 +00:00
Vinod Kumar Vavilapalli
f29bfa0e0e MAPREDUCE-2702. Added a new API in OutputCommitter for recovering the outputs of tasks from a crashed job so as to support MR Application Master recovery. Contributed by Sharad Agarwal and Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179188 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 12:15:44 +00:00
Vinod Kumar Vavilapalli
f24dcb3449 MAPREDUCE-2907. Changed log level for various messages in ResourceManager from INFO to DEBUG. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179178 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 11:56:54 +00:00
Vinod Kumar Vavilapalli
66137cf17c MAPREDUCE-3013. Removed YarnConfiguration.YARN_SECURITY_INFO and its usage as it doesn't affect security any more. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 11:43:05 +00:00
Amar Kamat
45e6fb00af MAPREDUCE-2950. [Gridmix] TestUserResolve fails in trunk. (Ravi Gummadi via amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179090 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 08:03:49 +00:00
Konstantin Shvachko
ff9d7de705 Move MAPREDUCE-2531 changes to Release 0.22 section. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179078 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 07:39:35 +00:00
Vinod Kumar Vavilapalli
e979a3ddb1 MAPREDUCE-3056. Changed the default staging directory to not include user.name to prevent issues with non-secure mode. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178740 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 09:37:56 +00:00
Eric Yang
a48e6f2519 MAPREDUCE-3112. Fixed recursive sourcing of HADOOP_OPTS environment 
variable. (Eric Yang)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178657 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:51:13 +00:00
Arun Murthy
4f97e37648 MAPREDUCE-3138. Add a utility to help applications bridge changes in Context Objects APIs due to MAPREDUCE-954. Contributed by omalley. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178650 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:42:37 +00:00
Arun Murthy
12743d2169 MAPREDUCE-2792. Replace usage of node ip-addresses with hostnames. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178631 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-03 23:21:20 +00:00
Arun Murthy
43b59b894b MAPREDUCE-3134. Added documentation the CapacityScheduler. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178530 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-03 19:35:41 +00:00
Arun Murthy
7aa8a3c9f8 MAPREDUCE-2889. Added documentation for writing new YARN applications. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177920 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-01 05:22:50 +00:00
Arun Murthy
f3b98a26c4 MAPREDUCE-3113. Ensure bin/yarn and bin/yarn-daemon.sh identify the root of the install properly. Contributed by Xie Xianshan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177874 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 23:58:58 +00:00
Arun Murthy
fad230a49d MAPREDUCE-2719. Add a simple, DistributedShell, application to illustrate alternate frameworks on YARN. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177864 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 22:25:32 +00:00
Arun Murthy
dcf9d475e0 MAPREDUCE-3050. Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177859 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 22:19:17 +00:00
Konstantin Shvachko
960d7643fb MAPREDUCE-2779. JobSplitWriter.java can't handle large job.split file. Contributed by Ming Ma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177779 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 19:04:56 +00:00
Vinod Kumar Vavilapalli
063e33a862 MAPREDUCE-3098. Fixed RM and MR AM to report YarnApplicationState and application's FinalStatus separately. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177633 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 12:46:32 +00:00
Arun Murthy
964d4a1666 MAPREDUCE-2996. Add uber-ness information to JobHistory. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177531 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 06:47:53 +00:00
Arun Murthy
0b3c654d83 MAPREDUCE-2791. Added missing info on 'job -status' output. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177487 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 03:08:35 +00:00
Vinod Kumar Vavilapalli
71886657f8 MAPREDUCE-3001. Added task-specific counters to AppMaster and JobHistory web-UIs. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177203 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-29 08:08:49 +00:00
Vinod Kumar Vavilapalli
44090734c9 MAPREDUCE-3114. Fixed invalid ApplicationURL on RM WebUI. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177187 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-29 07:28:29 +00:00
Arun Murthy
312a7e7100 MAPREDUCE-3078. Ensure MapReduce AM reports progress correctly for displaying on the RM Web-UI. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176762 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 07:31:03 +00:00
Vinod Kumar Vavilapalli
f4a425e10d MAPREDUCE-3110. Fixed TestRPC failure. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176739 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 06:03:18 +00:00
Vinod Kumar Vavilapalli
cc42ccf02a MAPREDUCE-2843. Fixed the node-table to be completely displayed and making node entries on RM UI to be sortable. Contributed by Abhijit Suresh Shingate. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176730 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 05:35:43 +00:00
Arun Murthy
eda0ad4fd0 MAPREDUCE-3041. Fixed ClientRMProtocol to provide min/max resource capabilities along-with new ApplicationId for application submission. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176715 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 04:20:16 +00:00
Arun Murthy
5f6adcf396 MAPREDUCE-3021. Change base urls for RM web-ui. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176681 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 01:23:04 +00:00
Mahadev Konar
8e4c70fb20 MAPREDUCE-3099. Add docs for setting up a single node MRv2 cluster. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176607 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 20:41:56 +00:00
Mahadev Konar
0d2bb06236 MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176600 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 20:30:24 +00:00
Mahadev Konar
359c17071b MAPREDUCE-3095. fairscheduler ivy including wrong version for hdfs. (John George via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176566 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 19:44:56 +00:00
Giridharan Kesavan
98cc200700 MAPREDUCE-3081. Fix vaidya startup script. Contributed by Suhas 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176550 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 19:35:29 +00:00
Vinod Kumar Vavilapalli
87b969c835 MAPREDUCE-2999. Fix YARN webapp framework to properly filter servlet paths. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176469 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 17:03:19 +00:00
Vinod Kumar Vavilapalli
6507a0bc35 MAPREDUCE-3092. Removed a special comparator for JobIDs in JobHistory as JobIDs are already comparable. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176453 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 16:11:54 +00:00
Arun Murthy
17da0bdb27 MAPREDUCE-3067. Ensure exit-code is set correctly for containers. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176235 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 06:56:50 +00:00
Mahadev Konar
70388cffe0 MAPREDUCE-3071. app master configuration web UI link under the Job menu opens up application menu. (thomas graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176203 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 05:30:01 +00:00
Arun Murthy
81926396f6 MAPREDUCE-2984. Better error message for displaying completed containers. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176045 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 20:45:18 +00:00
Vinod Kumar Vavilapalli
eff931a1b1 MAPREDUCE-3031. Proper handling of killed containers to prevent stuck containers/AMs on an external kill signal. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175960 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 17:27:15 +00:00
Vinod Kumar Vavilapalli
1e6dfa7472 MAPREDUCE-2646. Fixed AMRMProtocol to return containers based on priority. Contributed by Sharad Agarwal and Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175859 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 13:25:27 +00:00
Arun Murthy
c9a7d3dbf9 MAPREDUCE-3090. Fix MR AM to use ApplicationAttemptId rather than (ApplicationId, startCount) consistently. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175718 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 08:44:41 +00:00
Mahadev Konar
5a3040cad4 MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175449 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 18:02:51 +00:00
Vinod Kumar Vavilapalli
a5c9ede143 MAPREDUCE-2952. Fixed ResourceManager/MR-client to consume diagnostics for AM failures in a couple of corner cases. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175403 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 14:46:59 +00:00
Arun Murthy
c285cf3114 MAPREDUCE-3053. Better diagnostic message for unknown methods in ProtoBuf RPCs. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175357 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 09:50:02 +00:00
Arun Murthy
5ace0cabe5 MAPREDUCE-2990. Fixed display of NodeHealthStatus. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175351 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 09:36:12 +00:00
Arun Murthy
d09ceac1f7 MAPREDUCE-2691. Increase threadpool size for launching containers in MapReduce ApplicationMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175294 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-25 04:43:51 +00:00
Vinod Kumar Vavilapalli
b549c10782 MAPREDUCE-3055. Simplified ApplicationAttemptId passing to ApplicationMaster via environment variable. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1174785 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-23 14:07:42 +00:00
Arun Murthy
1eb7008753 MAPREDUCE-3073. Fixed build issues in MR1. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1174294 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-22 18:05:30 +00:00
Vinod Kumar Vavilapalli
4806d7ba74 MAPREDUCE-2754. Fixed MR AM stdout, stderr and syslog to redirect to correct log-files. Contributed by Ravi Teja Ch N V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1174194 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-22 15:14:31 +00:00
Mahadev Konar
3557202a92 MAPREDUCE-3044. Pipes jobs stuck without making progress. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173909 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 23:52:50 +00:00
Arun Murthy
ed7eda00cb MAPREDUCE-3066. Fixed default ResourceTracker address for the NodeManager. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173904 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 23:32:31 +00:00
Arun Murthy
32f28e6994 MAPREDUCE-3062. Fixed default RMAdmin address. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173838 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 20:37:08 +00:00
Vinod Kumar Vavilapalli
d00b3c49f6 MAPREDUCE-2880. Improved classpath-construction for mapreduce AM and containers. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173783 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 18:28:23 +00:00
Vinod Kumar Vavilapalli
af61f4ae15 MAPREDUCE-2970. Fixed NPEs in corner cases with different configurations for mapreduce.framework.name. Contributed by Venu Gopala Rao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173534 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 09:13:34 +00:00
Vinod Kumar Vavilapalli
01fbb0fb45 MAPREDUCE-2930. Added the ability to be able to generate graphs from the state-machine definitions. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173524 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 08:37:00 +00:00
Arun Murthy
0e870d7d18 MAPREDUCE-3023. Fixed clients to display queue state correctly. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173458 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 01:32:14 +00:00
Arun Murthy
1d067c6e2b MAPREDUCE-2998. Fixed a bug in TaskAttemptImpl which caused it to fork bin/mapred too many times. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173456 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 01:26:48 +00:00
Arun Murthy
339b85b88e MAPREDUCE-3036. Fixed metrics for reserved resources in CS. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173453 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 01:14:20 +00:00
Arun Murthy
b8e8b8da75 MAPREDUCE-3018. Fixed -file option for streaming. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173451 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 01:10:49 +00:00
Arun Murthy
f0fedda8ef MAPREDUCE-3040. Fixed extra copy of Configuration in YarnClientProtocolProvider and ensured MiniMRYarnCluster sets JobHistory configuration for tests. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172929 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-20 00:46:22 +00:00
Mahadev Konar
bbfd81503c MAPREDUCE-3017. The Web UI shows FINISHED for killed/successful/failed jobs. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172906 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-19 23:56:52 +00:00
Arun Murthy
482d840bcf MAPREDUCE-3004. Fix ReduceTask to not assume 'local' mode in YARN. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172893 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-19 23:17:27 +00:00
Mahadev Konar
e16f8a9fdf MAPREDUCE-3038. job history server not starting because conf() missing HsController (Jeffrey Naisbitt via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172875 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-19 22:51:46 +00:00
Arun Murthy
8ae3cdeac7 MAPREDUCE-3042. Fixed default ResourceTracker address. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172847 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-19 21:31:15 +00:00
Vinod Kumar Vavilapalli
b7660f2bc9 MAPREDUCE-2726. Added job-file to the AM and JobHistoryServer web interfaces. Contributed by Jeffrey Naisbitt. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172704 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-19 17:01:50 +00:00
Vinod Kumar Vavilapalli
2e61ed306f MAPREDUCE-3030. Fixed a bug in NodeId.equals() that was causing RM to reject all NMs. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172638 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-19 14:17:56 +00:00
Vinod Kumar Vavilapalli
29552eeb36 MAPREDUCE-2925. Fixed Yarn+MR client code to behave saner with completed jobs. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172587 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-19 12:26:14 +00:00
Vinod Kumar Vavilapalli
34d3936bcd MAPREDUCE-2965. Streamlined the methods hashCode(), equals(), compareTo() and toString() for all IDs. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172212 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-18 07:51:11 +00:00
Vinod Kumar Vavilapalli
61900651b1 MAPREDUCE-3006. Fixed MapReduce AM to exit only after properly writing out history file. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172206 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-18 07:16:18 +00:00
Mahadev Konar
e1acb1222d MAPREDUCE-1788. o.a.h.mapreduce.Job shouldn't make a copy of the JobConf. (Arun Murthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1172171 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-18 02:50:30 +00:00
Arun Murthy
2eab7a28c7 MAPREDUCE-2987. Fixed display of logged user on RM Web-UI. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171901 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-17 05:52:49 +00:00
Todd Lipcon
ca333f17c1 MAPREDUCE-2836. Provide option to fail jobs when submitted to non-existent fair scheduler pools. Contributed by Ahmed Radwan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171832 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-16 22:22:34 +00:00
Vinod Kumar Vavilapalli
341a15a23d MAPREDUCE-2994. Fixed a bug in ApplicationID parsing that affects RM UI. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171485 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-16 10:08:52 +00:00
Mahadev Konar
d9ba4670ed MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job. (Robert Evans via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171297 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-15 22:21:00 +00:00
Vinod Kumar Vavilapalli
cb48bc1c93 MAPREDUCE-3007. Fixed Yarn Mapreduce client to be able to connect to JobHistoryServer in secure mode. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1171051 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-15 10:53:05 +00:00
Konstantin Shvachko
9704960e42 MAPREDUCE-2991. queueinfo.jsp fails if queue names contain special symbols. Contributed by Priyo Mustafi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170917 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-15 01:38:44 +00:00
Arun Murthy
5183e88109 MAPREDUCE-3005. Fix both FifoScheduler and CapacityScheduler to correctly enforce locality constraints. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170879 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-14 22:46:57 +00:00
Vinod Kumar Vavilapalli
4ba2acf336 MAPREDUCE-2966. Added ShutDown hooks for MRV2 processes so that they can gracefully exit. Contributed by Abhijit Suresh Shingate. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170746 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-14 17:57:52 +00:00
Vinod Kumar Vavilapalli
47e04d3124 MAPREDUCE-2949. Fixed NodeManager to shut-down correctly if a service startup fails. Contributed by Ravi Teja Ch N V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170689 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-14 15:35:42 +00:00
Mahadev Konar
88b82a0f66 MAPREDUCE-2899. Replace major parts of ApplicationSubmissionContext with a ContainerLaunchContext (Arun Murthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170459 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-14 07:26:37 +00:00
Mahadev Konar
a3e8f6836b MAPREDUCE-2676. MR-279: JobHistory Job page needs reformatted. (Robert Evans via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170379 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-13 22:55:22 +00:00
Arun Murthy
88ff272cfd MAPREDUCE-2995. Fixed race condition in ContainerLauncher. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170281 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-13 18:15:14 +00:00
Arun Murthy
53f921418d MAPREDUCE-2995. Better handling of expired containers in MapReduce ApplicationMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170279 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-13 18:12:02 +00:00
Arun Murthy
6165875dc6 MAPREDUCE-2896. Simplify all apis to in org.apache.hadoop.yarn.api.records.* to be get/set only. Added javadocs to all public records. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169980 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-13 00:05:51 +00:00
Arun Murthy
7fa0bf3558 MAPREDUCE-2874. Fix formatting of ApplicationId in web-ui. Contributed by Eric Payne. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169973 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 23:17:27 +00:00
Arun Murthy
c8383a5f49 MAPREDUCE-2985. Fixed findbugs warnings in ResourceLocalizationService. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169955 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 21:53:51 +00:00
Konstantin Shvachko
1c01951264 MAPREDUCE-2669. Add new examples for Mean, Median, and Standard Deviation. Contributed by Plamen Jeliazkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169874 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 19:13:41 +00:00
Vinod Kumar Vavilapalli
fb65bfc661 MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169808 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 16:00:40 +00:00
Vinod Kumar Vavilapalli
be32d25c54 MAPREDUCE-2675. Reformat JobHistory Server main page to be more useful. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169763 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 14:24:32 +00:00
Vinod Kumar Vavilapalli
e14da3a72e MAPREDUCE-2978. Fixed test-patch to make Jenkins report findbugs warnings properly. Contributed by Tom White. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169696 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 11:22:38 +00:00
Arun Murthy
6b2f2efe4d MAPREDUCE-2749. Ensure NM registers with RM after starting all its services correctly. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169621 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 07:11:43 +00:00
Arun Murthy
817ead65b9 MAPREDUCE-2933. Change allocate call to return ContainerStatus for completed containers rather than Container. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169484 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-11 17:26:06 +00:00
Vinod Kumar Vavilapalli
8fb67650b1 MAPREDUCE-2691. Finish up the cleanup of distributed cache file resources and related tests. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1167676 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-11 06:21:39 +00:00
Mahadev Konar
b982e6205f MAPREDUCE-2971. ant build mapreduce fails protected access jc.displayJobList (jobs) (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1167383 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 22:42:57 +00:00
Arun Murthy
fbf0035ee6 MAPREDUCE-2975. Fixed YARNRunner to use YarnConfiguration rather than Configuration. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1167331 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 19:01:08 +00:00
Aaron Myers
68cb2b01b6 HADOOP-7607 and MAPREDUCE-2934. Simplify the RPC proxy cleanup process. (atm) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1167318 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 18:12:14 +00:00
Vinod Kumar Vavilapalli
1d84d983a2 MAPREDUCE-2954. Fixed a deadlock in NM caused due to wrong synchronization in protocol buffer records. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1167061 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 09:47:15 +00:00
Tsz-wo Sze
85e17fd2df MAPREDUCE-2711. Update TestBlockPlacementPolicyRaid for the new namesystem and block management APIs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1167001 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 04:48:35 +00:00
Arun Murthy
71e0515433 MAPREDUCE-2963. Fix hang in TestMRJobs. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166975 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 03:01:08 +00:00
Arun Murthy
ca853445e9 MAPREDUCE-2953. Fix a race condition on submission which caused client to incorrectly assume application was gone by making submission synchronous for RMAppManager. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166968 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 02:19:24 +00:00
Arun Murthy
a0ef2d7503 MAPREDUCE-2937. Ensure reason for application failure is displayed to the user. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166966 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 02:14:33 +00:00
Arun Murthy
035937bee8 MAPREDUCE-2690. Web-page for FifoScheduler. Contributed by Eric Payne. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166958 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 02:04:55 +00:00
Arun Murthy
fafe8cd28e MAPREDUCE-2864. Normalize configuration variable names for YARN. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166955 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 01:44:58 +00:00
Arun Murthy
b3c9c3c182 MAPREDUCE-2677. Fixed 404 for some links from HistoryServer. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166901 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 20:59:34 +00:00
Arun Murthy
1b294f206b MAPREDUCE-2844. Fixed display of nodes in UI. Contributed by Ravi Teja Ch N V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166855 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 18:48:10 +00:00
Arun Murthy
6b6ef7b8e3 MAPREDUCE-2947. Fixed race condition in AuxiliaryServices. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166849 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 18:40:27 +00:00
Mahadev Konar
8ac20f92f5 MAPREDUCE-2942. TestNMAuditLogger.testNMAuditLoggerWithIP failing (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166842 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 18:29:49 +00:00
Arun Murthy
1f46b991da MAPREDUCE-2908. Fix all findbugs warnings. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166838 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 18:26:20 +00:00
Amar Kamat
ae5e8e0104 MAPREDUCE-2784. [Gridmix] Bug fixes in ExecutionSummarizer and ResourceUsageMatcher. (amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166636 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 11:29:55 +00:00
Mahadev Konar
1c15670d53 MAPREDUCE-2948. Hadoop streaming test failure, post MR-2767 (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166402 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-07 21:18:09 +00:00
Arun Murthy
0ecba04727 MAPREDUCE-2938. Log application submission failure in CapacityScheduler 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166006 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-07 05:54:25 +00:00
Arun Murthy
cdfabf5ae2 MAPREDUCE-2655. Add audit logs to ResourceManager and NodeManager. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165949 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-07 01:24:21 +00:00
Arun Murthy
d51078eb7e MAPREDUCE-2774. Add startup message to ResourceManager & NodeManager on startup. Contributed by Venu Gopala Rao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165933 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 23:40:49 +00:00
Arun Murthy
03c1015d6e MAPREDUCE-2800. Set final progress for tasks to ensure all task information is correctly logged to JobHistory. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165930 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 23:36:19 +00:00
Konstantin Shvachko
3a3f277017 MAPREDUCE-2767. CHANGES.txt 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165894 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 21:59:09 +00:00
Arun Murthy
f560a1066f MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165890 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 21:49:46 +00:00
Vinod Kumar Vavilapalli
e20f135174 MAPREDUCE-2936. Contrib Raid compilation broken after HDFS-1620. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165657 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 13:19:10 +00:00
Todd Lipcon
ae367be8f3 MAPREDUCE-2882. TestLineRecordReader depends on ant jars. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165449 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-05 23:23:35 +00:00
Arun Murthy
6b608aad7d MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running applications per-queue & per-user. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165403 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-05 19:49:47 +00:00
Vinod Kumar Vavilapalli
6bdf5746d5 MAPREDUCE-2716. MRReliabilityTest job fails because of missing job-file. Contributed by Jeffrey Naisbitt. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1164805 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-03 06:22:03 +00:00
Sanjay Radia
b97a4d40c8 HADOOP-7524 and MapReduce-2887 Change RPC to allow multiple protocols including multuple versions of the same protocol (sanjay Radia) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1164771 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-03 00:31:05 +00:00
Arun Murthy
5d4f16b93c MAPREDUCE-2735. Add an applications summary log to ResourceManager. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1164301 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-01 23:58:12 +00:00
Arun Murthy
a9bf33445e MAPREDUCE-2756. Better error handling in JobControl for failed jobs. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1164255 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-01 20:24:38 +00:00
Arun Murthy
b9a5fd5190 MAPREDUCE-2917. Fixed corner case in container reservation which led to starvation and hung jobs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1163768 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-31 19:52:20 +00:00
Vinod Kumar Vavilapalli
ade0f0560f MAPREDUCE-2652. Enabled multiple NMs to be runnable on a single node by making shuffle service port to be truely configurable. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1163585 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-31 11:38:32 +00:00
Mahadev Konar
5786efbfa6 MAPREDUCE-2916. Ivy build for MRv1 fails with bad organization for common daemon. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1163490 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-31 05:48:02 +00:00
Arun Murthy
c2b851b0f2 MAPREDUCE-2897. Javadoc for ClientRMProtocol protocol and related records. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1163069 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-30 01:22:28 +00:00
Mahadev Konar
fb6ecb9b27 MAPREDUCE-2886. Fix Javadoc warnings in MapReduce. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1163050 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-29 23:27:47 +00:00
Mahadev Konar
aad81a15ec MAPREDUCE-2737. Update the progress of jobs on client side. (Siddharth Seth and Mahadev Konar via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1162979 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-29 20:23:17 +00:00
Arun Murthy
3380d8f70c MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1162954 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-29 19:36:26 +00:00
Arun Murthy
b9259a2710 MAPREDUCE-2898. Javadoc for ContainerManager protocol and related records. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1162613 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-29 00:04:06 +00:00
Arun Murthy
00d2263908 MAPREDUCE-2891. Javadoc for AMRMProtocol and related records. Contributed by acmurthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1162499 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-28 11:41:36 +00:00
Arun Murthy
c9cbd8e4e6 MAPREDUCE-2893. Remove duplicate entry of YarnClientProtocolProvider in ClientProtocolProvider services file. Contributed by Liang-Chi Hsieh. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1162491 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-28 10:37:24 +00:00
Arun Murthy
d8c3e63dce MAPREDUCE-2885. Fix mapred-config.sh to look for hadoop-config.sh in HADOOP_COMMON_HOME/libexec. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1162188 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-26 17:55:53 +00:00
Giridharan Kesavan
0ee0fa45d2 MAPREDUCE-2881. Fix to include log4j 1.2.16 depenency. Contributed by Giridharan Kesavan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161782 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 22:30:42 +00:00
Arun Murthy
a44ffcc0be Preparing for hadoop-0.24.0 development. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161736 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 20:32:04 +00:00
Arun Murthy
ee05510c42 Preparing for release 0.23.0 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161716 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 19:39:15 +00:00
Arun Murthy
58676edf98 MAPREDUCE-2879. Fix version for MR-279 to 0.23.0. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161705 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 19:13:50 +00:00
Arun Murthy
1dd113b24a MAPREDUCE-2796. Set start times for MR applications for clients to see. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161704 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 19:12:35 +00:00
Mahadev Konar
7c85f33ce1 MAPREDUCE-2877. Add missing Apache license header in some files in MR and also add the rat plugin to the poms. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161650 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 17:16:36 +00:00
Sharad Agarwal
f2b91a8367 MAPREDUCE-2807. Fix AM restart and client redirection. Contributed by Sharad Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161408 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 06:35:58 +00:00
Arun Murthy
cd7157784e HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161332 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 00:14:24 +00:00