big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,832 Commits 2 Branches 2 Tags 264 MiB
cda16311a9
Commit Graph

118 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
47a381e306 MAPREDUCE-3146. Added a MR specific command line to dump logs for a given TaskAttemptID. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195349 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 06:42:06 +00:00
Vinod Kumar Vavilapalli
5f9e67e226 MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195340 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 05:51:33 +00:00
Arun Murthy
85fce9017c Adding executable svn prop on mock-container-executor. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195336 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 05:29:13 +00:00
Arun Murthy
7f4dc27757 MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1194850 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 09:35:36 +00:00
Vinod Kumar Vavilapalli
c1e333f32f MAPREDUCE-3306. Fixed a bug in NodeManager ApplicationImpl that was causing NodeManager to crash. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190523 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 19:45:57 +00:00
Vinod Kumar Vavilapalli
cbdb07f4ca MAPREDUCE-2775. Fixed ResourceManager and NodeManager to force a decommissioned node to shutdown. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190467 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 17:31:06 +00:00
Vinod Kumar Vavilapalli
670fa24b48 MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the JobHistoryServer. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:45:04 +00:00
Vinod Kumar Vavilapalli
b9f6890db9 MAPREDUCE-3281. Fixed one more bug due to which TestLinuxContainerExecutorWithMocks was failing. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190007 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 20:38:40 +00:00
Vinod Kumar Vavilapalli
1b8511ef89 MAPREDUCE-3281. Fixed a bug in TestLinuxContainerExecutorWithMocks. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189818 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 15:55:41 +00:00
Vinod Kumar Vavilapalli
f114dad5ef MAPREDUCE-2986. Fixed MiniYARNCluster to support multiple NodeManagers. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189721 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 12:42:12 +00:00
Vinod Kumar Vavilapalli
2fe343f963 MAPREDUCE-3240. Fixed NodeManager to be able to forcefully cleanup its containers (process-trees) irrespective of whether the container succeeded, or killed. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189711 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 12:03:17 +00:00
Arun Murthy
fc74b5a25c MAPREDUCE-3259. Added java.library.path of NodeManager to ContainerLocalizer in LinuxContainerExecutor. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189667 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 08:28:34 +00:00
Arun Murthy
237154982b MAPREDUCE-3205. Fix memory specifications to be physical rather than virtual, allowing for a ratio between the two to be configurable. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189542 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-26 23:58:31 +00:00
Mahadev Konar
d19cfe0164 MAPREDUCE-2746. Yarn servers can't communicate with each other with hadoop.security.authorization set to true (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188522 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:07:13 +00:00
Todd Lipcon
bccfd6a09f Amend MAPREDUCE-3159. Fix JavaDoc warnings in previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188479 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:14:17 +00:00
Arun Murthy
c70ee2488c MAPREDUCE-3248. Fixed log4j properties. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188476 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:03:08 +00:00
Arun Murthy
6217e54718 MAPREDUCE-3159. Ensure DefaultContainerExecutor doesn't delete application directories during app-init. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188466 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 00:36:19 +00:00
Vinod Kumar Vavilapalli
7ce1c4ab35 MAPREDUCE-2747. Cleaned up LinuxContainerExecutor binary sources and changed the configuration to use yarn names. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188236 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 17:09:37 +00:00
Mahadev Konar
4186121c08 MAPREDUCE-3239. Use new createSocketAddr API in MRv2 to give better error messages on misconfig (Todd Lipcon via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187556 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 21:36:58 +00:00
Vinod Kumar Vavilapalli
df2991c0cb MAPREDUCE-3104. Implemented Application-acls. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1186748 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-20 11:45:38 +00:00
Vinod Kumar Vavilapalli
e3bb120e9f MAPREDUCE-3162. Separated application-init and container-init event types in NodeManager's Application state machine. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185988 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 06:40:03 +00:00
Vinod Kumar Vavilapalli
13e4562924 MAPREDUCE-3144. Augmented JobHistory with the information needed for serving aggregated logs. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185976 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 05:21:18 +00:00
Vinod Kumar Vavilapalli
3e3abcb769 MAPREDUCE-3199. Fixed pom files to include correct log4j configuration for tests. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185967 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 04:30:27 +00:00
Arun Murthy
c1d90772b6 MAPREDUCE-3068. Added a whitelist of environment variables for containers from the NodeManager and set MALLOC_ARENA_MAX for all daemons and containers. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185447 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-18 01:22:14 +00:00
Mahadev Konar
955d8eca21 MAPREDUCE-3196. TestLinuxContainerExecutorWithMocks fails on Mac OSX. (Arun Murthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185417 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 22:41:26 +00:00
Alejandro Abdelnur
15a1d9af19 MAPREDUCE-3003. Publish MR JARs to Maven snapshot repository. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185286 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 17:06:42 +00:00
Mahadev Konar
68328ae926 MAPREDUCE-3187. Add names for various unnamed threads in MR2. (Todd Lipcon and Siddharth Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184904 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-16 19:27:02 +00:00
Alejandro Abdelnur
327e02fdf4 MAPREDUCE-3171. normalize nodemanager native code compilation with common/hdfs native. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182575 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 20:38:51 +00:00
Vinod Kumar Vavilapalli
1c358b0796 MAPREDUCE-2668. Fixed AuxServices to send a signal on application-finish to all the services. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181803 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 14:26:17 +00:00
Vinod Kumar Vavilapalli
11b9dd4e84 MAPREDUCE-3161. Improved some javadocs and fixed some typos in YARN. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1181622 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 04:45:28 +00:00
Vinod Kumar Vavilapalli
a8190ce5c5 MAPREDUCE-2988. Reenabled TestLinuxContainerExecutor reflecting the current NodeManager code. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180833 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 09:18:44 +00:00
Arun Murthy
811a6a9e30 MAPREDUCE-3123. Fix NM to quote symlink names to escape special characters. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180772 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-10 03:27:01 +00:00
Alejandro Abdelnur
4972959173 Reverted MAPREDUCE-3003 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180283 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-08 00:38:41 +00:00
Alejandro Abdelnur
68d1162f7a MAPREDUCE-3003. Publish Yarn and MapReduce artifacts to Maven snapshot repository. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180155 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 19:01:01 +00:00
Thomas White
02ce6b3d07 MAPREDUCE-3014. Rename and invert logic of '-cbuild' profile to 'native' and off by default. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180116 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 17:33:10 +00:00
Vinod Kumar Vavilapalli
02a81203bd MAPREDUCE-2751. Modified NodeManager to stop leaving around local files after application finishes. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180071 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 15:23:20 +00:00
Vinod Kumar Vavilapalli
66137cf17c MAPREDUCE-3013. Removed YarnConfiguration.YARN_SECURITY_INFO and its usage as it doesn't affect security any more. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 11:43:05 +00:00
Arun Murthy
12743d2169 MAPREDUCE-2792. Replace usage of node ip-addresses with hostnames. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1178631 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-03 23:21:20 +00:00
Arun Murthy
5f6adcf396 MAPREDUCE-3021. Change base urls for RM web-ui. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176681 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 01:23:04 +00:00
Arun Murthy
17da0bdb27 MAPREDUCE-3067. Ensure exit-code is set correctly for containers. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176235 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-27 06:56:50 +00:00
Arun Murthy
81926396f6 MAPREDUCE-2984. Better error message for displaying completed containers. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1176045 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 20:45:18 +00:00
Vinod Kumar Vavilapalli
eff931a1b1 MAPREDUCE-3031. Proper handling of killed containers to prevent stuck containers/AMs on an external kill signal. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1175960 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 17:27:15 +00:00
Vinod Kumar Vavilapalli
b549c10782 MAPREDUCE-3055. Simplified ApplicationAttemptId passing to ApplicationMaster via environment variable. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1174785 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-23 14:07:42 +00:00
Mahadev Konar
3557202a92 MAPREDUCE-3044. Pipes jobs stuck without making progress. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173909 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 23:52:50 +00:00
Arun Murthy
ed7eda00cb MAPREDUCE-3066. Fixed default ResourceTracker address for the NodeManager. Contributed by Chris Riccomini. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173904 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 23:32:31 +00:00
Vinod Kumar Vavilapalli
d00b3c49f6 MAPREDUCE-2880. Improved classpath-construction for mapreduce AM and containers. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173783 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 18:28:23 +00:00
Vinod Kumar Vavilapalli
01fbb0fb45 MAPREDUCE-2930. Added the ability to be able to generate graphs from the state-machine definitions. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1173524 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-21 08:37:00 +00:00
Vinod Kumar Vavilapalli
4ba2acf336 MAPREDUCE-2966. Added ShutDown hooks for MRV2 processes so that they can gracefully exit. Contributed by Abhijit Suresh Shingate. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170746 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-14 17:57:52 +00:00
Vinod Kumar Vavilapalli
47e04d3124 MAPREDUCE-2949. Fixed NodeManager to shut-down correctly if a service startup fails. Contributed by Ravi Teja Ch N V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170689 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-14 15:35:42 +00:00
Mahadev Konar
88b82a0f66 MAPREDUCE-2899. Replace major parts of ApplicationSubmissionContext with a ContainerLaunchContext (Arun Murthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1170459 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-14 07:26:37 +00:00
Arun Murthy
6165875dc6 MAPREDUCE-2896. Simplify all apis to in org.apache.hadoop.yarn.api.records.* to be get/set only. Added javadocs to all public records. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169980 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-13 00:05:51 +00:00
Arun Murthy
c8383a5f49 MAPREDUCE-2985. Fixed findbugs warnings in ResourceLocalizationService. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169955 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 21:53:51 +00:00
Arun Murthy
6b2f2efe4d MAPREDUCE-2749. Ensure NM registers with RM after starting all its services correctly. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169621 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-12 07:11:43 +00:00
Arun Murthy
817ead65b9 MAPREDUCE-2933. Change allocate call to return ContainerStatus for completed containers rather than Container. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1169484 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-11 17:26:06 +00:00
Vinod Kumar Vavilapalli
8fb67650b1 MAPREDUCE-2691. Finish up the cleanup of distributed cache file resources and related tests. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1167676 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-11 06:21:39 +00:00
Arun Murthy
fafe8cd28e MAPREDUCE-2864. Normalize configuration variable names for YARN. Contributed by Robert Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166955 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 01:44:58 +00:00
Arun Murthy
6b6ef7b8e3 MAPREDUCE-2947. Fixed race condition in AuxiliaryServices. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166849 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 18:40:27 +00:00
Mahadev Konar
8ac20f92f5 MAPREDUCE-2942. TestNMAuditLogger.testNMAuditLoggerWithIP failing (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166842 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 18:29:49 +00:00
Arun Murthy
1f46b991da MAPREDUCE-2908. Fix all findbugs warnings. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1166838 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-08 18:26:20 +00:00
Arun Murthy
8815bdf304 Adding new files for MAPREDUCE-2655. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165950 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-07 01:26:21 +00:00
Arun Murthy
cdfabf5ae2 MAPREDUCE-2655. Add audit logs to ResourceManager and NodeManager. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165949 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-07 01:24:21 +00:00
Arun Murthy
d51078eb7e MAPREDUCE-2774. Add startup message to ResourceManager & NodeManager on startup. Contributed by Venu Gopala Rao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165933 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 23:40:49 +00:00
Arun Murthy
f560a1066f MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165890 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-06 21:49:46 +00:00
Vinod Kumar Vavilapalli
ade0f0560f MAPREDUCE-2652. Enabled multiple NMs to be runnable on a single node by making shuffle service port to be truely configurable. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1163585 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-31 11:38:32 +00:00
Mahadev Konar
fb6ecb9b27 MAPREDUCE-2886. Fix Javadoc warnings in MapReduce. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1163050 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-29 23:27:47 +00:00
Mahadev Konar
7c85f33ce1 MAPREDUCE-2877. Add missing Apache license header in some files in MR and also add the rat plugin to the poms. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161650 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 17:16:36 +00:00
Sharad Agarwal
f2b91a8367 MAPREDUCE-2807. Fix AM restart and client redirection. Contributed by Sharad Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161408 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 06:35:58 +00:00
Arun Murthy
cd7157784e HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1161332 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 00:14:24 +00:00