big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,979 Commits 2 Branches 2 Tags 264 MiB
e5ab158846
Commit Graph

268 Commits

Author SHA1 Message Date
Arun Murthy
f49ba355e5 MAPREDUCE-3762. Fixed default CapacityScheduler configs. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238987 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-01 07:57:35 +00:00
Vinod Kumar Vavilapalli
d1f805f942 MAPREDUCE-3754. Modified RM UI to filter applications based on state of the applications. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238887 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-01 00:51:40 +00:00
Arun Murthy
7f23d72352 MAPREDUCE-3764. Fixed resource usage metrics for queues and users. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238255 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-31 08:30:31 +00:00
Arun Murthy
7ed0f74997 MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238157 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-31 06:22:22 +00:00
Mahadev Konar
443d8f6c0b MAPREDUCE-3679. AM logs and others should not automatically refresh after every 1 second. (Vinod KV via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238146 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-31 04:53:49 +00:00
Vinod Kumar Vavilapalli
76b653a367 MAPREDUCE-3716. Fixing YARN+MR to allow MR jobs to be able to use java.io.File.createTempFile to create temporary files as part of their tasks. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1238136 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-31 02:23:38 +00:00
Mahadev Konar
7e4725db41 MAPREDUCE-3703. ResourceManager should provide node lists in JMX output. (Eric Payne via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1237920 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-30 18:47:55 +00:00
Vinod Kumar Vavilapalli
5262b7ba4d MAPREDUCE-3732. Modified CapacityScheduler to use only users with pending requests for computing user-limits. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236953 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-28 01:32:29 +00:00
Alejandro Abdelnur
8a234f394e HADOOP-7939. Improve Hadoop subcomponent integration in Hadoop 0.23. (rvs via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236929 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-27 23:53:35 +00:00
Vinod Kumar Vavilapalli
fe4bff77e1 MAPREDUCE-3740. Fixed broken mapreduce compilation after the patch for HADOOP-7965. Contributed by Devaraj K. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236901 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-27 21:49:15 +00:00
Mahadev Konar
fdb9d51cec MAPREDUCE-3742. "yarn logs" command fails with ClassNotFoundException. (Jason Lowe via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236816 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-27 18:46:56 +00:00
Vinod Kumar Vavilapalli
3cdc100369 MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236433 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-26 22:56:53 +00:00
Vinod Kumar Vavilapalli
54f738575c MAPREDUCE-3699. Increased RPC handlers for all YARN servers to reasonable values for working at scale. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236396 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-26 21:37:28 +00:00
Mahadev Konar
af6448e543 MAPREDUCE-3737. The Web Application Proxy's is not documented very well. (Robert Evans via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236371 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-26 20:10:50 +00:00
Vinod Kumar Vavilapalli
21c9116309 MAPREDUCE-3713. Fixed the way head-room is allocated to applications by CapacityScheduler so that it deducts current-usage per user and not per-application. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235989 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-25 23:31:14 +00:00
Arun Murthy
9d1621da52 MAPREDUCE-3683. Fixed maxCapacity of queues to be product of parent maxCapacities. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235858 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-25 18:17:55 +00:00
Alejandro Abdelnur
f73daf6af1 MAPREDUCE-3505. yarn APPLICATION_CLASSPATH needs to be overridable. (ahmed via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235391 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 18:21:27 +00:00
Eli Collins
5dfe62d845 HDFS-2397. Undeprecate SecondaryNameNode. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235135 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 06:05:52 +00:00
Arun Murthy
9a153334ac MAPREDUCE-3681. Fixed computation of queue's usedCapacity. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235103 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 01:00:51 +00:00
Mahadev Konar
c415dee2b7 MAPREDUCE-3691. webservices add support to compress response. (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1234989 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-23 21:15:55 +00:00
Mahadev Konar
8a1719e999 MAPREDUCE-3549. write api documentation for web service apis for RM, NM, mapreduce app master, and job history server (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1234222 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-21 00:53:02 +00:00
Mahadev Konar
b49aae0635 MAPREDUCE-3689. RM web UI doesn't handle newline in job name. (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1234148 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-20 21:27:39 +00:00
Mahadev Konar
c6923061d0 MAPREDUCE-3698. Client cannot talk to the history server in secure mode. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1234120 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-20 20:44:17 +00:00
Eli Collins
6861560098 MAPREDUCE-3692. yarn-resourcemanager out and log files can get big. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1233605 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-19 22:23:53 +00:00
Vinod Kumar Vavilapalli
4a343c9d4a MAPREDUCE-3641. Making CapacityScheduler more conservative so as to assign only one off-switch container in a single scheduling iteration. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1232182 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-16 21:56:14 +00:00
Mahadev Konar
c81995e370 MAPREDUCE-3657. State machine visualize build fails. (Jason Lowe via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1232167 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-16 21:22:21 +00:00
Vinod Kumar Vavilapalli
90f096d86c MAPREDUCE-3532. Modified NM to report correct http address when an ephemeral web port is configured. Contributed by Bhallamudi Venkata Siva Kamesh. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1231342 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-13 22:21:33 +00:00
Siddharth Seth
78ff0b720e MAPREDUCE-3596. Fix scheduler to handle cleaned up containers, which NMs may subsequently report as running. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1231297 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-13 21:15:22 +00:00
Jitendra Nath Pandey
f717c5785e MAPREDUCE-3664. Federation Documentation has incorrect configuration example. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1230708 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-12 19:11:33 +00:00
Mahadev Konar
0086014703 MAPREDUCE-3625. CapacityScheduler web-ui display of queue's used capacity is broken. (Jason Lowe via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1230336 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-12 00:15:35 +00:00
Mahadev Konar
5f79f180f6 MAPREDUCE-3553. Add support for data returned when exceptions thrown from web service apis to be in either xml or in JSON. (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1230330 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 23:59:23 +00:00
Mahadev Konar
bc4b1f48d3 MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229855 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 01:50:10 +00:00
Vinod Kumar Vavilapalli
7712e70090 MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR AM. Improved the earlier patch to not to JobHistoryServer repeatedly. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229787 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 23:07:54 +00:00
Arun Murthy
5ba6477099 MAPREDUCE-3624. Remove unnecessary dependency on JDK's tools.jar. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1228354 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-06 19:12:02 +00:00
Arun Murthy
2919ce8c8b MAPREDUCE-3617. Removed wrong default value for yarn.resourcemanager.principal and yarn.nodemanager.principal. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227808 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 20:11:09 +00:00
Arun Murthy
7440a8aa37 MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227801 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 20:01:20 +00:00
Arun Murthy
4404f20671 MAPREDUCE-3326. Added detailed information about queue's to the CapacityScheduler web-ui. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227788 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 19:50:18 +00:00
Alejandro Abdelnur
6d45b9e31d HADOOP-7934. Normalize dependencies versions across all modules. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227787 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 19:46:42 +00:00
Mahadev Konar
e793ba8cba MAPREDUCE-3478. Cannot build against ZooKeeper 3.4.0. (Tom White via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227408 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 00:13:39 +00:00
Arun Murthy
4ab6ae543e MAPREDUCE-3608. Fixed compile issue with MAPREDUCE-3522. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1225726 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-30 02:24:14 +00:00
Arun Murthy
743a5b78e3 MAPREDUCE-3522. Ensure queues inherit ACLs from parent if they aren't explicitly specified. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1225618 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-29 19:06:34 +00:00
Arun Murthy
f025652fda MAPREDUCE-3547. Added a bunch of unit tests for the the RM/NM webservices. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1225463 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-29 08:06:04 +00:00
Vinod Kumar Vavilapalli
cd90b82227 MAPREDUCE-3399. Modifying ContainerLocalizer to send a heartbeat to NM immediately after downloading a resource instead of always waiting for a second. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1224970 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-27 18:26:21 +00:00
Siddharth Seth
8fa0a3c737 MAPREDUCE-3567. Extraneous JobConf objects in AM heap. Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1222498 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-22 22:34:40 +00:00
Vinod Kumar Vavilapalli
ef9b974879 MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221950 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 23:47:54 +00:00
Vinod Kumar Vavilapalli
8626f1bc6c MAPREDUCE-3588. Fixed bin/yarn which was broken by MAPREDUCE-3366 so that yarn daemons can start. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221533 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 00:06:06 +00:00
Vinod Kumar Vavilapalli
e7543b944c MAPREDUCE-3339. Fixed MR AM to stop considering node blacklisting after the number of nodes blacklisted crosses a threshold. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221523 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-20 23:27:02 +00:00
Vinod Kumar Vavilapalli
4e1d5a0d71 MAPREDUCE-3391. Making a trivial change to correct a log message in DistributedShell app's AM. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221516 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-20 23:13:15 +00:00
Harsh J
2cf4aca02e MAPREDUCE-3579. ConverterUtils shouldn't include a port in a path from a url without a port (Contributed by atm) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1215528 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-17 17:10:12 +00:00
Vinod Kumar Vavilapalli
feb3a23c71 MAPREDUCE-3387. Fixed AM's tracking URL to always go through the proxy, even before the job started, so that it works properly with oozie throughout the job execution. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1215345 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-16 23:41:36 +00:00