big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,481 Commits 2 Branches 2 Tags 264 MiB
03ab24aa01
Commit Graph

292 Commits

Author SHA1 Message Date
Jason Lowe
03ab24aa01 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 2014-12-03 17:02:14 +00:00
Zhijie Shen
51af8d367d YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 2014-12-01 19:04:04 -08:00
Jason Lowe
0c588904f8 MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol interface implementation. Contributed by Rohith 2014-12-01 22:39:22 +00:00
Karthik Kambatla
a655973e78 Revert "MAPREDUCE-5785. Derive heap size or mapreduce.*.memory.mb automatically. (Gera Shegalov and Karthik Kambatla via kasha)" 
This reverts commit a4df9eed05.
 2014-11-25 16:24:26 -08:00
Karthik Kambatla
a4df9eed05 MAPREDUCE-5785. Derive heap size or mapreduce.*.memory.mb automatically. (Gera Shegalov and Karthik Kambatla via kasha) 2014-11-21 18:36:41 -08:00
Arun C. Murthy
43cd07b408 YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 2014-11-09 14:57:37 -08:00
Zhijie Shen
ed63b11646 MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du. 2014-11-01 00:47:57 -07:00
Vinod Kumar Vavilapalli
3f1441245d MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and TestMRTimelineEventHandling. Contributed by Zhijie Shen. 2014-10-29 09:57:39 -07:00
Zhijie Shen
6b2f11b54b MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server. Contributed by Robert Kanter. 2014-10-27 20:35:40 -07:00
Zhijie Shen
0f3b6900be YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 2014-10-23 21:56:03 -07:00
Jian He
0af1a2b5bc YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:22:07 -07:00
Jason Lowe
bbe80cdc7b MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai 2014-10-13 15:08:23 +00:00
Jason Lowe
2e789eb226 MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai 2014-10-07 14:10:39 +00:00
Zhijie Shen
376233cdd4 MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 09:28:47 -07:00
Zhijie Shen
6fe5c6b746 YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He. 2014-09-18 21:56:56 -07:00
Vinod Kumar Vavilapalli
14e2639fd0 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 2014-09-13 18:04:05 -07:00
Zhijie Shen
524a63e59e MAPREDUCE-6044. Fully qualified intermediate done dir path breaks per-user dir creation on Windows. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619863 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-22 17:06:23 +00:00
Jason Darrell Lowe
d51f81c3b6 MAPREDUCE-6036. TestJobEndNotifier fails intermittently in branch-2. Contributed by chang li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618691 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:18:22 +00:00
Eric Yang
08d6201b34 MAPREDUCE-6033. Updated access check for displaying job information 
(Yu Gao via Eric Yang)



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618648 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 16:13:56 +00:00
Zhijie Shen
84bc2fe402 MAPREDUCE-6032. Made MR jobs write job history files on the default FS when the current context’s FS is different. Contributed by Benjamin Zhitomirsky. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618269 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 20:17:45 +00:00
Jason Darrell Lowe
857d134fda MAPREDUCE-6014. New task status field in task attempts table can lead to an empty web page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616018 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 21:52:22 +00:00
Xuan Gong
e52f67e389 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 20:06:02 +00:00
Jian He
875592220f MAPREDUCE-5910. Make MR AM resync with RM in case of work-preserving RM-restart. Contributed by Rohith 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611434 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 18:46:20 +00:00
Karthik Kambatla
43a12f3c01 MAPREDUCE-5952. LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for mapOutIndex. (Gera Shegalov via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611196 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 21:24:51 +00:00
Zhijie Shen
64306aa1b5 MAPREDUCE-5956. Made MR AM not use maxAttempts to determine if the current attempt is the last retry. Contributed by Wangda Tan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609649 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-11 08:45:40 +00:00
Vinod Kumar Vavilapalli
4b2ded8202 MAPREDUCE-5517. Fixed MapReduce ApplicationMaster to not validate reduce side resource configuration for deciding uber-mode on map-only jobs. Contributed by Siqi Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608595 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:14:01 +00:00
Zhijie Shen
c6a09d2110 MAPREDUCE-5900. Changed to the interpret container preemption exit code as a task attempt killing event. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607512 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 01:43:56 +00:00
Vinod Kumar Vavilapalli
e285b98f0f YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:43:22 +00:00
Karthik Kambatla
7b9c074b76 MAPREDUCE-5844. Add a configurable delay to reducer-preemption. (Maysam Yabandeh via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603957 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 17:22:56 +00:00
Jian He
0abddac476 MAPREDUCE-5924. Changed TaskAttemptImpl to ignore TA_COMMIT_PENDING event at COMMIT_PENDING state. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603338 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 00:51:39 +00:00
Andrew Wang
4a839f0cc1 Fix CRLFs in TestKillAMPreemptionPolicy.java 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596058 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-19 21:18:55 +00:00
Devarajulu K
f74e446355 MAPREDUCE-5867. Fix NPE in KillAMPreemptionPolicy related to ProportionalCapacityPreemptionPolicy. Contributed by Sunil G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595754 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-19 06:42:29 +00:00
Jason Darrell Lowe
bb7ce82816 MAPREDUCE-5812. Make job context available to OutputCommitter.isRecoverySupported(). Contributed by Mohammad Kamrul Islam 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590668 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-28 15:18:02 +00:00
Jason Darrell Lowe
283a3a246a MAPREDUCE-5835. Killing Task might cause the job to go to ERROR state. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590168 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 22:26:24 +00:00
Jason Darrell Lowe
5492149c3c MAPREDUCE-5749. TestRMContainerAllocator#testReportedAppProgress Failed. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590046 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 14:53:17 +00:00
Jason Darrell Lowe
6957745c2c MAPREDUCE-5841. uber job doesn't terminate on getting mapred job kill. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589524 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-23 21:53:06 +00:00
Chris Nauroth
4a91b876db MAPREDUCE-5833. TestRMContainerAllocator fails ocassionally. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589248 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-22 18:44:54 +00:00
Chris Nauroth
06c15b6a3e MAPREDUCE-5852. Prepare MapReduce codebase for JUnit 4.11. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589006 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-21 23:44:41 +00:00
Jason Darrell Lowe
8d569c2220 MAPREDUCE-4937. MR AM handles an oversized split metainfo file poorly. Contributed by Eric Payne 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588559 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-18 20:33:28 +00:00
Jason Darrell Lowe
26421dd7dc MAPREDUCE-5825. Provide diagnostics for reducers killed during ramp down. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586479 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 22:02:48 +00:00
Vinod Kumar Vavilapalli
6015e95941 MAPREDUCE-5795. Fixed MRAppMaster to record the correct job-state after it recovers from a commit during a previous attempt. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581180 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 02:00:39 +00:00
Vinod Kumar Vavilapalli
8a06949df8 MAPREDUCE-5806. Fixed a bug in MRAppMaster so as to enable users to properly override HADOOP_ROOT_LOGGER or HADOOP_CLIENT_OPTS. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580100 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:32:57 +00:00
Vinod Kumar Vavilapalli
96e0ca2d27 YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. 
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578135 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-16 18:32:05 +00:00
Jason Darrell Lowe
8a2a741595 MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time. Contributed by Rushabh S Shah 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577692 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 20:33:03 +00:00
Jason Darrell Lowe
d9f723ed74 MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM. Contributed by Rohith 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577647 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 18:15:50 +00:00
Jason Darrell Lowe
cbf1c72d1b MAPREDUCE-5778. JobSummary does not escape newlines in the job name. Contributed by Akira AJISAKA 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576940 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 21:57:34 +00:00
Vinod Kumar Vavilapalli
94b29b3348 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572711 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 19:23:55 +00:00
Jason Darrell Lowe
658b1bf561 MAPREDUCE-5754. Preserve Job diagnostics in history. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572269 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 21:32:34 +00:00
Jonathan Turner Eagles
19f870858e MAPREDUCE-5688. TestStagingCleanup fails intermittently with JDK7 (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570631 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 16:43:11 +00:00
Zhijie Shen
b36cc29073 MAPREDUCE-5743. Fixed the test failure in TestRMContainerAllocator. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565087 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 06:20:41 +00:00