big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7,264 Commits 2 Branches 2 Tags 264 MiB
bc00bc966b
Commit Graph

954 Commits

Author SHA1 Message Date
Jian He
bc00bc966b YARN-1830. Fixed TestRMRestart#testQueueMetricsOnRMRestart failure due to race condition when app is submitted. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578486 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 18:03:06 +00:00
Jonathan Turner Eagles
1190f19ddd YARN-1845. Elapsed time for failed tasks that never started is wrong (Rushabh S Shah via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578459 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 16:53:17 +00:00
Jonathan Turner Eagles
f8e439105c YARN-1845. Elapsed time for failed tasks that never started is wrong (Rushabh S Shah via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578457 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 16:50:56 +00:00
Vinod Kumar Vavilapalli
96e0ca2d27 YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. 
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578135 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-16 18:32:05 +00:00
Jonathan Turner Eagles
fbdab6fab1 YARN-1520. update capacity scheduler docs to include necessary parameters (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577777 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-15 04:08:25 +00:00
Jonathan Turner Eagles
95b7f50813 YARN-1833. TestRMAdminService Fails in trunk and branch-2 (Mit Desais via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577737 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 21:58:42 +00:00
Zhijie Shen
b3ea4aebff YARN-1717. Enabled periodically discarding old data in LeveldbTimelineStore. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577693 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 20:35:00 +00:00
Vinod Kumar Vavilapalli
d9cdcb9474 YARN-1658. Modified web-app framework to let standby RMs redirect web-service calls to the active RM. Contributed by Cindy Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577408 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 02:39:59 +00:00
Christopher Douglas
53790d3300 YARN-1771. Reduce the number of NameNode operations during localization of 
public resources using a cache. Contributed by Sangjin Lee



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577391 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 00:30:35 +00:00
Zhijie Shen
6af044bb75 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577052 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 07:42:29 +00:00
Jason Darrell Lowe
135700a611 YARN-1789. ApplicationSummary does not escape newlines in the app name. Contributed by Tsuyoshi OZAWA 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576960 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 22:25:01 +00:00
Vinod Kumar Vavilapalli
197217c95d YARN-1816. Fixed ResourceManager to get RMApp correctly handle ATTEMPT_FINISHED event at ACCEPTED state that can happen after RM restarts. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576911 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 21:18:55 +00:00
Vinod Kumar Vavilapalli
4de17c6052 YARN-1812. Fixed ResourceManager to synchrously renew tokens after recovery and thus recover app itself synchronously and avoid races with resyncing NodeManagers. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576843 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 18:25:33 +00:00
Arun Murthy
4ce0e4bf2e YARN-1444. Fix CapacityScheduler to deal with cases where applications specify host/rack requests without off-switch request. Contributed by Wangda Tan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576751 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 14:36:27 +00:00
Vinod Kumar Vavilapalli
8aab8533a1 YARN-1800. Fixed NodeManager to gracefully handle RejectedExecutionException in the public-localizer thread-pool. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576545 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 23:33:56 +00:00
Karthik Kambatla
477ed62b3f YARN-1821. NPE on registerNodeManager if the request has containers for UnmanagedAMs (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576525 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 22:18:54 +00:00
Vinod Kumar Vavilapalli
0edda25373 YARN-1764. Modified YarnClient to correctly handle failover of ResourceManager after the submitApplication call goes through. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576160 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 00:52:03 +00:00
Vinod Kumar Vavilapalli
e4190b2a97 YARN-1788. Fixed a bug in ResourceManager to set the apps-completed and apps-killed metrics correctly for killed applications. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576072 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 20:36:34 +00:00
Jian He
7da07461ff YARN-1793. Fixed ClientRMService#forceKillApplication not killing unmanaged application. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576023 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-10 17:58:55 +00:00
Sanford Ryza
e2530de4ea YARN-1790. Fair Scheduler UI not showing apps table (bc Wong via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575487 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 05:12:45 +00:00
Vinod Kumar Vavilapalli
874cc955e0 YARN-1787. Fixed help messages for applicationattempt and container sub-commands in bin/yarn. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575482 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 04:49:52 +00:00
Vinod Kumar Vavilapalli
8497b870af YARN-1410. Added tests to validate that clients can fail-over to a new RM 
after getting an application-ID but before submission and can still submit
to the newly active RM with no issues. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575478 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 04:43:29 +00:00
Vinod Kumar Vavilapalli
0b1304d098 YARN-1781. Modified NodeManagers to allow admins to specify max disk utilization for local disks so as to be able to offline full disks. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575463 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 00:52:06 +00:00
Vinod Kumar Vavilapalli
1c4047b0e4 YARN-1783. Fixed a bug in NodeManager's status-updater that was losing completed container statuses when NodeManager is forced to resync by the ResourceManager. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575437 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 22:36:47 +00:00
Karthik Kambatla
b06cc16f7d YARN-1774. FS: Submitting to non-leaf queue throws NPE. (Anubhav Dhoot and Karthik Kambatla via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575415 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 21:34:19 +00:00
Karthik Kambatla
9b15c5b11a YARN-1525. Web UI should redirect to active RM when HA is enabled. (Cindy Li via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575166 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 04:34:16 +00:00
Vinod Kumar Vavilapalli
f4eb6ddec9 YARN-1780. Improved logging in the Timeline client and server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575141 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 03:52:53 +00:00
Haohui Mai
ad61eec072 HADOOP-10386. Log proxy hostname in various exceptions being thrown in a HA setup. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574716 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 23:09:47 +00:00
Vinod Kumar Vavilapalli
d963b3cd52 YARN-1761. Modified RMAdmin CLI to check whether HA is enabled or not before it executes any of the HA admin related commands. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574661 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 21:07:50 +00:00
Jian He
8cd23c5b48 YARN-1752. Fixed ApplicationMasterService to reject unregister request if AM did not register before. Contributed by Rohith Sharma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574623 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 19:04:58 +00:00
Karthik Kambatla
2e7c512270 YARN-1785. FairScheduler treats app lookup failures as ERRORs. (bc Wong via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574604 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 18:22:25 +00:00
Ravi Prakash
240b4e9a7f YARN-1768. Fixed error message being too verbose when killing a non-existent application. Contributed by Tsuyoshi Ozawa 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574401 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 09:29:16 +00:00
Vinod Kumar Vavilapalli
82e8c8bd71 YARN-1766. Fixed a bug in ResourceManager to use configuration loaded from the configuration-provider when booting up. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574252 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 23:35:20 +00:00
Vinod Kumar Vavilapalli
88245b6a41 YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574190 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 20:39:06 +00:00
Vinod Kumar Vavilapalli
40464fba22 YARN-1730. Implemented simple write-locking in the LevelDB based timeline-store. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574145 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 17:32:41 +00:00
Zhijie Shen
33714d9ad6 YARN-1729. Made TimelineWebServices deserialize the string primary- and secondary-filters param into the JSON-compatible object. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573825 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 00:20:43 +00:00
Vinod Kumar Vavilapalli
5bf449e402 YARN-1748. Excluded core-site.xml from hadoop-yarn-server-tests package's jar and thus avoid breaking downstream tests. Contributed by Sravya Tirukkovalur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573795 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 23:09:08 +00:00
Vinod Kumar Vavilapalli
c26bdddcc3 YARN-1675. Added the previously missed new file. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573736 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 20:39:01 +00:00
Vinod Kumar Vavilapalli
b35051c0f0 YARN-1765. Added test cases to verify that killApplication API works across ResourceManager failover. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573735 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 20:37:54 +00:00
Vinod Kumar Vavilapalli
43f9bf2f25 YARN-1704. Modified LICENSE and NOTICE files to reflect newly used levelDB related libraries. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573702 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 19:52:48 +00:00
Vinod Kumar Vavilapalli
a60434368c YARN-1758. Fixed ResourceManager to not mandate the presence of site specific configuration files and thus fix failures in downstream tests. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573695 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 19:41:52 +00:00
Karthik Kambatla
8caeb0af9d YARN-1760. Addendum patch. TestRMAdminService assumes Capacity Scheduler. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573134 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-01 08:46:29 +00:00
Karthik Kambatla
1ba26e3e68 YARN-1528. Allow setting auth for ZK connections. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573014 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 17:34:08 +00:00
Vinod Kumar Vavilapalli
94b29b3348 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572711 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 19:23:55 +00:00
Karthik Kambatla
44c3bbf872 YARN-1429. *nix: Allow a way for users to augment classpath of YARN daemons. (Jarek Jarcec Cecho via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572405 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 02:18:34 +00:00
Zhijie Shen
faddccc216 YARN-1301. Added the INFO level log of the non-empty blacklist additions and removals inside ApplicationMasterService. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572400 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 02:07:16 +00:00
Vinod Kumar Vavilapalli
0fbc7fe816 YARN-1588. Enhanced RM and the scheduling protocol to also send NMTokens of transferred containers from previous app-attempts to new AMs after YARN-1490. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572230 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:20:07 +00:00
Junping Du
c7142e7761 YARN-1561. Fix a generic type warning in FairScheduler. (Chen He via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571924 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 05:14:51 +00:00
Karthik Kambatla
5a42e1b7c3 YARN-1760. TestRMAdminService assumes CapacityScheduler. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571777 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 18:06:45 +00:00
Vinod Kumar Vavilapalli
ad70f26b1f YARN-1734. Fixed ResourceManager to update the configurations when it transits from standby to active mode so as to assimilate any changes that happened while it was in standby mode. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571539 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 02:07:19 +00:00