big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,004 Commits 2 Branches 2 Tags 264 MiB
a8acdd65b3
Commit Graph

517 Commits

Author SHA1 Message Date
Xuan Gong
eeb4acd955 YARN-2212: ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616892 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:38:24 +00:00
Jian He
8437df8ba9 YARN-2008. Fixed CapacityScheduler to calculate headroom based on max available capacity instead of configured max capacity. Contributed by Craig Welch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616580 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 20:00:04 +00:00
Junping Du
b47f65214c YARN-2298. Move TimelineClient to yarn-common project (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616100 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 05:48:32 +00:00
Junping Du
1141edc424 YARN-2051. Fix bug in PBimpls and add more unit tests with reflection. (Contributed by Binglin Chang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615025 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 04:16:08 +00:00
Xuan Gong
e52f67e389 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 20:06:02 +00:00
Zhijie Shen
d6532d3a77 YARN-2247. Made RM web services authenticate users via kerberos and delegation token. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613821 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 17:55:06 +00:00
Zhijie Shen
70dededdc9 YARN-2270. Made TestFSDownload#testDownloadPublicWithStatCache be skipped when there’s no ancestor permissions. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612460 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 04:58:00 +00:00
Xuan Gong
f1b831ccfb YARN-2208. AMRMTokenManager need to have a way to roll over AMRMToken. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611820 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 21:46:29 +00:00
Jian He
40e1bb9d31 YARN-2264. Fixed a race condition in DrainDispatcher which may cause random test failures. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611126 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 18:25:05 +00:00
Vinod Kumar Vavilapalli
a6cd712ca7 YARN-2228. Augmented TimelineServer to load pseudo authentication filter when authentication = simple. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610575 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 04:33:48 +00:00
Xuan Gong
c0439b732c YARN-2269. Remove external links from YARN UI. Contributed by Craig Welch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609590 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 23:31:55 +00:00
Jian He
50be8561a7 YARN-2088. Fixed a bug in GetApplicationsRequestPBImpl#mergeLocalToBuilder. Contributed by Binglin Chang 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609584 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 22:36:13 +00:00
Junping Du
4074843fea YARN-2251. Avoid negative elapsed time in JHS/MRAM web UI and services (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607833 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-04 11:18:05 +00:00
Vinod Kumar Vavilapalli
075ff276ca YARN-1713. Added get-new-app and submit-app functionality to RM web services. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607216 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 00:23:07 +00:00
Ravi Prakash
f084170a0a YARN 2163. WebUI: Order of AppId in apps table should be consistent with ApplicationId.compareTo() 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605964 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 06:43:46 +00:00
Vinod Kumar Vavilapalli
e285b98f0f YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:43:22 +00:00
Vinod Kumar Vavilapalli
424fd9494f YARN-1368. Added core functionality of recovering container state into schedulers after ResourceManager Restart so as to preserve running work in the cluster. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601303 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-09 03:09:21 +00:00
Junping Du
790cbbf3e2 YARN-2103. Inconsistency between viaProto flag and initial value of SerializedExceptionProto.Builder (Contributed by Binglin Chang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599115 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-02 08:15:30 +00:00
Chris Nauroth
d599adcc6a YARN-1868. YARN status web ui does not show correctly in IE 11. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598686 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 18:07:23 +00:00
Karthik Kambatla
75112e97b4 YARN-2010. Document yarn.resourcemanager.zk-auth and its scope. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598636 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:32:23 +00:00
Karthik Kambatla
49a3a0cd0c YARN-2054. Better defaults for YARN ZK configs for retries and retry-inteval when HA is enabled. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598630 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:24:49 +00:00
Vinod Kumar Vavilapalli
cd24e955cf YARN-2112. Fixed yarn-common's pom.xml to include jackson dependencies so that both Timeline Server and client can access them. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598373 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 19:06:36 +00:00
Zhijie Shen
001078e067 YARN-2107. Refactored timeline classes into o.a.h.y.s.timeline package. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598094 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-28 18:09:04 +00:00
Vinod Kumar Vavilapalli
af8b09af75 YARN-1936. Added security support for the Timeline Client. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597153 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 18:47:57 +00:00
Vinod Kumar Vavilapalli
b867b69556 YARN-2049. Added delegation-token support for the Timeline Server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597130 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 17:09:47 +00:00
Vinod Kumar Vavilapalli
fdb5870d49 YARN-1938. Added kerberos login for the Timeline Server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596710 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-21 23:20:08 +00:00
Jason Darrell Lowe
64ec743e6c YARN-2050. Fix LogCLIHelpers to create the correct FileContext. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596310 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-20 16:19:55 +00:00
Junping Du
461ded8bd4 YARN-2066. Wrong field is referenced in GetApplicationsRequestPBImpl#mergeLocalToBuilder (Contributed by Hong Zhiguo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595413 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-17 01:43:15 +00:00
Jason Darrell Lowe
b0e80d1a47 MAPREDUCE-5814. fat jar with *-default.xml may fail when mapreduce.job.classloader=true. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594326 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 18:55:05 +00:00
Jian He
4f6493bcf9 YARN-2016. Fix a bug in GetApplicationsRequestPBImpl to add the missed fields to proto. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594085 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-12 20:45:21 +00:00
Vinod Kumar Vavilapalli
58c81d56e9 YARN-1962. Changed Timeline Service client configuration to be off by default given the non-readiness of the feature yet. Contributed by Mohammad Kamrul Islam. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593750 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 00:15:12 +00:00
Zhijie Shen
0c437e90b8 YARN-1701. Improved default paths of the timeline store and the generic history store. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593481 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-09 04:23:27 +00:00
Haohui Mai
d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-02 22:26:02 +00:00
Sanford Ryza
faffedeb24 YARN-483. Improve documentation on log aggregation in yarn-default.xml (Akira Ajisaka via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590150 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 21:14:03 +00:00
Jason Darrell Lowe
d667df4ed0 YARN-1932. Javascript injection on the job status page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588572 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-18 21:41:37 +00:00
Junping Du
36ed8c74ea YARN-1870. FileInputStream is not closed in ProcfsBasedProcessTree#constructProcessSMAPInfo. (Fengdong Yu via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588324 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 18:05:43 +00:00
Thomas Graves
ad9edb02b9 YARN-1931. Private API change in YARN-1824 in 2.4 broke compatibility with previous releases (Sandy Ryza via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588281 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 15:23:22 +00:00
Vinod Kumar Vavilapalli
cde7b17566 YARN-1914. Fixed resource-download on NodeManagers to skip permission verification of public cache files in Windows+local file-system environment. Contribued by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586434 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 19:28:31 +00:00
Steve Loughran
b8f250a99d HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585932 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 11:11:20 +00:00
Karthik Kambatla
245012a9d9 YARN-1757. NM Recovery. Auxiliary service support. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585783 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-08 17:15:58 +00:00
Chris Nauroth
4eacde67f8 YARN-1905. TestProcfsBasedProcessTree must only run on Linux. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585376 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-07 03:04:09 +00:00
Vinod Kumar Vavilapalli
dce1d20383 YARN-1452. Added documentation about the configuration and usage of generic application history and the timeline data service. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581656 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 02:52:58 +00:00
Vinod Kumar Vavilapalli
09f383254c YARN-1850. Introduced the ability to optionally disable sending out timeline-events in the TimelineClient. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581189 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 02:22:39 +00:00
Jonathan Turner Eagles
2d89dcb121 YARN-1670. aggregated log writer can write more log data then it says is the log length (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580957 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 18:11:48 +00:00
Vinod Kumar Vavilapalli
06579878df YARN-1775. Enhanced ProcfsBasedProcessTree to optionally add the ability to use smaps for obtaining used memory information. Contributed by Rajesh Balamohan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580087 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:01:59 +00:00
Vinod Kumar Vavilapalli
2826a35469 YARN-1670. Fixed a bug in log-aggregation that can cause the writer to write more log-data than the log-length that it records. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580005 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 18:35:54 +00:00
Vinod Kumar Vavilapalli
4224e61340 YARN-1811. Fixed AMFilters in YARN to correctly accept requests from either web-app proxy or the RMs when HA is enabled. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579877 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 04:20:46 +00:00
Vinod Kumar Vavilapalli
81a456e638 YARN-1591. Fixed AsyncDispatcher to handle interrupts on shutdown in a sane manner and thus fix failure of TestResourceTrackerService. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578628 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 22:17:43 +00:00
Vinod Kumar Vavilapalli
8314674947 YARN-1685. Fixed few bugs related to handling of containers' log-URLs on ResourceManager and history-service. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578602 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 21:36:21 +00:00
Jonathan Turner Eagles
7944eab64a YARN-1136. Replace junit.framework.Assert with org.junit.Assert (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578539 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 20:03:35 +00:00
Jonathan Turner Eagles
1190f19ddd YARN-1845. Elapsed time for failed tasks that never started is wrong (Rushabh S Shah via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578459 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 16:53:17 +00:00
Jonathan Turner Eagles
f8e439105c YARN-1845. Elapsed time for failed tasks that never started is wrong (Rushabh S Shah via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578457 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 16:50:56 +00:00
Vinod Kumar Vavilapalli
96e0ca2d27 YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. 
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578135 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-16 18:32:05 +00:00
Zhijie Shen
b3ea4aebff YARN-1717. Enabled periodically discarding old data in LeveldbTimelineStore. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577693 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 20:35:00 +00:00
Vinod Kumar Vavilapalli
d9cdcb9474 YARN-1658. Modified web-app framework to let standby RMs redirect web-service calls to the active RM. Contributed by Cindy Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577408 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 02:39:59 +00:00
Christopher Douglas
53790d3300 YARN-1771. Reduce the number of NameNode operations during localization of 
public resources using a cache. Contributed by Sangjin Lee



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577391 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 00:30:35 +00:00
Zhijie Shen
6af044bb75 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577052 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 07:42:29 +00:00
Vinod Kumar Vavilapalli
0b1304d098 YARN-1781. Modified NodeManagers to allow admins to specify max disk utilization for local disks so as to be able to offline full disks. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575463 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 00:52:06 +00:00
Karthik Kambatla
9b15c5b11a YARN-1525. Web UI should redirect to active RM when HA is enabled. (Cindy Li via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575166 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 04:34:16 +00:00
Haohui Mai
ad61eec072 HADOOP-10386. Log proxy hostname in various exceptions being thrown in a HA setup. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574716 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 23:09:47 +00:00
Vinod Kumar Vavilapalli
88245b6a41 YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574190 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 20:39:06 +00:00
Vinod Kumar Vavilapalli
a60434368c YARN-1758. Fixed ResourceManager to not mandate the presence of site specific configuration files and thus fix failures in downstream tests. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573695 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 19:41:52 +00:00
Vinod Kumar Vavilapalli
94b29b3348 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572711 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 19:23:55 +00:00
Vinod Kumar Vavilapalli
0fbc7fe816 YARN-1588. Enhanced RM and the scheduling protocol to also send NMTokens of transferred containers from previous app-attempts to new AMs after YARN-1490. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572230 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:20:07 +00:00
Vinod Kumar Vavilapalli
e167e585e9 YARN-1749. Updated application-history related configs to reflect the latest reality and to be consistently named. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570948 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 02:04:58 +00:00
Vinod Kumar Vavilapalli
84425fb435 YARN-1687. Renamed user-facing records for the timeline-service to be simply named after 'timeline' instead of 'apptimeline'. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570922 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-22 20:55:06 +00:00
Vinod Kumar Vavilapalli
9da9f7d4d8 YARN-1666. Modified RM HA handling of include/exclude node-lists to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569856 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:52:57 +00:00
Vinod Kumar Vavilapalli
990cffdcfa YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568501 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:02 +00:00
Zhijie Shen
3587b6774c YARN-1706. Created an utility method to dump timeline records to JSON strings. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566982 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-11 04:39:37 +00:00
Vinod Kumar Vavilapalli
1fa6ab249b YARN-1459. Changed ResourceManager to depend its service initialization on the configuration-provider mechanism during startup too. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566791 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 22:50:15 +00:00
Zhijie Shen
23b2e43f5d YARN-1635. Implemented a Leveldb based ApplicationTimelineStore. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565868 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 02:15:46 +00:00
Vinod Kumar Vavilapalli
8b2336fcef YARN-1660. Simplified the RM HA configuration to accept and be able to simply depend just on 
configuration properties of the form yarn.resourcemanager.hostname.RMID and use the default
ports for all service addresses. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 03:18:39 +00:00
Vinod Kumar Vavilapalli
30294a2196 YARN-1665. Simplify the configuration of RM HA by having better default values. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565517 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:39:32 +00:00
Vinod Kumar Vavilapalli
484faadffe YARN-1636. Augmented Application-history server's web-services to also expose new APIs for retrieving and storing timeline information. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564829 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 17:02:26 +00:00
Zhijie Shen
ebe0c17a95 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564633 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:55:00 +00:00
Vinod Kumar Vavilapalli
39ecc46ff6 YARN-1285. Changed the default value of yarn.acl.enable in yarn-default.xml to be consistent with what exists (false) in the code and documentation. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564190 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-04 04:21:40 +00:00
Vinod Kumar Vavilapalli
eac066b13f YARN-1659. Defined the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564037 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:07:01 +00:00
Vinod Kumar Vavilapalli
3e7d56678c YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564032 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:53:46 +00:00
Vinod Kumar Vavilapalli
ca72e11158 YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564002 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:07:55 +00:00
Karthik Kambatla
88d8ea9509 HADOOP-10085. CompositeService should allow adding services while being inited. (Steve Loughran via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563694 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-02 19:51:39 +00:00
Vinod Kumar Vavilapalli
6e8c1bf200 YARN-1633. Defined user-facing entity, entity-info and event objects related to Application Timeline feature. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563356 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 00:09:53 +00:00
Jason Darrell Lowe
4de2bd3fbf YARN-1600. RM does not startup when security is enabled without spnego configured. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562482 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 15:30:04 +00:00
Karthik Kambatla
8006467e58 YARN-1642. RMDTRenewer#getRMClient should use ClientRMProxy (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561990 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 08:26:33 +00:00
Suresh Srinivas
33a47743a5 HADOOP-10255. Rename HttpServer to HttpServer2 to retain older HttpServer in branch-2 for compatibility. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561959 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 07:32:52 +00:00
Vinod Kumar Vavilapalli
df000e69e8 YARN-1625. Fixed RAT warnings after YARN-321 merge. Contributed by Shinichi Yamashita. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561458 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 07:20:51 +00:00
Vinod Kumar Vavilapalli
cbee889711 YARN-321. Merging YARN-321 branch to trunk. 
svn merge ../branches/YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561452 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 04:51:10 +00:00
Karthik Kambatla
482993ac53 YARN-1573. ZK store should use a private password for root-node-acls. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560594 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 06:22:05 +00:00
Sanford Ryza
163348701c YARN-1623. Include queue name in RegisterApplicationMasterResponse (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560545 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:25:03 +00:00
Jason Darrell Lowe
d1963ad4bd MAPREDUCE-5693. Restore MRv1 behavior for log flush. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560148 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-21 19:23:34 +00:00
Sanford Ryza
67cbde3008 YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559050 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 08:56:22 +00:00
Jason Darrell Lowe
a6ea460a91 MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j (syslog). Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558948 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 22:56:09 +00:00
Karthik Kambatla
3928de29fc YARN-1606. Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558882 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 18:54:49 +00:00
Vinod Kumar Vavilapalli
7c8b654ba5 YARN-888. Cleaned up POM files so that non-leaf modules don't include any dependencies and thus compact the dependency list for leaf modules. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557801 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-13 18:34:02 +00:00
Vinod Kumar Vavilapalli
25bc68d15e YARN-1041. Added the ApplicationMasterProtocol API for applications to use the ability in ResourceManager to optionally not kill containers when the ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557318 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 07:07:17 +00:00
Karthik Kambatla
6608b75829 YARN-1574. RMDispatcher should be reset on transition to standby. (Xuan Gong via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557248 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 20:15:00 +00:00
Vinod Kumar Vavilapalli
1393581bce YARN-1490. Introduced the ability to make ResourceManager optionally not kill all containers when an ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557143 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 15:04:39 +00:00
Chris Nauroth
8b2c8a657e YARN-1138. yarn.application.classpath is set to point to etc., which does not work on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556978 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-09 23:04:18 +00:00
Vinod Kumar Vavilapalli
4931545f76 YARN-1482. Modified WebApplicationProxy to make it work across ResourceManager fail-over. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556380 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 22:16:30 +00:00
Vinod Kumar Vavilapalli
c3cc855d27 YARN-1029. Added embedded leader election in the ResourceManager. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556103 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-07 01:56:11 +00:00
Karthik Kambatla
2cddd21db9 YARN-1559. Race between ServerRMProxy and ClientRMProxy setting RMProxy#INSTANCE. (kasha and vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555970 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 18:40:15 +00:00
Vinod Kumar Vavilapalli
460ac8cb50 YARN-1121. Addendum patch. Fixed AsyncDispatcher hang issue during stop due to a race condition caused by the previous patch. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554344 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-31 01:10:01 +00:00
Karthik Kambatla
1152e23ed0 YARN-1463. Tests should avoid starting http-server where possible or creates spnego keytab/principals (vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553790 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-28 06:55:30 +00:00
Karthik Kambatla
624703ed7b YARN-1028. Addendum patch. Added FailoverProxyProvider capability to ResourceManager to help with RM failover. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552920 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-21 16:29:10 +00:00
Vinod Kumar Vavilapalli
ca125153b3 YARN-1028. Added FailoverProxyProvider capability to ResourceManager to help with RM failover. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551739 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 22:32:38 +00:00
Vinod Kumar Vavilapalli
b774d7b3de YARN-1446. Changed client API to retry killing application till RM acknowledges so as to account for RM crashes/failover. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551444 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 02:16:20 +00:00
Vinod Kumar Vavilapalli
49ad07af97 YARN-312. Introduced ResourceManagerAdministrationProtocol changes to support changing resources on node. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551403 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 23:36:16 +00:00
Vinod Kumar Vavilapalli
5a1b33507b YARN-1145. Fixed a potential file-handle leak in the web interface for displaying aggregated logs. Contributed by Rohith Sharma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551326 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-16 19:27:48 +00:00
Vinod Kumar Vavilapalli
1182ca04d4 YARN-1485. Modified RM HA configuration validation to also ensure that service-address configuration are configured for every RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550854 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-13 23:54:44 +00:00
Vinod Kumar Vavilapalli
8a79be36bd YARN-1325. Modified RM HA configuration validation to also ensure that multiple RMs are configured. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550524 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-12 20:50:11 +00:00
Jonathan Turner Eagles
f3db30ccd8 YARN-1491. Upgrade JUnit3 TestCase to JUnit 4 (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550204 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-11 17:50:26 +00:00
Sanford Ryza
9c4168f45a YARN-1448. AM-RM protocol changes to support container resizing (Wangda Tan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1549627 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-09 17:44:29 +00:00
Vinod Kumar Vavilapalli
305ae48136 YARN-1378. Implemented a cleaner of old finished applications from the RM state-store. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548990 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-08 04:17:51 +00:00
Sanford Ryza
1cdeb83167 YARN-1447: Forgot to add new files 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548320 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-05 22:16:48 +00:00
Vinod Kumar Vavilapalli
9c95015bb4 YARN-895. Changed RM state-store to not crash immediately if RM restarts while the state-store is down. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547538 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-03 19:28:02 +00:00
Vinod Kumar Vavilapalli
c58ae266e9 YARN-1318. Promoted AdminService to an Always-On service and merged it into RMHAProtocolService. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547212 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 22:18:44 +00:00
Jing Zhao
2214871d91 HDFS-5545. Allow specifying endpoints for listeners in HttpServer. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546151 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-27 18:20:14 +00:00
Bikas Saha
f7efa0b86e YARN-1411. HA config shouldn't affect NodeManager RPC addresses (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542367 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-15 19:02:30 +00:00
Bikas Saha
797159bbd4 YARN-1222. Make improvements in ZKRMStateStore for fencing (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541995 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-14 17:53:25 +00:00
Sanford Ryza
72c6d6255a YARN-1387. RMWebServices should use ClientRMService for filtering applications (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540851 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-11 21:49:43 +00:00
Vinod Kumar Vavilapalli
e28015ed1b YARN-1121. Changed ResourceManager's state-store to drain all events on shut-down. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540232 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-09 00:55:21 +00:00
Luke Lu
6990355e57 YARN-311. RM/scheduler support for dynamic resource configuration. (Junping Du via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539134 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-05 21:23:53 +00:00
Bikas Saha
03510d00f4 YARN-1068. Add admin support for HA operations (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536888 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 21:03:54 +00:00
Sanford Ryza
defa7af4d3 YARN-1109. Demote NodeManager "Sending out status for container" logs to debug (haosdent via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536573 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 00:54:12 +00:00
Bikas Saha
116b459d22 YARN-1305. RMHAProtocolService#serviceInit should handle HAUtil's IllegalArgumentException (Tsuyoshi Ozawa via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534884 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-23 01:10:17 +00:00
Vinod Kumar Vavilapalli
9b9ddf29e2 YARN-1283. Fixed RM to give a fully-qualified proxy URL for an application so that clients don't need to do scheme-mangling. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530819 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 22:49:06 +00:00
Vinod Kumar Vavilapalli
21181b6553 YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related 
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529662 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 18:43:36 +00:00
Arun Murthy
d55f3780fb YARN-1032. Fixed NPE in RackResolver. Contributed by Lohit Vijayarenu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529534 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-05 22:20:18 +00:00
Alejandro Abdelnur
80c5bffc4b YARN-1253. Changes to LinuxContainerExecutor to run containers as a single dedicated user in non-secure mode. (rvs via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529325 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 21:59:43 +00:00
Bikas Saha
cbab04727b YARN-1232. Configuration to support multiple RMs (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529251 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 18:40:18 +00:00
Chris Nauroth
6be30a7799 YARN-1219. FSDownload changes file suffix making FileUtil.unTar() throw exception. Contributed by Shanyu Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529084 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 05:59:07 +00:00
Hitesh Shah
6ff600d9e3 YARN-1131. logs command should return an appropriate error message if YARN application is still running. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529068 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 02:36:17 +00:00
Bikas Saha
8ebf37f369 YARN-1256. NM silently ignores non-existent service in StartContainerRequest (Xuan Gong via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529039 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 00:23:35 +00:00
Vinod Kumar Vavilapalli
1608d8b527 YARN-621. Changed YARN web app to not add paths that can cause duplicate additions of authenticated filters there by causing kerberos replay errors. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529030 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-03 23:21:32 +00:00
Vinod Kumar Vavilapalli
cb44a4b850 YARN-1260. Added webapp.http.address to yarn-default.xml so that default install with https enabled doesn't have broken link on NM UI. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528312 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 06:31:27 +00:00
Chris Nauroth
b2a7811fe7 YARN-1215. Yarn URL should include userinfo. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528233 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 22:28:02 +00:00
Jonathan Turner Eagles
5a15c392a1 YARN-819. ResourceManager and NodeManager should check for a minimum allowed version (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526660 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 19:55:26 +00:00
Colin McCabe
8a66e493ba HADOOP-9998. Provide methods to clear only part of the DNSToSwitchMapping. (Junping Du via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526567 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 15:24:30 +00:00
Siddharth Seth
10a4289ebf YARN-1229. Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526065 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-25 00:35:03 +00:00
Vinod Kumar Vavilapalli
78e995a37d YARN-1204. Added separate configuration properties for https for RM and NM without which servers enabled with https will also start on http ports. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5523. Added separate configuration properties for https for JHS without which even when https is enabled, it starts on http port itself. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525947 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-24 17:20:18 +00:00
Hitesh Shah
8628c1704b YARN-353. Add Zookeeper-based store implementation for RMStateStore. Contributed by Bikas Saha, Jian He and Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524829 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-19 20:39:10 +00:00
Bikas Saha
ead3dcc3de YARN-1027. Implement RMHAProtocolService (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523750 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 18:33:59 +00:00
Bikas Saha
ec010a2936 YARN-540. Race condition causing RM to potentially relaunch already unregistered AMs on RM restart (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523376 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-15 01:01:06 +00:00
Bikas Saha
49afc64cd4 YARN-1065. NM should provide AuxillaryService data to the container (Xuan Gong via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520135 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-04 20:42:11 +00:00
Vinod Kumar Vavilapalli
76cb07ee20 YARN-981. Fixed YARN webapp so that /logs servlet works like before. Addendum patch to fix bugs in the first patch. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519208 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-31 19:55:37 +00:00
Sanford Ryza
236b8530bd HADOOP-9918. Add addIfService to CompositeService (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519129 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-31 02:09:45 +00:00
Vinod Kumar Vavilapalli
6d100eb794 YARN-1117. Improved help messages for "yarn application" and "yarn node" commands. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519117 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-30 23:26:01 +00:00
Bikas Saha
bafd302208 YARN-771. AMRMClient support for resource blacklisting (Junping Du via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519107 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-30 22:58:41 +00:00
Vinod Kumar Vavilapalli
89fb4d8ffd YARN-707. Added user information also in the YARN ClientToken so that AMs can implement authorization based on incoming users. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518868 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-30 01:12:40 +00:00
Vinod Kumar Vavilapalli
febedd64e9 YARN-1080. Improved help message for "yarn logs" command. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518731 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-29 17:55:30 +00:00
Vinod Kumar Vavilapalli
914a0e5172 YARN-981. Fixed YARN webapp so that /logs servlet works like before. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518030 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-27 23:36:42 +00:00
Vinod Kumar Vavilapalli
5adba5597c YARN-1085. Addendum patch to address issues with the earlier patch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517721 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-26 22:59:09 +00:00
Vinod Kumar Vavilapalli
962da4dcc7 YARN-1074. Cleaned up YARN CLI application list to only display running applications by default. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517196 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-24 21:16:40 +00:00
Vinod Kumar Vavilapalli
e86036662c YARN-1085. Modified YARN and MR2 web-apps to do HTTP authentication in secure setup with kerberos. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517101 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-24 02:47:47 +00:00
Jason Darrell Lowe
c660339c09 Revert MAPREDUCE-5475 and YARN-707 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517097 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-24 01:15:37 +00:00
Jason Darrell Lowe
f1638fdf94 YARN-707. Add user info in the YARN ClientToken. Contributed by Vinod Kumar Vavilapalli 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517073 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-23 22:11:54 +00:00
Siddharth Seth
b776bd46ae YARN-1045. Improve toString implementation for PBImpls. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514185 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 07:20:14 +00:00
Arun Murthy
472d8bc983 YARN-1056. Remove dual use of string 'resourcemanager' in yarn.resourcemanager.connect.{max.wait.secs|retry_interval.secs}. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514135 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 02:35:48 +00:00
Vinod Kumar Vavilapalli
817a654346 YARN-945. Removed setting of AMRMToken's service from ResourceManager and changed client libraries do it all the time and correctly. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508232 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-29 23:17:39 +00:00
Alejandro Abdelnur
f179afc68d YARN-937. Fix unmanaged AM in non-secure/secure setup post YARN-701. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507706 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-27 20:04:38 +00:00
Kihwal Lee
605fafb4c0 YARN-968. RM admin commands don't work. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506675 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-24 19:43:48 +00:00
Vinod Kumar Vavilapalli
3ca3137179 YARN-926. Modified ContainerManagerProtcol APIs to take in requests for multiple containers. Contributed by Jian He. 
MAPREDUCE-5412. Update MR app to use multiple containers API of ContainerManager after YARN-926. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506391 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-24 03:41:03 +00:00
Jason Darrell Lowe
a42e459b9e MAPREDUCE-5356. Ability to refresh aggregated log retention period and check interval. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506226 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-23 19:20:57 +00:00
Arun Murthy
ac914f79bc YARN-918. Remove ApplicationAttemptId from RegisterApplicationMasterRequestProto. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504735 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-19 00:57:40 +00:00
Bikas Saha
33b5a81509 YARN-513. Create common proxy client for communicating with RM (Xuan Gong & Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503933 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 22:47:40 +00:00
Christopher Douglas
85f0efb68f YARN-569. Add support for requesting and enforcing preemption requests via 
a capacity monitor. Contributed by Carlo Curino, Chris Douglas


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502083 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-11 01:20:23 +00:00
Chris Nauroth
413bddf596 MAPREDUCE-4374. Fix child task environment variable config and add support for Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502046 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-10 22:59:08 +00:00
Hitesh Shah
e2cce7bd26 YARN-727, MAPREDUCE-5325. ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501599 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-09 23:01:01 +00:00
Vinod Kumar Vavilapalli
5e4f6ad1d9 YARN-791. Changed RM APIs and web-services related to nodes to ensure that both are consistent with each other. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500994 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-08 22:30:52 +00:00
Arun Murthy
f3a893b696 YARN-869. Move ResourceManagerAdministrationProtocol out of main YARN api. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496751 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-26 05:57:05 +00:00
Alejandro Abdelnur
e60fbbcc2e YARN-736. Add a multi-resource fair sharing metric. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496153 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-24 18:33:45 +00:00
Bikas Saha
37d7935a9d YARN-827. Need to make Resource arithmetic methods accessible^CJian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495533 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-21 18:28:57 +00:00
Jonathan Turner Eagles
90fd863cb8 YARN-478. fix coverage org.apache.hadoop.yarn.webapp.log (Aleksey Gorshkov via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495129 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-20 18:56:12 +00:00
Vinod Kumar Vavilapalli
3f809667bc YARN-854. Fixing YARN bugs that are failing applications in secure environment. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494845 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-20 03:41:36 +00:00
Chris Nauroth
51c85ca067 YARN-852. TestAggregatedLogFormat.testContainerLogsFileAccess fails on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494733 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-19 18:45:37 +00:00
Vinod Kumar Vavilapalli
243bcd367f YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 23:19:49 +00:00
Vinod Kumar Vavilapalli
2fd7f28230 YARN-846. Move pb Impl classes from yarn-api to yarn-common. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494052 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 07:36:43 +00:00
Arun Murthy
acc0d3eb52 YARN-840. Moved ProtoUtils to yarn.api.records.pb.impl. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494027 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 05:47:54 +00:00
Vinod Kumar Vavilapalli
5d1b453b85 YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 23:13:17 +00:00
Vinod Kumar Vavilapalli
7d0c43d7cf YARN-822. Renamed ApplicationToken to be AMRMToken, and similarly the corresponding TokenSelector and SecretManager. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493889 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 19:11:29 +00:00
Vinod Kumar Vavilapalli
cd30058193 YARN-837. Moved yarn.ClusterInfo into MapReduce project as it doesn't belong to YARN. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493676 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 08:13:48 +00:00
Vinod Kumar Vavilapalli
4662b8b443 YARN-833. Moved Graph and VisualizeStateMachine into yarn.state package. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493654 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 06:51:51 +00:00
Vinod Kumar Vavilapalli
b9efe6bd4a HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common for usage across all Hadoop projects. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493650 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 06:39:33 +00:00
Vinod Kumar Vavilapalli
d4a811edb2 YARN-826. Moved Clock and SystemClock into yarn.util package. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493639 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 04:49:13 +00:00
Vinod Kumar Vavilapalli
f5f8f3bca4 YARN-825. Fixed javadoc and annotations for yarn-common module. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493634 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 03:32:49 +00:00
Arun Murthy
77e134d13f YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 02:27:18 +00:00
Vinod Kumar Vavilapalli
4aa7503342 YARN-828. Removed the unsed YarnVersionAnnotation. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493592 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 21:34:53 +00:00
Vinod Kumar Vavilapalli
e3769ea1d2 YARN-829. Renamed RMTokenSelector to be RMDelegationTokenSelector. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493577 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 20:33:30 +00:00
Arun Murthy
98d97d316c YARN-787. Removed minimum resource from RegisterApplicationMasterResponse. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493509 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 13:59:41 +00:00
Vinod Kumar Vavilapalli
769a0bd831 YARN-693. Modified RM to send NMTokens on allocate call so that AMs can then use them for authentication with NMs. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493448 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 03:11:33 +00:00
Alejandro Abdelnur
710a5eea62 YARN-782. vcores-pcores ratio functions differently from vmem-pmem ratio in misleading way. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493064 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 13:41:45 +00:00
Vinod Kumar Vavilapalli
f0eb4bc342 YARN-692. Creating NMToken master key on RM and sharing it with NM as a part of RM-NM heartbeat. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492907 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 00:06:42 +00:00
Vinod Kumar Vavilapalli
c0cd68d8fb YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 23:25:14 +00:00
Vinod Kumar Vavilapalli
686ac311c9 YARN-746. Renamed Service.register() and Service.unregister() to registerServiceListener() & unregisterServiceListener() respectively. Contributed by Steve Loughran. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492780 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 17:49:14 +00:00
Vinod Kumar Vavilapalli
0928502029 YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran. 
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 15:54:38 +00:00
Chris Nauroth
1ed916f6c6 YARN-700. TestInfoBlock fails on Windows because of line ending missmatch. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492383 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-12 19:18:39 +00:00
Siddharth Seth
6d4d87008c YARN-731. RPCUtil.unwrapAndThrowException should unwrap remote RuntimeExceptions. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492000 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-11 22:34:39 +00:00
Siddharth Seth
c6c41abf68 YARN-737. Throw some specific exceptions directly instead of wrapping them in YarnException. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491896 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-11 18:47:16 +00:00
Vinod Kumar Vavilapalli
a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth. 
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:05:50 +00:00
Vinod Kumar Vavilapalli
d33534c4fb YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He. 
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 00:34:18 +00:00
Arun Murthy
f27e484636 YARN-749. Rename ResourceRequest.(get,set)HostName to ResourceRequest.(get,set)ResourceName. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488806 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-02 22:53:58 +00:00
Vinod Kumar Vavilapalli
a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He. 
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 21:43:17 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Siddharth Seth
2692675fc3 YARN-684. ContainerManager.startContainer should use ContainerTokenIdentifier instead of the entire Container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488085 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 04:13:16 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Konstantin Boudnik
2fb2a5ab42 HADOOP-9607. Fixes in Javadoc build. Contributed by Timothy St. Clair 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487979 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 19:06:49 +00:00
Siddharth Seth
b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli
5420f287cc YARN-578. Fixed NM to use SecureIOUtils for reading and aggregating logs. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487672 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 23:14:59 +00:00
Vinod Kumar Vavilapalli
edc6d7d3ab YARN-422. Add a NM Client library to help application-writers. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487184 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 01:41:41 +00:00
Vinod Kumar Vavilapalli
259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 03:22:54 +00:00
Jonathan Turner Eagles
52cbb906d9 YARN-548. Add tests for YarnUncaughtExceptionHandler (Vadim Bondarev via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485877 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 22:21:56 +00:00
Vinod Kumar Vavilapalli
43876770d9 YARN-563. Add the concept of an application-type for each application. Contributed by Mayank Bansal. 
MAPREDUCE-5246. Specify application-type at the time of job submission after YARN-563. Contributed by Mayank Bansal.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485790 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 18:02:04 +00:00
Vinod Kumar Vavilapalli
132e38ef47 YARN-708. Deleting files that were supposed to removed in the previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485481 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 23:06:58 +00:00
Vinod Kumar Vavilapalli
1fac1ac98c YARN-708. Moved RecordFactory classes to hadoop-yarn-api, and put some miscellaneous fixes to the interfaces. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485478 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 22:51:04 +00:00
Vinod Kumar Vavilapalli
c665b0e86b YARN-706. Fixed race conditions in TestFSDownload. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485028 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 01:50:34 +00:00
Vinod Kumar Vavilapalli
e5ed91aff7 YARN-695. Remove masterContainer and status unused fields from ApplicationReportProto and fix bugs in ApplicationReportPBImpl. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484612 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-20 21:50:47 +00:00
Vinod Kumar Vavilapalli
065747efab YARN-628. Fix the way YarnRemoteException is being unrolled to extract out the underlying exception. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483207 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-16 06:58:34 +00:00
Siddharth Seth
1a119f87b4 YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 04:11:55 +00:00
Arun Murthy
6f236b28ff YARN-597. TestFSDownload fails on Windows due to dependencies on tar/gzip/jar tools. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482149 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-13 23:49:05 +00:00
Vinod Kumar Vavilapalli
2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Vinod Kumar Vavilapalli
9c4f86879c YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli
0727ecaf50 YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli
33ed6e2838 YARN-651. Changed PBClientImpls of ContainerManager and RMAdmin to throw IOExceptions also. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480154 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 04:17:02 +00:00
Christopher Douglas
1f5ac02a0d YARN-45. Add protocol for schedulers to request containers back from 
ApplicationMasters. Contributed by Carlo Curino and Chris Douglas.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479771 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 06:12:36 +00:00
Vinod Kumar Vavilapalli
9eb53b9c84 YARN-632. Changed ContainerManager api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479740 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 00:42:48 +00:00
Vinod Kumar Vavilapalli
674cf7f742 YARN-633. Changed RMAdminProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479738 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 00:37:46 +00:00
Vinod Kumar Vavilapalli
92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli
fc275fed66 YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 22:44:12 +00:00
Vinod Kumar Vavilapalli
fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Vinod Kumar Vavilapalli
448df18646 YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah. 
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 05:42:22 +00:00
Vinod Kumar Vavilapalli
a2f97fd9cb YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471608 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 19:42:47 +00:00
Bikas Saha
a1ee2145cf YARN-549. YarnClient.submitApplication should wait for application to be accepted by the RM (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470797 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 03:06:41 +00:00
Vinod Kumar Vavilapalli
a91067fc5e YARN-542. Changed the default global AM max-attempts value to be not one. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470094 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 00:35:29 +00:00
Vinod Kumar Vavilapalli
8e1c2823fc YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong. 
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 01:33:13 +00:00
Alejandro Abdelnur
de6d05c8b0 YARN-476. ProcfsBasedProcessTree info message confuses users. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469093 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 23:11:15 +00:00
Vinod Kumar Vavilapalli
dbcda89fef YARN-500. Fixed YARN webapps to not roll-over ports when explicitly asked to use non-ephemeral ports. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468739 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 04:16:51 +00:00
Vinod Kumar Vavilapalli
0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:54:09 +00:00
Vinod Kumar Vavilapalli
e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong. 
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:28:51 +00:00
Vinod Kumar Vavilapalli
3a54a5653b YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466196 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 19:56:10 +00:00
Bikas Saha
bc6777dd5b YARN-193. Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465067 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 17:45:44 +00:00
Vinod Kumar Vavilapalli
0836d68abf YARN-532. Change RMAdmin and Localization client protocol PB implementations to implement closeable so that they can be stopped when needed via RPC.stopProxy(). Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464788 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 23:30:15 +00:00
Vinod Kumar Vavilapalli
1fd462b118 YARN-536. Removed the unused objects ContainerStatus and ContainerStatus from Container which also don't belong to the container. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464271 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 03:41:18 +00:00
Alejandro Abdelnur
3a089cf356 YARN-458. YARN daemon addresses must be placed in many different configs. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464204 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 21:04:23 +00:00
Alejandro Abdelnur
2100e93b04 YARN-538. RM address DNS lookup can cause unnecessary slowness on every JHS page load. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464197 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 20:41:57 +00:00
Vinod Kumar Vavilapalli
e67e3ff05d YARN-467. Modify public distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463823 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 05:00:28 +00:00
Vinod Kumar Vavilapalli
7ca9fe73a0 YARN-309. Changed NodeManager to obtain heart-beat interval from the ResourceManager. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463346 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-02 01:04:07 +00:00
Steve Loughran
7c65dbb938 YARN-524 TestYarnVersionInfo failing if generated properties doesn't include an SVN URL 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463300 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-01 20:57:02 +00:00
Vinod Kumar Vavilapalli
0757364014 YARN-378. Fix RM to make the AM max attempts/retries to be configurable per application by clients. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460895 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 21:11:33 +00:00
Robert Joseph Evans
c02dd9aa4b YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460723 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 15:46:25 +00:00
Hitesh Shah
b15b44e722 YARN-470. Support a way to disable resource monitoring on the NodeManager. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460001 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-22 21:28:31 +00:00
Bikas Saha
28bac40295 YARN-417. Create AMRMClient wrapper that provides asynchronous callbacks. (Sandy Ryza via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459555 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 22:26:06 +00:00