big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
25,146 Commits 2 Branches 2 Tags 264 MiB
4627e9c7ef
Commit Graph

522 Commits

Author SHA1 Message Date
Eric Yang
438c1d333e YARN-7540. Route YARN service CLI function through YARN Service API. (Contributed by Eric Yang) 2017-12-13 19:29:04 -05:00
Jian He
3ebe6a7819 YARN-7565. Yarn service pre-maturely releases the container after AM restart. Contributed by Chandni Singh 2017-12-12 13:35:56 -08:00
Weiwei Yang
40b0045ebe YARN-7610. Extend Distributed Shell to support launching job with opportunistic containers. Contributed by Weiwei Yang. 2017-12-06 17:52:41 +08:00
Eric Yang
d30d57828f YARN-6669. Implemented Kerberos security for YARN service framework. (Contributed by Jian He) 2017-12-04 15:11:00 -05:00
Billie Rinaldi
6f9d7a146d YARN-7529. TestYarnNativeServices#testRecoverComponentsAfterRMRestart() fails intermittently. Contributed by Chandni Singh 2017-11-20 07:37:04 -08:00
Eric Yang
0940e4f692 YARN-7218. Decouple YARN Services REST API namespace from RM. (Contributed by Eric Yang) 2017-11-17 12:28:12 -05:00
Jian He
6bf2c30192 YARN-7503. Configurable heap size / JVM opts in service AM. Contributed by Jonathan Hung 2017-11-16 10:53:55 -08:00
Billie Rinaldi
f4d5d20286 YARN-7486. Race condition in service AM that can cause NPE. Contributed by Jian He 2017-11-16 07:58:06 -08:00
Jian He
4f40cd314a YARN-7488. Make ServiceClient.getAppId method public to return ApplicationId for a service name. Contributed by Gour Saha 2017-11-13 18:57:56 -08:00
Jian He
a55d0738f1 YARN-7371. Added allocateRequestId in NMContainerStatus for recovery. Contributed by Chandni Singh 2017-11-06 13:30:20 -08:00
Billie Rinaldi
a127f7b0fb YARN-7423. Improve service client loggings. Contributed by Jian He 2017-11-06 13:30:20 -08:00
Billie Rinaldi
a6c4bd74b6 YARN-6394. Support specifying YARN related params in the service REST API. Contributed by Jian He 2017-11-06 13:30:20 -08:00
Jian He
ba7ed7b66a YARN-7384. Remove apiserver cmd and merge service cmd into application cmd. Contributed by Billie Rinaldi 2017-11-06 13:30:19 -08:00
Billie Rinaldi
68acd88dcb YARN-7323. Data structure update in service REST API. Contributed by Jian He 2017-11-06 13:30:18 -08:00
Jian He
b57144a172 YARN-7202. Add UT for api-server. Contributed by Eric Yang 2017-11-06 13:30:18 -08:00
Jian He
b8a7ef1b64 YARN-6744. Recover component information on YARN native services AM restart. Contributed by Billie Rinaldi 2017-11-06 13:30:18 -08:00
Jian He
c723021579 YARN-6626. Embed REST API service into RM. Contributed by Eric Yang 2017-11-06 13:30:17 -08:00
Jian He
673c0db43c Revert "YARN-6626. Embed REST API service into RM. Contributed by Eric Yang" 
This reverts commit 63d1084e9781e0fee876916190b69f6242dd00e4.
 2017-11-06 13:30:17 -08:00
Jian He
9e677fa05c YARN-6626. Embed REST API service into RM. Contributed by Eric Yang 2017-11-06 13:30:17 -08:00
Jian He
c70e5df100 YARN-7201. Added an apache httpd example YARN service. Contributed by Billie Rinaldi 2017-11-06 13:30:16 -08:00
Billie Rinaldi
ce74e64363 YARN-7210. Some NPE fixes in Registry DNS. Contributed by Jian He 2017-11-06 13:30:16 -08:00
Billie Rinaldi
37c9b7327d YARN-7186. Add examples in yarn-service. Contributed by Jian He 2017-11-06 13:30:16 -08:00
Jian He
307d55b3e1 Rebased onto trunk - fix conflicts 2017-11-06 13:30:15 -08:00
Gour Saha
541b64c798 YARN-7165. Miscellaneous fixes in yarn-native-services. Contributed by Jian He 2017-11-06 13:30:15 -08:00
Jian He
eed9e3e8f4 Rebase onto trunk - fix some issues. Contributed by Rohith Sharma K S 2017-11-06 13:30:14 -08:00
Jian He
7dd4490a44 YARN-7073. Yarn native services rest API documentation. Contributed by Gour Saha 2017-11-06 13:30:14 -08:00
Jian He
59fe98cd50 YARN-7100. Added JsonIgnore annotation to Resource#getMemoryMB. Contributed by Eric Yang 2017-11-06 13:30:13 -08:00
Jian He
fecf22b2fd YARN-7113. Clean up packaging and dependencies for yarn-native-services. Contributed by Billie Rinaldi 2017-11-06 13:30:13 -08:00
Billie Rinaldi
40ab068eab YARN-7091. Rename application to service in yarn-native-services. Contributed by Jian He 2017-11-06 13:30:13 -08:00
Billie Rinaldi
2191f5b533 YARN-7050. Post cleanup after YARN-6903, removal of org.apache.slider package. Contributed by Jian He 2017-11-06 13:30:12 -08:00
Billie Rinaldi
79bf5c084d YARN-7029. Add more UTs in yarn-native-services. Contributed by Jian He 2017-11-06 13:30:11 -08:00
Billie Rinaldi
1888318c89 YARN-6903. Yarn-native-service framework core rewrite. Contributed by Jian He 2017-11-06 13:30:11 -08:00
Jian He
27dcc67959 Update pom version to 3.0.0-beta1-SNAPSHOT 2017-11-06 13:30:10 -08:00
Jian He
8d335e59cf YARN-6804. [yarn-native-services changes] Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi 2017-11-06 13:30:10 -08:00
Billie Rinaldi
14f3f3b42e YARN-6655. Fix compilation failure in TestServiceApiUtil due to changes in YARN-6716. 2017-11-06 13:30:10 -08:00
Jian He
59a9b4f438 YARN-6655. Validate yarn native services application submission side to ensure that the hostname should be less than 63 characters. Contributed by Billie Rinaldi 2017-11-06 13:30:09 -08:00
Jian He
e86d828548 YARN-6716. Native services support for specifying component start order. Contributed by Billie Rinaldi 2017-11-06 13:30:09 -08:00
Akira Ajisaka
b7e5d739ba YARN-6722. Bumping up pom file hadoop version. Contributed by Panagiotis Garefalakis. 2017-11-06 13:30:09 -08:00
Billie Rinaldi
a0b9e8a630 YARN-6617. Services API delete call first attempt usually fails. Contributed by Jian He 2017-11-06 13:30:09 -08:00
Jian He
a0574e7f4f YARN-6613. Update json validation for new native services providers. Contributed by Billie Rinaldi 2017-11-06 13:30:08 -08:00
Jian He
9085cd5c6e YARN-6160. Create an agent-less docker-less provider in the native services framework. Contributed by Billie Rinaldi 2017-11-06 13:30:08 -08:00
Billie Rinaldi
02d68c4bcf YARN-6588. Add native-service AM log4j properties in classpath. Contributed by Jian He 2017-11-06 13:30:08 -08:00
Jian He
a041373dd4 YARN-6533. Race condition in writing service record to registry in yarn native services. Contributed by Billie Rinaldi 2017-11-06 13:30:08 -08:00
Billie Rinaldi
ce05c6e981 YARN-6545. Followup fix for YARN-6405. Contributed by Jian He 2017-11-06 13:30:07 -08:00
Sunil G
ba812bf67f YARN-6446. addendum patch. Contributed by Rohith Sharma K S. 2017-11-06 13:30:07 -08:00
Jian He
a8a273b064 YARN-6446. Revisit ATSv2 integration to ensure all required information is published. Contributed by Rohith Sharma K S 2017-11-06 13:30:07 -08:00
Billie Rinaldi
845764da14 YARN-6405. Improve configuring services through REST API. Contributed by Jian He 2017-11-06 13:28:42 -08:00
Jian He
384ee13eec YARN-6335. Port slider's groovy unit tests to yarn native services. Contributed by Billie Rinaldi 2017-11-06 13:28:42 -08:00
Jian He
c3ac67e3e1 Rebase to latest trunk, fix conflicts and pom.xml 2017-11-06 13:28:41 -08:00
Jian He
fd9d0d4ead YARN-6400. Remove some unneeded code after YARN-6255. Contributed by Jian He 2017-11-06 13:28:41 -08:00
Jian He
f4216b7bba YARN-6395. Integrate service app master to write data into ATSv2. Contributed by Rohith Sharma K S 2017-11-06 13:28:41 -08:00
Jian He
c31cd981eb YARN-6255. Refactor yarn-native-services framework. Contributed by Jian He 2017-11-06 13:28:41 -08:00
Billie Rinaldi
024e90a508 YARN-6192. Move yarn-native-service API records into hadoop-yarn-slider-core module. Contributed by Jian He 2017-11-06 13:28:41 -08:00
Gour Saha
0ed8d4ebe6 YARN-6205. Default lifetime for native services app is invalid. Contributed by Billie Rinaldi 2017-11-06 13:28:40 -08:00
Gour Saha
500695d726 YARN-6185. Apply SLIDER-1199 to yarn native services for blacklisting nodes. Contributed by Billie Rinaldi 2017-11-06 13:28:40 -08:00
Gour Saha
8967a1b812 YARN-6186 Handle InvalidResourceRequestException in native services AM onError 2017-11-06 13:28:40 -08:00
Billie Rinaldi
e34ca6b02d YARN-6173. Add artifact info and privileged container details to the container info in API GET response. Contributed by Gour Saha 2017-11-06 13:28:40 -08:00
Jian He
68940ab8f6 Updated pom to point to 3.0.0-alpha3-SNAPSHOT 2017-11-06 13:28:40 -08:00
Billie Rinaldi
d943eb4c15 YARN-6132. SliderClient bondToCluster should call findInstance with live states filter. Contributed by Gour Saha 2017-11-06 13:28:40 -08:00
Billie Rinaldi
7edc154e0e YARN-6115. Few additional paths in Slider client still uses get all Applications without tags/states filter. Contributed by Gour Saha 2017-11-06 13:28:40 -08:00
Gour Saha
a5e20f0fc1 YARN-5993. Allow native services quicklinks to be exported for each component. Contributed by Billie Rinaldi 2017-11-06 13:28:39 -08:00
Billie Rinaldi
74fff4086e YARN-6014. Followup fix for slider core module findbugs. Contributed by Jian He 2017-11-06 13:28:39 -08:00
Billie Rinaldi
1ea43c443e YARN-6010. Fix findbugs, site warnings in yarn-services-api module. Contributed by Jian He 2017-11-06 13:28:38 -08:00
Billie Rinaldi
db96e8aa21 YARN-5967. Fix slider core module findbugs warnings. Contributed by Jian He 2017-11-06 13:28:38 -08:00
Gour Saha
c11f4b3c21 YARN-5996. Native services AM kills app on AMRMClientAsync onError call. Contributed by Billie Rinaldi 2017-11-06 13:28:38 -08:00
Gour Saha
2e1fa61253 YARN-5740. Add a new field in Slider status output - lifetime (remaining). Contributed by Jian He 2017-11-06 13:28:37 -08:00
Billie Rinaldi
2cea59dc54 YARN-5975. Remove the agent - slider AM ssl related code. Contributed by Jian He 2017-11-06 13:28:37 -08:00
Jian He
efe7d44ead YARN-5961. Generate native services protobuf classes during build. Contributed by Billie Rinaldi 2017-11-06 13:28:37 -08:00
Gour Saha
e4f0263e51 YARN-5958. Fix ASF license warnings for slider core module. Contributed by Billie Rinaldi 2017-11-06 13:28:37 -08:00
Gour Saha
1f44e6a5f4 YARN-5944. Native services AM should remain up if RM is down. Contributed by Billie Rinaldi 2017-11-06 13:28:37 -08:00
Gour Saha
6143aa135b YARN-5769. Integrate update app lifetime using feature implemented in YARN-5611. Contributed by Jian He 2017-11-06 13:28:37 -08:00
Gour Saha
bd0e9f91b6 YARN-5941. Slider handles "per.component" for multiple components incorrectly. Contributed by Billie Rinaldi 2017-11-06 13:28:37 -08:00
Gour Saha
4f43ebdb9f YARN-5943. Write native services container stderr file to log directory. Contributed by Billie Rinaldi 2017-11-06 13:28:36 -08:00
Billie Rinaldi
7b8fd3abd6 YARN-5883 Avoid or eliminate expensive YARN get all applications call. Contributed by Gour Saha 2017-11-06 13:28:36 -08:00
Jian He
234dba84a1 YARN-5909. Remove agent related code in slider AM. Contributed by Jian He 2017-11-06 13:28:36 -08:00
Gour Saha
d1ee95981e YARN-5828. Native services client errors out when config formats are uppercase. Contributed by Billie Rinaldi 2017-11-06 13:28:35 -08:00
Jian He
7757f6fe4a YARN-5812. Exception during GET call - "Failed to retrieve application: null". Contributed by Gour Saha 2017-11-06 13:28:35 -08:00
Gour Saha
25411a1742 YARN-5813. Slider should not try to set a negative lifetime timeout value. Contributed by Jian He 2017-11-06 13:28:35 -08:00
Jian He
3741e5518f YARN-5796. Convert enums values in service code to upper case and special handling of an error. Contributed by Gour Saha 2017-11-06 13:28:34 -08:00
Gour Saha
09e4b9e897 YARN-5690. Integrate native services modules into maven build. Contributed by Billie Rinaldi 2017-11-06 13:28:34 -08:00
Billie Rinaldi
ef5a3628c2 YARN-5770. Performance improvement of native-services REST API service. Contributed by Gour Saha 2017-11-06 13:28:34 -08:00
Gour Saha
1d7a42e795 YARN-5680. Add 2 new fields in Slider status output - image-name and is-privileged-container. Contributed by Billie Rinaldi 2017-11-06 13:28:34 -08:00
Billie Rinaldi
773c565ac5 YARN-5775. Convert enums in swagger definition to uppercase. Contributed by Gour Saha 2017-11-06 13:28:34 -08:00
Gour Saha
6bdfce3ac7 YARN-5778. Add .keep file for yarn native services AM web app. Contributed by Billie Rinaldi 2017-11-06 13:28:34 -08:00
Jian He
f206a43954 YARN-5701. Fix issues in yarn native services apps-of-apps. Contributed by Billie Rinaldi 2017-11-06 13:28:33 -08:00
Gour Saha
4ec1cbe86d YARN-5735. Make the service REST API use the app timeout feature YARN-4205. Contributed by Jian He 2017-11-06 13:28:33 -08:00
Jian He
7da243ebe0 YARN-5729. Bug fixes for the service Rest API. Contributed by Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
c0120e51b9 YARN-5689. Update native services REST API to use agentless docker provider. Contributed by Billie Rinaldi & Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
f66ecfecf6 Addendum patch for YARN-5610. Contributed by Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
295b87d0fb YARN-5675. Swagger definition for YARN service API. Contributed by Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
18bf0dce28 YARN-5610. Initial code for native services REST API. Contributed by Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
9dc46aa379 YARN-5623. Apply SLIDER-1166 to yarn-native-services branch. Contributed by Gour Saha 2017-11-06 13:28:32 -08:00
Jian He
bce06ed1af YARN-5505. Create an agent-less docker provider in the native-services framework. Contributed by Billie Rinaldi 2017-11-06 13:28:32 -08:00
Jian He
3279baecb9 YARN-5538. Apply SLIDER-875 to yarn-native-services. Contributed by Billie Rinaldi 2017-11-06 13:28:32 -08:00
Jian He
02e2740bdf YARN-5513. Move Java only tests from slider develop to yarn-native-services. Contributed by Gour Saha 2017-11-06 13:28:32 -08:00
Jian He
54ec9e2440 Modify pom file for slider 2017-11-06 13:28:32 -08:00
Jian He
804bb82d1b Rename org.apache.slider.core.build to org.apache.slider.core.buildutils 2017-11-06 13:28:32 -08:00
Jian He
f47df51791 YARN-5461. Initial code ported from slider-core module. (jianhe) 2017-11-06 13:28:31 -08:00
Akira Ajisaka
785f1b0d11
YARN-7407. Moving logging APIs over to slf4j in hadoop-yarn-applications. Contributed by Yeliang Cang. 2017-10-31 16:36:02 +09:00
Yufei Gu
6b7c87c945 YARN-7355. TestDistributedShell should be scheduler agnostic. (Contributed by Haibo) 2017-10-20 11:15:20 -07:00
Wangda Tan
e490602e9b YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
 2017-09-12 09:19:12 -07:00
Sunil G
7805deed48 YARN-5588. [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Arun Suresh
c41118a7f8 YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh) 2017-09-07 10:23:12 -07:00
Andrew Wang
0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Varun Saxena
3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Sangjin Lee
092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Daniel Templeton
d81372dfad YARN-6646. Modifier 'static' is redundant for inner enums 
(Contributed by ZhangBing Lin via Daniel Templeton)
 2017-05-26 12:05:48 -07:00
Andrew Wang
539ef5aa2e HFDS-11596. hadoop-hdfs-client jar is in the wrong directory in release tarball. Contributed by Yuanbo Liu. 2017-04-05 16:04:09 -07:00
Chris Douglas
6eba79232f HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang 2017-04-03 20:13:14 -07:00
Sangjin Lee
4fa1afdb88 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-02-16 11:41:04 -08:00
Andrew Wang
5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
Daniel Templeton
71a4acf74b YARN-5991. Yarn Distributed Shell does not print throwable t to App Master When failed to start container 
(Contributed by Jim Frankola via Daniel Templeton)
 2017-01-06 14:26:08 -08:00
Robert Kanter
9262797e86 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 2016-12-27 14:14:08 -08:00
Akira Ajisaka
209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Akira Ajisaka
d3bb69a667 YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton. 2016-10-26 15:32:07 +09:00
Arun Suresh
d6d9cff21b YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 2016-08-30 15:52:29 -07:00
Andrew Wang
da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Varun Saxena
a3ac1c1943 YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 2016-07-12 20:41:52 +05:30
Sangjin Lee
6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Varun Saxena
1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Vrushali Channapattan
3832795e3c YARN-5210 NPE in Distributed Shell while publishing DS_CONTAINER_START event (Varun Saxena via Vrushali C) 2016-07-10 08:46:02 -07:00
Li Lu
0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Sangjin Lee
d5dfee24b1 YARN-5050. Code cleanup for TestDistributedShell (Li Lu via sjlee) 2016-07-10 08:45:59 -07:00
Varun Saxena
a3cf40e532 YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:55 -07:00
Sangjin Lee
0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Varun Saxena
eb0ac8efb1 YARN-4350. TestDistributedShell fails for V2 scenarios. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:50 -07:00
Li Lu
8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase. 
Contributed by Zhan Zhang.
 2016-07-10 08:45:49 -07:00
Li Lu
89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Junping Du
477a30f536 YARN-3045. Implement NM writing container lifecycle events to Timeline Service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:42 -07:00
Sangjin Lee
22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee) 
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
 2016-07-10 08:45:38 -07:00
Zhijie Shen
f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Junping Du
47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen 
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
 2016-07-10 08:45:33 -07:00
Zhijie Shen
5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du
d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen
f0e752c14b YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C. 
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
bf54d32750 YARN-3210. Refactored timeline aggregator according to new code organization proposed in YARN-3166. Contributed by Li Lu. 
(cherry picked from commit d3ff7f06cbc66d3a23c2551e7d4c752689f46afe)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
d45ff878c4 YARN-3125. Made the distributed shell use timeline service next gen and add an integration test for it. Contributed by Junping Du and Li Lu. 
(cherry picked from commit bf08f7f0ed4900ce52f98137297dd1a47ba2a536)
 2016-07-10 08:45:30 -07:00
Junping Du
9bdb5bebea Fix OOM issue in DistributedShell. Contributed by Jian He. 2016-07-09 05:09:53 -07:00
Wangda Tan
c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Varun Vasudev
42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Junping Du
48c931331c YARN-5131. Distributed shell AM fails when extra container arrives during finishing. Contributed by Wangda Tan. 2016-05-25 06:18:11 -07:00
Andrew Wang
3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang
ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Varun Vasudev
0f25a1bb52 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 2016-04-29 16:09:07 +05:30
Xuan
e6c0742012 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 2016-04-16 19:39:18 -07:00
Robert Kanter
8286270466 YARN-4436. DistShell ApplicationMaster.ExecBatScripStringtPath is misspelled (mattlamantia via rkanter) 2016-03-29 17:21:29 -07:00
Junping Du
f291d82cd4 YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du 2016-03-14 08:28:38 -07:00
Steve Loughran
e9a622606f YARN-4629. Distributed shell breaks under strong security. (Daniel Templeton via stevel) 2016-02-10 13:01:10 +00:00
Jian He
f385851141 YARN-4596. SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities. Contributed by Li Lu 2016-01-18 16:58:39 -08:00
Wangda Tan
7ff280fca9 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda) 2015-11-12 11:50:00 -08:00
Wangda Tan
c99925d6dd YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda) 2015-11-10 11:45:46 -08:00
Wangda Tan
0783184f4b YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) 2015-11-03 11:18:34 -08:00
Steve Loughran
446212a39e YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via stevel) 2015-10-24 12:55:36 +01:00
Steve Loughran
14f8dd0423 YARN-4289. TestDistributedShell failing with bind exception. (Brahma Reddy Battula via stevel) 2015-10-24 12:48:15 +01:00
Jason Lowe
2b2bd92146 YARN-3950. Add unique SHELL_ID environment variable to DistributedShell. Contributed by Robert Kanter 2015-07-29 15:16:40 +00:00
Jian He
7438966586 YARN-2821. Fixed a problem that DistributedShell AM may hang if restarted. Contributed by Varun Vasudev 2015-05-19 14:20:31 -07:00
Devaraj K
241a72af0d YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:23:53 +05:30
Akira Ajisaka
918af8efff YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 13:33:03 +09:00
Akira Ajisaka
5ce3a77f3c YARN-3444. Fix typo capabililty. Contributed by Gabor Liptak. 2015-04-25 06:08:16 +09:00
Vinod Kumar Vavilapalli
f5fe35e297 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 2015-04-23 11:19:55 -07:00
Jian He
aa4a192feb YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the correct user. Contributed by Zhijie Shen 2015-04-23 11:07:26 -07:00
Hitesh Shah
265ed1fe80 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 2015-04-08 14:13:10 -07:00
Jonathan Eagles
8180e676ab YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 2015-03-13 12:04:30 -05:00
Tsuyoshi Ozawa
24aa462673 YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature. Contributed by Wangda Tan. 2015-01-23 20:37:05 +09:00
Junping Du
d468c9aaf1 YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 2014-12-23 20:04:33 -08:00
Junping Du
856b46cf47 Revert ca3381de61 and 20674255ad as missing JIRA number in commit message 2014-12-23 20:02:33 -08:00
Junping Du
ca3381de61 Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 2014-12-23 18:57:16 -08:00
Jian He
a4f2995b9e Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367d.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-12-15 11:04:09 -08:00
Zhijie Shen
51af8d367d YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 2014-12-01 19:04:04 -08:00
Jonathan Eagles
c298a9a845 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 2014-11-20 23:34:35 -06:00
Arun C. Murthy
43cd07b408 YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 2014-11-09 14:57:37 -08:00
Vinod Kumar Vavilapalli
737d9284c1 YARN-2607. Fixed issues in TestDistributedShell. Contributed by Wangda Tan. 2014-11-08 11:00:57 -08:00
Vinod Kumar Vavilapalli
2ac1be7dec YARN-2824. Fixed Capacity Scheduler to not crash when some node-labels are not mapped to queues by making default capacities per label to be zero. Contributed by Wangda Tan. 2014-11-07 10:39:37 -08:00
Zhijie Shen
a8c1202220 YARN-2769. Fixed the problem that timeline domain is not set in distributed shell AM when using shell_command on Windows. Contributed by Varun Vasudev. 2014-10-29 12:12:30 -07:00
Jian He
f6b963fdfc YARN-2502. Changed DistributedShell to support node labels. Contributed by Wangda Tan 2014-10-27 20:13:00 -07:00
Zhijie Shen
1d4612f5ad YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 2014-10-09 12:59:47 -07:00
Zhijie Shen
52bbe0f11b YARN-2630. Prevented previous AM container status from being acquired by the current restarted AM. Contributed by Jian He. 2014-10-01 15:38:11 -07:00
Vinod Kumar Vavilapalli
a3d9934f91 YARN-1779. Fixed AMRMClient to handle AMRMTokens correctly across ResourceManager work-preserving-restart or failover. Contributed by Jian He. 2014-09-18 10:16:18 -07:00
XuanGong
8e5d6713cf YARN-2557. Add a parameter "attempt_Failures_Validity_Interval" into 
DistributedShell. Contributed by Xuan Gong
 2014-09-16 10:58:18 -07:00
Hitesh Shah
3de66011c2 YARN-2450. Fix typos in log messages. Contributed by Ray Chiang. 2014-08-29 11:16:36 -07:00
Zhijie Shen
1a535f8dcd YARN-2070. Made DistributedShell publish the short user name to the timeline server. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617837 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 21:07:18 +00:00
Jian He
9ebeac16eb YARN-2374. Fixed TestDistributedShell#testDSShell failure due to hostname dismatch. Contributed by Varun Vasudev 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616302 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 18:58:50 +00:00
Jian He
7e54b1c6d9 YARN-2354. DistributedShell may allocate more containers than client specified after AM restarts. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614538 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 03:58:59 +00:00
Jian He
f6c723ff0c YARN-2295. Refactored DistributedShell to use public APIs of protocol records. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612626 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 17:49:14 +00:00
Zhijie Shen
28058894dd YARN-2081. Fixed TestDistributedShell failure after YARN-1962. Contributed by Zhiguo Hong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596724 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 01:16:14 +00:00
Haohui Mai
d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-02 22:26:02 +00:00
Vinod Kumar Vavilapalli
d6a6e982a8 YARN-1926. Changed DistributedShell to use appIDs as unique identifiers for HDFS paths and thus avoid test failures on Windows. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586562 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 04:06:40 +00:00
Vinod Kumar Vavilapalli
6c2a0ce30b YARN-1908. Fixed DistributedShell to not fail in secure clusters. Contributed by Vinod Kumar Vavilapalli and Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585849 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-08 22:11:52 +00:00
Zhijie Shen
3af3b0bda9 YARN-1873. Fixed TestDistributedShell failure when the test cases are out of order. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582060 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 21:18:38 +00:00
Zhijie Shen
0f1eda6bbf YARN-1838. Enhanced timeline service getEntities API to get entities from a given entity ID or insertion timestamp. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580960 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 18:14:13 +00:00
Zhijie Shen
ef9634bf79 YARN-1577. Made UnmanagedAMLauncher do launchAM after the attempt reaches the LAUNCHED state. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580164 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 04:35:46 +00:00
Vinod Kumar Vavilapalli
bf428cc474 YARN-1855. Made Application-history server to be optional in MiniYARNCluster and thus avoid the failure of TestRMFailover#testRMWebAppRedirect. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579838 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 00:18:33 +00:00
Zhijie Shen
fd1c424548 YARN-1690. Made DistributedShell send timeline entities+events. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579123 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-19 03:49:29 +00:00
Jonathan Turner Eagles
7944eab64a YARN-1136. Replace junit.framework.Assert with org.junit.Assert (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578539 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 20:03:35 +00:00
Vinod Kumar Vavilapalli
96e0ca2d27 YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. 
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578135 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-16 18:32:05 +00:00
Vinod Kumar Vavilapalli
0fbc7fe816 YARN-1588. Enhanced RM and the scheduling protocol to also send NMTokens of transferred containers from previous app-attempts to new AMs after YARN-1490. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572230 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:20:07 +00:00
Zhijie Shen
6182e7592d YARN-1661. Fixed DS ApplicationMaster to write the correct exit log. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565394 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 18:56:46 +00:00
Vinod Kumar Vavilapalli
7c8b654ba5 YARN-888. Cleaned up POM files so that non-leaf modules don't include any dependencies and thus compact the dependency list for leaf modules. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557801 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-13 18:34:02 +00:00
Vinod Kumar Vavilapalli
aa0c489a28 YARN-1566. Changed Distributed Shell to retain containers across application attempts. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557322 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 07:42:16 +00:00
Vinod Kumar Vavilapalli
bb2e2fee60 YARN-1549. Fixed a bug in ResourceManager's ApplicationMasterService that was causing unamanged AMs to not finish correctly. Contributed by haosdent. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554886 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-02 19:54:07 +00:00
Zhijie Shen
d63cfdbf1a YARN-1435. Modified Distributed Shell to accept either the command or the custom script. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550867 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-14 02:00:41 +00:00
Vinod Kumar Vavilapalli
8897b0cc9b YARN-1450. Fixed test failure in TestUnmanagedAMLauncher by removing its dependency on distributed-shell. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548619 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-06 17:53:28 +00:00
Vinod Kumar Vavilapalli
c4bdddeab5 YARN-1314. Fixed DistributedShell to not fail with multiple arguments for a shell command separated by spaces. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1545486 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-26 01:10:21 +00:00
Vinod Kumar Vavilapalli
33a8234040 YARN-1320. Fixed Distributed Shell application to respect custom log4j properties file. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544364 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 23:16:04 +00:00
Vinod Kumar Vavilapalli
a802ef4a5f YARN-1303. Reverted the wrong patch committed earlier and committing the correct patch now. In one go. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544029 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 04:09:46 +00:00
Vinod Kumar Vavilapalli
ad558cf2b3 YARN-1303. Fixed DistributedShell to not fail with multiple commands separated by a semi-colon as shell-command. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544023 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-21 03:47:02 +00:00
Chris Nauroth
5a5ba62a85 YARN-1395. Distributed shell application master launched with debug flag can hang waiting for external ls process. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540839 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-11 20:46:17 +00:00
Luke Lu
735d8b27f7 YARN-7. Support CPU resource for DistributedShell. (Junping Du via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531222 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-11 08:18:57 +00:00
Vinod Kumar Vavilapalli
be3edccf0a YARN-1273. Fixed Distributed-shell to account for containers that failed to start. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529389 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-05 06:08:48 +00:00
Vinod Kumar Vavilapalli
fc23fd3121 YARN-1167. Fixed Distributed Shell to not incorrectly show empty hostname on RM UI. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529376 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-05 01:43:29 +00:00
Hitesh Shah
f104665f7e YARN-1251. TestDistributedShell#TestDSShell failed with timeout. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529369 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-05 00:48:03 +00:00
Hitesh Shah
42c3cd3d13 YARN-49. Improve distributed shell application to work on a secure cluster. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526330 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-25 23:39:17 +00:00
Vinod Kumar Vavilapalli
817a654346 YARN-945. Removed setting of AMRMToken's service from ResourceManager and changed client libraries do it all the time and correctly. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508232 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-29 23:17:39 +00:00
Alejandro Abdelnur
f179afc68d YARN-937. Fix unmanaged AM in non-secure/secure setup post YARN-701. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507706 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-27 20:04:38 +00:00
Bikas Saha
9c453d4432 YARN-875. Application can hang if AMRMClientAsync callback thread has exception (Xuan Gong via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506750 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-24 22:13:17 +00:00
Arun Murthy
ac914f79bc YARN-918. Remove ApplicationAttemptId from RegisterApplicationMasterRequestProto. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504735 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-19 00:57:40 +00:00
Arun Murthy
32bc200d54 YARN-701. Use application tokens irrespective of secure or non-secure mode. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504604 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-18 19:03:48 +00:00
Bikas Saha
3520039979 YARN-927. Change ContainerRequest to not have more than 1 container count and remove StoreContainerRequest (bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503960 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-17 00:47:46 +00:00
Vinod Kumar Vavilapalli
5e4f6ad1d9 YARN-791. Changed RM APIs and web-services related to nodes to ensure that both are consistent with each other. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500994 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-08 22:30:52 +00:00
Vinod Kumar Vavilapalli
1a06175440 YARN-851. Share NMTokens using NMTokenCache (api-based) between AMRMClient and NMClient instead of memory based approach which is used currently. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495247 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-21 00:08:13 +00:00
Arun Murthy
d3198dddc8 YARN-553. Replaced YarnClient.getNewApplication with YarnClient.createApplication which provides a directly usable ApplicationSubmissionContext to simplify the api. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494476 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-19 07:06:54 +00:00
Vinod Kumar Vavilapalli
243bcd367f YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 23:19:49 +00:00
Vinod Kumar Vavilapalli
7ef54faad4 YARN-834. Fixed annotations for yarn-client module, reorganized packages and clearly differentiated *Async apis. Contributed by Arun C Murthy and Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494017 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 04:02:47 +00:00
Vinod Kumar Vavilapalli
5d1b453b85 YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 23:13:17 +00:00
Arun Murthy
ff70f912f7 YARN-824. Added static factory methods to hadoop-yarn-client interfaces. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493631 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 03:19:21 +00:00
Arun Murthy
6b75a5c3b5 YARN-831. Removed minimum resource from GetNewApplicationResponse as a follow-up to YARN-787. Contributed Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493626 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 02:33:11 +00:00
Arun Murthy
77e134d13f YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 02:27:18 +00:00
Arun Murthy
98d97d316c YARN-787. Removed minimum resource from RegisterApplicationMasterResponse. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493509 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 13:59:41 +00:00
Vinod Kumar Vavilapalli
b503b6a07d YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493280 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 23:18:34 +00:00
Vinod Kumar Vavilapalli
fdc9412a81 YARN-806. Moved ContainerExitStatus from yarn.api to yarn.api.records. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493138 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 16:10:37 +00:00
Vinod Kumar Vavilapalli
0928502029 YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran. 
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 15:54:38 +00:00
Alejandro Abdelnur
af8514eef2 YARN-686. Flatten NodeReport. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490827 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-07 20:59:23 +00:00
Bikas Saha
9fcfbf5f51 YARN-759. Create Command enum in AllocateResponse (bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490470 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-06 23:33:48 +00:00
Vinod Kumar Vavilapalli
a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth. 
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:05:50 +00:00
Bikas Saha
3492f5eff1 YARN-660. Improve AMRMClient with matching requests (bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488485 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 08:23:30 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Siddharth Seth
2692675fc3 YARN-684. ContainerManager.startContainer should use ContainerTokenIdentifier instead of the entire Container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488085 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 04:13:16 +00:00
Vinod Kumar Vavilapalli
259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 03:22:54 +00:00
Siddharth Seth
8a430d0643 YARN-716. Fixed unit test failures - TestDistributedShell and TestUnmanagedAMLauncher. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485579 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 05:48:22 +00:00
Vinod Kumar Vavilapalli
9c4f86879c YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli
0727ecaf50 YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli
9eb53b9c84 YARN-632. Changed ContainerManager api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479740 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 00:42:48 +00:00
Vinod Kumar Vavilapalli
92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Suresh Srinivas
fe3853484e HADOOP-9290. Some tests cannot load native library on windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476586 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:24:02 +00:00
Vinod Kumar Vavilapalli
27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00
Vinod Kumar Vavilapalli
0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:54:09 +00:00
Vinod Kumar Vavilapalli
e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong. 
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:28:51 +00:00
Vinod Kumar Vavilapalli
55865f42c4 YARN-557. Fix TestUnmanagedAMLauncher failure on Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465869 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 03:26:38 +00:00
Vinod Kumar Vavilapalli
1fd462b118 YARN-536. Removed the unused objects ContainerStatus and ContainerStatus from Container which also don't belong to the container. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464271 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 03:41:18 +00:00
Vinod Kumar Vavilapalli
aa634e0814 YARN-475. Remove a unused constant in the public API - ApplicationConstants.AM_APP_ATTEMPT_ID_ENV. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463033 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-31 19:17:44 +00:00
Bikas Saha
2aed48a67f YARN-498. Unmanaged AM launcher does not set various constants in env for an AM, also does not handle failed AMs properly (Hitesh Shah via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460954 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 02:34:27 +00:00
Bikas Saha
1acb805058 YARN-497. Yarn unmanaged-am launcher jar does not define a main class in its manifest (Hitesh Shah via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460846 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-25 18:57:15 +00:00
Bikas Saha
7c325a5d71 YARN-417. Addendum patch to fix broken test (Sandy Ryza via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460064 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-23 00:08:24 +00:00
Bikas Saha
28bac40295 YARN-417. Create AMRMClient wrapper that provides asynchronous callbacks. (Sandy Ryza via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459555 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 22:26:06 +00:00
Hitesh Shah
d6b33ee981 YARN-490. TestDistributedShell fails on Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459520 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-21 20:48:18 +00:00
Hitesh Shah
1bd345d6e3 YARN-396. Rationalize AllocateResponse in RM Scheduler API. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459040 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-20 20:44:35 +00:00
Suresh Srinivas
638801cce1 HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 19:15:18 +00:00
Hitesh Shah
f5551bf809 YARN-277. Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437156 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 20:49:09 +00:00
Thomas White
e464607b7d YARN-129. Simplify classpath construction for mini YARN tests. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411235 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-19 15:12:22 +00:00
Siddharth Seth
88d326f0a4 YARN-179. Fix some unit test failures. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401481 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-23 21:06:36 +00:00
Vinod Kumar Vavilapalli
bca57d471f YARN-94. Modify DistributedShell to point to main-class by default, clean up the help message, and hard-code the AM class. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396226 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 19:40:31 +00:00
Arun Murthy
70c295d90d YARN-137. Change the default YARN scheduler to be the CapacityScheduler. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391929 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-30 00:41:24 +00:00
Vinod Kumar Vavilapalli
11cf00b08a YARN-78. Changed UnManagedAM application to use YarnClient. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383705 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-12 00:49:17 +00:00
Vinod Kumar Vavilapalli
87c5d243db YARN-83. Change package of YarnClient to org.apache.hadoop. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381273 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 17:54:32 +00:00
Vinod Kumar Vavilapalli
b1632b4a77 YARN-10. Fix DistributedShell module to not have a dependency on hadoop-mapreduce-client-core. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380879 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-04 20:56:53 +00:00
Siddharth Seth
10e704c50b YARN-29. Add a yarn-client module. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377781 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-27 18:27:18 +00:00
Arun Murthy
e1fdf62123 YARN-1. Promote YARN to be a sub-project of Apache Hadoop. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370666 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-08 05:22:27 +00:00