big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,045 Commits 2 Branches 2 Tags 264 MiB
4e0aa2ceac
Commit Graph

427 Commits

Author SHA1 Message Date
Jian He
836e3c45e8 Revert "YARN-7540. Route YARN service CLI function through YARN Service API. (Contributed by Eric Yang)" 
This reverts commit 438c1d333e.
 2018-01-05 18:25:00 -08:00
Wangda Tan
7467e8fe5a YARN-7555. Support multiple resource types in YARN native services. (wangda) 
Change-Id: I330e6ee17a73962dcaadd766a3e72d2888681731
 2017-12-29 15:34:08 -08:00
Eric Yang
41b581012a YARN-7616. Map YARN application status to Service Status more accurately. (Contributed by Gour Saha) 2017-12-19 19:14:45 -05:00
Eric Yang
989c75109a YARN-7543. Add check for max cpu limit and missing file for YARN service. 
(Contributed by Jian He)
 2017-12-19 16:45:04 -05:00
Weiwei Yang
9289641020 YARN-7617. Add a flag in distributed shell to automatically PROMOTE opportunistic containers to guaranteed once they are started. Contributed by Weiwei Yang. 2017-12-18 10:07:16 +08:00
Eric Yang
438c1d333e YARN-7540. Route YARN service CLI function through YARN Service API. (Contributed by Eric Yang) 2017-12-13 19:29:04 -05:00
Jian He
3ebe6a7819 YARN-7565. Yarn service pre-maturely releases the container after AM restart. Contributed by Chandni Singh 2017-12-12 13:35:56 -08:00
Weiwei Yang
40b0045ebe YARN-7610. Extend Distributed Shell to support launching job with opportunistic containers. Contributed by Weiwei Yang. 2017-12-06 17:52:41 +08:00
Eric Yang
d30d57828f YARN-6669. Implemented Kerberos security for YARN service framework. (Contributed by Jian He) 2017-12-04 15:11:00 -05:00
Billie Rinaldi
6f9d7a146d YARN-7529. TestYarnNativeServices#testRecoverComponentsAfterRMRestart() fails intermittently. Contributed by Chandni Singh 2017-11-20 07:37:04 -08:00
Eric Yang
0940e4f692 YARN-7218. Decouple YARN Services REST API namespace from RM. (Contributed by Eric Yang) 2017-11-17 12:28:12 -05:00
Jian He
6bf2c30192 YARN-7503. Configurable heap size / JVM opts in service AM. Contributed by Jonathan Hung 2017-11-16 10:53:55 -08:00
Billie Rinaldi
f4d5d20286 YARN-7486. Race condition in service AM that can cause NPE. Contributed by Jian He 2017-11-16 07:58:06 -08:00
Jian He
4f40cd314a YARN-7488. Make ServiceClient.getAppId method public to return ApplicationId for a service name. Contributed by Gour Saha 2017-11-13 18:57:56 -08:00
Jian He
a55d0738f1 YARN-7371. Added allocateRequestId in NMContainerStatus for recovery. Contributed by Chandni Singh 2017-11-06 13:30:20 -08:00
Billie Rinaldi
a127f7b0fb YARN-7423. Improve service client loggings. Contributed by Jian He 2017-11-06 13:30:20 -08:00
Billie Rinaldi
a6c4bd74b6 YARN-6394. Support specifying YARN related params in the service REST API. Contributed by Jian He 2017-11-06 13:30:20 -08:00
Jian He
ba7ed7b66a YARN-7384. Remove apiserver cmd and merge service cmd into application cmd. Contributed by Billie Rinaldi 2017-11-06 13:30:19 -08:00
Billie Rinaldi
68acd88dcb YARN-7323. Data structure update in service REST API. Contributed by Jian He 2017-11-06 13:30:18 -08:00
Jian He
b57144a172 YARN-7202. Add UT for api-server. Contributed by Eric Yang 2017-11-06 13:30:18 -08:00
Jian He
b8a7ef1b64 YARN-6744. Recover component information on YARN native services AM restart. Contributed by Billie Rinaldi 2017-11-06 13:30:18 -08:00
Jian He
c723021579 YARN-6626. Embed REST API service into RM. Contributed by Eric Yang 2017-11-06 13:30:17 -08:00
Jian He
673c0db43c Revert "YARN-6626. Embed REST API service into RM. Contributed by Eric Yang" 
This reverts commit 63d1084e9781e0fee876916190b69f6242dd00e4.
 2017-11-06 13:30:17 -08:00
Jian He
9e677fa05c YARN-6626. Embed REST API service into RM. Contributed by Eric Yang 2017-11-06 13:30:17 -08:00
Jian He
c70e5df100 YARN-7201. Added an apache httpd example YARN service. Contributed by Billie Rinaldi 2017-11-06 13:30:16 -08:00
Billie Rinaldi
ce74e64363 YARN-7210. Some NPE fixes in Registry DNS. Contributed by Jian He 2017-11-06 13:30:16 -08:00
Billie Rinaldi
37c9b7327d YARN-7186. Add examples in yarn-service. Contributed by Jian He 2017-11-06 13:30:16 -08:00
Jian He
307d55b3e1 Rebased onto trunk - fix conflicts 2017-11-06 13:30:15 -08:00
Gour Saha
541b64c798 YARN-7165. Miscellaneous fixes in yarn-native-services. Contributed by Jian He 2017-11-06 13:30:15 -08:00
Jian He
eed9e3e8f4 Rebase onto trunk - fix some issues. Contributed by Rohith Sharma K S 2017-11-06 13:30:14 -08:00
Jian He
7dd4490a44 YARN-7073. Yarn native services rest API documentation. Contributed by Gour Saha 2017-11-06 13:30:14 -08:00
Jian He
59fe98cd50 YARN-7100. Added JsonIgnore annotation to Resource#getMemoryMB. Contributed by Eric Yang 2017-11-06 13:30:13 -08:00
Jian He
fecf22b2fd YARN-7113. Clean up packaging and dependencies for yarn-native-services. Contributed by Billie Rinaldi 2017-11-06 13:30:13 -08:00
Billie Rinaldi
40ab068eab YARN-7091. Rename application to service in yarn-native-services. Contributed by Jian He 2017-11-06 13:30:13 -08:00
Billie Rinaldi
2191f5b533 YARN-7050. Post cleanup after YARN-6903, removal of org.apache.slider package. Contributed by Jian He 2017-11-06 13:30:12 -08:00
Billie Rinaldi
79bf5c084d YARN-7029. Add more UTs in yarn-native-services. Contributed by Jian He 2017-11-06 13:30:11 -08:00
Billie Rinaldi
1888318c89 YARN-6903. Yarn-native-service framework core rewrite. Contributed by Jian He 2017-11-06 13:30:11 -08:00
Jian He
27dcc67959 Update pom version to 3.0.0-beta1-SNAPSHOT 2017-11-06 13:30:10 -08:00
Jian He
8d335e59cf YARN-6804. [yarn-native-services changes] Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi 2017-11-06 13:30:10 -08:00
Billie Rinaldi
14f3f3b42e YARN-6655. Fix compilation failure in TestServiceApiUtil due to changes in YARN-6716. 2017-11-06 13:30:10 -08:00
Jian He
59a9b4f438 YARN-6655. Validate yarn native services application submission side to ensure that the hostname should be less than 63 characters. Contributed by Billie Rinaldi 2017-11-06 13:30:09 -08:00
Jian He
e86d828548 YARN-6716. Native services support for specifying component start order. Contributed by Billie Rinaldi 2017-11-06 13:30:09 -08:00
Akira Ajisaka
b7e5d739ba YARN-6722. Bumping up pom file hadoop version. Contributed by Panagiotis Garefalakis. 2017-11-06 13:30:09 -08:00
Billie Rinaldi
a0b9e8a630 YARN-6617. Services API delete call first attempt usually fails. Contributed by Jian He 2017-11-06 13:30:09 -08:00
Jian He
a0574e7f4f YARN-6613. Update json validation for new native services providers. Contributed by Billie Rinaldi 2017-11-06 13:30:08 -08:00
Jian He
9085cd5c6e YARN-6160. Create an agent-less docker-less provider in the native services framework. Contributed by Billie Rinaldi 2017-11-06 13:30:08 -08:00
Billie Rinaldi
02d68c4bcf YARN-6588. Add native-service AM log4j properties in classpath. Contributed by Jian He 2017-11-06 13:30:08 -08:00
Jian He
a041373dd4 YARN-6533. Race condition in writing service record to registry in yarn native services. Contributed by Billie Rinaldi 2017-11-06 13:30:08 -08:00
Billie Rinaldi
ce05c6e981 YARN-6545. Followup fix for YARN-6405. Contributed by Jian He 2017-11-06 13:30:07 -08:00
Sunil G
ba812bf67f YARN-6446. addendum patch. Contributed by Rohith Sharma K S. 2017-11-06 13:30:07 -08:00
Jian He
a8a273b064 YARN-6446. Revisit ATSv2 integration to ensure all required information is published. Contributed by Rohith Sharma K S 2017-11-06 13:30:07 -08:00
Billie Rinaldi
845764da14 YARN-6405. Improve configuring services through REST API. Contributed by Jian He 2017-11-06 13:28:42 -08:00
Jian He
384ee13eec YARN-6335. Port slider's groovy unit tests to yarn native services. Contributed by Billie Rinaldi 2017-11-06 13:28:42 -08:00
Jian He
c3ac67e3e1 Rebase to latest trunk, fix conflicts and pom.xml 2017-11-06 13:28:41 -08:00
Jian He
fd9d0d4ead YARN-6400. Remove some unneeded code after YARN-6255. Contributed by Jian He 2017-11-06 13:28:41 -08:00
Jian He
f4216b7bba YARN-6395. Integrate service app master to write data into ATSv2. Contributed by Rohith Sharma K S 2017-11-06 13:28:41 -08:00
Jian He
c31cd981eb YARN-6255. Refactor yarn-native-services framework. Contributed by Jian He 2017-11-06 13:28:41 -08:00
Billie Rinaldi
024e90a508 YARN-6192. Move yarn-native-service API records into hadoop-yarn-slider-core module. Contributed by Jian He 2017-11-06 13:28:41 -08:00
Gour Saha
0ed8d4ebe6 YARN-6205. Default lifetime for native services app is invalid. Contributed by Billie Rinaldi 2017-11-06 13:28:40 -08:00
Gour Saha
500695d726 YARN-6185. Apply SLIDER-1199 to yarn native services for blacklisting nodes. Contributed by Billie Rinaldi 2017-11-06 13:28:40 -08:00
Gour Saha
8967a1b812 YARN-6186 Handle InvalidResourceRequestException in native services AM onError 2017-11-06 13:28:40 -08:00
Billie Rinaldi
e34ca6b02d YARN-6173. Add artifact info and privileged container details to the container info in API GET response. Contributed by Gour Saha 2017-11-06 13:28:40 -08:00
Jian He
68940ab8f6 Updated pom to point to 3.0.0-alpha3-SNAPSHOT 2017-11-06 13:28:40 -08:00
Billie Rinaldi
d943eb4c15 YARN-6132. SliderClient bondToCluster should call findInstance with live states filter. Contributed by Gour Saha 2017-11-06 13:28:40 -08:00
Billie Rinaldi
7edc154e0e YARN-6115. Few additional paths in Slider client still uses get all Applications without tags/states filter. Contributed by Gour Saha 2017-11-06 13:28:40 -08:00
Gour Saha
a5e20f0fc1 YARN-5993. Allow native services quicklinks to be exported for each component. Contributed by Billie Rinaldi 2017-11-06 13:28:39 -08:00
Billie Rinaldi
74fff4086e YARN-6014. Followup fix for slider core module findbugs. Contributed by Jian He 2017-11-06 13:28:39 -08:00
Billie Rinaldi
1ea43c443e YARN-6010. Fix findbugs, site warnings in yarn-services-api module. Contributed by Jian He 2017-11-06 13:28:38 -08:00
Billie Rinaldi
db96e8aa21 YARN-5967. Fix slider core module findbugs warnings. Contributed by Jian He 2017-11-06 13:28:38 -08:00
Gour Saha
c11f4b3c21 YARN-5996. Native services AM kills app on AMRMClientAsync onError call. Contributed by Billie Rinaldi 2017-11-06 13:28:38 -08:00
Gour Saha
2e1fa61253 YARN-5740. Add a new field in Slider status output - lifetime (remaining). Contributed by Jian He 2017-11-06 13:28:37 -08:00
Billie Rinaldi
2cea59dc54 YARN-5975. Remove the agent - slider AM ssl related code. Contributed by Jian He 2017-11-06 13:28:37 -08:00
Jian He
efe7d44ead YARN-5961. Generate native services protobuf classes during build. Contributed by Billie Rinaldi 2017-11-06 13:28:37 -08:00
Gour Saha
e4f0263e51 YARN-5958. Fix ASF license warnings for slider core module. Contributed by Billie Rinaldi 2017-11-06 13:28:37 -08:00
Gour Saha
1f44e6a5f4 YARN-5944. Native services AM should remain up if RM is down. Contributed by Billie Rinaldi 2017-11-06 13:28:37 -08:00
Gour Saha
6143aa135b YARN-5769. Integrate update app lifetime using feature implemented in YARN-5611. Contributed by Jian He 2017-11-06 13:28:37 -08:00
Gour Saha
bd0e9f91b6 YARN-5941. Slider handles "per.component" for multiple components incorrectly. Contributed by Billie Rinaldi 2017-11-06 13:28:37 -08:00
Gour Saha
4f43ebdb9f YARN-5943. Write native services container stderr file to log directory. Contributed by Billie Rinaldi 2017-11-06 13:28:36 -08:00
Billie Rinaldi
7b8fd3abd6 YARN-5883 Avoid or eliminate expensive YARN get all applications call. Contributed by Gour Saha 2017-11-06 13:28:36 -08:00
Jian He
234dba84a1 YARN-5909. Remove agent related code in slider AM. Contributed by Jian He 2017-11-06 13:28:36 -08:00
Gour Saha
d1ee95981e YARN-5828. Native services client errors out when config formats are uppercase. Contributed by Billie Rinaldi 2017-11-06 13:28:35 -08:00
Jian He
7757f6fe4a YARN-5812. Exception during GET call - "Failed to retrieve application: null". Contributed by Gour Saha 2017-11-06 13:28:35 -08:00
Gour Saha
25411a1742 YARN-5813. Slider should not try to set a negative lifetime timeout value. Contributed by Jian He 2017-11-06 13:28:35 -08:00
Jian He
3741e5518f YARN-5796. Convert enums values in service code to upper case and special handling of an error. Contributed by Gour Saha 2017-11-06 13:28:34 -08:00
Gour Saha
09e4b9e897 YARN-5690. Integrate native services modules into maven build. Contributed by Billie Rinaldi 2017-11-06 13:28:34 -08:00
Billie Rinaldi
ef5a3628c2 YARN-5770. Performance improvement of native-services REST API service. Contributed by Gour Saha 2017-11-06 13:28:34 -08:00
Gour Saha
1d7a42e795 YARN-5680. Add 2 new fields in Slider status output - image-name and is-privileged-container. Contributed by Billie Rinaldi 2017-11-06 13:28:34 -08:00
Billie Rinaldi
773c565ac5 YARN-5775. Convert enums in swagger definition to uppercase. Contributed by Gour Saha 2017-11-06 13:28:34 -08:00
Gour Saha
6bdfce3ac7 YARN-5778. Add .keep file for yarn native services AM web app. Contributed by Billie Rinaldi 2017-11-06 13:28:34 -08:00
Jian He
f206a43954 YARN-5701. Fix issues in yarn native services apps-of-apps. Contributed by Billie Rinaldi 2017-11-06 13:28:33 -08:00
Gour Saha
4ec1cbe86d YARN-5735. Make the service REST API use the app timeout feature YARN-4205. Contributed by Jian He 2017-11-06 13:28:33 -08:00
Jian He
7da243ebe0 YARN-5729. Bug fixes for the service Rest API. Contributed by Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
c0120e51b9 YARN-5689. Update native services REST API to use agentless docker provider. Contributed by Billie Rinaldi & Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
f66ecfecf6 Addendum patch for YARN-5610. Contributed by Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
295b87d0fb YARN-5675. Swagger definition for YARN service API. Contributed by Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
18bf0dce28 YARN-5610. Initial code for native services REST API. Contributed by Gour Saha 2017-11-06 13:28:33 -08:00
Jian He
9dc46aa379 YARN-5623. Apply SLIDER-1166 to yarn-native-services branch. Contributed by Gour Saha 2017-11-06 13:28:32 -08:00
Jian He
bce06ed1af YARN-5505. Create an agent-less docker provider in the native-services framework. Contributed by Billie Rinaldi 2017-11-06 13:28:32 -08:00
Jian He
3279baecb9 YARN-5538. Apply SLIDER-875 to yarn-native-services. Contributed by Billie Rinaldi 2017-11-06 13:28:32 -08:00
Jian He
02e2740bdf YARN-5513. Move Java only tests from slider develop to yarn-native-services. Contributed by Gour Saha 2017-11-06 13:28:32 -08:00
Jian He
54ec9e2440 Modify pom file for slider 2017-11-06 13:28:32 -08:00
Jian He
804bb82d1b Rename org.apache.slider.core.build to org.apache.slider.core.buildutils 2017-11-06 13:28:32 -08:00
Jian He
f47df51791 YARN-5461. Initial code ported from slider-core module. (jianhe) 2017-11-06 13:28:31 -08:00
Akira Ajisaka
785f1b0d11
YARN-7407. Moving logging APIs over to slf4j in hadoop-yarn-applications. Contributed by Yeliang Cang. 2017-10-31 16:36:02 +09:00
Yufei Gu
6b7c87c945 YARN-7355. TestDistributedShell should be scheduler agnostic. (Contributed by Haibo) 2017-10-20 11:15:20 -07:00
Wangda Tan
e490602e9b YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
 2017-09-12 09:19:12 -07:00
Sunil G
7805deed48 YARN-5588. [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Arun Suresh
c41118a7f8 YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh) 2017-09-07 10:23:12 -07:00
Andrew Wang
0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Varun Saxena
3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Sangjin Lee
092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Daniel Templeton
d81372dfad YARN-6646. Modifier 'static' is redundant for inner enums 
(Contributed by ZhangBing Lin via Daniel Templeton)
 2017-05-26 12:05:48 -07:00
Andrew Wang
539ef5aa2e HFDS-11596. hadoop-hdfs-client jar is in the wrong directory in release tarball. Contributed by Yuanbo Liu. 2017-04-05 16:04:09 -07:00
Chris Douglas
6eba79232f HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang 2017-04-03 20:13:14 -07:00
Sangjin Lee
4fa1afdb88 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-02-16 11:41:04 -08:00
Andrew Wang
5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
Daniel Templeton
71a4acf74b YARN-5991. Yarn Distributed Shell does not print throwable t to App Master When failed to start container 
(Contributed by Jim Frankola via Daniel Templeton)
 2017-01-06 14:26:08 -08:00
Robert Kanter
9262797e86 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 2016-12-27 14:14:08 -08:00
Akira Ajisaka
209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Akira Ajisaka
d3bb69a667 YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton. 2016-10-26 15:32:07 +09:00
Arun Suresh
d6d9cff21b YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 2016-08-30 15:52:29 -07:00
Andrew Wang
da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Varun Saxena
a3ac1c1943 YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 2016-07-12 20:41:52 +05:30
Sangjin Lee
6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Varun Saxena
1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Vrushali Channapattan
3832795e3c YARN-5210 NPE in Distributed Shell while publishing DS_CONTAINER_START event (Varun Saxena via Vrushali C) 2016-07-10 08:46:02 -07:00
Li Lu
0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Sangjin Lee
d5dfee24b1 YARN-5050. Code cleanup for TestDistributedShell (Li Lu via sjlee) 2016-07-10 08:45:59 -07:00
Varun Saxena
a3cf40e532 YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:55 -07:00
Sangjin Lee
0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Varun Saxena
eb0ac8efb1 YARN-4350. TestDistributedShell fails for V2 scenarios. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:50 -07:00
Li Lu
8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase. 
Contributed by Zhan Zhang.
 2016-07-10 08:45:49 -07:00
Li Lu
89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Junping Du
477a30f536 YARN-3045. Implement NM writing container lifecycle events to Timeline Service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:42 -07:00
Sangjin Lee
22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee) 
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
 2016-07-10 08:45:38 -07:00
Zhijie Shen
f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Junping Du
47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen 
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
 2016-07-10 08:45:33 -07:00
Zhijie Shen
5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du
d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen
f0e752c14b YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C. 
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
bf54d32750 YARN-3210. Refactored timeline aggregator according to new code organization proposed in YARN-3166. Contributed by Li Lu. 
(cherry picked from commit d3ff7f06cbc66d3a23c2551e7d4c752689f46afe)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
d45ff878c4 YARN-3125. Made the distributed shell use timeline service next gen and add an integration test for it. Contributed by Junping Du and Li Lu. 
(cherry picked from commit bf08f7f0ed4900ce52f98137297dd1a47ba2a536)
 2016-07-10 08:45:30 -07:00
Junping Du
9bdb5bebea Fix OOM issue in DistributedShell. Contributed by Jian He. 2016-07-09 05:09:53 -07:00
Wangda Tan
c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Varun Vasudev
42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Junping Du
48c931331c YARN-5131. Distributed shell AM fails when extra container arrives during finishing. Contributed by Wangda Tan. 2016-05-25 06:18:11 -07:00
Andrew Wang
3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang
ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Varun Vasudev
0f25a1bb52 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 2016-04-29 16:09:07 +05:30
Xuan
e6c0742012 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 2016-04-16 19:39:18 -07:00
Robert Kanter
8286270466 YARN-4436. DistShell ApplicationMaster.ExecBatScripStringtPath is misspelled (mattlamantia via rkanter) 2016-03-29 17:21:29 -07:00
Junping Du
f291d82cd4 YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du 2016-03-14 08:28:38 -07:00
Steve Loughran
e9a622606f YARN-4629. Distributed shell breaks under strong security. (Daniel Templeton via stevel) 2016-02-10 13:01:10 +00:00
Jian He
f385851141 YARN-4596. SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities. Contributed by Li Lu 2016-01-18 16:58:39 -08:00
Wangda Tan
7ff280fca9 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda) 2015-11-12 11:50:00 -08:00
Wangda Tan
c99925d6dd YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda) 2015-11-10 11:45:46 -08:00
Wangda Tan
0783184f4b YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) 2015-11-03 11:18:34 -08:00
Steve Loughran
446212a39e YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via stevel) 2015-10-24 12:55:36 +01:00
Steve Loughran
14f8dd0423 YARN-4289. TestDistributedShell failing with bind exception. (Brahma Reddy Battula via stevel) 2015-10-24 12:48:15 +01:00
Jason Lowe
2b2bd92146 YARN-3950. Add unique SHELL_ID environment variable to DistributedShell. Contributed by Robert Kanter 2015-07-29 15:16:40 +00:00
Jian He
7438966586 YARN-2821. Fixed a problem that DistributedShell AM may hang if restarted. Contributed by Varun Vasudev 2015-05-19 14:20:31 -07:00
Devaraj K
241a72af0d YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:23:53 +05:30
Akira Ajisaka
918af8efff YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 13:33:03 +09:00
Akira Ajisaka
5ce3a77f3c YARN-3444. Fix typo capabililty. Contributed by Gabor Liptak. 2015-04-25 06:08:16 +09:00
Vinod Kumar Vavilapalli
f5fe35e297 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 2015-04-23 11:19:55 -07:00
Jian He
aa4a192feb YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the correct user. Contributed by Zhijie Shen 2015-04-23 11:07:26 -07:00
Hitesh Shah
265ed1fe80 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 2015-04-08 14:13:10 -07:00
Jonathan Eagles
8180e676ab YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 2015-03-13 12:04:30 -05:00
Tsuyoshi Ozawa
24aa462673 YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature. Contributed by Wangda Tan. 2015-01-23 20:37:05 +09:00
Junping Du
d468c9aaf1 YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 2014-12-23 20:04:33 -08:00
Junping Du
856b46cf47 Revert ca3381de61 and 20674255ad as missing JIRA number in commit message 2014-12-23 20:02:33 -08:00
Junping Du
ca3381de61 Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 2014-12-23 18:57:16 -08:00
Jian He
a4f2995b9e Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367d.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-12-15 11:04:09 -08:00
Zhijie Shen
51af8d367d YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 2014-12-01 19:04:04 -08:00
Jonathan Eagles
c298a9a845 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 2014-11-20 23:34:35 -06:00
Arun C. Murthy
43cd07b408 YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 2014-11-09 14:57:37 -08:00
Vinod Kumar Vavilapalli
737d9284c1 YARN-2607. Fixed issues in TestDistributedShell. Contributed by Wangda Tan. 2014-11-08 11:00:57 -08:00
Vinod Kumar Vavilapalli
2ac1be7dec YARN-2824. Fixed Capacity Scheduler to not crash when some node-labels are not mapped to queues by making default capacities per label to be zero. Contributed by Wangda Tan. 2014-11-07 10:39:37 -08:00
Zhijie Shen
a8c1202220 YARN-2769. Fixed the problem that timeline domain is not set in distributed shell AM when using shell_command on Windows. Contributed by Varun Vasudev. 2014-10-29 12:12:30 -07:00
Jian He
f6b963fdfc YARN-2502. Changed DistributedShell to support node labels. Contributed by Wangda Tan 2014-10-27 20:13:00 -07:00
Zhijie Shen
1d4612f5ad YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 2014-10-09 12:59:47 -07:00
Zhijie Shen
52bbe0f11b YARN-2630. Prevented previous AM container status from being acquired by the current restarted AM. Contributed by Jian He. 2014-10-01 15:38:11 -07:00
Vinod Kumar Vavilapalli
a3d9934f91 YARN-1779. Fixed AMRMClient to handle AMRMTokens correctly across ResourceManager work-preserving-restart or failover. Contributed by Jian He. 2014-09-18 10:16:18 -07:00
XuanGong
8e5d6713cf YARN-2557. Add a parameter "attempt_Failures_Validity_Interval" into 
DistributedShell. Contributed by Xuan Gong
 2014-09-16 10:58:18 -07:00
Hitesh Shah
3de66011c2 YARN-2450. Fix typos in log messages. Contributed by Ray Chiang. 2014-08-29 11:16:36 -07:00
Zhijie Shen
1a535f8dcd YARN-2070. Made DistributedShell publish the short user name to the timeline server. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617837 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 21:07:18 +00:00
Jian He
9ebeac16eb YARN-2374. Fixed TestDistributedShell#testDSShell failure due to hostname dismatch. Contributed by Varun Vasudev 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616302 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 18:58:50 +00:00
Jian He
7e54b1c6d9 YARN-2354. DistributedShell may allocate more containers than client specified after AM restarts. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614538 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 03:58:59 +00:00
Jian He
f6c723ff0c YARN-2295. Refactored DistributedShell to use public APIs of protocol records. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612626 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 17:49:14 +00:00
Zhijie Shen
28058894dd YARN-2081. Fixed TestDistributedShell failure after YARN-1962. Contributed by Zhiguo Hong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596724 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 01:16:14 +00:00
Haohui Mai
d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-02 22:26:02 +00:00
Vinod Kumar Vavilapalli
d6a6e982a8 YARN-1926. Changed DistributedShell to use appIDs as unique identifiers for HDFS paths and thus avoid test failures on Windows. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586562 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 04:06:40 +00:00
Vinod Kumar Vavilapalli
6c2a0ce30b YARN-1908. Fixed DistributedShell to not fail in secure clusters. Contributed by Vinod Kumar Vavilapalli and Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585849 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-08 22:11:52 +00:00
Zhijie Shen
3af3b0bda9 YARN-1873. Fixed TestDistributedShell failure when the test cases are out of order. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582060 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 21:18:38 +00:00
Zhijie Shen
0f1eda6bbf YARN-1838. Enhanced timeline service getEntities API to get entities from a given entity ID or insertion timestamp. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580960 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 18:14:13 +00:00