hadoop

Author	SHA1	Message	Date
Szilard Nemeth	4df4389325	YARN-10911. AbstractCSQueue: Create a separate class for usernames and weights that are travelling in a Map. Contributed by Szilard Nemeth	2021-09-20 16:47:46 +02:00
Adam Antal	a9b2469a53	YARN-10950. Code cleanup in QueueCapacities (#3454 )	2021-09-19 14:42:02 +02:00
TiborKovacsCloudera	5dcb23c1f0	YARN-10937. Fix log message arguments in LogAggregationFileController (#3450 )	2021-09-19 14:35:34 +02:00
Szilard Nemeth	aa74a303ed	YARN-10913. AbstractCSQueue: Group preemption methods and fields into a separate class (#3420 )	2021-09-19 13:11:56 +02:00
adamantal	f5c76c8e31	YARN-10936. Log typo corrected. (#3449 ) Contributed by: Tibor Kovacs	2021-09-17 08:27:51 +02:00
Eric Badger	43f0a34dd4	YARN-10935. AM Total Queue Limit goes below per-user AM Limit if parent is full. Contributed by Eric Payne.	2021-09-15 20:03:45 +00:00
Benjamin Teke	5dc2f7b137	YARN-10915. AbstractCSQueue: Simplify complex logic in methods: deriveCapacityFromAbsoluteConfigurations and updateEffectiveResources (#3418 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-09-14 18:05:40 +02:00
Tamas Domok	783d94f5cd	YARN-10917. Investigate and simplify CapacitySchedulerConfigValidator#validateQueueHierarchy (#3403 ) * YARN-10917. Investigate and simplify CapacitySchedulerConfigValidator#validateQueueHierarchy. Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-14 17:54:25 +02:00
Tamas Domok	63c892278f	YARN-10912. AbstractCSQueue#updateConfigurableResourceRequirement: Separate validation logic from initialization logic (#3390 ) - capacityConfigType update is extracted to a separate method - validation logic is extracted to a helper function - min resource must not be greater than max resource is now checked after the max resource is updated Change-Id: I731c2639281721afed32c30854bafcf048d6ee28 Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-14 17:30:44 +02:00
Weihao Zheng	ad1d40970a	YARN-10928. Support default queue config for minimum-user-limit-percent/user-limit-factor (#3389 ) Contributed by Weihao Zheng	2021-09-13 11:06:53 +08:00
Jack	d8026e387e	YARN-10903. Fix the headroom check in ParentQueue and RegularContainerAllocator for DRF (#3352 ) Contributed by Jie Wang <jie.wang@hulu.com>	2021-09-13 10:54:11 +08:00
Benjamin Teke	971f1b8b0a	YARN-10872. Replace getPropsWithPrefix calls in AutoCreatedQueueTemplate (#3396 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-09-10 17:32:42 +02:00
9uapaw	811fd23f23	YARN-10852. Optimise CSConfiguration getAllUserWeightsForQueue (#3392 )	2021-09-10 16:59:46 +02:00
Benjamin Teke	b229e5a345	YARN-10910. AbstractCSQueue#setupQueueConfigs: Separate validation logic from initialization logic (#3407 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-09-10 16:48:58 +02:00
Tamas Domok	29a6f141d4	YARN-10914. Simplify duplicated code for tracking ResourceUsage in AbstractCSQueue (#3402 ) Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-10 15:57:46 +02:00
Akshat Bordia	dee6dc2f89	YARN-10893. Adding metrics for getClusterMetrics and getApplications APIs in FederationClientInterceptor (#3325 )	2021-09-09 09:20:57 -07:00
Akshat Bordia	a186460004	YARN-10829. Follow up: Adding null checks before merging ResourceUsage Report (#3252 )	2021-09-08 10:36:56 -07:00
Szilard Nemeth	2ff3fc50e4	YARN-10870. Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page. Contributed by Gergely Pollak	2021-09-08 18:01:39 +02:00
Tamas Domok	5e166898aa	YARN-10901. Permission checking error on an existing directory in LogAggregationFileController#verifyAndCreateRemoteLogDir (#3355 ) Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-08 17:04:32 +02:00
Jack	4e209a31da	YARN-10919. Remove LeafQueue#scheduler field (#3382 ) Co-authored-by: Jie Wang <jie.wang@hulu.com>	2021-09-08 16:19:29 +02:00
Benjamin Teke	40e639ad07	YARN-10646. TestCapacitySchedulerWeightMode test descriptor comments doesnt reflect the correct scenario (#3339 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-09-08 16:11:04 +02:00
zhuqi	3024a47026	Add documentation for YARN-10623 auto refresh queue conf in CS (#3279 )	2021-09-08 16:03:15 +02:00
Benjamin Teke	d9cb69853b	YARN-10522. Document for Flexible Auto Queue Creation in Capacity Scheduler	2021-09-08 15:43:57 +02:00
Szilard Nemeth	9c8fe1e512	YARN-10576. Update Capacity Scheduler documentation with JSON-based placement mapping. Contributed by Benjamin Teke	2021-09-08 15:17:27 +02:00
Swathi Chandrashekar	9512c774c0	YARN-10884: Handle empty owners to parse log files (#3318 ) * YARN 10884 : Parse log files which has empty owner * Removing the whitespace * Added a test case for null user * Fixed indentations * Fixed the indentation for test cases Co-authored-by: Swathi C <cswathi@microsoft.com>	2021-09-07 21:37:51 +05:30
Tamas Domok	16e6030e25	YARN-10891. Extend QueueInfo with max-parallel-apps in CS. (#3314 ) Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-08-27 23:09:54 +02:00
Szilard Nemeth	e06a5cb197	YARN-10838. Implement an optimised version of Configuration getPropsWithPrefix. Contributed by Andras Gyori, Benjamin Teke	2021-08-24 15:27:34 +02:00
srinivasst	4f3f26ce09	YARN-10873: Account for scheduled AM containers before deactivating node (#3287 ) * Account for scheduled AM containers before deactivating node * Move AM container check to separate method. * Fix UTs * Fix UTs * Remove unnecessary import * Add timeout for UT	2021-08-17 14:18:55 +05:30
Gergely Pollak	44bab51be4	YARN-10849 Clarify testcase documentation for TestServiceAM#testContainersReleasedWhenPreLaunchFails. Contributed by Szilard Nemeth	2021-08-04 15:43:57 +02:00
Tamas Domok	0ba6f35dc2	YARN-10355. Refactor NM ContainerLaunch.java#orderEnvByDependencies Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-08-04 15:27:46 +02:00
Tamas Domok	a5811dda7b	YARN-10874. Refactor NM ContainerLaunch#getEnvDependencies's unit tests (#3248 ) Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-08-03 16:19:04 +02:00
Viraj Jasani	ccfa072dc7	HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (#3241 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-08-03 14:44:00 +09:00
zhuqi-lucas	efb3fa2bf5	YARN-10854. Support marking inactive node as untracked without configured include path. Contributed by Tao Yang.	2021-08-02 18:23:33 +08:00
Benjamin Teke	8f750c54aa	YARN-10856. Prevent ATS v2 health check REST API call if the ATS service itself is disabled. (#3236 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-07-29 19:15:27 +02:00
Benjamin Teke	ac0a4e7f58	YARN-10869. CS considers only the default maximum-allocation-mb/vcore property as a maximum when it creates dynamic queues (#3225 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-07-29 17:56:14 +02:00
minni31	d78b300ed4	YARN-10841. Fix token reset synchronization for UAM response token. (#3194 ) YARN-10841. Fix token reset synchronization for UAM response token. Contributed by Minni Mittal	2021-07-29 14:55:39 +05:30
Szilard Nemeth	f2b6c03fc1	YARN-6272. TestAMRMClient#testAMRMClientWithContainerResourceChange fails intermittently. Contributed by Andras Gyory & Prabhu Joseph	2021-07-28 17:02:15 +02:00
Szilard Nemeth	1b9efe58c9	YARN-10790. CS Flexible AQC: Add separate parent and leaf template property. Contributed by Andras Gyori	2021-07-28 16:50:14 +02:00
Szilard Nemeth	8d0297c213	YARN-10727. ParentQueue does not validate the queue on removal. Contributed by Andras Gyori	2021-07-28 14:49:10 +02:00
9uapaw	b4a524722a	YARN-9551. TestTimelineClientV2Impl.testSyncCall fails intermittent (#3212 ) Co-authored-by: Andras Gyori <gandras@cloudera.com>	2021-07-27 19:35:20 -07:00
Szilard Nemeth	fa0289b022	YARN-6221. Entities missing from ATS when summary log file info got returned to the ATS before the domain log. Contributed by Xiaomin Zhang	2021-07-27 16:47:41 +02:00
dependabot[bot]	b7431c3ed5	[UI2] Bump http-proxy to 1.18.1 (#2891 ) Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1. - [Release notes](https://github.com/http-party/node-http-proxy/releases) - [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md) - [Commits](https://github.com/http-party/node-http-proxy/compare/1.18.0...1.18.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2021-07-26 02:16:21 +09:00
Benjamin Teke	05b6a1a06a	YARN-10833. Set the X-FRAME-OPTIONS header for the default contexts. (#3203 ) * YARN-10833. Set the X-FRAME-OPTIONS header for the default contexts. * fixup: YARN-10833. Set the X-FRAME-OPTIONS header for the default contexts. Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-07-24 11:44:21 +08:00
Akshat Bordia	aa1a5dd413	YARN-10829. Support getApplications API in FederationClientInterceptor (#3135 ) YARN-10829. Support getApplications API in FederationClientInterceptor (#3135)	2021-07-23 17:07:51 +05:30
zhuqi-lucas	2da9b95d4d	YARN-10657. We should make max application per queue to support node label. Contributed by Andras Gyori.	2021-07-22 20:30:43 +08:00
zhuqi-lucas	0441efe1fc	YARN-10860. Make max container per heartbeat configs refreshable. Contributed by Eric Badger.	2021-07-21 15:31:44 +08:00
9uapaw	e634bf33f3	YARN-10630. [UI2] Ambiguous queue name resolution (#3214 ) Co-authored-by: Andras Gyori <gandras@cloudera.com>	2021-07-21 10:06:46 +08:00
Masatake Iwasaki	17bf2fcbc5	YARN-10858. [UI2] YARN-10826 breaks Queue view. (#3213 )	2021-07-20 11:52:52 +08:00
zhuqi-lucas	0ac443b1f8	YARN-10855. yarn logs cli fails to retrieve logs if any TFile is corrupt or empty. Contributed by Jim Brennan.	2021-07-19 10:11:20 +08:00
Viraj Jasani	d0ee065cc4	HADOOP-16272. Upgrade HikariCP to 4.0.3 (#3204 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-07-16 12:17:17 +09:00
Jim Brennan	632f64cadb	YARN-10456. RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry. Contributed by Eric Payne.	2021-07-15 14:23:31 +00:00
Artem Smotrakov	ba325a8ada	HADOOP-17793. Better token validation (#3189 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-07-10 13:42:31 +09:00
Viraj Jasani	618c9218ee	HADOOP-17788. Replace IOUtils#closeQuietly usages by Hadoop's own utility (#3171 ) Reviewed-by: Steve Loughran <stevel@apache.org> Reviewed-by: Akira Ajisaka <aajisaka@apache.org> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-07-08 16:03:40 +09:00
Jim Brennan	dc6f456e95	YARN-10834. Intra-queue preemption: apps that don't use defined custom resource won't be preempted. Contributed by Eric Payne.	2021-06-28 14:52:19 +00:00
Prabhu Joseph	6b4987f09c	YARN-10820. Make GetClusterNodesRequestPBImpl thread safe. Contributed by Swathi Chandrashekar.	2021-06-27 16:28:32 +05:30
Jim Brennan	7c7d02edbd	YARN-10824. Title not set for JHS and NM webpages. Contributed by Bilwa S T.	2021-06-25 20:32:08 +00:00
Peter Bacsko	0934e783cf	YARN-10780. Optimise retrieval of configured node labels in CS queues. Contributed by Andras Gyori.	2021-06-24 20:15:10 +02:00
Masatake Iwasaki	11d144d228	YARN-10826. [UI2] Upgrade Node.js to v12.22.1. (#3128 )	2021-06-23 19:33:03 +09:00
Szilard Nemeth	6562391737	YARN-10813. Set default capacity of root for node labels. Contributed by Andras Gyori	2021-06-16 18:55:09 +02:00
Szilard Nemeth	428478bbe2	YARN-10801. Fix Auto Queue template to properly set all configuration properties. Contributed by Andras Gyori	2021-06-16 18:26:58 +02:00
Jim Brennan	9a6a11c452	YARN-10767. Yarn Logs Command retrying on Standby RM for 30 times. Contributed by D M Murali Krishna Reddy.	2021-06-15 18:58:42 +00:00
Szilard Nemeth	e31d06032b	YARN-10802. Change Capacity Scheduler minimum-user-limit-percent to accept decimal values. Contributed by Benjamin Teke	2021-06-14 22:33:04 +02:00
Szilard Nemeth	f0bdc422aa	YARN-10816. Avoid doing delegation token ops when yarn.timeline-service.http-authentication.type=simple. Contributed by Tarun Parimi	2021-06-12 15:18:41 +02:00
Szilard Nemeth	7003997e36	YARN-10789. RM HA startup can fail due to race conditions in ZKConfigurationStore. Contributed by Tarun Parimi	2021-06-12 14:49:52 +02:00
Viraj Jasani	4ef27a596f	HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom (#3087 )	2021-06-11 12:15:52 +09:00
Akira Ajisaka	9a17f97133	YARN-10803. [JDK 11] TestRMFailoverProxyProvider and TestNoHaRMFailoverProxyProvider fails by ClassCastException. (#3068 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>	2021-06-10 14:29:47 +09:00
Viraj Jasani	81d7069316	YARN-10805. Replace Guava Lists usage by Hadoop's own Lists in hadoop-yarn-project (#3075 )	2021-06-09 15:15:47 +09:00
Viraj Jasani	ca27744770	YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3081 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>	2021-06-08 19:37:23 -07:00
Prabhu Josephraj	9445abb500	YARN-10792. Set Completed AppAttempt LogsLink to Log Server URL. Contributed by Abhinaba Sarkar	2021-06-08 20:37:40 +05:30
zhuqi-lucas	ec16b1d3b9	YARN-10807. Parents node labels are incorrectly added to child queues in weight mode. Contributed by Benjamin Teke.	2021-06-08 21:03:43 +08:00
Hideyuki Furue	9c7b8cf54e	Fix container-executor Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-06-06 07:05:29 +09:00
Szilard Nemeth	200eec8f2e	YARN-10796. Capacity Scheduler: dynamic queue cannot scale out properly if its capacity is 0%. Contributed by Peter Bacsko	2021-06-03 13:27:36 +02:00
Szilard Nemeth	2707f69251	YARN-10787. Queue submit ACL check is wrong when CS queue is ambiguous. Contributed by Gergely Pollak	2021-06-01 16:01:39 +02:00
Gergely Pollak	e9339aa376	YARN-10797. Logging parameter issues in scheduler package. Contributed by Szilard Nemeth	2021-06-01 15:57:22 +02:00
Viraj Jasani	986d0a4f1d	HADOOP-17732. Keep restrict-imports-enforcer-rule for Guava Sets in hadoop-main pom (#3049 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-05-26 17:14:31 +09:00
Jim Brennan	c4581827a9	YARN-10691. DominantResourceCalculator isInvalidDivisor should consider only countable resource types. Contributed by Bilwa S T.	2021-05-25 18:07:28 +00:00
Szilard Nemeth	b86a6eb871	YARN-10782. Extend /scheduler endpoint with template properties. Contributed by Andras Gyori	2021-05-25 18:27:53 +02:00
Szilard Nemeth	2541efa496	YARN-10783. Allow definition of auto queue template properties in root. Contributed by Andras Gyori	2021-05-25 13:55:59 +02:00
Akira Ajisaka	1cbcde04f3	YARN-10770. container-executor permission is wrong in SecureContainer.md. Contributed by Siddharth Ahuja.	2021-05-25 18:28:09 +09:00
Viraj Jasani	996d31f2dc	HADOOP-17721. Replace Guava Sets usage by Hadoop's own Sets in hadoop-yarn-project (#3033 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-05-25 18:10:20 +09:00
zhuqi-lucas	59172ada90	YARN-10771. Add cluster metric for size of SchedulerEventQueue and RMEventQueue. Contributed by chaosju.	2021-05-24 23:12:07 +08:00
Szilard Nemeth	6bb0892f6e	YARN-10753. Document the removal of FS default queue creation. Contributed by Benjamin Teke	2021-05-22 15:00:08 +02:00
Szilard Nemeth	1e44bdb84c	YARN-7769. FS QueueManager should not create default queue at init. Contributed by Benjamin Teke	2021-05-22 14:55:01 +02:00
Szilard Nemeth	d146ab3af7	YARN-10766. [UI2] Bump moment-timezone to 0.5.33. Contributed by Andras Gyori	2021-05-21 18:07:32 +02:00
Szilard Nemeth	43e77caa71	YARN-10779. Add option to disable lowercase conversion in GetApplicationsRequestPBImpl and ApplicationSubmissionContextPBImpl. Contributed by Peter Bacsko	2021-05-21 18:04:24 +02:00
Akira Ajisaka	c8073811b2	YARN-9279. Remove the old hamlet package. (#2986 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-05-21 12:40:50 +09:00
dependabot[bot]	f8b0063201	Bump node-sass (#3004 ) Bumps [node-sass](https://github.com/sass/node-sass) from 4.13.0 to 4.14.1. - [Release notes](https://github.com/sass/node-sass/releases) - [Changelog](https://github.com/sass/node-sass/blob/master/CHANGELOG.md) - [Commits](https://github.com/sass/node-sass/compare/v4.13.0...v4.14.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>	2021-05-18 20:14:18 -07:00
Peter Bacsko	eb72628e15	YARN-10258. Add metrics for 'ApplicationsRunning' in NodeManager. Contributed by ANANDA G B.	2021-05-17 21:07:29 +02:00
Peter Bacsko	8891e5c028	YARN-10763. Add the number of containers assigned per second metrics to ClusterMetrics. Contributed by chaosju.	2021-05-17 13:30:12 +02:00
lujiefsi	d92a25b790	YARN-10555. Missing access check before getAppAttempts (#2608 ) Co-authored-by: lujie <lujie@foxmail.com> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-05-17 13:44:38 +09:00
kyoty	344cacce23	YARN-10545. Improve the readability of diagnostics log in yarn-ui2 web page. #2540 Contributed by kyoty.	2021-05-14 23:57:54 +08:00
zhuqi	e7f0e8073b	YARN-10761: Add more event type to RM Dispatcher event metrics. Contributed by Qi Zhu.	2021-05-14 13:36:07 +08:00
zhuqi	d2b0675d61	YARN-10737: Fix typos in CapacityScheduler#schedule. (#2911 ) Contributed by Qi Zhu.	2021-05-14 13:12:28 +08:00
dependabot[bot]	881ab4ed39	Bump underscore (#2997 ) Bumps [underscore](https://github.com/jashkenas/underscore) from 1.9.1 to 1.13.1. - [Release notes](https://github.com/jashkenas/underscore/releases) - [Commits](https://github.com/jashkenas/underscore/compare/1.9.1...1.13.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>	2021-05-13 05:54:07 -07:00
dependabot[bot]	2f67a2646e	Bump acorn (#3003 ) Bumps [acorn](https://github.com/acornjs/acorn) from 5.7.3 to 5.7.4. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](https://github.com/acornjs/acorn/compare/5.7.3...5.7.4) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>	2021-05-13 05:51:43 -07:00
Peter Bacsko	626be24c3e	YARN-10571. Refactor dynamic queue handling logic. Contributed by Andras Gyori.	2021-05-12 14:54:47 +02:00
Borislav Iordanov	91430889a5	HADOOP-17665 Ignore missing keystore configuration in reloading mechanism	2021-05-10 13:31:48 -07:00
Akira Ajisaka	7cb7b8adc1	YARN-10756. Remove additional junit 4.11 dependency from javadoc. (#2960 ) Reviewed-by: Viraj Jasani <vjasani@apache.org> Reviewed-by: Bilwa S T <stbilwa@gmail.com> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-05-06 23:37:45 +09:00
Wei-Chiu Chuang	f1e1809029	HADOOP-17653. Do not use guava's Files.createTempDir(). (#2945 ) Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-05-02 11:10:32 +09:00
Eric Badger	803ac4b1a0	YARN-10707. Support custom resources in ResourceUtilization, and update Node GPU Utilization to use. Contributed by Qi Zhu	2021-04-29 17:42:13 +00:00
Siyao Meng	5b09ba75ec	YARN-10747. Bump YARN CSI protobuf version to 3.7.1 (#2946 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org> Change-Id: Ib2a4bf775a5804641a91c6f9a486772c9d87c17e	2021-04-28 12:44:13 +09:00
Peter Bacsko	a967ab06f2	YARN-10739. GenericEventHandler.printEventQueueDetails causes RM recovery to take too much time. Contributed by Qi Zhu.	2021-04-27 14:01:53 +02:00
Peter Bacsko	9166bfeb74	YARN-10637. fs2cs: add queue autorefresh policy during conversion. Contributed by Qi Zhu.	2021-04-24 10:10:10 +02:00
Viraj Jasani	538ce9c354	HADOOP-17650. Bump solr to unblock build failure with Maven 3.8.1 (#2939 ) Reviewed-by: Siyao Meng <siyao@apache.org>	2021-04-25 23:48:39 -07:00
Eric Badger	6857a05d6a	YARN-10479. Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647. Contributed by D M Murali Krishna Reddy	2021-04-23 22:02:04 +00:00
Jim Brennan	1cbe35946b	YARN-10743. Add a policy for not aggregating for containers which are killed because exceeding container log size limit. Contributed by Qi Zhu.	2021-04-23 21:35:09 +00:00
Szilard Nemeth	ebf6b14c67	YARN-10123. Error message around yarn app -stop/start can be improved. Contributed by Siddharth Ahuja This is required to highlight that an implementation at framework level is needed for the stop/start functionality to work.	2021-04-23 22:49:26 +02:00
Peter Bacsko	14a84c47b0	YARN-10705. Misleading DEBUG log for container assignment needs to be removed when the container is actually reserved, not assigned in FairScheduler. Contributed by Siddharth Ahuja.	2021-04-23 17:38:47 +02:00
Szilard Nemeth	3b6b21e459	YARN-10750. TestMetricsInvariantChecker.testManyRuns is broken since HADOOP-17524. Contributed by Gergely Pollak	2021-04-23 16:17:29 +02:00
Szilard Nemeth	919daec36b	YARN-10746. RmWebApp add default-node-label-expression to the queue info. Contributed by Gergely Pollak	2021-04-23 16:12:12 +02:00
Szilard Nemeth	f76a2a7606	YARN-10654. Dots '.' in CSMappingRule path variables should be replaced. Contributed by Peter Bacsko	2021-04-23 16:07:58 +02:00
Eric Badger	6cb90005a7	YARN-10723. Change CS nodes page in UI to support custom resource. Contributed by Qi Zhu	2021-04-20 17:34:49 +00:00
Viraj Jasani	9a261b0a16	HADOOP-17642. Remove appender EventCounter to avoid instantiation (#2922 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-04-17 20:47:44 +09:00
Miklos Gergely	ba3bc53f4e	YARN-10736. Fix GetApplicationsRequest JavaDoc. Contributed by Miklos Gergely. (#2906 )	2021-04-14 17:16:30 +02:00
dependabot[bot]	6a9a5ae5fd	Bump y18n (#2843 ) Bumps [y18n](https://github.com/yargs/y18n) from 3.2.1 to 3.2.2. - [Release notes](https://github.com/yargs/y18n/releases) - [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md) - [Commits](https://github.com/yargs/y18n/commits) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-04-11 01:30:49 +09:00
Eric Badger	213d3deb26	YARN-10503. Support queue capacity in terms of absolute resources with custom resourceType. Contributed by Qi Zhu.	2021-04-09 00:34:15 +00:00
Peter Bacsko	ca9aa91d10	YARN-10564. Support Auto Queue Creation template configurations. Contributed by Andras Gyori.	2021-04-08 12:42:48 +02:00
Szilard Nemeth	9cd69c20c4	YARN-10714. Remove dangling dynamic queues on reinitialization. Contributed by Andras Gyori	2021-04-07 11:52:21 +02:00
Eric Badger	26b8f678b2	YARN-10702. Add cluster metric for amount of CPU used by RM Event Processor. Contributed by Jim Brennan.	2021-04-06 01:16:14 +00:00
Peter Bacsko	158758c5bf	YARN-10726. Log the size of DelegationTokenRenewer event queue in case of too many pending events. Contributed by Qi Zhu.	2021-04-01 16:09:52 +02:00
Peter Bacsko	9f1655baf2	YARN-9618. NodesListManager event improvement. Contributed by Qi Zhu.	2021-04-01 11:39:40 +02:00
Peter Bacsko	a0deda1a77	YARN-10720. YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging. Contributed by Qi Zhu.	2021-04-01 09:21:15 +02:00
Borislav Iordanov	2c482fbacf	HADOOP-16524. Automatic keystore reloading for HttpServer2 Reapply of issue reverted first because it caused yarn failures and then again because the commit message was incorrectly formatted (and yet again because of commit message format). Signed-off-by: stack <stack@apache.org>	2021-03-31 10:46:35 -07:00
stack	22961a615d	Revert "HADOOP-16524. Automatic keystore reloading for HttpServer2" This reverts commit `a2975d2153`.	2021-03-31 10:43:09 -07:00
stack	a2975d2153	HADOOP-16524. Automatic keystore reloading for HttpServer2 Reapply of issue reverted first because it caused yarn failures and then again because the commit message was incorrectly formatted.	2021-03-31 10:40:20 -07:00
stack	5183aaeda2	Revert "Hadoop 16524 - resubmission following some unit test fixes (#2693 )" Revert to fix the summary message. This reverts commit `9509bebf7f`.	2021-03-31 10:39:55 -07:00
Borislav Iordanov	9509bebf7f	Hadoop 16524 - resubmission following some unit test fixes (#2693 ) Signed-off-by: stack <stack@apache.org>	2021-03-31 10:07:42 -07:00
Szilard Nemeth	6fd0c661b6	YARN-10597. CSMappingPlacementRule should not create new instance of Groups. Contributed by Gergely Pollak	2021-03-31 16:14:21 +02:00
Peter Bacsko	ff6ec20d84	YARN-10718. Fix CapacityScheduler#initScheduler log error. Contributed by Qi Zhu.	2021-03-31 10:55:14 +02:00
Akira Ajisaka	8ad77a26c9	YARN-10716. Fix typo in ContainerRuntime. Contributed by xishuhai.	2021-03-30 17:46:51 +09:00
Brahma Reddy Battula	413a4c3c05	YARN-10466.Fix NullPointerException in yarn-services Component.java. Contributed by D M Murali Krishna Reddy	2021-03-30 13:52:07 +05:30
Brahma Reddy Battula	6577bf1891	YARN-10439. addendum fix for shaded guva.	2021-03-30 13:48:40 +05:30
Brahma Reddy Battula	b61f52ec56	YARN-10441. Add support for hadoop.http.rmwebapp.scheduler.page.class. Contributed by D M Murali Krishna Reddy	2021-03-30 09:51:47 +05:30
Brahma Reddy Battula	d0dcfc405c	YARN-10439. Yarn Service AM listens on all IP's on the machine. Contributed by D M Murali Krishna Reddy	2021-03-30 09:46:12 +05:30
Brahma Reddy Battula	2d62dced4b	YARN-10437. Destroy yarn service if any YarnException occurs during submitApp.Contributed by D M Murali Krishna Reddy	2021-03-30 09:39:00 +05:30
Eric Badger	19e418c10d	YARN-10713. ClusterMetrics should support custom resource capacity related metrics. Contributed by Qi Zhu.	2021-03-25 22:33:58 +00:00
Peter Bacsko	ceb75e1e2a	YARN-10674. fs2cs should generate auto-created queue deletion properties. Contributed by Qi Zhu.	2021-03-24 08:15:06 +01:00
Jim Brennan	174f3a96b1	YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T.	2021-03-23 18:21:45 +00:00
Jack Jiang	d8ec8ab965	HADOOP-17599. Remove NULL checks before instanceof (#2804 )	2021-03-23 08:46:11 -07:00
Cyrus Jackson	cd44e917d0	YARN-10476. Queue metrics for Unmanaged applications (#2674 ). Contributed by Cyrus Jackson	2021-03-19 15:49:05 +05:30
Eric Badger	3d4ea4b336	YARN-10701. The yarn.resource-types should support multi types without trimmed. Contributed by Qi Zhu.	2021-03-18 19:36:50 +00:00
Eric Badger	181b9ddc51	YARN-10703. Fix potential null pointer error of gpuNodeResourceUpdateHandler in NodeResourceMonitorImpl. Contributed by Qi Zhu.	2021-03-18 17:45:09 +00:00
Peter Bacsko	ce6bfd5718	YARN-10641. Refactor the max app related update, and fix maxApllications update error when add new queues. Contributed by Qi Zhu.	2021-03-18 13:40:16 +01:00
Peter Bacsko	38495af325	YARN-10692. Add Node GPU Utilization and apply to NodeMetrics. Contributed by Qi Zhu.	2021-03-18 12:46:11 +01:00
Szilard Nemeth	a5745711dd	YARN-10659. Improve CS MappingRule %secondary_group evaluation. Contributed by Gergely Pollak	2021-03-18 12:43:01 +01:00
Peter Bacsko	d7eeca4d0c	YARN-10685. Fix typos in AbstractCSQueue. Contributed by Qi Zhu.	2021-03-18 11:49:16 +01:00
Eric Badger	49f89f1d3d	YARN-10688. ClusterMetrics should support GPU capacity related metrics.. Contributed by Qi Zhu.	2021-03-17 18:11:37 +00:00
Peter Bacsko	3e58d5611d	YARN-10497. Fix an issue in CapacityScheduler which fails to delete queues. Contributed by Wangda Tan and Qi Zhu.	2021-03-17 13:38:20 +01:00
Wilfred Spiegelenburg	f276f1af80	YARN-10652. Capacity Scheduler fails to handle user weights for a user that has a "." (dot) in it This only fixes the user name resolution for weights in the queues. It does not add generic support for user names with dots in all use cases in the capacity scheduler. Contributed by: Siddharth Ahuja	2021-03-17 10:55:05 +11:00
Peter Bacsko	73394fabc7	YARN-10686. Fix TestCapacitySchedulerAutoQueueCreation#testAutoQueueCreationFailsForEmptyPathWithAQCAndWeightMode. Contributed by Qi Zhu.	2021-03-16 16:09:12 +01:00
Peter Bacsko	b80588b688	YARN-10682. The scheduler monitor policies conf should trim values separated by comma. Contributed by Qi Zhu.	2021-03-16 15:23:27 +01:00
zhuqi	e9c98548e9	YARN-10689. Fix the finding bugs in extractFloatValueFromWeightConfig. (#2760 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-03-16 13:50:29 +09:00
Eric Payne	aa4c17b9d7	YARN-10588. Percentage of queue and cluster is zero in WebUI . Contributed by Bilwa S T	2021-03-15 19:09:40 +00:00
Jim Brennan	5f067cf0f3	[YARN-10687] Add option to disable/enable free disk space checking and percentage checking for full and not-full disks. Contributed by Qi Zhu.	2021-03-12 17:17:31 +00:00
Szilard Nemeth	5db4c0bf70	YARN-10412. Move CS placement rule related changes to a separate package. Contributed by Gergely Pollak	2021-03-12 14:10:16 +01:00
Akira Ajisaka	23b343aed1	HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753 ) Removed findbugs from the hadoop build images and added spotbugs instead. Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0. Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>	2021-03-11 10:56:07 +09:00
Brahma Reddy Battula	b2a565629d	YARN-10671.Fix Typo in TestSchedulingRequestContainerAllocation. Contributed by D M Murali Krishna Reddy.	2021-03-09 20:26:07 +05:30
Peter Bacsko	d5e035dbe1	YARN-9615. Add dispatcher metrics to RM. Contributed by Jonathan Hung and Qi Zhu.	2021-03-09 14:33:14 +01:00
Peter Bacsko	3851994cd6	Revert "YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu." This reverts commit `369f75b7a7`.	2021-03-09 14:32:02 +01:00
Peter Bacsko	369f75b7a7	YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu.	2021-03-09 14:28:23 +01:00
Peter Bacsko	20416bc62d	YARN-10675. Consolidate YARN-10672 and YARN-10447. Contributed by Szilard Nemeth.	2021-03-09 11:34:37 +01:00
Peter Bacsko	e472ee2aa5	YARN-10676. Improve code quality in TestTimelineAuthenticationFilterForV1. Contributed by Szilard Nemeth.	2021-03-09 11:28:24 +01:00
Peter Bacsko	04cd3115ba	YARN-10658. CapacityScheduler QueueInfo add queue path field to avoid ambiguous QueueName. Contributed by Qi Zhu.	2021-03-08 12:29:44 +01:00
Peter Bacsko	c6b30a59dd	YARN-10672. All testcases in TestReservations are flaky. Contributed By Szilard Nemeth.	2021-03-05 23:13:35 +01:00
Eric Badger	9d088639bf	YARN-10664. Allow parameter expansion in NM_ADMIN_USER_ENV. Contributed by Jim Brennan.	2021-03-05 19:46:40 +00:00
Peter Bacsko	e82e7c597a	YARN-10640. Adjust the queue Configured capacity to Configured weight number for weight mode in UI. Contributed by Qi Zhu.	2021-03-05 15:56:56 +01:00
Peter Bacsko	e7cad3811f	YARN-10642. Race condition: AsyncDispatcher can get stuck by the changes introduced in YARN-8995. Contributed by zhengchenyu.	2021-03-05 13:50:45 +01:00
Peter Bacsko	e19c00925f	YARN-10639. Queueinfo related capacity, should adjusted to weight mode. Contributed by Qi Zhu.	2021-03-05 13:18:06 +01:00
Peter Bacsko	6699198b54	YARN-10532. Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used. Contributed by Qi Zhu.	2021-03-04 17:18:35 +01:00
Peter Bacsko	a85aeee876	YARN-10623. Capacity scheduler should support refresh queue automatically by a thread policy. Contributed by Qi Zhu.	2021-03-04 12:23:11 +01:00
Neil	d615e2d3bd	YARN-10649. Fix RMNodeImpl.updateExistContainers leak (#2719 ). Contributed by Max Xie	2021-03-04 14:52:58 +05:30
Peter Bacsko	7e8040e6ad	YARN-10655. Limit queue creation depth relative to its first static parent. Contributed by Andras Gyori.	2021-03-03 17:44:30 +01:00
JJiaguodong	8af56de1fa	HADOOP-17560. Fix some spelling errors (#2730 ) Co-authored-by: jiaguodong5 <jiaguodong5@jd.com>	2021-03-03 11:41:05 +09:00
Akira Ajisaka	3101c39b49	YARN-10656. Parsing error in CapacityScheduler.md (#2725 ) Reviewed-by: Peter Bacsko <pbacsko@apache.org>	2021-02-27 12:46:36 +09:00
Peter Bacsko	947b50489d	YARN-10627. Extend logging to give more information about weight mode. Contributed by Benjamin Teke.	2021-02-26 21:57:54 +01:00
Ahmed Hussein	e04bcb3a06	MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722 ). Contributed by Ahmed Hussein	2021-02-26 13:42:33 -06:00
Eric Badger	fcdb1e7fec	YARN-10653. Fixed the findbugs issues introduced by YARN-10647. Contributed by Qi Zhu	2021-02-26 19:03:59 +00:00
Jonathan Hung	ac37ca0b67	YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen	2021-02-25 16:01:18 -08:00
Jim Brennan	c373da9f88	[YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne	2021-02-25 16:46:19 +00:00
Peter Bacsko	a89c624002	YARN-10609. Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue). Contributed by Qi Zhu.	2021-02-24 20:41:35 +01:00
Eric Badger	47420ae3ed	YARN-10647. Fix TestRMNodeLabelsManager failed after YARN-10501. Contributed by Qi Zhu.	2021-02-22 18:48:14 +00:00
Peter Bacsko	b722c52f4e	YARN-10513. CS Flexible Auto Queue Creation RM UIv2 modifications. Contributed by Andras Gyori.	2021-02-22 12:55:45 +01:00
Eric Badger	4891e68c2b	YARN-10501. Can't remove all node labels after add node label without nodemanager port. Contributed by caozhiqiang.	2021-02-19 22:29:10 +00:00
Peter Bacsko	d28b6f90c8	YARN-10636. CS Auto Queue creation should reject submissions with empty path parts. Contributed by Gergely Pollak.	2021-02-19 13:57:30 +01:00
Peter Bacsko	4383726d19	YARN-10635. CSMapping rule can return paths with empty parts. Contributed by Gergely Pollak.	2021-02-19 12:01:31 +01:00
Akira Ajisaka	9a298d180d	Revert "HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454 )" This reverts commit `4cf3531583`.	2021-02-19 11:09:10 +09:00
Akira Ajisaka	4cf3531583	HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454 ) Use spotbugs instead of findbugs. Removed findbugs from the hadoop build images, and added spotbugs in the images instead. Reviewed-by: Masatake Iwasaki <iwasakims@apache.org> Reviewed-by: Inigo Goiri <inigoiri@apache.org> Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>	2021-02-17 10:38:20 +09:00
Szilard Nemeth	9882a22365	YARN-10625. FairScheduler: add global flag to disable AM-preemption. Contributed by Peter Bacsko	2021-02-16 18:06:57 +01:00
Jim Brennan	3fcc7a55b6	[YARN-10626] Log resource allocation in NM log at container start time. Contributed by Eric Badger	2021-02-16 16:45:35 +00:00
Szilard Nemeth	bad6038a48	YARN-10622. Fix preemption policy to exclude childless ParentQueues. Contributed by Andras Gyori	2021-02-15 14:48:42 +01:00
Szilard Nemeth	852aac34f2	Support max queues limit configuration in new auto created queue, consistent with old auto created. Contributed by Qi Zhu	2021-02-15 14:20:50 +01:00
Masatake Iwasaki	f9a073c6c1	YARN-10500. TestDelegationTokenRenewer fails intermittently. (#2619 )	2021-02-11 13:55:04 -06:00
Szilard Nemeth	98ca6afd17	YARN-10618. RM UI2 Application page shows the AM preempted containers instead of the nonAM ones. Contributed by Benjamin Teke	2021-02-11 09:14:37 +01:00
Szilard Nemeth	cacc870389	YARN-10593. Fix incorrect string comparison in GpuDiscoverer. Contributed by Peter Bacsko	2021-02-10 13:09:32 +01:00
Szilard Nemeth	a8bd516e39	YARN-10620. fs2cs: parentQueue for certain placement rules are not set during conversion. Contributed by Peter Bacsko	2021-02-10 11:15:53 +01:00
Szilard Nemeth	fc5f241535	YARN-10619. CS Mapping Rule %specified rule catches default submissions. Contributed by Gergely Pollak	2021-02-09 18:16:42 +01:00
Amogh Rajesh Desai	0b05dd8802	YARN-9650. Set thread names for CapacityScheduler AsyncScheduleThread (#2665 ) Amogh Rajesh Desai via Zhankun Tang	2021-02-09 11:05:14 +08:00
Jim Brennan	c22c77af43	[YARN-10607] User environment is unable to prepend PATH when mapreduce.admin.user.env also sets PATH. Contributed by Eric Badger.	2021-02-05 17:33:01 +00:00
Szilard Nemeth	5aa9866ec2	YARN-10615. Fix Auto Queue Creation hierarchy construction to use queue path instead of short queue name. Contributed by Andras Gyori	2021-02-05 17:43:01 +01:00
Szilard Nemeth	c19326c051	YARN-10610. Add queuePath to RESTful API for CapacityScheduler consistent with FairScheduler queuePath. Contributed by Qi Zhu	2021-02-05 17:34:45 +01:00
Szilard Nemeth	79a46599f7	YARN-10428. Zombie applications in the YARN queue using FAIR + sizebasedweight. Contributed by Guang Yang, Andras Gyori	2021-02-05 17:09:08 +01:00
Akira Ajisaka	52217fce3d	HADOOP-17432. [JDK 16] KerberosUtil#getOidInstance is broken by JEP 396 (#2546 ) Reviewed-by: Steve Loughran <stevel@apache.org>	2021-02-05 16:14:10 +09:00
Eric Badger	5e719bf588	YARN-10611. Fix that shaded should be used for google guava imports in YARN-10352. Contributed by Qi Zhu	2021-02-04 18:30:39 +00:00
Szilard Nemeth	c17414111b	YARN-10612. Fix findbugs issue introduced in YARN-10585. Contributed by Gergely Pollak	2021-02-04 08:12:13 +01:00
bibinchundatt	6fc26ad539	YARN-10352 Skip schedule on not heartbeated nodes in Multi Node Placement. Contributed by Prabhu Joseph and Qi Zhu	2021-02-03 08:50:45 +05:30
Szilard Nemeth	9bf2ac07bb	YARN-10605. Add queue-mappings-override.enable property in FS2CS conversions. Contributed by Andras Gyori	2021-02-02 09:13:46 +01:00
Szilard Nemeth	7d88953768	YARN-10604. Support auto queue creation without mapping rules. Contributed by Andras Gyori	2021-02-02 09:07:25 +01:00
belugabehr	21a3fc3d2d	HADOOP-17482: Remove Commons Logger from FileSystem Class (#2633 )	2021-02-01 09:40:01 -08:00
Peter Bacsko	fa15594ae6	YARN-10600. Convert root queue in fs2cs weight mode conversion. Contributed by Benjamin Teke.	2021-01-28 20:58:51 +01:00
Szilard Nemeth	9628aa87bf	YARN-10598. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to extend the creation type with additional information. Contributed by Benjamin Teke	2021-01-27 18:14:06 +01:00
Szilard Nemeth	7c4ef42837	YARN-10599. fs2cs should generate new 'auto-queue-creation-v2.enabled' properties for all parents. Contributed by Peter Bacsko	2021-01-27 15:49:26 +01:00
Szilard Nemeth	e2a7008d50	YARN-10585. Create a class which can convert from legacy mapping rule format to the new JSON format. Contributed by Gergely Pollak	2021-01-26 18:31:39 +01:00
Wangda Tan	4f008153ef	YARN-10587. Fix AutoCreateLeafQueueCreation cap related caculation when in absolute mode. (Qi Zhu via wangda) Change-Id: I8889c393a9f08b633c72a2e873c6f45ca12cc82b	2021-01-26 08:24:29 -08:00
Wangda Tan	b7384a8d02	YARN-10531. Be able to disable user limit factor for CapacityScheduler Leaf Queue. (Qi Zhu via wangda) Change-Id: I670e5525619b320745254609c48e7e1afb084835	2021-01-26 08:24:28 -08:00
Szilard Nemeth	f1766e5bb4	YARN-10596. Allow static definition of childless ParentQueues with auto-queue-creation-v2 enabled. Contributed by Andras Gyori	2021-01-26 16:22:35 +01:00
Szilard Nemeth	8620984b8d	YARN-10515. Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags. Contributed by Peter Bacsko	2021-01-21 20:58:57 +01:00
Szilard Nemeth	1ccba3734d	YARN-10490. yarn top command not quitting completely with ctrl+c. Contributed by Agshin Kazimli	2021-01-21 20:30:19 +01:00
Peter Bacsko	17fdac8de5	YARN-10581. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include queue creation type for queues. Contributed by Szilard Nemeth.	2021-01-21 18:06:53 +01:00
Szilard Nemeth	06fef5ee43	YARN-10579. CS Flexible AQC: Modify RM /scheduler endpoint to include weight values for queues. Contributed by Szilard Nemeth	2021-01-21 09:23:11 +01:00
bibinchundatt	8bc2dfbf36	YARN-10519. Refactor QueueMetricsForCustomResources class to move to yarn-common package. Contributed by Minni Mittal	2021-01-21 11:23:30 +05:30
Szilard Nemeth	cfe6e1f7da	YARN-10578. Fix Auto Queue Creation parent handling. Contributed by Andras Gyori	2021-01-20 15:22:44 +01:00
Sunil G	4b5bc05a78	YARN-10512. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include mode of operation for CS. Contributed by Szilard Nemeth.	2021-01-20 12:28:59 +05:30
Szilard Nemeth	a326f22606	YARN-10573. Enhance placement rule conversion in fs2cs in weight mode and enable it by default. Contributed by Peter Bacsko	2021-01-19 09:42:40 +01:00
Szilard Nemeth	6abdb148e4	YARN-10535. Make queue placement in CapacityScheduler compliant with auto-queue-placement. Contributed by Gergely Pollak	2021-01-18 20:19:36 +01:00
Szilard Nemeth	acea862d59	YARN-10574. Fix the FindBugs warning introduced in YARN-10506. Contributed by Gergely Pollak	2021-01-18 16:04:45 +01:00
Wangda Tan	3d46141583	YARN-10506. Update queue creation logic to use weight mode and allow the flexible static/dynamic creation. (Contributed by Qi Zhu, Andras Gyori) Change-Id: I118862fd5e11ee6888275e2bcf667fedfa56c5d7	2021-01-15 14:20:08 -08:00
Eric Badger	768e2f42ba	YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan.	2021-01-13 23:41:52 +00:00
Eric Payne	d8fadec944	YARN-4589: Diagnostics for localization timeouts is lacking. Contributed by Chang Li (lichangleo) and Jim Brennan (Jim_Brennan)	2021-01-13 17:01:28 +00:00
Szilard Nemeth	836c630430	YARN-10525. Add weight mode conversion to fs2cs. Contributed by Peter Bacsko	2021-01-13 14:24:12 +01:00
bibinchundatt	61f77b7674	YARN-8529. Add timeout to RouterWebServiceUtil#invokeRMWebService. Contributed by Minni Mittal	2021-01-13 16:00:11 +05:30
Szilard Nemeth	85b1c017ee	YARN-10563. Fix dependency exclusion problem in poms. Contributed by Peter Bacsko	2021-01-12 15:29:20 +01:00
Szilard Nemeth	9c6e093783	YARN-10570. Remove experimental warning message from fs2cs. Contributed by Peter Bacsko	2021-01-12 15:21:27 +01:00
Wangda Tan	b0eec09097	YARN-10504. Implement weight mode in Capacity Scheduler. (Contributed by Wangda Tan, Benjamin Teke, zhuqi, Andras Gyori) Change-Id: Ic49c730b0ab502ba86527fb662d25c4c8b1c2588	2021-01-11 17:46:09 -08:00
He Xiaoqiao	ec22850dbe	Make upstream aware of 3.2.2 release.	2021-01-09 18:06:08 +08:00
Ahmed Hussein	890f2da624	YARN-10553. Refactor TestDistributedShell (#2581 )	2021-01-09 08:39:21 +09:00
Szilard Nemeth	4176759909	YARN-10507. Add the capability to fs2cs to write the converted placement rules inside capacity-scheduler.xml. Contributed by Peter Bacsko	2021-01-08 13:40:33 +01:00
Szilard Nemeth	580a6a75a3	YARN-10528. maxAMShare should only be accepted for leaf queues, not parent queues. Contributed by Siddharth Ahuja	2021-01-08 12:38:43 +01:00
srinivasst	1b1791075a	YARN-10538: Add RECOMMISSIONING nodes to the list of updated nodes returned to the AM (#2564 ) Contributed by Srinivas S T	2021-01-08 10:52:52 +05:30
Akira Ajisaka	ae4945fb2c	YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2. (#2592 )	2021-01-06 12:12:46 +09:00
Masatake Iwasaki	5abeecd33f	YARN-10558. Fix failure of TestDistributedShell#testDSShellWithOpportunisticContainers. (#2586 )	2021-01-05 13:38:34 +09:00
Neil	fa4cf91b57	YARN-10541. capture the performance metrics of ZKRMStateStore (#2568 )	2020-12-30 22:20:12 -08:00
Ahmed Hussein	513f1995ad	YARN-10334. Close clients in TestDistributedShell (#2571 )	2020-12-28 11:09:10 -08:00
Eric Badger	4c5d88e230	YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes. Contributed by Jim Brennan.	2020-12-21 23:10:31 +00:00
zhuqi	bb528e3239	YARN-10463: For Federation, we should support getApplicationAttemptReport. (#2563 ) Qi Zhu via Zhankun Tang	2020-12-21 10:04:16 +08:00
Ahmed Hussein	7a88f45366	YARN-10536. Client in distributedShell swallows interrupt exceptions (#2554 )	2020-12-17 15:13:28 -08:00
Akira Ajisaka	df7f1e5199	YARN-10499. TestRouterWebServiceREST fails (#2490 ). Contributed by Akira Ajisaka	2020-12-16 17:26:15 +01:00
Szilard Nemeth	df35c7f519	YARN-10526. RMAppManager CS Placement ignores parent path. Contributed by Gergely Pollak	2020-12-15 17:30:14 +01:00
adamantal	3234e5eaf3	YARN-10031. Create a general purpose log request with additional query parameters. Contributed by Andras Gyori	2020-12-12 09:42:22 +01:00
adamantal	0a45bd034e	YARN-10520. Deprecated the residual nested class for the LCEResourceHandler (#2527 ). Contributed by Wanqiang Ji	2020-12-09 12:55:24 +01:00
zhuqi	d67ccd03e3	YARN-10380: Import logic of multi-node allocation in CapacityScheduler (#2494 ) Contributed by Qi Zhu.	2020-12-09 19:48:39 +08:00
Eric Badger	7d3c8ef606	YARN-10495. make the rpath of container-executor configurable. Contributed by angerszhu.	2020-12-08 00:53:24 +00:00
Andrea Scarpino	9170eb566b	YARN-10511. Update yarn.nodemanager.env-whitelist value in docs (#2512 ) Reviewed-by: Adam Antal <adamantal@apache.org> Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2020-12-04 00:14:04 +09:00
adamantal	9969745343	YARN-9883. Reshape SchedulerHealth class. Contributed by D M Murali Krishna Reddy	2020-12-03 09:55:06 +01:00
Eric Payne	fa773a8326	YARN-10278: CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework. Contributed by Szilard Nemeth (snemeth)	2020-12-01 22:06:47 +00:00
kevinzhao1661	4d2ae5b398	YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484 ) Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2020-11-30 11:17:10 +09:00
Akira Ajisaka	2ce2198287	HADOOP-17394. [JDK 11] Fix error in mvn package -Pdocs (#2488 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>	2020-11-26 11:32:10 +09:00
Akira Ajisaka	65002c9486	Revert "[JDK 11] Fix error in mvn package -Pdocs (#2488 )" because JIRA ID is missing. This reverts commit `ce5b3d7975`.	2020-11-26 11:30:53 +09:00
Akira Ajisaka	ce5b3d7975	[JDK 11] Fix error in mvn package -Pdocs (#2488 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>	2020-11-26 11:27:30 +09:00
Ankit Kumar	08b2e285db	YARN-10488. Several typos in package: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair (#2485 ) Co-authored-by: akumar <akumar@cloudera.com> Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2020-11-25 13:00:09 +09:00
Ahmed Hussein	569b20e31c	YARN-10468. Fix TestNodeStatusUpdater timeouts and broken conditions (#2461 )	2020-11-24 11:09:30 -08:00
Akira Ajisaka	c4ba0ab7df	YARN-10470. When building new web ui with root user, the bower install should support it. Contributed by zhuqi.	2020-11-24 15:22:07 +09:00
Szilard Nemeth	0d3155a687	YARN-10457. Add a configuration switch to change between legacy and JSON placement rule format. Contributed by Gergely Pollak	2020-11-18 17:02:47 +01:00
Szilard Nemeth	5ff70a59c4	YARN-10486. FS-CS converter: handle case when weight=0 and allow more lenient capacity checks in Capacity Scheduler. Contributed by Peter Bacsko	2020-11-18 16:51:42 +01:00
Ahmed Hussein	0b2510ee1f	YARN-10485. TimelineConnector swallows InterruptedException (#2450 ). Contributed by Ahmed Hussein	2020-11-16 14:55:40 -06:00
Peter Bacsko	567600fd80	YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak.	2020-11-11 17:10:15 +01:00
Peter Bacsko	cd0490e8c6	Revert "YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak." This reverts commit `b0ab222a6c`.	2020-11-11 17:06:12 +01:00
Peter Bacsko	b0ab222a6c	YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak.	2020-11-11 17:03:16 +01:00
Gabriel Medeiros Coelho	61f8c5767e	YARN-10480. replace href tags with ng-href (#2426 )	2020-11-10 10:54:28 -08:00
Akira Ajisaka	4331c88352	HADOOP-17344. Harmonize guava version and shade guava in yarn-csi. (#2434 ) Reviewed-by: Ayush Saxena <ayushsaxena@apache.org> Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-11-10 15:19:40 +09:00
Eric E Payne	55339c2bdd	YARN-10479. RMProxy should retry on SocketTimeout Exceptions. Contributed by Jim Brennan (Jim_Brennan)	2020-11-05 21:50:46 +00:00
Eric E Payne	0461a07c01	YARN-10475: Scale RM-NM heartbeat interval based on node utilization. Contributed by Jim Brennan (Jim_Brennan).	2020-11-02 16:55:06 +00:00
neko	deea5d8f2b	YARN-10469. YARN-UI2 The accuracy of the percentage values in the same chart on the YARN 'Cluster OverView' page are inconsistent (#2401 )	2020-11-02 17:52:58 +08:00
Szilard Nemeth	4890b855ec	YARN-10420. Update CS MappingRule documentation with the new format and features. Contributed by Peter Bacsko	2020-10-31 15:17:01 +01:00
Surendra Singh Lilhore	7169ec4509	YARN-10442. RM should make sure node label file highly available. (#2390 ) * YARN-10442. RM should make sure node label file highly available. * YARN-10442. Corrected method name. Added license header for file TestNodeLabelFileReplication.java Added Test code. * YARN-10442. Changed property name and log. * YARN-10442. Changed default replication for FS store file.	2020-10-30 23:05:16 +05:30
Wangda Tan	c47c9fd65d	YARN-10458. Hive On Tez queries fails upon submission to dynamically created pools. (Peter Bacsko via wangda) Change-Id: I518dc925187ce55e9d35a37ba20878c0f4e37e5c	2020-10-30 09:49:48 -07:00
Jim Brennan	8ee6bc2518	YARN-10471. Prevent logs for any container from becoming larger than a configurable size. Contributed by Eric Payne	2020-10-29 19:49:18 +00:00
Jonathan Hung	bab5bf9743	YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen	2020-10-28 10:33:18 -07:00
Sunil G	7435604a91	YARN-10453. Add partition resource info to get-node-labels and label-mappings api responses. Contributed Akhil PB.	2020-10-22 12:52:58 +05:30
Szilard Nemeth	31477a224f	YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko	2020-10-19 13:17:51 +02:00
Ayush Saxena	1e3a6efcef	HADOOP-17288. Use shaded guava from thirdparty. (#2342 ). Contributed by Ayush Saxena.	2020-10-17 12:01:18 +05:30
Eric Badger	8b8c672780	YARN-10450. Add cpu and memory utilization per node and cluster-wide metrics. Contributed by Jim Brennan.	2020-10-15 17:57:43 +00:00
Szilard Nemeth	42358169f6	YARN-10454: Add applicationName policy. Contributed by Peter Bacsko	2020-10-13 10:11:10 +02:00
Jim Brennan	deb35a32ba	YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by Ahmed Hussein	2020-10-08 18:46:47 +00:00
Jim Brennan	b361f29dda	YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne	2020-10-06 18:18:08 +00:00
Adam Antal	a1f7e760df	YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan	2020-10-05 10:09:14 +02:00
Fei Hui	d0d10f7e8f	HADOOP-17276. Extend CallerContext to make it include many items (#2327 )	2020-10-04 02:02:28 +09:00
Adam Antal	bb8446e80c	YARN-10447. TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing. Contributed by Peter Bacsko	2020-10-01 11:42:04 +02:00
Szilard Nemeth	a7a1f1541a	YARN-10413. Change fs2cs to generate mapping rules in the new format. Contributed by Peter Bacsko	2020-09-29 20:18:26 +02:00
Aryan Gupta	bbbfa7d415	HADOOP-17286. Updated jQuery in yarn-common. (#2340 )	2020-09-27 03:18:01 +09:00
Shubham Gupta	e3cd627069	YARN-10438. Handle null containerId in ClientRMService#getContainerReport() (#2313 ) Co-authored-by: Shubham Gupta <gshubham@microsoft.com>	2020-09-25 12:20:39 +05:30
Aryan Gupta	486ddb73f9	HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2020-09-24 21:15:55 +09:00
Adam Antal	7ec16ac7f6	YARN-10443. Document options of logs CLI. Contributed by Ankit Kumar. Fixes #2325	2020-09-23 17:42:56 +02:00
tkuramoto33	18f9d77c46	YARN-6754. Fair scheduler docs should explain meaning of weight=0 for a queue. (#2300 )	2020-09-23 14:50:28 +09:00
Hui Fei	474fa80bfb	HADOOP-17277. Correct spelling errors for separator (#2322 ) Contributed by Hui Fei.	2020-09-22 17:10:33 +01:00
Szilard Nemeth	eacbe07b56	YARN-9333. TestFairSchedulerPreemption.testRelaxLocalityPreemptionWithNoLessAMInRemainingNodes fails intermittently. Contributed by Peter Bacsko	2020-09-17 19:57:19 +02:00
Jim Brennan	90894ea641	YARN-10430. Log improvements in NodeStatusUpdaterImpl. Contributed by Bilwa S T.	2020-09-14 21:19:14 +00:00
Eric E Payne	9afec2ed17	YARN-10390: LeafQueue: retain user limits cache across assignContainers() calls. Contributed by Samir Khan (samkhan).	2020-09-11 13:29:26 +00:00
Szilard Nemeth	773ac799c6	YARN-10387. Implement logic which returns MappingRule objects based on mapping rules. Contributed by Peter Bacsko	2020-09-09 15:12:58 +02:00
Brahma Reddy Battula	43572fc7f8	YARN-10397. SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints. Contributed by Bilwa S T.	2020-09-09 17:08:13 +05:30
Peter Bacsko	aba4a506d6	YARN-10424. Adapt existing AppName and UserGroupMapping unittests to ensure backwards compatibility. Contributed by Benjamin Teke.	2020-09-09 09:41:11 +02:00
Peter Bacsko	c4fb4044b2	YARN-10415. Create a group matcher which checks ALL groups of the user. Contributed by Gergely Pollak.	2020-09-08 10:57:00 +02:00
Adam Antal	34fe74da0e	YARN-10332. RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. Contributed by yehuanhuan	2020-09-07 11:44:09 +02:00
Peter Bacsko	360bbcd3bc	YARN-10411. Create an allowCreate flag for MappingRuleAction. Contributed by Gergely Pollak.	2020-09-07 11:39:03 +02:00
Adam Antal	dd6c66556e	YARN-9136. getNMResourceInfo NodeManager REST API method is not documented	2020-09-07 10:29:20 +02:00
Peter Bacsko	1841a5bb03	YARN-10376. Create a class that covers the functionality of UserGroupMappingPlacementRule and AppNameMappingPlacementRule using the new mapping rules. Contributed by Gergely Pollak.	2020-09-05 10:00:33 +02:00
Szilard Nemeth	696e4fe50e	Typos in YarnScheduler#allocate method's doc comment. Contributed by Siddharth Ahuja	2020-09-04 09:59:54 +02:00
Peter Bacsko	5c15815773	YARN-10375. CS Mapping rule config parser should return MappingRule objects. Contributed by Gergely Pollak.	2020-09-03 18:54:22 +02:00
Peter Bacsko	288dab7563	YARN-10372. Create MappingRule class to represent each CS mapping rule. Contributed by Gergely Pollak	2020-09-03 15:02:48 +02:00
Masatake Iwasaki	5e12dc51f0	YARN-10419. Javadoc error in hadoop-yarn-server-common module. (#2271 )	2020-09-03 16:17:52 +09:00
Peter Bacsko	c7dab2b23e	YARN-10374. Create Actions for CS mapping rules. Contributed by Gergely Pollak.	2020-09-01 17:38:49 +02:00
Adam Antal	73a0d149e2	YARN-10386. Addendum patch: Fixing ASF Warnings. Contributed by Peter Bacsko.	2020-09-01 14:39:24 +02:00
Szilard Nemeth	4ffe26f9b8	[UI1] Provide a way to hide Tools section in Web UIv1. Contributed by Andras Gyori	2020-08-31 16:00:39 +02:00
Szilard Nemeth	56a5c360a1	YARN-10373. Create Matchers for CS mapping rules. Contributed by Gergely Pollak	2020-08-29 21:34:55 +02:00
Szilard Nemeth	f4f872b778	YARN-10371. Create variable context class for CS queue mapping rules. Contributed by Gergely Pollak	2020-08-29 17:31:48 +02:00
Szilard Nemeth	a888d580d8	YARN-10409. Improve MockQueueHierarchyBuilder to detect queue ambiguity. Contributed by Gergely Pollak	2020-08-29 15:24:50 +02:00
Szilard Nemeth	44542863f4	YARN-10408. Extract MockQueueHierarchyBuilder to a separate class. Contributed by Gergely Pollak	2020-08-28 17:35:45 +02:00
Szilard Nemeth	6e3d7050a9	YARN-10386. Create new JSON schema for Placement Rules. Contributed by Peter Bacsko and Benjamin Teke	2020-08-28 17:09:31 +02:00
Prabhu Joseph	75db5526b5	YARN-1806. Add ThreadDump Option in YARN UI2 to fetch for running containers Contributed by Siddharth Ahuja. Reviewed by Akhil PB.	2020-08-26 13:08:17 +05:30
Adam Antal	82a7505646	YARN-10304. Create an endpoint for remote application log directory path query. Contributed by Andras Gyori	2020-08-25 13:29:12 +02:00
Adam Antal	c049296235	YARN-10106. Yarn logs CLI filtering by application attempt. Contributed by Hudáky Márton Gyula	2020-08-25 09:53:39 +02:00
Prabhu Joseph	5c927eb550	YARN-10360. Support Multi Node Placement in SingleConstraintAppPlacementAllocator Reviewed by Sunil G.	2020-08-24 15:00:02 +05:30
Zhengbo Li	9b9f7ea16a	YARN-10399 Refactor NodeQueueLoadMonitor class to make it extendable (#2228 ) Refactor NodeQueueLoadMonitor class to make it extendable	2020-08-19 10:14:50 -07:00
Sunil G	82ec28f442	YARN-10396. Max applications calculation per queue disregards queue level settings in absolute mode. Contributed by Benjamin Teke.	2020-08-19 11:54:48 +05:30
Jim Brennan	4a7deae478	YARN-10391. --module-gpu functionality is broken in container-executor. Contributed by Eric Badger	2020-08-17 16:41:15 +00:00
hemanthboyina	86bbd38c8d	YARN-10336. RM page should throw exception when command injected in RM REST API to get applications. Contributed by Bilwa S T.	2020-08-13 23:44:42 +05:30
Prabhu Joseph	6c2ce3d56b	YARN-10389. Option to override RMWebServices with custom WebService class Contributed by Tanu Ajmera. Reviewed by Bilwa ST and Sunil G.	2020-08-11 15:46:47 +05:30
Prabhu Joseph	5e0f879779	YARN-10364. Fix logic of queue capacity is absolute resource or percentage. Contributed by Bilwa ST. Reviewed by Sunil G.	2020-08-08 22:33:58 +05:30
Jonathan Hung	64753addba	YARN-10251. Show extended resources on legacy RM UI. Contributed by Eric Payne	2020-08-07 17:43:01 -07:00
bibinchundatt	81da221c75	YARN-10388. RMNode updatedCapability flag not set while RecommissionNodeTransition. Contributed by Pranjal Protim Borah	2020-08-07 08:36:52 +05:30
Prabhu Joseph	c7e71a6c0b	YARN-10361. Make custom DAO classes configurable into RMWebApp#JAXBContextResolver. Contributed by Bilwa ST.	2020-08-06 13:19:52 +05:30
Eric Badger	5edd8b925e	YARN-4575. ApplicationResourceUsageReport should return ALL reserved resource. Contributed by Bibin Chundatt and Eric Payne.	2020-08-05 18:47:49 +00:00
Prabhu Joseph	0277856738	YARN-10377. Fix filter index to show apps while clicking on queue in RM UI Contributed by Tarun Parimi.	2020-08-04 23:10:45 +05:30
Prabhu Joseph	aa5afa72c5	YARN-10381. Add application attempt state in AppAttempts RM REST API Contributed by Siddharth Ahuja. Reviewed by Bilwa ST.	2020-08-04 23:07:28 +05:30
kevinzhao1661	ab2b3df2de	YARN-10383. YarnCommands.md is inconsistent with the source code (#2177 )	2020-08-04 12:02:16 +09:00
Brahma Reddy Battula	eac558380f	YARN-10229. [Federation] Client should be able to submit application to RM directly using normal client conf. Contributed by Bilwa S T.	2020-08-03 12:54:36 +05:30
bibinchundatt	5323e83edf	YARN-10359. Log container report only if list is not empty. Contributed by Bilwa S T.	2020-08-01 13:03:46 +05:30
Eric E Payne	e0c9653166	YARN-1529: Add Localization overhead metrics to NM. Contributed by Jim_Brennan.	2020-07-30 15:30:22 +00:00
bibinchundatt	5d8600e80a	YARN-10369. Make NMTokenSecretManagerInRM sending NMToken for nodeId DEBUG. Contributed by Jim Brennan.	2020-07-29 23:26:08 +05:30
Jonathan Hung	3eaf62726f	YARN-10343. Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. Contributed by Eric Payne	2020-07-28 13:43:19 -07:00
bibinchundatt	5dadf963d3	YARN-10208. Add capacityScheduler metric for NODE_UPDATE interval. Contributed by Pranjal Protim Borah.	2020-07-28 11:55:47 +05:30
Prabhu Joseph	60a254621a	YARN-10366. Fix Yarn rmadmin help message shows two labels for one node for --replaceLabelsOnNode. Contributed by Tanu Ajmera.	2020-07-27 12:23:21 +05:30
Akira Ajisaka	d02be17a26	YARN-10362. Javadoc for TimelineReaderAuthenticationFilterInitializer is broken. Contributed by Xieming Li.	2020-07-27 01:55:04 +09:00
Eric Badger	ac5f21dbef	YARN-4771. Some containers can be skipped during log aggregation after NM restart. Contributed by Jason Lowe and Jim Brennan.	2020-07-24 22:35:16 +00:00
Prabhu Joseph	247eb0979b	YARN-10319. Record Last N Scheduler Activities from ActivitiesManager Reviewed by Tao Yang and Adam Antal.	2020-07-24 11:43:31 +05:30
bibinchundatt	bfcd775381	YARN-10315. Avoid sending RMNodeResourceupdate event if resource is same. Contributed by Sushil Ks.	2020-07-23 16:41:35 +05:30
Masatake Iwasaki	1b29c9bfee	HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. (#2155 )	2020-07-22 13:40:20 +09:00
Eric Badger	736bed6d6d	[YARN-10353] Log vcores used and cumulative cpu in containers monitor. Contributed by Jim Brennan	2020-07-20 18:08:27 +00:00
Ayush Saxena	6bcb24d269	HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.	2020-07-18 14:33:43 +05:30
Prabhu Joseph	cc71d50b21	YARN-10339. Fix TimelineClient in NodeManager failing when Simple Http Auth used in Secure Cluster Contributed by Tarun Parimi.	2020-07-16 23:36:49 +05:30
Ahmed Hussein	1f71c4ae71	HADOOP-17099. Replace Guava Predicate with Java8+ Predicate Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-07-15 11:39:06 -05:00
Ahmed Hussein	98fcffe93f	HADOOP-17101. Replace Guava Function with Java8+ Function Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-07-15 09:53:18 -05:00
Bilwa S T	8980c23fc2	YARN-10350. TestUserGroupMappingPlacementRule fails Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2020-07-15 14:31:34 +09:00
Eric Badger	48f90115b5	YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by Jim Brennan	2020-07-13 23:10:39 +00:00
Eric E Payne	0427100b75	YARN-10297. TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently. Contributed by Jim Brennan (Jim_Brennan)	2020-07-13 18:57:50 +00:00
Xiaoyu Yao	f91a8ad88b	HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. (#2085 )	2020-07-09 11:33:37 -07:00
Sunil G	5dd270e208	YARN-10333. YarnClient obtain Delegation Token for Log Aggregation Path. Contributed by Prabhu Joseph.	2020-07-09 12:50:25 +05:30
Brahma Reddy Battula	dfe60392c9	YARN-10341. Yarn Service Container Completed event doesn't get processed. Contributed by Bilwa S T.	2020-07-09 12:34:52 +05:30
Akira Ajisaka	10d218934c	YARN-10344. Sync netty versions in hadoop-yarn-csi. (#2126 )	2020-07-09 10:29:47 +05:30
Prabhu Joseph	3a4d05b850	YARN-8047. RMWebApp make external class pluggable. Contributed by Bilwa S T.	2020-07-08 12:43:20 +05:30
Prabhu Joseph	2bbd00dff4	YARN-10337. Fix failing testcase TestRMHATimelineCollectors. Contributed by Bilwa S T.	2020-07-07 17:10:59 +05:30
Akira Ajisaka	639acb6d89	HADOOP-17111. Replace Guava Optional with Java8+ Optional. Contributed by Ahmed Hussein.	2020-07-06 16:08:36 +09:00
Szilard Nemeth	04abd0eb17	YARN-10330. Add missing test scenarios to TestUserGroupMappingPlacementRule and TestAppNameMappingPlacementRule. Contributed by Peter Bacsko	2020-07-01 14:10:55 +02:00
Szilard Nemeth	9b5557a9e8	YARN-10325. Document max-parallel-apps for Capacity Scheduler. Contributed by Peter Bacsko	2020-07-01 13:41:30 +02:00
Eric Yang	e8dc862d38	YARN-9809. Added node manager health status to resource manager registration call. Contributed by Eric Badger via eyang	2020-06-30 11:39:16 -07:00
Akira Ajisaka	cd188ea9f0	YARN-10331. Upgrade node.js to 10.21.0. (#2106 )	2020-06-30 16:52:57 +09:00
Eric E Payne	74fc13cf91	YARN-9903: Support reservations continue looking for Node Labels. Contributed by Jim Brennan (Jim_Brennan).	2020-06-29 18:39:53 +00:00
Eric Yang	0be26811f3	YARN-10328. Fixed ZK Curator NodeExists exception in YARN service AM logs Contributed by Bilwa S T via eyang	2020-06-29 09:21:24 -07:00
Brahma Reddy Battula	2c03524fa4	YARN-6526. Refactoring SQLFederationStateStore by avoiding to recreate a connection at every call. COntributed by Bilwa S T.	2020-06-26 20:43:27 +05:30
Szilard Nemeth	e0c1d8a969	YARN-10327. Remove duplication of checking for invalid application ID in TestLogsCLI. Contributed by Marton Hudaky	2020-06-26 13:21:55 +02:00
Szilard Nemeth	6459cc73e6	YARN-10318. ApplicationHistory Web UI incorrect column indexing. Contributed by Andras Gyori	2020-06-26 13:15:14 +02:00
Szilard Nemeth	33d3df7ee2	YARN-10277. CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy. Contributed by Szilard Nemeth	2020-06-26 11:14:24 +02:00
Szilard Nemeth	6a8fd73b27	YARN-10279. Avoid unnecessary QueueMappingEntity creations. Contributed by Marton Hudaky	2020-06-25 17:27:58 +02:00
Szilard Nemeth	03f855e3e7	YARN-10316. FS-CS converter: convert maxAppsDefault, maxRunningApps settings. Contributed by Peter Bacsko	2020-06-23 12:13:04 +02:00
Prabhu Joseph	ce1008fe61	YARN-10321. Break down TestUserGroupMappingPlacementRule#testMapping into test scenarios Contributed by Szilard Nemeth. Reviewed by Gergely Pollak.	2020-06-22 12:49:14 +05:30
Surendra Singh Lilhore	b2facc84a1	YARN-9460. QueueACLsManager and ReservationsACLManager should not use instanceof checks. Contributed by Bilwa S T.	2020-06-20 20:09:31 +05:30
Szilard Nemeth	469841446f	YARN-9930. Support max running app logic for CapacityScheduler. Contributed by Peter Bacsko	2020-06-19 14:50:24 +02:00
Eric Yang	89689c52c3	YARN-10308. Update javadoc and variable names for YARN service. Contributed by Bilwa S T via eyang	2020-06-17 09:04:26 -07:00
Szilard Nemeth	5b1a56f9f1	YARN-10281. Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule. Contributed by Gergely Pollak	2020-06-17 14:34:40 +02:00
Vinayakumar B	e154084770	HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026 )	2020-06-12 23:16:33 +05:30
Tao Yang	7c4de59fc1	YARN-10293. Reserved Containers not allocated from available space of other nodes in CandidateNodeSet for MultiNodePlacement. Contributed by Prabhu Joseph.	2020-06-12 22:10:25 +08:00
Eric Badger	fed6fecd3a	YARN-10312. Add support for yarn logs -logFile to retain backward compatibility. Contributed by Jim Brennan.	2020-06-11 21:02:41 +00:00
Eric E Payne	56247db302	YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger).	2020-06-09 18:43:16 +00:00
Szilard Nemeth	8abff5151a	YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke	2020-06-05 15:00:39 +02:00
Szilard Nemeth	8b146c17b3	YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak	2020-06-05 11:38:32 +02:00
Szilard Nemeth	a4835db95a	YARN-10296. Make ContainerPBImpl#getId/setId synchronized. Contributed by Benjamin Teke	2020-06-05 10:11:40 +02:00
Szilard Nemeth	b5efdea4fd	YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak	2020-06-02 18:29:15 +02:00
Szilard Nemeth	aa6d13455b	YARN-10284. Add lazy initialization of LogAggregationFileControllerFactory in LogServlet. Contributed by Adam Antal	2020-06-02 15:53:48 +02:00
Szilard Nemeth	e0a0741ac8	YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori	2020-06-02 15:42:39 +02:00
Szilard Nemeth	593af878c0	YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak	2020-05-27 13:05:52 +02:00
Szilard Nemeth	fb030f3239	Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak" This reverts commit `ac4540dd8e`. Reverting because of missing Jira-ID in the commit message.	2020-05-27 13:04:48 +02:00
Jonathan Hung	c30c23cb66	YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R	2020-05-26 16:29:42 -07:00
Szilard Nemeth	ac4540dd8e	FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak	2020-05-21 13:11:44 +02:00
Eric Yang	726b8e324b	YARN-10228. Relax restriction of file path character in yarn.service.am.java.opts. Contributed by Bilwa S T via eyang	2020-05-20 09:23:56 -07:00
Prabhu Joseph	cef0756929	YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. Contributed by Bilwa S T. Reviewed by Akira Ajisaka.	2020-05-20 11:36:52 +05:30
Akira Ajisaka	b65815d691	Revert "YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient." This reverts commit `7836bc4c35`.	2020-05-18 16:29:07 +09:00
Prabhu Joseph	7836bc4c35	YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. Contributed by Bilwa S T. Reviewed by Brahma Reddy Battula.	2020-05-14 19:40:42 +05:30
Prabhu Joseph	6ce295b787	YARN-10259. Fix reservation logic in Multi Node Placement. Reviewed by Wangda Tan.	2020-05-14 16:52:11 +05:30
Surendra Singh Lilhore	1958cb7c2b	YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng	2020-05-14 15:09:34 +05:30
Ayush Saxena	0918433b4d	YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.	2020-05-14 00:36:20 +05:30
Inigo Goiri	108ecf992f	YARN-8942. PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value. Contributed by Bilwa S T.	2020-05-13 10:04:12 -07:00
Prabhu Joseph	450e5aa9dd	YARN-10154. Addendum Patch which fixes below bugs 1. RM fails to start when LeafQueueTemplate max capacity is not specified. 2. Job stuck in ACCEPTED state with DominantResourceCalculator as Queue Capacity is set to NaN during RM startup with clusterResource is zero. Reviewed by Sunil G and Manikandan R.	2020-05-13 14:35:37 +05:30
Jonathan Hung	fff1d2c122	YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T	2020-05-12 10:48:54 -07:00
Elixir Kook	a3f945fb84	HADOOP-17035. fixed typos (timeout, interruped) (#2007 ) Co-authored-by: Sungpeo Kook <elixir.kook@kakaocorp.com>	2020-05-12 10:50:04 -05:00
Inigo Goiri	35010120fb	YARN-9017. PlacementRule order is not maintained in CS. Contributed by Bilwa S T.	2020-05-06 13:22:54 -07:00
Ahmed Hussein	92e3ebb401	YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-05-06 12:27:17 -05:00
Szilard Nemeth	cb6399c109	YARN-10257. FS-CS converter: skip increment properties for mem/vcores and fix DRF check. Contributed by Peter Bacsko	2020-05-05 16:00:43 +02:00
Szilard Nemeth	0debe55d6c	YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph	2020-05-05 13:11:28 +02:00
Ahmed Hussein	f5081a9a5d	YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles) Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-05-04 10:13:51 -05:00
Surendra Singh Lilhore	257b04ce4e	YARN-6553. Replace MockResourceManagerFacade with MockRM for AMRMProxy/Router tests. Contributed by Bilwa S T.	2020-05-02 12:43:20 +05:30
Inigo Goiri	d125d39108	YARN-6973. Adding RM Cluster Id in ApplicationReport. Contributed by Bilwa S T.	2020-04-29 11:23:58 -07:00
Szilard Nemeth	410c605aec	YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G	2020-04-29 15:53:30 +02:00
Szilard Nemeth	ab36429559	YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori	2020-04-28 18:53:28 +02:00
Akira Ajisaka	6ef01646ba	YARN-9848. Revert YARN-4946. Contributed by Steven Rand.	2020-04-28 05:35:36 +09:00
Prabhu Joseph	9224568b0e	YARN-10156. Destroy Jersey Client in TimelineConnector. Contributed by Tanu Ajmera.	2020-04-27 18:49:15 +05:30
Szilard Nemeth	f91e21ac10	YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph	2020-04-27 08:26:11 +02:00
Szilard Nemeth	d4874585f4	YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph	2020-04-24 17:04:47 +02:00
Akira Ajisaka	9827ff2961	YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. (#1939 )	2020-04-24 09:32:25 +09:00
Prabhu Joseph	60fa15366e	YARN-10240. Prevent Fatal CancelledException in TimelineV2Client when stopping Contributed by Tarun Parimi.	2020-04-21 12:01:33 +05:30
Szilard Nemeth	92b28e38b7	YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja	2020-04-18 10:13:37 +02:00
Szilard Nemeth	76900b4f5b	YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke	2020-04-18 09:57:27 +02:00
Jonathan Hung	3601054822	YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T	2020-04-17 10:34:44 -07:00
Sunil G	2fe122e322	YARN-10154. CS Dynamic Queues cannot be configured with absolute resources. Contributed by Manikandan R.	2020-04-17 18:51:19 +05:30
Szilard Nemeth	4db598e0e6	YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke	2020-04-15 13:18:44 +02:00
Prabhu Joseph	94f7470c71	YARN-10233. Fix YARN UI2 Daemon Logs Contributed by Akhil PB.	2020-04-15 13:39:37 +05:30
Szilard Nemeth	55fcbcb5c2	YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko	2020-04-15 07:24:04 +02:00
Prabhu Joseph	c791b0e90e	YARN-10219. Fix YARN Native Service Placement Constraints with Node Attributes. Contributed by Eric Yang.	2020-04-14 12:34:53 +05:30
Jonathan Hung	23481ad378	YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T	2020-04-09 11:06:11 -07:00
Szilard Nemeth	c04a484aed	YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T.	2020-04-09 15:46:25 +02:00
Sunil G	6fb29d0f17	YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko.	2020-04-09 18:51:42 +05:30
Szilard Nemeth	8e4517f251	YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja	2020-04-09 12:31:01 +02:00
Akira Ajisaka	e53d472bb0	HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 (#1933 )	2020-04-09 01:47:31 +09:00
Wilfred Spiegelenburg	2214005c0f	YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja	2020-04-08 11:30:03 +10:00
Szilard Nemeth	bffb43b00e	YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja	2020-04-07 17:03:17 +02:00
Szilard Nemeth	45362a9f4c	YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja	2020-04-07 15:51:55 +02:00
Prabhu Joseph	7a39da3537	YARN-10120. Amendment fix for Java Doc. Contributed by Bilwa S T.	2020-04-03 13:07:41 +05:30
Prabhu Joseph	25361b077b	YARN-10120. Https Support in Router WebServiceClient. Contributed by Bilwa S T.	2020-04-02 18:11:43 +05:30
Akira Ajisaka	c162648aff	YARN-10202. Fix documentation about NodeAttributes. Contributed by Sen Zhao.	2020-04-01 16:05:47 +09:00
Brahma Reddy Battula	8914cf9167	Preparing for 3.4.0 development	2020-03-29 23:24:25 +05:30
RogPodge	50f7f6dfd1	YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover.	2020-03-26 08:14:30 -07:00
Szilard Nemeth	3d5ade1839	YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R	2020-03-26 13:23:43 +01:00
Szilard Nemeth	9a297ff31d	YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko	2020-03-26 12:35:10 +01:00
Szilard Nemeth	348685dcb9	YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke	2020-03-26 12:18:15 +01:00
Jonathan Hung	6ce189c621	YARN-10200. Add number of containers to RMAppManager summary	2020-03-25 10:41:59 -07:00
RogPodge	2d294bd575	HADOOP-16938. Make non-HA proxy providers pluggable	2020-03-25 08:06:58 -07:00
Sunil G	cdb2107066	YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak.	2020-03-25 16:50:19 +05:30
Prabhu Joseph	d312991ead	YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue Contributed by Peter Bacsko. Reviewed by Manikandan R.	2020-03-23 18:01:14 +05:30
Prabhu Joseph	f2d3ac2a3f	YARN-10034. Remove Allocation Tags from released container from Decommission node Contributed by Kyungwan Nam. Reviewed by Adam Antal.	2020-03-19 12:55:14 +05:30
Eric Badger	1975479285	YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed Hussein.	2020-03-16 22:24:02 +00:00
Weiwei Yang	b9d825f178	YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang.	2020-03-13 14:39:36 -07:00
Weiwei Yang	d273f6a2b4	YARN-9538. Document scheduler/app activities and REST APIs. Contributed by Tao Yang.	2020-03-13 14:36:52 -07:00
Bilwa	c83644deac	YARN-10110. Adding RouterPolicyProvider for RM Federation. Contributed by Bilwa S T.	2020-03-13 16:56:44 +05:30
Szilard Nemeth	5ead9c15ca	YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori	2020-03-12 12:29:03 +01:00
Szilard Nemeth	38d87883b6	YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko	2020-03-12 12:18:37 +01:00
Sunil G	0fd8bf5f6b	YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko.	2020-03-12 15:03:12 +05:30
Szilard Nemeth	cf9cf83a43	YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori	2020-03-10 16:44:48 +01:00
Szilard Nemeth	61f4cf3055	YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke	2020-03-10 16:35:04 +01:00
Szilard Nemeth	9314ef947f	YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko	2020-03-10 16:07:46 +01:00
Eric E Payne	ede05b19d1	YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein)	2020-03-10 14:17:45 +00:00
Szilard Nemeth	44afe1154d	YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori	2020-03-09 16:08:24 +01:00
bibinchundatt	3859fa76d0	YARN-6924. Metrics for Federation AMRMProxy. Contributed by Young Chen	2020-03-07 09:34:42 +05:30
Szilard Nemeth	ea0444851d	YARN.10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke	2020-03-06 17:49:45 +01:00
Szilard Nemeth	004e955348	YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. Contributed by Peter Bacsko	2020-03-05 09:56:42 +01:00
Eric Badger	2649f8b327	YARN-10173. Make pid file generation timeout configurable in case of reacquired container. Contributed by Adam Antal.	2020-03-04 23:31:57 +00:00
Szilard Nemeth	c0d084247c	YARN-10175. FS-CS converter: only convert placement rules if a cmd line switch is defined. Contributed by Peter Bacsko	2020-03-03 09:46:32 +01:00
Szilard Nemeth	e9eecedf69	YARN-10148. addendum: Fix method call parameter order of setAdminAndSubmitACL in TestCapacitySchedulerQueueACLs. Contributed by Kinga Marton	2020-03-02 16:25:14 +01:00
Inigo Goiri	a43510e21d	YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan.	2020-02-27 13:18:30 -08:00
Inigo Goiri	b420ddeada	YARN-10155. TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk. Contributed by Manikandan R.	2020-02-27 12:53:20 -08:00
Szilard Nemeth	10461e0193	YARN-10148. Add Unit test for queue ACL for both FS and CS. Contributed by Kinga Marton	2020-02-27 20:10:32 +01:00
Akira Ajisaka	7dfa37e8f0	YARN-10152. Fix findbugs warnings in hadoop-yarn-applications-mawo-core module (#1852 )	2020-02-27 10:36:49 +09:00
Brahma Reddy Battula	3a9ccf7f6d	YARN-10141.Interceptor in FederationInterceptorREST doesnt update on RM switchover. Contributed by D M Murali Krishna Reddy.	2020-02-26 23:24:00 +05:30
tanu.ajmera	b5df1da3a1	YARN-9593. Support Comma in the value of Scheduler Configuration Mutation API.	2020-02-26 15:04:29 +05:30
Elixir Kook	d608e94f92	YARN-10156. Fix typo 'complaint' which means quite different in Federation.md (#1856 )	2020-02-26 17:29:26 +09:00
Szilard Nemeth	d68616b045	YARN-10130. FS-CS converter: Do not allow output dir to be the same as input dir. Contributed by Adam Antal	2020-02-25 21:30:04 +01:00
Wei-Chiu Chuang	dda00d3ff5	YARN-10074. Update netty to 4.1.42Final in yarn-csi. Contributed by Wei-Chiu Chuang. (#1807 )	2020-02-25 13:47:52 +09:00
Szilard Nemeth	72fa9c7f1b	YARN-10135. FS-CS converter tool: issue warning on dynamic auto-create mapping rules. Contributed by Peter Bacsko	2020-02-24 21:54:07 +01:00
Szilard Nemeth	34cf63c7d0	YARN-10157. FS-CS converter: initPropertyActions() is not called without rules file. Contributed by Peter Bacsko	2020-02-24 21:45:49 +01:00
Prabhu Joseph	181e6d033e	YARN-10119. Option to reset AM failure count for YARN Service Contributed by kyungwan nam.	2020-02-20 20:34:06 +05:30
Szilard Nemeth	4af2556b48	YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal	2020-02-20 15:27:15 +01:00
Szilard Nemeth	3f1aad05f0	YARN-10147. FPGA plugin can't find the localized aocx file. Contributed by Peter Bacsko	2020-02-19 15:54:25 +01:00
Sunil G	6526f95bd2	YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph.	2020-02-19 11:17:22 +05:30
Akira Ajisaka	d6d7f8d8c5	YARN-8374. Upgrade objenesis to 2.6 (#1798 )	2020-02-19 09:50:37 +09:00
Brahma Reddy Battula	20add89718	YARN-10136. [Router] : Application metrics are hardcode as N/A in UI. Contributed by Bilwa S T.	2020-02-14 16:50:29 +05:30
Szilard Nemeth	da99ac7e93	YARN-10137. UIv2 build is broken in trunk. Contributed by Adam Antal	2020-02-13 16:31:35 +01:00
Szilard Nemeth	f1b1b332f5	YARN-10029. Add option to UIv2 to get container logs from the new JHS API. Contributed by Adam Antal	2020-02-13 12:08:54 +01:00
Prabhu Joseph	fe7d67a8a2	YARN-9521. Handle FileSystem close in ApiServiceClient Contributed by kyungwan nam. Reviewed by Eric Yang.	2020-02-13 09:39:13 +05:30
Prabhu Joseph	e637797211	YARN-10127. Remove setting App Ordering Policy to ParentQueue in FSQueueConverter Contributed by Peter Bacsko.	2020-02-11 22:01:58 +05:30
Sunil G	28f730b317	YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph	2020-02-09 21:14:53 +05:30
Ayush Saxena	3f0a7cd17a	YARN-9624. Use switch case for ProtoUtils#convertFromProtoFormat containerState. Contributed by Bilwa S T	2020-02-09 19:14:18 +05:30
Vinayakumar B	7dac7e1d13	HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635 ). Contributed by Vinayakumar B.	2020-02-07 14:51:24 +05:30
Szilard Nemeth	71b2c2ffe9	YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal	2020-02-06 12:25:06 +01:00
Jonathan Hung	314e2f9d2e	YARN-10116. Expose diagnostics in RMAppManager summary	2020-02-04 17:44:05 -08:00
Giovanni Matteo Fumarola	bf8686f43f	YARN-8982. [Router] Add locality policy. Contributed by Young Chen.	2020-01-30 16:59:36 -08:00
Szilard Nemeth	a7d72c523a	YARN-10099. FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues. Contributed by Peter Bacsko	2020-01-30 16:03:38 +01:00
Akira Ajisaka	a5ef08b619	YARN-9743. [JDK11] TestTimelineWebServices.testContextFactory fails. (#1824 ) Contributed by Akira Ajisaka and Kinga Marton.	2020-01-30 14:10:31 +09:00
Prabhu Joseph	825db8fe2a	YARN-10107. Fix GpuResourcePlugin#getNMResourceInfo to honor Auto Discovery Enabled Contributed by Szilard Nemeth.	2020-01-29 13:30:00 +05:30
Eric Badger	e578e52aae	YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne.	2020-01-29 03:54:43 +00:00
Inigo Goiri	5abd0148eb	YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.	2020-01-28 10:06:37 -08:00
Prabhu Joseph	1ab9c692fa	YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change Contributed by Kinga Marton.	2020-01-28 23:16:04 +05:30
Szilard Nemeth	94f060242e	YARN-10104. FS-CS converter: dry run should work without output defined. Contributed by Peter Bacsko	2020-01-28 15:36:04 +01:00
Szilard Nemeth	ca29768035	YARN-10085. FS-CS converter: remove mixed ordering policy check. Contributed by Peter Bacsko	2020-01-28 15:22:48 +01:00
Szilard Nemeth	9520b2ad79	YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal	2020-01-22 16:25:07 +01:00
Inigo Goiri	b4870bce3a	Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R." This reverts commit `0696828a09`.	2020-01-21 17:45:17 -08:00
Inigo Goiri	0696828a09	YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.	2020-01-21 13:41:01 -08:00
Akira Ajisaka	1defe3a65a	HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li.	2020-01-21 10:58:32 +09:00
Szilard Nemeth	581072a8f0	YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg	2020-01-20 13:10:32 +01:00
Szilard Nemeth	6d52bbbfcf	YARN-9525. IFile format is not working against s3a remote folder. Contributed by Adam Antal	2020-01-20 12:36:55 +01:00
Szilard Nemeth	8b3ee2f7e9	YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph	2020-01-20 09:54:22 +01:00
Szilard Nemeth	57aad0f43a	YARN-10081. Exception message from ClientRMProxy#getRMAddress is misleading. Contributed by Ravuri Sushma Sree	2020-01-20 09:41:06 +01:00
Szilard Nemeth	14d0f9a775	YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph	2020-01-20 09:28:23 +01:00
Prabhu Joseph	a0ff42d761	YARN-10070. Fix NPE if no queue mapping defined for proxy user when using application tag based placement. Contributed by Kinga Marton. Reviewed by Adam Antal.	2020-01-16 12:55:27 +05:30
Akira Ajisaka	14c2c3d69d	YARN-9512. [JDK11] TestAuxServices#testCustomizedAuxServiceClassPath fails because of ClassCastException. (#1806 )	2020-01-16 12:21:50 +09:00
Szilard Nemeth	2aa065d98f	YARN-10082. FS-CS converter: disable terminal placement rule checking. Contributed by Peter Bacsko	2020-01-15 12:42:13 +01:00
Szilard Nemeth	7c5cecc3b3	YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R	2020-01-15 09:55:41 +01:00
Szilard Nemeth	1c51f36be7	YARN-9788. Queue Management API does not support parallel updates. Contributed by Prabhu Joseph	2020-01-14 12:26:03 +01:00
Szilard Nemeth	13cea0412c	YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal	2020-01-14 11:00:08 +01:00
Szilard Nemeth	621c5eac38	YARN-9912. Capacity scheduler: support u:user2:%secondary_group queue mapping. Contributed by Manikandan R	2020-01-13 17:23:00 +01:00
Szilard Nemeth	b7ef8a333f	YARN-9868. Validate %primary_group queue in CS queue manager. Contributed by Manikandan R	2020-01-13 17:15:09 +01:00
Szilard Nemeth	2576c31644	YARN-9989. Typo in CapacityScheduler documentation: Runtime Configuration. Contributed by Kevin Su	2020-01-13 16:50:07 +01:00
Szilard Nemeth	d842dfffa5	YARN-9866. u:user2:%primary_group is not working as expected. Contributed by Manikandan R	2020-01-12 14:04:15 +01:00
Szilard Nemeth	24e6a9e43a	YARN-10067. Add dry-run feature to FS-CS converter tool. Contributed by Peter Bacsko	2020-01-12 13:48:39 +01:00
Eric Yang	2301b25899	YARN-10019. Improved container-executor exec() calls. Contributed by Peter Bacsko	2020-01-10 19:04:04 -05:00
Adam Antal	20a90c0b5b	YARN-10071. Sync Mockito version with other modules Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2020-01-10 17:41:04 +09:00
Eric E Payne	93233a7d6e	YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla)	2020-01-09 17:18:44 +00:00
Eric E Payne	b1e07d27cc	YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan)	2020-01-08 19:26:01 +00:00
Eric E Payne	6899be5a17	YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan)	2020-01-08 17:29:56 +00:00
Prabhu Joseph	571795cd18	YARN-10068. Fix TimelineV2Client leaking File Descriptors. Contributed by Anand Srinivasan. Reviewed by Adam Antal.	2020-01-08 12:01:30 +05:30
Eric Yang	d81d45ff2f	YARN-9956. Improved connection error message for YARN ApiServerClient. Contributed by Prabhu Joseph	2020-01-06 13:24:16 -05:00
Szilard Nemeth	dd2607e3ec	YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal	2020-01-06 17:16:11 +01:00
Szilard Nemeth	768ee22e9e	YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal	2020-01-06 16:26:33 +01:00
Prabhu Joseph	eca7e14c2f	YARN-10053. Use Shared Group Mapping Service in Placement Rules. Contributed by Wilfred Spiegelenburg.	2020-01-02 14:13:57 +05:30
Prabhu Joseph	21ada4d1b0	Revert "YARN-10053. Use Shared Group Mapping Service in Placement Rules." This reverts commit `217b56ffdd`.	2020-01-02 14:12:43 +05:30
Prabhu Joseph	217b56ffdd	YARN-10053. Use Shared Group Mapping Service in Placement Rules. Contributed by Wilfred Spiegelenburg.	2020-01-02 14:07:49 +05:30
Liu sheng	0fed874adf	YARN-10041. Should not use AbstractPath to create unix domain socket (#1771 )	2019-12-27 16:50:15 +05:30
Akira Ajisaka	40887c9b12	YARN-10057. Upgrade the dependencies managed by yarnpkg. (#1780 )	2019-12-25 14:10:44 +09:00
Akira Ajisaka	34ff7dbaf5	YARN-10055. bower install fails. (#1778 )	2019-12-23 19:45:47 +09:00
Eric Badger	412035b47a	YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne.	2019-12-20 18:07:00 +00:00
Zhankun Tang	12722ab0c7	YARN-10042. Upgrade grpc-xxx depdencies to 1.26.0. Contributed by Sheng Liu.	2019-12-20 11:10:27 +08:00
Giovanni Matteo Fumarola	ef59ffd362	YARN-10038. [UI] Finish Time is not correctly parsed in the RM Apps page. Contributed by Inigo Goiri.	2019-12-19 11:37:17 -08:00
Masatake Iwasaki	7868da894a	YARN-10037. Upgrade build tools for YARN Web UI v2.	2019-12-19 19:27:31 +09:00
Jonathan Hung	7b93575b92	YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne	2019-12-18 13:18:11 -08:00
Jonathan Hung	fddc3d55c3	YARN-10039. Allow disabling app submission from REST endpoints	2019-12-18 10:48:05 -08:00
Masatake Iwasaki	92c8962f98	YARN-10020. Fix build instruction of hadoop-yarn-ui. (#1750 )	2019-12-18 13:34:12 +09:00
Eric Badger	f47dcf2d4c	YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne.	2019-12-17 17:11:23 +00:00
Szilard Nemeth	631dbbc6f2	YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal	2019-12-15 17:28:04 +01:00
prabhujoseph	d4bde134e3	YARN-7721. Fix TestContinuousScheduling fails sporadically with NPE. Contributed by Wilfred Spiegelenburg.	2019-12-10 15:00:23 +05:30
Akira Ajisaka	dc66de7448	YARN-9985. Unsupported transitionToObserver option displaying for rmadmin command. Contributed by Ayush Saxena.	2019-12-09 18:37:34 +09:00
dependabot[bot]	e9c5bb8078	Bump bower from 1.7.7 to 1.8.8 (#1683 ) Bumps [bower](https://github.com/bower/bower) from 1.7.7 to 1.8.8. - [Release notes](https://github.com/bower/bower/releases) - [Changelog](https://github.com/bower/bower/blob/master/CHANGELOG.md) - [Commits](https://github.com/bower/bower/compare/v1.7.7...v1.8.8) Signed-off-by: dependabot[bot] <support@github.com>	2019-12-09 15:46:08 +09:00
Eric Badger	8dffd8dc89	YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger	2019-12-09 01:25:10 +00:00
Eric Badger	ab2cc5ac92	Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger" This reverts commit `76bb297db9`.	2019-12-09 01:25:10 +00:00
Jonathan Hung	6d970d8100	YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R	2019-12-08 16:35:02 -08:00
Jonathan Hung	cd4779f317	Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources" This reverts commit `92bce918dc`.	2019-12-08 16:34:46 -08:00
Jonathan Hung	92bce918dc	YARN-10012. Guaranteed and max capacity queue metrics for custom resources	2019-12-08 15:52:17 -08:00
Eric Badger	76bb297db9	YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger	2019-12-06 18:33:58 +00:00
prabhujoseph	e575df5ca6	YARN-10006. IOException used in place of YARNException in CapaitySheduler Contributed by Adam Antal. Reviewed by Szilard Nemeth.	2019-12-06 12:51:28 +05:30
Szilard Nemeth	4f758dd4c6	YARN-9607. Auto-configuring rollover-size of IFile format for non-appendable filesystems. Contributed by Adam Antal	2019-12-05 21:49:00 +01:00
Szilard Nemeth	cf68857631	YARN-9938. Validate Parent Queue for QueueMapping contains dynamic group as parent queue. Contributed by Manikandan R	2019-12-05 21:04:42 +01:00
Szilard Nemeth	0ab9c0398d	YARN-9969. Improve yarn.scheduler.capacity.queue-mappings documentation. Contributed by Manikandan R	2019-12-05 20:53:43 +01:00
Szilard Nemeth	5cc6f945da	YARN-9789. Disable Option for Write Ahead Logs of LogMutation. Contributed by Prabhu Joseph	2019-12-05 20:43:26 +01:00
Szilard Nemeth	4627dd6708	YARN-9780. SchedulerConf Mutation API does not Allow Stop and Remove Queue in a single call. Contributed by Prabhu Joseph	2019-12-05 20:38:37 +01:00
Szilard Nemeth	c71befaf8f	YARN-9781. SchedConfCli to get current stored scheduler configuration. Contributed by Prabhu Joseph	2019-12-05 20:31:24 +01:00
Szilard Nemeth	520fe2c99b	YARN-5106. Provide a builder interface for FairScheduler allocations for use in tests. Contributed by Adam Antal	2019-12-05 17:37:40 +01:00
Sunil G	682e6fdeda	YARN-9052. Replace all MockRM submit method definitions with a builder. Contributed by Szilard Nemeth.	2019-12-05 13:26:23 +05:30
Wanqiang Ji	c48de9aa2d	YARN-9958. Remove the invalid lock in ContainerExecutor (#1704 )	2019-12-04 14:35:41 +09:00
Abhishek Modi	a2dadac790	YARN-9990. Testcase fails with Insufficient configured threads: required=16 < max=10. Contributed by Prabhu Joseph.	2019-11-29 11:54:33 +05:30
Szilard Nemeth	ef950b0863	YARN-9290. Invalid SchedulingRequest not rejected in Scheduler PlacementConstraintsHandler. Contributed by Prabhu Joseph	2019-11-26 22:04:07 +01:00
Szilard Nemeth	828ab400ee	YARN-9362. Code cleanup in TestNMLeveldbStateStoreService. Contributed by Denes Gerencser	2019-11-26 21:45:12 +01:00
prabhujoseph	aa7ab2719f	YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth.	2019-11-27 01:59:12 +05:30
Szilard Nemeth	8c9018d5c7	YARN-9899. Migration tool that help to generate CS config based on FS config [Phase 2]. Contributed by Peter Bacsko	2019-11-26 21:22:35 +01:00
Szilard Nemeth	52e9ee39a1	YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak	2019-11-26 16:42:59 +01:00
Szilard Nemeth	448ffb12ec	YARN-9937. addendum: Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph	2019-11-26 13:41:41 +01:00
Eric Badger	6c20512b68	Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger" This reverts commit `289bbca870`.	2019-11-25 17:46:42 +00:00
HUAN-PING SU	f8e36e03b4	YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709 )	2019-11-25 15:28:53 +09:00
Szilard Nemeth	4c1a1287bc	YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi	2019-11-22 12:58:10 +01:00
Tao Yang	8afabe41e4	YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu.	2019-11-22 16:06:14 +08:00
Szilard Nemeth	dfdc6d6dd9	YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton	2019-11-19 21:14:15 +01:00
Eric Yang	ea8ffac121	YARN-9983. Fixed typo in YARN Service overview. Contributed by Denes Gerencser	2019-11-19 14:16:51 -05:00
Prabhu Joseph	30b2096933	YARN-9986. Allow app owner to signal container via RMWebServices. Contributed by kyungwan nam.	2019-11-19 11:35:00 +00:00
Sunil G	ea68756c0c	YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg.	2019-11-19 14:10:41 +05:30
Sunil G	215f2052fc	YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg.	2019-11-19 14:03:02 +05:30
Abhishek Modi	dc3f4fc2f4	YARN-9965. Fix NodeManager failing to start on subsequent times when Hdfs Auxillary Jar is set (addendum). Contributed by Prabhu Joseph.	2019-11-19 08:08:40 +05:30
Eric Badger	0e22e9ab83	YARN-9562. Add Java changes for the new RuncContainerRuntime. Contributed by Eric Badger	2019-11-18 21:56:31 +00:00
Eric Badger	289bbca870	YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger	2019-11-18 21:54:16 +00:00
Szilard Nemeth	27642367ef	YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko	2019-11-18 16:29:39 +01:00
prabhujoseph	bd454348b0	YARN-9982. Fix Container API example link in NodeManager REST API doc. Contributed by Charan Hebri.	2019-11-18 15:19:47 +05:30
Yufei Gu	b83b9ab418	YARN-9537. Add configuration to disable AM preemption. Contributed by ZhouKang	2019-11-12 09:40:16 -08:00
Abhishek Modi	fb512f5087	YARN-9697. Efficient allocation of Opportunistic containers. Contributed by Abhishek Modi.	2019-11-12 16:34:04 +05:30
Szilard Nemeth	30b93f914b	YARN-9865. Capacity scheduler: add support for combined %user + %secondary_group mapping. Contributed by Manikandan R	2019-11-11 13:28:02 +01:00
Abhishek Modi	516377bfa6	YARN-9965. Fix NodeManager failing to start when Hdfs Auxillary Jar is set. Contributed by Prabhu Joseph.	2019-11-11 16:01:44 +05:30
Szilard Nemeth	ceb9c6175e	YARN-9890. [UI2] Add Application tag to the app table and app detail page. Contributed by Julia Kinga Marton	2019-11-09 14:12:04 +01:00
Szilard Nemeth	31f172fd96	YARN-9677. Make FpgaDevice and GpuDevice classes more similar to each other. Contributed by kevin su	2019-11-09 14:04:14 +01:00
zhoukang	7d0adddf09	YARN-9689: Support proxy user for Router to support kerberos (#1654 )	2019-11-04 17:13:29 -08:00
Sunil G	51e7d1b37e	YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph.	2019-11-04 23:27:49 +05:30
Sunil G	d462308e04	YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. Contributed by Prabhu Joseph.	2019-11-03 08:47:27 +05:30
Jonathan Hung	e6137d0e50	Make upstream aware of 2.10.0 release	2019-10-30 20:58:32 -07:00
Sunil G	9a2e43e29e	YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph.	2019-10-31 00:34:27 +05:30
Szilard Nemeth	e3e7daa4f5	YARN-9889. [UI] Add Application Tag column to RM All Applications table. Contributed by Kinga Marton	2019-10-30 16:26:56 +01:00
Abhishek Modi	ed45c13f67	YARN-2442. ResourceManager JMX UI does not give HA State. Contributed by Rohith Sharma and Cyrus Jackson.	2019-10-29 15:54:55 +05:30
Eric Badger	eef34f2d87	YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan	2019-10-25 17:13:04 +00:00
Eric Yang	ac6b6a6a85	HADOOP-16614. Add aarch64 support for dependent leveldbjni. Contributed by liusheng close #1546	2019-10-24 11:45:57 -04:00
Zhankun Tang	fd84ca5161	YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi	2019-10-24 10:05:00 +08:00
Eric E Payne	83d148074f	YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph.	2019-10-21 20:30:50 +00:00
Eric Yang	3f7756dc6c	YARN-9922. Fixed YARN javadoc errors from YARN-9699. Contributed by Peter Bacsko	2019-10-21 16:16:49 -04:00
Eric Yang	155864da00	YARN-9875. Improve fair scheduler configuration store on HDFS. Contributed by Prabhu Joseph	2019-10-18 20:30:11 -04:00
Eric Yang	72b1bed998	YARN-9884. Make container-executor mount logic modular Contributed by Eric Badger	2019-10-18 19:10:32 -04:00
Mate Szalay-Beko	6d92aa7c30	HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 (#1656 ). Contributed by Norbert Kalmár, Mate Szalay-Beko * HADOOP-16579 - Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.5 - Add a static initializer for the unit tests using ZooKeeper to enable the four-letter-words diagnostic telnet commands. (this is an interface that become disabled by default, so to keep the ZooKeeper 3.4.x behavior we enabled it for the tests) - Also fix ZKFailoverController to look for relevant fail-over ActiveAttempt records. The new ZooKeeper seems to respond quicker during the fail-over tests than the ZooKeeper, so we made sure to catch all the relevant records by adding a new parameter to ZKFailoverontroller.waitForActiveAttempt(). Co-authored-by: Norbert Kalmár <nkalmar@cloudera.com>	2019-10-18 13:26:20 -07:00
Yuming Wang	3d41f33018	HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng. Co-authored-By: Siyao Meng <smeng@cloudera.com> Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-10-18 08:25:02 -07:00
Szilard Nemeth	f0699a7406	YARN-9841. Capacity scheduler: add support for combined %user + %primary_group mapping. Contributed by Manikandan R	2019-10-18 11:19:49 +02:00
Sunil G	3990ffa083	YARN-9909. Offline format of YarnConfigurationStore. Contributed by Prabhu Joseph.	2019-10-17 06:43:06 +05:30
Eric E Payne	a5034c7988	YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R.	2019-10-16 21:10:08 +00:00
Sunil G	090f73a9aa	YARN-9900. Revert to previous state when Invalid Config is applied and Refresh Support in SchedulerConfig Format. Contributed by Prabhu Joseph.	2019-10-16 18:14:31 +05:30
Szilard Nemeth	a8ef03e961	YARN-9840. Capacity scheduler: add support for Secondary Group rule mapping. Contributed by Manikandan R	2019-10-15 17:22:52 +02:00
Szilard Nemeth	5cc7873a47	YARN-9699. Migration tool that help to generate CS config based on FS config [Phase 1]. Contributed by Peter Bacsko	2019-10-14 17:19:39 +02:00
Szilard Nemeth	ec86f42e40	YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal	2019-10-11 14:01:19 +02:00
Eric Yang	31e0122f4d	YARN-9860. Enable service mode for Docker containers on YARN Contributed by Prabhu Joseph and Shane Kumpf	2019-10-10 19:02:02 -04:00
Szilard Nemeth	6f1ab95168	YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl	2019-10-09 13:26:26 +02:00
Szilard Nemeth	35f093f5b3	YARN-9356. Add more tests to ratio method in TestResourceCalculator. Contributed by Zoltan Siegl	2019-10-09 13:09:09 +02:00
Sunil G	be901f4962	YARN-9873. Mutation API Config Change need to update Version Number. Contributed by Prabhu Joseph	2019-10-09 15:53:14 +05:30
Jonathan Hung	5462d2176f	YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena	2019-10-08 11:20:13 -07:00
Sunil G	fb1ecff6a2	Revert "YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph" This reverts commit `4510970e2f`.	2019-10-05 09:15:17 +05:30
Sunil G	4510970e2f	YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph	2019-10-04 21:49:07 +05:30
Sunil G	0d2d6f9267	YARN-9792. Document examples of SchedulerConf with Node Labels. Contributed by Prabhu Joseph	2019-10-02 23:43:47 +05:30
Abhishek Modi	61a8436004	YARN-9870. Remove unused function from OpportunisticContainerAllocatorAMService. Contributed by Abhishek Modi.	2019-10-02 17:55:07 +05:30
Eric Yang	d1ddba60e5	YARN-9862. Increase yarn-services-core test timeout value. Contributed by Prabhu Joseph	2019-10-01 21:15:26 -04:00
Sunil G	99cd7572f1	YARN-9801. SchedConfCli does not work wiwith https mode. Contributed by Prabhu Joseph	2019-10-01 20:06:21 +05:30
bibinchundatt	425a6c89ae	YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung.	2019-10-01 16:02:26 +05:30
Sunil G	137546a78a	YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph	2019-10-01 09:08:41 +05:30
Abhishek Modi	4d3c580b03	YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi.	2019-09-30 23:40:15 +05:30
Eric Badger	18a8c2404e	YARN-9857. TestDelegationTokenRenewer throws NPE but tests pass. Contributed by Ahmed Hussein	2019-09-26 16:15:08 +00:00
Jonathan Hung	606e341c1a	Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label	2019-09-25 17:54:53 -07:00
Szilard Nemeth	c72457787d	YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko	2019-09-25 10:36:22 +02:00
Szilard Nemeth	bec0864394	YARN-9808. Zero length files in container log output haven't got a header. Contributed by Adam Antal	2019-09-25 10:28:34 +02:00
Jonathan Hung	c2731d4b63	YARN-9730. Support forcing configured partitions to be exclusive based on app node label	2019-09-24 13:51:32 -07:00
Duo Zhang	bfe1dac29d	HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes (#1496 ) Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>	2019-09-24 08:50:36 +08:00
Jonathan Hung	3d78b1223d	YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar	2019-09-23 13:12:16 -07:00
Vinayakumar B	1654497f98	HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432 ) HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.	2019-09-20 16:08:30 +05:30
Malcolm Taylor	56248f9d87	HADOOP-16556. Fix some alerts raised by LGTM. Contributed by Malcolm Taylor. Change-Id: Ic60c3f4681dd9d48b3afcba7520bd1e4d3cc4231	2019-09-19 16:00:05 +01:00
Sunil G	01d7924473	YARN-9814. JobHistoryServer can't delete aggregated files, if remote app root directory is created by NodeManager. Contributed by Adam Antal.	2019-09-18 09:27:41 +05:30
Weiwei Yang	f8c14326ee	YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu.	2019-09-18 09:12:28 +08:00
Eric Yang	eefe9bc85c	YARN-9837. Fixed reading YARN Service JSON spec file larger than 128k. Contributed by Tarun Parimi	2019-09-17 13:13:35 -04:00
Sunil G	c474e24c0b	YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko.	2019-09-17 19:41:38 +05:30
Abhishek Modi	da2b4fe204	YARN-9794. RM crashes due to runtime errors in TimelineServiceV2Publisher. Contributed by Tarun Parimi.	2019-09-15 14:16:18 +05:30
HUAN-PING SU	4239007349	YARN-9787. Typo in analysesErrorMsg. Contributed by kevin su. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Surendra Singh Lilhore <surendralilhore@apache.org>	2019-09-13 10:36:45 -07:00
Abhishek Modi	44850f6784	YARN-9816. EntityGroupFSTimelineStore#scanActiveLogs fails when undesired files are present under /ats/active. Contribued by Prabhu Joseph.	2019-09-12 12:50:10 +05:30
Abhishek Modi	3b06f0bf9e	YARN-9819. Make TestOpportunisticContainerAllocatorAMService more resilient. Contribued by Abhishek Modi	2019-09-12 08:09:57 +05:30
Eric Badger	56b7571131	YARN-9815 ReservationACLsTestBase fails with NPE. Contributed by Ahmed Hussein	2019-09-11 21:59:28 +00:00
Jonathan Hung	f8f8598ea5	YARN-9824. Fall back to configured queue ordering policy class name	2019-09-10 15:19:07 -07:00
Eric Yang	10144a580e	YARN-9728. Bugfix for escaping illegal xml characters for Resource Manager REST API. Contributed by Prabhu Joseph	2019-09-10 17:04:39 -04:00
Abhishek Modi	60af8793b4	YARN-9821. NM hangs at serviceStop when ATSV2 Backend Hbase is Down. Contributed by Prabhu Joseph.	2019-09-09 14:26:00 +05:30
Jonathan Hung	387c332b64	YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph	2019-09-09 00:23:04 -07:00
Jonathan Hung	43e389b980	YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar	2019-09-08 19:07:11 -07:00

... 11 12 13 14 15 ...

6676 Commits