hadoop

Author	SHA1	Message	Date
Gera Shegalov	a94e9fcbde	YARN-11055. Add missing newline in cgroups-operations.c (#3851 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2022-01-17 16:20:48 +09:00
luoyuan3471	e2d620192a	HADOOP-18044. Hadoop - Upgrade to jQuery 3.6.0 (#3791 ) Co-authored-by: luoyuan <luoyuan@shopee.com>	2022-01-12 11:40:32 +08:00
Andras Gyori	21fe0e63fd	YARN-10632. Make auto queue creation maximum allowed depth configurable (#3859 )	2022-01-11 14:38:59 +08:00
minni31	3bf014d871	YARN-8859. Add audit logs for router service (#3607 ) Co-authored-by: Minni Mittal <mimittal@microsoft.com>	2022-01-03 10:38:59 +05:30
Cheng Pan	4db27acf06	YARN-11053. AuxService should not use class name as default system classes (#3816 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-12-24 11:08:04 +09:00
Ashutosh Gupta	00e2405fbd	YARN-8234. Improve RM system metrics publisher's performance by pushing events to timeline server in batch (#3793 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-12-23 17:14:51 +09:00
Eric Payne	e2d6fd075d	YARN-10178: Global Scheduler async thread crash caused by 'Comparison method violates its general contract. Contributed by Andras Gyori (gandras) and Qi Zhu (zhuqi).	2021-12-21 19:05:39 +00:00
Viraj Jasani	6185056f4d	YARN-11047. ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled (#3802 )	2021-12-20 18:02:57 +08:00
Viraj Jasani	04b6b9a87b	HADOOP-16908. Prune Jackson 1 from the codebase and restrict it's usage for future (#3789 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-12-20 16:01:34 +09:00
Szilard Nemeth	07141426e0	YARN-11050 (#3805 )	2021-12-18 11:52:03 +08:00
Szilard Nemeth	8d214cb785	YARN-10951. CapacityScheduler: Move all fields and initializer code that belongs to async scheduling to a new class (#3800 ). Contributed by Szilard Nemeth	2021-12-17 00:18:14 +01:00
Szilard Nemeth	aec9cdb467	YARN-10963. Split TestCapacityScheduler by test categories. Contributed by Tamas Domok	2021-12-16 23:39:18 +01:00
Szilard Nemeth	a9a5830f31	YARN-11048. Add tests that shows how to delete config values with Mutation API (#3799 ). Contributed by Szilard Nemeth	2021-12-16 15:53:08 +01:00
Szilard Nemeth	6ea1685abd	YARN-11044. Fix TestApplicationLimits.testLimitsComputation() ineffective asserts. Contributed by Benjamin Teke	2021-12-16 00:01:09 +01:00
Viraj Jasani	f43ac31b44	YARN-11045. ATSv2 storage monitor fails to read from hbase cluster (#3796 )	2021-12-15 16:47:51 +08:00
Szilard Nemeth	843f66f4dc	Clean up checkstyle warnings from YARN-11024/10907/10929. Contributed by Benjamin Teke	2021-12-14 22:00:46 +01:00
Szilard Nemeth	364d38f00d	YARN-10929. Do not use a separate config in legacy CS AQC. Contributed by Benjamin Teke	2021-12-14 14:57:08 +01:00
Szilard Nemeth	a5bcf4c792	YARN-10907. Minimize usages of AbstractCSQueue#csContext. Contributed by Benjamin Teke	2021-12-13 21:57:46 +01:00
Szilard Nemeth	898055e204	YARN-11024. Create an AbstractLeafQueue to store the common LeafQueue + AutoCreatedLeafQueue functionality. Contributed by Benjamin Teke	2021-12-13 16:12:55 +01:00
Viraj Jasani	4e326f0c97	HADOOP-18039. Upgrade hbase2 version and fix TestTimelineWriterHBaseDown (#3768 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-12-13 14:03:32 +09:00
Szilard Nemeth	f544ef4a51	YARN-11033. isAbsoluteResource is not correct for dynamically created queues. Contributed by Tamas Domok	2021-12-10 15:09:53 +01:00
Szilard Nemeth	a0d8cde133	YARN-10982. Replace all occurences of queuePath with the new QueuePath class. Contributed by Tibor Kovacs	2021-12-09 17:51:44 +01:00
Andrew Chung	ffee92bbf1	YARN-11005. Implement the core QUEUE_LENGTH_THEN_RESOURCES OContainer allocation policy (#3717 )	2021-12-08 10:02:10 -08:00
Szilard Nemeth	66a08e933a	YARN-11038. Fix testQueueSubmitWithACL* tests in TestAppManager. Contributed by Tamas Domok	2021-12-08 16:41:35 +01:00
Szilard Nemeth	19430118c6	YARN-11031. Improve the maintainability of RM webapp tests like TestRMWebServicesCapacitySched. Contributed by Tamas Domok	2021-12-08 16:03:42 +01:00
Viraj Jasani	53edd0de5a	HADOOP-18033. Upgrade fasterxml Jackson to 2.13.0 (#3749 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-12-08 16:52:22 +09:00
Szilard Nemeth	126079612c	YARN-11016. Queue weight is incorrectly reset to zero. Contributed by Andras Gyori	2021-12-07 15:55:16 +01:00
Szilard Nemeth	7284d23476	YARN-11006. Allow overriding user limit factor and maxAMResourcePercent with AQCv2 templates. Contributed by Benjamin Teke	2021-12-07 15:43:55 +01:00
Szilard Nemeth	60440a836a	YARN-11023. Extend the root QueueInfo with max-parallel-apps in CapacityScheduler. Contributed by Tamas Domok	2021-12-07 15:26:07 +01:00
Szilard Nemeth	7a8c2e0097	YARN-11020. [UI2] No container is found for an application attempt with a single AM container. Contributed by Andras Gyori	2021-12-07 08:51:03 +01:00
Szilard Nemeth	d75f2db2f9	YARN-11014. YARN incorrectly validates maximum capacity resources on the validation API. Contributed by Benjamin Teke	2021-12-07 08:42:57 +01:00
Ashutosh Gupta	5a950b8900	YARN-9063. ATS 1.5 fails to start if RollingLevelDb files are corrupt or missing (#3728 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-12-06 19:45:44 +09:00
Andras Gyori	47ea0d734f	HADOOP-18021. Provide a public wrapper of Configuration#substituteVars (#3710 ) Contributed by Andras Gyori	2021-12-03 16:44:58 +00:00
Viraj Jasani	b34dcb5b3a	HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile (#3722 )	2021-12-02 10:40:30 +08:00
Sushmasree-28	7b840f2693	YARN-10975 EntityGroupFSTimelineStore#ActiveLogParser parses already processed files (#3735 ) Contributed by sravuri <sravuri@microsoft.com> and reviewed by Prabhu Joseph	2021-11-30 00:00:25 +05:30
Viraj Jasani	215388beea	HADOOP-18022. Add restrict-imports-enforcer-rule for Guava Preconditions and remove remaining usages (#3712 ) Reviewed-by: Akira Ajisaka <aajisaka@apache.org> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-11-29 17:37:30 +09:00
GuoPhilipse	c9d64bad37	YARN-11007. Correct words in YARN documents (#3680 ) Reviewed-by: cxorm <lianp964@gmail.com> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-11-25 17:48:54 +09:00
Ashutosh Gupta	08f3df3ea2	YARN-10991. Fix to ignore the grouping "[]" for resourcesStr in parseResourcesString method (#3592 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-11-24 18:41:22 +09:00
Andrew Chung	d3f0b7eab7	YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0 (#3642 )	2021-11-23 13:21:51 -08:00
Andrew Chung	5b1b2c8ef6	YARN-11003. Make RMNode aware of all (OContainer inclusive) allocated resources (#3646 )	2021-11-23 13:20:08 -08:00
Viraj Jasani	c0bdba8fac	HADOOP-18017. unguava: remove Preconditions from hadoop-yarn-project modules (#3687 )	2021-11-23 13:36:22 +09:00
Szilard Nemeth	b933f5f54b	YARN-11001. Add docs on removing node label mapping from a node. Contributed by Manu Zhang	2021-11-15 16:02:18 +01:00
Szilard Nemeth	7cb887e6c2	YARN-10997. Revisit allocation and reservation logging. Contributed by Andras Gyori	2021-11-12 15:43:45 +01:00
Szilard Nemeth	e220e88eca	YARN-10996. Fix race condition of User object acquisitions. Contributed by Andras Gyori	2021-11-12 15:33:39 +01:00
minni31	8b159d296d	[YARN-10998] Add YARN_ROUTER_HEAPSIZE to yarn-env for routers (#3603 )	2021-11-11 09:49:29 -08:00
Szilard Nemeth	2e32cc68fc	YARN-10985. Add some tests to verify ACL behaviour in CapacitySchedulerConfiguration (#3570 ). Contributed by Szilard Nemeth	2021-10-28 15:19:32 +02:00
Szilard Nemeth	d598904046	YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue (#3551 ) contributed by Szilard Nemeth	2021-10-27 19:03:45 +02:00
Szilard Nemeth	66ac476b48	YARN-10924. Clean up CapacityScheduler#initScheduler (#3581 ) Contributed by Szilard Nemeth	2021-10-27 17:13:49 +02:00
Jack	9cfd8d0a83	YARN-10909. AbstractCSQueue: Annotate all methods with VisibleForTesting that are only used by test code. Contributed by JackWangCS, Szilard Nemeth	2021-10-23 14:47:09 +02:00
9uapaw	32ecaed9c3	YARN-10930. Introduce universal capacity resource vector. Contributed by Andras Gyori	2021-10-22 17:32:33 +02:00
Adam Antal	23772d946b	YARN-10948. Rename SchedulerQueue#activeQueue to activateQueue. Contributed by Adam Antal	2021-10-22 16:33:03 +02:00
Viraj Jasani	516f36c6f1	HADOOP-17967. Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom (#3555 )	2021-10-21 16:54:25 +09:00
Ahmed Hussein	d286994009	YARN-1115: Provide optional means for a scheduler to check real user ACLs. Contributed by Eric Payne (epayne)	2021-10-20 22:18:36 +00:00
Szilard Nemeth	20aeb5ecc3	YARN-10916. Investigate and simplify GuaranteedOrZeroCapacityOverTimePolicy#computeQueueManagementChanges. Contributed by Szilard Nemeth	2021-10-20 15:52:37 +02:00
Andras Gyori	35b8441fd9	YARN-10949. Simplify AbstractCSQueue#updateMaxAppRelatedField and find a more meaningful name for this method. Contributed by Andras Gyori	2021-10-20 12:56:41 +02:00
Szilard Nemeth	414d40155c	YARN-10958. Use correct configuration for Group service init in CSMappingPlacementRule (#3560 ) * YARN-10958. Initial commit * Fix javadoc + behaviour * Fix review comments * fix checkstyle + blanks * fix checkstyle + blanks * Fix checkstyle + blanks	2021-10-20 10:48:42 +02:00
9uapaw	616cea2e80	YARN-10954. Remove commented code block from CSQueueUtils#loadCapacitiesByLabelsFromConf. Contributed by Andras Gyori	2021-10-19 13:06:45 +02:00
Szilard Nemeth	025f97c8c2	YARN-10942. Move AbstractCSQueue fields to separate objects that are tracking usage. Contributed by Szilard Nemeth	2021-10-19 12:24:58 +02:00
lujiefsi	ae95caa60e	YARN-10976. Fix resource leak due to Files.walk (#3552 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-10-18 15:24:15 +09:00
Viraj Jasani	d336227e5c	HADOOP-17963. Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-yarn-project modules (#3541 )	2021-10-14 18:03:01 +09:00
Viraj Jasani	1151edf12e	HADOOP-17956. Replace all default Charset usage with UTF-8 (#3529 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-10-14 13:07:24 +09:00
Szilard Nemeth	8c271caa81	YARN-6862. Nodemanager resource usage metrics sometimes are negative. Contributed by Benjamin Teke	2021-10-12 17:35:06 +02:00
Benjamin Teke	35eff54556	YARN-10934. Fix LeafQueue#activateApplication NPE when the user of the pending application is missing from usersManager. Contributed by Benjamin Teke Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-10-07 20:11:42 +02:00
9uapaw	4b1b6b858a	YARN-10953. Make CapacityScheduler#getOrCreateQueueFromPlacementConte… Contributed by Andras Gyori	2021-10-07 17:09:38 +02:00
Tamas Domok	5535d66fb5	YARN-10692. Do not extend from CapacitySchedulerTestBase when not needed. Contributed by Tamas Domok Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-10-07 16:21:05 +02:00
Viraj Jasani	8071dbb9c6	HADOOP-17950. Provide replacement for deprecated APIs of commons-io IOUtils (#3515 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-10-07 10:58:29 +09:00
9uapaw	ed8e879320	YARN-10823. Expose all node labels for root without explicit configurations. Contributed by Andras Gyori	2021-10-01 04:20:36 +02:00
Tamas Domok	3c9e3d5321	YARN-10960. Extract test queues and related methods from TestCapacityScheduler. Contributed by Tamas Domok Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-10-01 04:05:52 +02:00
Neil	4bd0c36189	YARN-10970. Standby RM should expose prom endpoint (#3480 ) Reviewed-by: Adam Antal <adamantal@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-09-29 15:46:55 +09:00
Akshat Bordia	51bd528bd3	YARN-10957. Using invokeConcurrent Overload with Collection in getClusterMetrics (#3439 )	2021-09-28 09:51:18 -07:00
Tamas Domok	8f4456d4a1	YARN-10961. TestCapacityScheduler: reuse appHelper where feasible. Contributed by Tamas Domok Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-21 16:13:04 +02:00
9uapaw	9f6430c9ed	YARN-10897. Introduce QueuePath class. Contributed by Andras Gyori	2021-09-21 16:08:24 +02:00
Szilard Nemeth	4df4389325	YARN-10911. AbstractCSQueue: Create a separate class for usernames and weights that are travelling in a Map. Contributed by Szilard Nemeth	2021-09-20 16:47:46 +02:00
Adam Antal	a9b2469a53	YARN-10950. Code cleanup in QueueCapacities (#3454 )	2021-09-19 14:42:02 +02:00
TiborKovacsCloudera	5dcb23c1f0	YARN-10937. Fix log message arguments in LogAggregationFileController (#3450 )	2021-09-19 14:35:34 +02:00
Szilard Nemeth	aa74a303ed	YARN-10913. AbstractCSQueue: Group preemption methods and fields into a separate class (#3420 )	2021-09-19 13:11:56 +02:00
adamantal	f5c76c8e31	YARN-10936. Log typo corrected. (#3449 ) Contributed by: Tibor Kovacs	2021-09-17 08:27:51 +02:00
Eric Badger	43f0a34dd4	YARN-10935. AM Total Queue Limit goes below per-user AM Limit if parent is full. Contributed by Eric Payne.	2021-09-15 20:03:45 +00:00
Benjamin Teke	5dc2f7b137	YARN-10915. AbstractCSQueue: Simplify complex logic in methods: deriveCapacityFromAbsoluteConfigurations and updateEffectiveResources (#3418 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-09-14 18:05:40 +02:00
Tamas Domok	783d94f5cd	YARN-10917. Investigate and simplify CapacitySchedulerConfigValidator#validateQueueHierarchy (#3403 ) * YARN-10917. Investigate and simplify CapacitySchedulerConfigValidator#validateQueueHierarchy. Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-14 17:54:25 +02:00
Tamas Domok	63c892278f	YARN-10912. AbstractCSQueue#updateConfigurableResourceRequirement: Separate validation logic from initialization logic (#3390 ) - capacityConfigType update is extracted to a separate method - validation logic is extracted to a helper function - min resource must not be greater than max resource is now checked after the max resource is updated Change-Id: I731c2639281721afed32c30854bafcf048d6ee28 Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-14 17:30:44 +02:00
Weihao Zheng	ad1d40970a	YARN-10928. Support default queue config for minimum-user-limit-percent/user-limit-factor (#3389 ) Contributed by Weihao Zheng	2021-09-13 11:06:53 +08:00
Jack	d8026e387e	YARN-10903. Fix the headroom check in ParentQueue and RegularContainerAllocator for DRF (#3352 ) Contributed by Jie Wang <jie.wang@hulu.com>	2021-09-13 10:54:11 +08:00
Benjamin Teke	971f1b8b0a	YARN-10872. Replace getPropsWithPrefix calls in AutoCreatedQueueTemplate (#3396 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-09-10 17:32:42 +02:00
9uapaw	811fd23f23	YARN-10852. Optimise CSConfiguration getAllUserWeightsForQueue (#3392 )	2021-09-10 16:59:46 +02:00
Benjamin Teke	b229e5a345	YARN-10910. AbstractCSQueue#setupQueueConfigs: Separate validation logic from initialization logic (#3407 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-09-10 16:48:58 +02:00
Tamas Domok	29a6f141d4	YARN-10914. Simplify duplicated code for tracking ResourceUsage in AbstractCSQueue (#3402 ) Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-10 15:57:46 +02:00
Akshat Bordia	dee6dc2f89	YARN-10893. Adding metrics for getClusterMetrics and getApplications APIs in FederationClientInterceptor (#3325 )	2021-09-09 09:20:57 -07:00
Akshat Bordia	a186460004	YARN-10829. Follow up: Adding null checks before merging ResourceUsage Report (#3252 )	2021-09-08 10:36:56 -07:00
Szilard Nemeth	2ff3fc50e4	YARN-10870. Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page. Contributed by Gergely Pollak	2021-09-08 18:01:39 +02:00
Tamas Domok	5e166898aa	YARN-10901. Permission checking error on an existing directory in LogAggregationFileController#verifyAndCreateRemoteLogDir (#3355 ) Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-09-08 17:04:32 +02:00
Jack	4e209a31da	YARN-10919. Remove LeafQueue#scheduler field (#3382 ) Co-authored-by: Jie Wang <jie.wang@hulu.com>	2021-09-08 16:19:29 +02:00
Benjamin Teke	40e639ad07	YARN-10646. TestCapacitySchedulerWeightMode test descriptor comments doesnt reflect the correct scenario (#3339 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-09-08 16:11:04 +02:00
zhuqi	3024a47026	Add documentation for YARN-10623 auto refresh queue conf in CS (#3279 )	2021-09-08 16:03:15 +02:00
Benjamin Teke	d9cb69853b	YARN-10522. Document for Flexible Auto Queue Creation in Capacity Scheduler	2021-09-08 15:43:57 +02:00
Szilard Nemeth	9c8fe1e512	YARN-10576. Update Capacity Scheduler documentation with JSON-based placement mapping. Contributed by Benjamin Teke	2021-09-08 15:17:27 +02:00
Swathi Chandrashekar	9512c774c0	YARN-10884: Handle empty owners to parse log files (#3318 ) * YARN 10884 : Parse log files which has empty owner * Removing the whitespace * Added a test case for null user * Fixed indentations * Fixed the indentation for test cases Co-authored-by: Swathi C <cswathi@microsoft.com>	2021-09-07 21:37:51 +05:30
Tamas Domok	16e6030e25	YARN-10891. Extend QueueInfo with max-parallel-apps in CS. (#3314 ) Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-08-27 23:09:54 +02:00
Szilard Nemeth	e06a5cb197	YARN-10838. Implement an optimised version of Configuration getPropsWithPrefix. Contributed by Andras Gyori, Benjamin Teke	2021-08-24 15:27:34 +02:00
srinivasst	4f3f26ce09	YARN-10873: Account for scheduled AM containers before deactivating node (#3287 ) * Account for scheduled AM containers before deactivating node * Move AM container check to separate method. * Fix UTs * Fix UTs * Remove unnecessary import * Add timeout for UT	2021-08-17 14:18:55 +05:30
Gergely Pollak	44bab51be4	YARN-10849 Clarify testcase documentation for TestServiceAM#testContainersReleasedWhenPreLaunchFails. Contributed by Szilard Nemeth	2021-08-04 15:43:57 +02:00
Tamas Domok	0ba6f35dc2	YARN-10355. Refactor NM ContainerLaunch.java#orderEnvByDependencies Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-08-04 15:27:46 +02:00
Tamas Domok	a5811dda7b	YARN-10874. Refactor NM ContainerLaunch#getEnvDependencies's unit tests (#3248 ) Co-authored-by: Tamas Domok <tdomok@cloudera.com>	2021-08-03 16:19:04 +02:00
Viraj Jasani	ccfa072dc7	HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (#3241 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-08-03 14:44:00 +09:00
zhuqi-lucas	efb3fa2bf5	YARN-10854. Support marking inactive node as untracked without configured include path. Contributed by Tao Yang.	2021-08-02 18:23:33 +08:00
Benjamin Teke	8f750c54aa	YARN-10856. Prevent ATS v2 health check REST API call if the ATS service itself is disabled. (#3236 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-07-29 19:15:27 +02:00
Benjamin Teke	ac0a4e7f58	YARN-10869. CS considers only the default maximum-allocation-mb/vcore property as a maximum when it creates dynamic queues (#3225 ) Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-07-29 17:56:14 +02:00
minni31	d78b300ed4	YARN-10841. Fix token reset synchronization for UAM response token. (#3194 ) YARN-10841. Fix token reset synchronization for UAM response token. Contributed by Minni Mittal	2021-07-29 14:55:39 +05:30
Szilard Nemeth	f2b6c03fc1	YARN-6272. TestAMRMClient#testAMRMClientWithContainerResourceChange fails intermittently. Contributed by Andras Gyory & Prabhu Joseph	2021-07-28 17:02:15 +02:00
Szilard Nemeth	1b9efe58c9	YARN-10790. CS Flexible AQC: Add separate parent and leaf template property. Contributed by Andras Gyori	2021-07-28 16:50:14 +02:00
Szilard Nemeth	8d0297c213	YARN-10727. ParentQueue does not validate the queue on removal. Contributed by Andras Gyori	2021-07-28 14:49:10 +02:00
9uapaw	b4a524722a	YARN-9551. TestTimelineClientV2Impl.testSyncCall fails intermittent (#3212 ) Co-authored-by: Andras Gyori <gandras@cloudera.com>	2021-07-27 19:35:20 -07:00
Szilard Nemeth	fa0289b022	YARN-6221. Entities missing from ATS when summary log file info got returned to the ATS before the domain log. Contributed by Xiaomin Zhang	2021-07-27 16:47:41 +02:00
dependabot[bot]	b7431c3ed5	[UI2] Bump http-proxy to 1.18.1 (#2891 ) Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1. - [Release notes](https://github.com/http-party/node-http-proxy/releases) - [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md) - [Commits](https://github.com/http-party/node-http-proxy/compare/1.18.0...1.18.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2021-07-26 02:16:21 +09:00
Benjamin Teke	05b6a1a06a	YARN-10833. Set the X-FRAME-OPTIONS header for the default contexts. (#3203 ) * YARN-10833. Set the X-FRAME-OPTIONS header for the default contexts. * fixup: YARN-10833. Set the X-FRAME-OPTIONS header for the default contexts. Co-authored-by: Benjamin Teke <bteke@cloudera.com>	2021-07-24 11:44:21 +08:00
Akshat Bordia	aa1a5dd413	YARN-10829. Support getApplications API in FederationClientInterceptor (#3135 ) YARN-10829. Support getApplications API in FederationClientInterceptor (#3135)	2021-07-23 17:07:51 +05:30
zhuqi-lucas	2da9b95d4d	YARN-10657. We should make max application per queue to support node label. Contributed by Andras Gyori.	2021-07-22 20:30:43 +08:00
zhuqi-lucas	0441efe1fc	YARN-10860. Make max container per heartbeat configs refreshable. Contributed by Eric Badger.	2021-07-21 15:31:44 +08:00
9uapaw	e634bf33f3	YARN-10630. [UI2] Ambiguous queue name resolution (#3214 ) Co-authored-by: Andras Gyori <gandras@cloudera.com>	2021-07-21 10:06:46 +08:00
Masatake Iwasaki	17bf2fcbc5	YARN-10858. [UI2] YARN-10826 breaks Queue view. (#3213 )	2021-07-20 11:52:52 +08:00
zhuqi-lucas	0ac443b1f8	YARN-10855. yarn logs cli fails to retrieve logs if any TFile is corrupt or empty. Contributed by Jim Brennan.	2021-07-19 10:11:20 +08:00
Viraj Jasani	d0ee065cc4	HADOOP-16272. Upgrade HikariCP to 4.0.3 (#3204 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-07-16 12:17:17 +09:00
Jim Brennan	632f64cadb	YARN-10456. RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry. Contributed by Eric Payne.	2021-07-15 14:23:31 +00:00
Artem Smotrakov	ba325a8ada	HADOOP-17793. Better token validation (#3189 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-07-10 13:42:31 +09:00
Viraj Jasani	618c9218ee	HADOOP-17788. Replace IOUtils#closeQuietly usages by Hadoop's own utility (#3171 ) Reviewed-by: Steve Loughran <stevel@apache.org> Reviewed-by: Akira Ajisaka <aajisaka@apache.org> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-07-08 16:03:40 +09:00
Jim Brennan	dc6f456e95	YARN-10834. Intra-queue preemption: apps that don't use defined custom resource won't be preempted. Contributed by Eric Payne.	2021-06-28 14:52:19 +00:00
Prabhu Joseph	6b4987f09c	YARN-10820. Make GetClusterNodesRequestPBImpl thread safe. Contributed by Swathi Chandrashekar.	2021-06-27 16:28:32 +05:30
Jim Brennan	7c7d02edbd	YARN-10824. Title not set for JHS and NM webpages. Contributed by Bilwa S T.	2021-06-25 20:32:08 +00:00
Peter Bacsko	0934e783cf	YARN-10780. Optimise retrieval of configured node labels in CS queues. Contributed by Andras Gyori.	2021-06-24 20:15:10 +02:00
Masatake Iwasaki	11d144d228	YARN-10826. [UI2] Upgrade Node.js to v12.22.1. (#3128 )	2021-06-23 19:33:03 +09:00
Szilard Nemeth	6562391737	YARN-10813. Set default capacity of root for node labels. Contributed by Andras Gyori	2021-06-16 18:55:09 +02:00
Szilard Nemeth	428478bbe2	YARN-10801. Fix Auto Queue template to properly set all configuration properties. Contributed by Andras Gyori	2021-06-16 18:26:58 +02:00
Jim Brennan	9a6a11c452	YARN-10767. Yarn Logs Command retrying on Standby RM for 30 times. Contributed by D M Murali Krishna Reddy.	2021-06-15 18:58:42 +00:00
Szilard Nemeth	e31d06032b	YARN-10802. Change Capacity Scheduler minimum-user-limit-percent to accept decimal values. Contributed by Benjamin Teke	2021-06-14 22:33:04 +02:00
Szilard Nemeth	f0bdc422aa	YARN-10816. Avoid doing delegation token ops when yarn.timeline-service.http-authentication.type=simple. Contributed by Tarun Parimi	2021-06-12 15:18:41 +02:00
Szilard Nemeth	7003997e36	YARN-10789. RM HA startup can fail due to race conditions in ZKConfigurationStore. Contributed by Tarun Parimi	2021-06-12 14:49:52 +02:00
Viraj Jasani	4ef27a596f	HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom (#3087 )	2021-06-11 12:15:52 +09:00
Akira Ajisaka	9a17f97133	YARN-10803. [JDK 11] TestRMFailoverProxyProvider and TestNoHaRMFailoverProxyProvider fails by ClassCastException. (#3068 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>	2021-06-10 14:29:47 +09:00
Viraj Jasani	81d7069316	YARN-10805. Replace Guava Lists usage by Hadoop's own Lists in hadoop-yarn-project (#3075 )	2021-06-09 15:15:47 +09:00
Viraj Jasani	ca27744770	YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3081 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>	2021-06-08 19:37:23 -07:00
Prabhu Josephraj	9445abb500	YARN-10792. Set Completed AppAttempt LogsLink to Log Server URL. Contributed by Abhinaba Sarkar	2021-06-08 20:37:40 +05:30
zhuqi-lucas	ec16b1d3b9	YARN-10807. Parents node labels are incorrectly added to child queues in weight mode. Contributed by Benjamin Teke.	2021-06-08 21:03:43 +08:00
Hideyuki Furue	9c7b8cf54e	Fix container-executor Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-06-06 07:05:29 +09:00
Szilard Nemeth	200eec8f2e	YARN-10796. Capacity Scheduler: dynamic queue cannot scale out properly if its capacity is 0%. Contributed by Peter Bacsko	2021-06-03 13:27:36 +02:00
Szilard Nemeth	2707f69251	YARN-10787. Queue submit ACL check is wrong when CS queue is ambiguous. Contributed by Gergely Pollak	2021-06-01 16:01:39 +02:00
Gergely Pollak	e9339aa376	YARN-10797. Logging parameter issues in scheduler package. Contributed by Szilard Nemeth	2021-06-01 15:57:22 +02:00
Viraj Jasani	986d0a4f1d	HADOOP-17732. Keep restrict-imports-enforcer-rule for Guava Sets in hadoop-main pom (#3049 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-05-26 17:14:31 +09:00
Jim Brennan	c4581827a9	YARN-10691. DominantResourceCalculator isInvalidDivisor should consider only countable resource types. Contributed by Bilwa S T.	2021-05-25 18:07:28 +00:00
Szilard Nemeth	b86a6eb871	YARN-10782. Extend /scheduler endpoint with template properties. Contributed by Andras Gyori	2021-05-25 18:27:53 +02:00
Szilard Nemeth	2541efa496	YARN-10783. Allow definition of auto queue template properties in root. Contributed by Andras Gyori	2021-05-25 13:55:59 +02:00
Akira Ajisaka	1cbcde04f3	YARN-10770. container-executor permission is wrong in SecureContainer.md. Contributed by Siddharth Ahuja.	2021-05-25 18:28:09 +09:00
Viraj Jasani	996d31f2dc	HADOOP-17721. Replace Guava Sets usage by Hadoop's own Sets in hadoop-yarn-project (#3033 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-05-25 18:10:20 +09:00
zhuqi-lucas	59172ada90	YARN-10771. Add cluster metric for size of SchedulerEventQueue and RMEventQueue. Contributed by chaosju.	2021-05-24 23:12:07 +08:00
Szilard Nemeth	6bb0892f6e	YARN-10753. Document the removal of FS default queue creation. Contributed by Benjamin Teke	2021-05-22 15:00:08 +02:00
Szilard Nemeth	1e44bdb84c	YARN-7769. FS QueueManager should not create default queue at init. Contributed by Benjamin Teke	2021-05-22 14:55:01 +02:00
Szilard Nemeth	d146ab3af7	YARN-10766. [UI2] Bump moment-timezone to 0.5.33. Contributed by Andras Gyori	2021-05-21 18:07:32 +02:00
Szilard Nemeth	43e77caa71	YARN-10779. Add option to disable lowercase conversion in GetApplicationsRequestPBImpl and ApplicationSubmissionContextPBImpl. Contributed by Peter Bacsko	2021-05-21 18:04:24 +02:00
Akira Ajisaka	c8073811b2	YARN-9279. Remove the old hamlet package. (#2986 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-05-21 12:40:50 +09:00
dependabot[bot]	f8b0063201	Bump node-sass (#3004 ) Bumps [node-sass](https://github.com/sass/node-sass) from 4.13.0 to 4.14.1. - [Release notes](https://github.com/sass/node-sass/releases) - [Changelog](https://github.com/sass/node-sass/blob/master/CHANGELOG.md) - [Commits](https://github.com/sass/node-sass/compare/v4.13.0...v4.14.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>	2021-05-18 20:14:18 -07:00
Peter Bacsko	eb72628e15	YARN-10258. Add metrics for 'ApplicationsRunning' in NodeManager. Contributed by ANANDA G B.	2021-05-17 21:07:29 +02:00
Peter Bacsko	8891e5c028	YARN-10763. Add the number of containers assigned per second metrics to ClusterMetrics. Contributed by chaosju.	2021-05-17 13:30:12 +02:00
lujiefsi	d92a25b790	YARN-10555. Missing access check before getAppAttempts (#2608 ) Co-authored-by: lujie <lujie@foxmail.com> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-05-17 13:44:38 +09:00
kyoty	344cacce23	YARN-10545. Improve the readability of diagnostics log in yarn-ui2 web page. #2540 Contributed by kyoty.	2021-05-14 23:57:54 +08:00
zhuqi	e7f0e8073b	YARN-10761: Add more event type to RM Dispatcher event metrics. Contributed by Qi Zhu.	2021-05-14 13:36:07 +08:00
zhuqi	d2b0675d61	YARN-10737: Fix typos in CapacityScheduler#schedule. (#2911 ) Contributed by Qi Zhu.	2021-05-14 13:12:28 +08:00
dependabot[bot]	881ab4ed39	Bump underscore (#2997 ) Bumps [underscore](https://github.com/jashkenas/underscore) from 1.9.1 to 1.13.1. - [Release notes](https://github.com/jashkenas/underscore/releases) - [Commits](https://github.com/jashkenas/underscore/compare/1.9.1...1.13.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>	2021-05-13 05:54:07 -07:00
dependabot[bot]	2f67a2646e	Bump acorn (#3003 ) Bumps [acorn](https://github.com/acornjs/acorn) from 5.7.3 to 5.7.4. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](https://github.com/acornjs/acorn/compare/5.7.3...5.7.4) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>	2021-05-13 05:51:43 -07:00
Peter Bacsko	626be24c3e	YARN-10571. Refactor dynamic queue handling logic. Contributed by Andras Gyori.	2021-05-12 14:54:47 +02:00
Borislav Iordanov	91430889a5	HADOOP-17665 Ignore missing keystore configuration in reloading mechanism	2021-05-10 13:31:48 -07:00
Akira Ajisaka	7cb7b8adc1	YARN-10756. Remove additional junit 4.11 dependency from javadoc. (#2960 ) Reviewed-by: Viraj Jasani <vjasani@apache.org> Reviewed-by: Bilwa S T <stbilwa@gmail.com> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2021-05-06 23:37:45 +09:00
Wei-Chiu Chuang	f1e1809029	HADOOP-17653. Do not use guava's Files.createTempDir(). (#2945 ) Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-05-02 11:10:32 +09:00
Eric Badger	803ac4b1a0	YARN-10707. Support custom resources in ResourceUtilization, and update Node GPU Utilization to use. Contributed by Qi Zhu	2021-04-29 17:42:13 +00:00
Siyao Meng	5b09ba75ec	YARN-10747. Bump YARN CSI protobuf version to 3.7.1 (#2946 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org> Change-Id: Ib2a4bf775a5804641a91c6f9a486772c9d87c17e	2021-04-28 12:44:13 +09:00
Peter Bacsko	a967ab06f2	YARN-10739. GenericEventHandler.printEventQueueDetails causes RM recovery to take too much time. Contributed by Qi Zhu.	2021-04-27 14:01:53 +02:00
Peter Bacsko	9166bfeb74	YARN-10637. fs2cs: add queue autorefresh policy during conversion. Contributed by Qi Zhu.	2021-04-24 10:10:10 +02:00
Viraj Jasani	538ce9c354	HADOOP-17650. Bump solr to unblock build failure with Maven 3.8.1 (#2939 ) Reviewed-by: Siyao Meng <siyao@apache.org>	2021-04-25 23:48:39 -07:00
Eric Badger	6857a05d6a	YARN-10479. Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647. Contributed by D M Murali Krishna Reddy	2021-04-23 22:02:04 +00:00
Jim Brennan	1cbe35946b	YARN-10743. Add a policy for not aggregating for containers which are killed because exceeding container log size limit. Contributed by Qi Zhu.	2021-04-23 21:35:09 +00:00
Szilard Nemeth	ebf6b14c67	YARN-10123. Error message around yarn app -stop/start can be improved. Contributed by Siddharth Ahuja This is required to highlight that an implementation at framework level is needed for the stop/start functionality to work.	2021-04-23 22:49:26 +02:00
Peter Bacsko	14a84c47b0	YARN-10705. Misleading DEBUG log for container assignment needs to be removed when the container is actually reserved, not assigned in FairScheduler. Contributed by Siddharth Ahuja.	2021-04-23 17:38:47 +02:00
Szilard Nemeth	3b6b21e459	YARN-10750. TestMetricsInvariantChecker.testManyRuns is broken since HADOOP-17524. Contributed by Gergely Pollak	2021-04-23 16:17:29 +02:00
Szilard Nemeth	919daec36b	YARN-10746. RmWebApp add default-node-label-expression to the queue info. Contributed by Gergely Pollak	2021-04-23 16:12:12 +02:00
Szilard Nemeth	f76a2a7606	YARN-10654. Dots '.' in CSMappingRule path variables should be replaced. Contributed by Peter Bacsko	2021-04-23 16:07:58 +02:00
Eric Badger	6cb90005a7	YARN-10723. Change CS nodes page in UI to support custom resource. Contributed by Qi Zhu	2021-04-20 17:34:49 +00:00
Viraj Jasani	9a261b0a16	HADOOP-17642. Remove appender EventCounter to avoid instantiation (#2922 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-04-17 20:47:44 +09:00
Miklos Gergely	ba3bc53f4e	YARN-10736. Fix GetApplicationsRequest JavaDoc. Contributed by Miklos Gergely. (#2906 )	2021-04-14 17:16:30 +02:00
dependabot[bot]	6a9a5ae5fd	Bump y18n (#2843 ) Bumps [y18n](https://github.com/yargs/y18n) from 3.2.1 to 3.2.2. - [Release notes](https://github.com/yargs/y18n/releases) - [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md) - [Commits](https://github.com/yargs/y18n/commits) Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-04-11 01:30:49 +09:00
Eric Badger	213d3deb26	YARN-10503. Support queue capacity in terms of absolute resources with custom resourceType. Contributed by Qi Zhu.	2021-04-09 00:34:15 +00:00
Peter Bacsko	ca9aa91d10	YARN-10564. Support Auto Queue Creation template configurations. Contributed by Andras Gyori.	2021-04-08 12:42:48 +02:00
Szilard Nemeth	9cd69c20c4	YARN-10714. Remove dangling dynamic queues on reinitialization. Contributed by Andras Gyori	2021-04-07 11:52:21 +02:00
Eric Badger	26b8f678b2	YARN-10702. Add cluster metric for amount of CPU used by RM Event Processor. Contributed by Jim Brennan.	2021-04-06 01:16:14 +00:00
Peter Bacsko	158758c5bf	YARN-10726. Log the size of DelegationTokenRenewer event queue in case of too many pending events. Contributed by Qi Zhu.	2021-04-01 16:09:52 +02:00
Peter Bacsko	9f1655baf2	YARN-9618. NodesListManager event improvement. Contributed by Qi Zhu.	2021-04-01 11:39:40 +02:00
Peter Bacsko	a0deda1a77	YARN-10720. YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging. Contributed by Qi Zhu.	2021-04-01 09:21:15 +02:00
Borislav Iordanov	2c482fbacf	HADOOP-16524. Automatic keystore reloading for HttpServer2 Reapply of issue reverted first because it caused yarn failures and then again because the commit message was incorrectly formatted (and yet again because of commit message format). Signed-off-by: stack <stack@apache.org>	2021-03-31 10:46:35 -07:00
stack	22961a615d	Revert "HADOOP-16524. Automatic keystore reloading for HttpServer2" This reverts commit `a2975d2153`.	2021-03-31 10:43:09 -07:00
stack	a2975d2153	HADOOP-16524. Automatic keystore reloading for HttpServer2 Reapply of issue reverted first because it caused yarn failures and then again because the commit message was incorrectly formatted.	2021-03-31 10:40:20 -07:00
stack	5183aaeda2	Revert "Hadoop 16524 - resubmission following some unit test fixes (#2693 )" Revert to fix the summary message. This reverts commit `9509bebf7f`.	2021-03-31 10:39:55 -07:00
Borislav Iordanov	9509bebf7f	Hadoop 16524 - resubmission following some unit test fixes (#2693 ) Signed-off-by: stack <stack@apache.org>	2021-03-31 10:07:42 -07:00
Szilard Nemeth	6fd0c661b6	YARN-10597. CSMappingPlacementRule should not create new instance of Groups. Contributed by Gergely Pollak	2021-03-31 16:14:21 +02:00
Peter Bacsko	ff6ec20d84	YARN-10718. Fix CapacityScheduler#initScheduler log error. Contributed by Qi Zhu.	2021-03-31 10:55:14 +02:00
Akira Ajisaka	8ad77a26c9	YARN-10716. Fix typo in ContainerRuntime. Contributed by xishuhai.	2021-03-30 17:46:51 +09:00
Brahma Reddy Battula	413a4c3c05	YARN-10466.Fix NullPointerException in yarn-services Component.java. Contributed by D M Murali Krishna Reddy	2021-03-30 13:52:07 +05:30
Brahma Reddy Battula	6577bf1891	YARN-10439. addendum fix for shaded guva.	2021-03-30 13:48:40 +05:30
Brahma Reddy Battula	b61f52ec56	YARN-10441. Add support for hadoop.http.rmwebapp.scheduler.page.class. Contributed by D M Murali Krishna Reddy	2021-03-30 09:51:47 +05:30
Brahma Reddy Battula	d0dcfc405c	YARN-10439. Yarn Service AM listens on all IP's on the machine. Contributed by D M Murali Krishna Reddy	2021-03-30 09:46:12 +05:30
Brahma Reddy Battula	2d62dced4b	YARN-10437. Destroy yarn service if any YarnException occurs during submitApp.Contributed by D M Murali Krishna Reddy	2021-03-30 09:39:00 +05:30
Eric Badger	19e418c10d	YARN-10713. ClusterMetrics should support custom resource capacity related metrics. Contributed by Qi Zhu.	2021-03-25 22:33:58 +00:00
Peter Bacsko	ceb75e1e2a	YARN-10674. fs2cs should generate auto-created queue deletion properties. Contributed by Qi Zhu.	2021-03-24 08:15:06 +01:00
Jim Brennan	174f3a96b1	YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T.	2021-03-23 18:21:45 +00:00
Jack Jiang	d8ec8ab965	HADOOP-17599. Remove NULL checks before instanceof (#2804 )	2021-03-23 08:46:11 -07:00
Cyrus Jackson	cd44e917d0	YARN-10476. Queue metrics for Unmanaged applications (#2674 ). Contributed by Cyrus Jackson	2021-03-19 15:49:05 +05:30
Eric Badger	3d4ea4b336	YARN-10701. The yarn.resource-types should support multi types without trimmed. Contributed by Qi Zhu.	2021-03-18 19:36:50 +00:00
Eric Badger	181b9ddc51	YARN-10703. Fix potential null pointer error of gpuNodeResourceUpdateHandler in NodeResourceMonitorImpl. Contributed by Qi Zhu.	2021-03-18 17:45:09 +00:00
Peter Bacsko	ce6bfd5718	YARN-10641. Refactor the max app related update, and fix maxApllications update error when add new queues. Contributed by Qi Zhu.	2021-03-18 13:40:16 +01:00
Peter Bacsko	38495af325	YARN-10692. Add Node GPU Utilization and apply to NodeMetrics. Contributed by Qi Zhu.	2021-03-18 12:46:11 +01:00
Szilard Nemeth	a5745711dd	YARN-10659. Improve CS MappingRule %secondary_group evaluation. Contributed by Gergely Pollak	2021-03-18 12:43:01 +01:00
Peter Bacsko	d7eeca4d0c	YARN-10685. Fix typos in AbstractCSQueue. Contributed by Qi Zhu.	2021-03-18 11:49:16 +01:00
Eric Badger	49f89f1d3d	YARN-10688. ClusterMetrics should support GPU capacity related metrics.. Contributed by Qi Zhu.	2021-03-17 18:11:37 +00:00
Peter Bacsko	3e58d5611d	YARN-10497. Fix an issue in CapacityScheduler which fails to delete queues. Contributed by Wangda Tan and Qi Zhu.	2021-03-17 13:38:20 +01:00
Wilfred Spiegelenburg	f276f1af80	YARN-10652. Capacity Scheduler fails to handle user weights for a user that has a "." (dot) in it This only fixes the user name resolution for weights in the queues. It does not add generic support for user names with dots in all use cases in the capacity scheduler. Contributed by: Siddharth Ahuja	2021-03-17 10:55:05 +11:00
Peter Bacsko	73394fabc7	YARN-10686. Fix TestCapacitySchedulerAutoQueueCreation#testAutoQueueCreationFailsForEmptyPathWithAQCAndWeightMode. Contributed by Qi Zhu.	2021-03-16 16:09:12 +01:00
Peter Bacsko	b80588b688	YARN-10682. The scheduler monitor policies conf should trim values separated by comma. Contributed by Qi Zhu.	2021-03-16 15:23:27 +01:00
zhuqi	e9c98548e9	YARN-10689. Fix the finding bugs in extractFloatValueFromWeightConfig. (#2760 ) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2021-03-16 13:50:29 +09:00
Eric Payne	aa4c17b9d7	YARN-10588. Percentage of queue and cluster is zero in WebUI . Contributed by Bilwa S T	2021-03-15 19:09:40 +00:00
Jim Brennan	5f067cf0f3	[YARN-10687] Add option to disable/enable free disk space checking and percentage checking for full and not-full disks. Contributed by Qi Zhu.	2021-03-12 17:17:31 +00:00
Szilard Nemeth	5db4c0bf70	YARN-10412. Move CS placement rule related changes to a separate package. Contributed by Gergely Pollak	2021-03-12 14:10:16 +01:00
Akira Ajisaka	23b343aed1	HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753 ) Removed findbugs from the hadoop build images and added spotbugs instead. Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0. Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>	2021-03-11 10:56:07 +09:00
Brahma Reddy Battula	b2a565629d	YARN-10671.Fix Typo in TestSchedulingRequestContainerAllocation. Contributed by D M Murali Krishna Reddy.	2021-03-09 20:26:07 +05:30
Peter Bacsko	d5e035dbe1	YARN-9615. Add dispatcher metrics to RM. Contributed by Jonathan Hung and Qi Zhu.	2021-03-09 14:33:14 +01:00
Peter Bacsko	3851994cd6	Revert "YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu." This reverts commit `369f75b7a7`.	2021-03-09 14:32:02 +01:00
Peter Bacsko	369f75b7a7	YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu.	2021-03-09 14:28:23 +01:00
Peter Bacsko	20416bc62d	YARN-10675. Consolidate YARN-10672 and YARN-10447. Contributed by Szilard Nemeth.	2021-03-09 11:34:37 +01:00
Peter Bacsko	e472ee2aa5	YARN-10676. Improve code quality in TestTimelineAuthenticationFilterForV1. Contributed by Szilard Nemeth.	2021-03-09 11:28:24 +01:00
Peter Bacsko	04cd3115ba	YARN-10658. CapacityScheduler QueueInfo add queue path field to avoid ambiguous QueueName. Contributed by Qi Zhu.	2021-03-08 12:29:44 +01:00
Peter Bacsko	c6b30a59dd	YARN-10672. All testcases in TestReservations are flaky. Contributed By Szilard Nemeth.	2021-03-05 23:13:35 +01:00
Eric Badger	9d088639bf	YARN-10664. Allow parameter expansion in NM_ADMIN_USER_ENV. Contributed by Jim Brennan.	2021-03-05 19:46:40 +00:00
Peter Bacsko	e82e7c597a	YARN-10640. Adjust the queue Configured capacity to Configured weight number for weight mode in UI. Contributed by Qi Zhu.	2021-03-05 15:56:56 +01:00
Peter Bacsko	e7cad3811f	YARN-10642. Race condition: AsyncDispatcher can get stuck by the changes introduced in YARN-8995. Contributed by zhengchenyu.	2021-03-05 13:50:45 +01:00
Peter Bacsko	e19c00925f	YARN-10639. Queueinfo related capacity, should adjusted to weight mode. Contributed by Qi Zhu.	2021-03-05 13:18:06 +01:00
Peter Bacsko	6699198b54	YARN-10532. Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used. Contributed by Qi Zhu.	2021-03-04 17:18:35 +01:00
Peter Bacsko	a85aeee876	YARN-10623. Capacity scheduler should support refresh queue automatically by a thread policy. Contributed by Qi Zhu.	2021-03-04 12:23:11 +01:00
Neil	d615e2d3bd	YARN-10649. Fix RMNodeImpl.updateExistContainers leak (#2719 ). Contributed by Max Xie	2021-03-04 14:52:58 +05:30
Peter Bacsko	7e8040e6ad	YARN-10655. Limit queue creation depth relative to its first static parent. Contributed by Andras Gyori.	2021-03-03 17:44:30 +01:00
JJiaguodong	8af56de1fa	HADOOP-17560. Fix some spelling errors (#2730 ) Co-authored-by: jiaguodong5 <jiaguodong5@jd.com>	2021-03-03 11:41:05 +09:00
Akira Ajisaka	3101c39b49	YARN-10656. Parsing error in CapacityScheduler.md (#2725 ) Reviewed-by: Peter Bacsko <pbacsko@apache.org>	2021-02-27 12:46:36 +09:00
Peter Bacsko	947b50489d	YARN-10627. Extend logging to give more information about weight mode. Contributed by Benjamin Teke.	2021-02-26 21:57:54 +01:00
Ahmed Hussein	e04bcb3a06	MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722 ). Contributed by Ahmed Hussein	2021-02-26 13:42:33 -06:00
Eric Badger	fcdb1e7fec	YARN-10653. Fixed the findbugs issues introduced by YARN-10647. Contributed by Qi Zhu	2021-02-26 19:03:59 +00:00
Jonathan Hung	ac37ca0b67	YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen	2021-02-25 16:01:18 -08:00
Jim Brennan	c373da9f88	[YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne	2021-02-25 16:46:19 +00:00
Peter Bacsko	a89c624002	YARN-10609. Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue). Contributed by Qi Zhu.	2021-02-24 20:41:35 +01:00
Eric Badger	47420ae3ed	YARN-10647. Fix TestRMNodeLabelsManager failed after YARN-10501. Contributed by Qi Zhu.	2021-02-22 18:48:14 +00:00
Peter Bacsko	b722c52f4e	YARN-10513. CS Flexible Auto Queue Creation RM UIv2 modifications. Contributed by Andras Gyori.	2021-02-22 12:55:45 +01:00
Eric Badger	4891e68c2b	YARN-10501. Can't remove all node labels after add node label without nodemanager port. Contributed by caozhiqiang.	2021-02-19 22:29:10 +00:00
Peter Bacsko	d28b6f90c8	YARN-10636. CS Auto Queue creation should reject submissions with empty path parts. Contributed by Gergely Pollak.	2021-02-19 13:57:30 +01:00
Peter Bacsko	4383726d19	YARN-10635. CSMapping rule can return paths with empty parts. Contributed by Gergely Pollak.	2021-02-19 12:01:31 +01:00
Akira Ajisaka	9a298d180d	Revert "HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454 )" This reverts commit `4cf3531583`.	2021-02-19 11:09:10 +09:00
Akira Ajisaka	4cf3531583	HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454 ) Use spotbugs instead of findbugs. Removed findbugs from the hadoop build images, and added spotbugs in the images instead. Reviewed-by: Masatake Iwasaki <iwasakims@apache.org> Reviewed-by: Inigo Goiri <inigoiri@apache.org> Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>	2021-02-17 10:38:20 +09:00
Szilard Nemeth	9882a22365	YARN-10625. FairScheduler: add global flag to disable AM-preemption. Contributed by Peter Bacsko	2021-02-16 18:06:57 +01:00
Jim Brennan	3fcc7a55b6	[YARN-10626] Log resource allocation in NM log at container start time. Contributed by Eric Badger	2021-02-16 16:45:35 +00:00
Szilard Nemeth	bad6038a48	YARN-10622. Fix preemption policy to exclude childless ParentQueues. Contributed by Andras Gyori	2021-02-15 14:48:42 +01:00
Szilard Nemeth	852aac34f2	Support max queues limit configuration in new auto created queue, consistent with old auto created. Contributed by Qi Zhu	2021-02-15 14:20:50 +01:00
Masatake Iwasaki	f9a073c6c1	YARN-10500. TestDelegationTokenRenewer fails intermittently. (#2619 )	2021-02-11 13:55:04 -06:00
Szilard Nemeth	98ca6afd17	YARN-10618. RM UI2 Application page shows the AM preempted containers instead of the nonAM ones. Contributed by Benjamin Teke	2021-02-11 09:14:37 +01:00
Szilard Nemeth	cacc870389	YARN-10593. Fix incorrect string comparison in GpuDiscoverer. Contributed by Peter Bacsko	2021-02-10 13:09:32 +01:00
Szilard Nemeth	a8bd516e39	YARN-10620. fs2cs: parentQueue for certain placement rules are not set during conversion. Contributed by Peter Bacsko	2021-02-10 11:15:53 +01:00
Szilard Nemeth	fc5f241535	YARN-10619. CS Mapping Rule %specified rule catches default submissions. Contributed by Gergely Pollak	2021-02-09 18:16:42 +01:00
Amogh Rajesh Desai	0b05dd8802	YARN-9650. Set thread names for CapacityScheduler AsyncScheduleThread (#2665 ) Amogh Rajesh Desai via Zhankun Tang	2021-02-09 11:05:14 +08:00
Jim Brennan	c22c77af43	[YARN-10607] User environment is unable to prepend PATH when mapreduce.admin.user.env also sets PATH. Contributed by Eric Badger.	2021-02-05 17:33:01 +00:00
Szilard Nemeth	5aa9866ec2	YARN-10615. Fix Auto Queue Creation hierarchy construction to use queue path instead of short queue name. Contributed by Andras Gyori	2021-02-05 17:43:01 +01:00
Szilard Nemeth	c19326c051	YARN-10610. Add queuePath to RESTful API for CapacityScheduler consistent with FairScheduler queuePath. Contributed by Qi Zhu	2021-02-05 17:34:45 +01:00
Szilard Nemeth	79a46599f7	YARN-10428. Zombie applications in the YARN queue using FAIR + sizebasedweight. Contributed by Guang Yang, Andras Gyori	2021-02-05 17:09:08 +01:00
Akira Ajisaka	52217fce3d	HADOOP-17432. [JDK 16] KerberosUtil#getOidInstance is broken by JEP 396 (#2546 ) Reviewed-by: Steve Loughran <stevel@apache.org>	2021-02-05 16:14:10 +09:00
Eric Badger	5e719bf588	YARN-10611. Fix that shaded should be used for google guava imports in YARN-10352. Contributed by Qi Zhu	2021-02-04 18:30:39 +00:00
Szilard Nemeth	c17414111b	YARN-10612. Fix findbugs issue introduced in YARN-10585. Contributed by Gergely Pollak	2021-02-04 08:12:13 +01:00
bibinchundatt	6fc26ad539	YARN-10352 Skip schedule on not heartbeated nodes in Multi Node Placement. Contributed by Prabhu Joseph and Qi Zhu	2021-02-03 08:50:45 +05:30
Szilard Nemeth	9bf2ac07bb	YARN-10605. Add queue-mappings-override.enable property in FS2CS conversions. Contributed by Andras Gyori	2021-02-02 09:13:46 +01:00
Szilard Nemeth	7d88953768	YARN-10604. Support auto queue creation without mapping rules. Contributed by Andras Gyori	2021-02-02 09:07:25 +01:00
belugabehr	21a3fc3d2d	HADOOP-17482: Remove Commons Logger from FileSystem Class (#2633 )	2021-02-01 09:40:01 -08:00
Peter Bacsko	fa15594ae6	YARN-10600. Convert root queue in fs2cs weight mode conversion. Contributed by Benjamin Teke.	2021-01-28 20:58:51 +01:00
Szilard Nemeth	9628aa87bf	YARN-10598. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to extend the creation type with additional information. Contributed by Benjamin Teke	2021-01-27 18:14:06 +01:00
Szilard Nemeth	7c4ef42837	YARN-10599. fs2cs should generate new 'auto-queue-creation-v2.enabled' properties for all parents. Contributed by Peter Bacsko	2021-01-27 15:49:26 +01:00
Szilard Nemeth	e2a7008d50	YARN-10585. Create a class which can convert from legacy mapping rule format to the new JSON format. Contributed by Gergely Pollak	2021-01-26 18:31:39 +01:00
Wangda Tan	4f008153ef	YARN-10587. Fix AutoCreateLeafQueueCreation cap related caculation when in absolute mode. (Qi Zhu via wangda) Change-Id: I8889c393a9f08b633c72a2e873c6f45ca12cc82b	2021-01-26 08:24:29 -08:00
Wangda Tan	b7384a8d02	YARN-10531. Be able to disable user limit factor for CapacityScheduler Leaf Queue. (Qi Zhu via wangda) Change-Id: I670e5525619b320745254609c48e7e1afb084835	2021-01-26 08:24:28 -08:00
Szilard Nemeth	f1766e5bb4	YARN-10596. Allow static definition of childless ParentQueues with auto-queue-creation-v2 enabled. Contributed by Andras Gyori	2021-01-26 16:22:35 +01:00
Szilard Nemeth	8620984b8d	YARN-10515. Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags. Contributed by Peter Bacsko	2021-01-21 20:58:57 +01:00
Szilard Nemeth	1ccba3734d	YARN-10490. yarn top command not quitting completely with ctrl+c. Contributed by Agshin Kazimli	2021-01-21 20:30:19 +01:00
Peter Bacsko	17fdac8de5	YARN-10581. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include queue creation type for queues. Contributed by Szilard Nemeth.	2021-01-21 18:06:53 +01:00
Szilard Nemeth	06fef5ee43	YARN-10579. CS Flexible AQC: Modify RM /scheduler endpoint to include weight values for queues. Contributed by Szilard Nemeth	2021-01-21 09:23:11 +01:00
bibinchundatt	8bc2dfbf36	YARN-10519. Refactor QueueMetricsForCustomResources class to move to yarn-common package. Contributed by Minni Mittal	2021-01-21 11:23:30 +05:30
Szilard Nemeth	cfe6e1f7da	YARN-10578. Fix Auto Queue Creation parent handling. Contributed by Andras Gyori	2021-01-20 15:22:44 +01:00
Sunil G	4b5bc05a78	YARN-10512. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include mode of operation for CS. Contributed by Szilard Nemeth.	2021-01-20 12:28:59 +05:30
Szilard Nemeth	a326f22606	YARN-10573. Enhance placement rule conversion in fs2cs in weight mode and enable it by default. Contributed by Peter Bacsko	2021-01-19 09:42:40 +01:00
Szilard Nemeth	6abdb148e4	YARN-10535. Make queue placement in CapacityScheduler compliant with auto-queue-placement. Contributed by Gergely Pollak	2021-01-18 20:19:36 +01:00
Szilard Nemeth	acea862d59	YARN-10574. Fix the FindBugs warning introduced in YARN-10506. Contributed by Gergely Pollak	2021-01-18 16:04:45 +01:00
Wangda Tan	3d46141583	YARN-10506. Update queue creation logic to use weight mode and allow the flexible static/dynamic creation. (Contributed by Qi Zhu, Andras Gyori) Change-Id: I118862fd5e11ee6888275e2bcf667fedfa56c5d7	2021-01-15 14:20:08 -08:00
Eric Badger	768e2f42ba	YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan.	2021-01-13 23:41:52 +00:00
Eric Payne	d8fadec944	YARN-4589: Diagnostics for localization timeouts is lacking. Contributed by Chang Li (lichangleo) and Jim Brennan (Jim_Brennan)	2021-01-13 17:01:28 +00:00
Szilard Nemeth	836c630430	YARN-10525. Add weight mode conversion to fs2cs. Contributed by Peter Bacsko	2021-01-13 14:24:12 +01:00
bibinchundatt	61f77b7674	YARN-8529. Add timeout to RouterWebServiceUtil#invokeRMWebService. Contributed by Minni Mittal	2021-01-13 16:00:11 +05:30
Szilard Nemeth	85b1c017ee	YARN-10563. Fix dependency exclusion problem in poms. Contributed by Peter Bacsko	2021-01-12 15:29:20 +01:00
Szilard Nemeth	9c6e093783	YARN-10570. Remove experimental warning message from fs2cs. Contributed by Peter Bacsko	2021-01-12 15:21:27 +01:00
Wangda Tan	b0eec09097	YARN-10504. Implement weight mode in Capacity Scheduler. (Contributed by Wangda Tan, Benjamin Teke, zhuqi, Andras Gyori) Change-Id: Ic49c730b0ab502ba86527fb662d25c4c8b1c2588	2021-01-11 17:46:09 -08:00
He Xiaoqiao	ec22850dbe	Make upstream aware of 3.2.2 release.	2021-01-09 18:06:08 +08:00
Ahmed Hussein	890f2da624	YARN-10553. Refactor TestDistributedShell (#2581 )	2021-01-09 08:39:21 +09:00
Szilard Nemeth	4176759909	YARN-10507. Add the capability to fs2cs to write the converted placement rules inside capacity-scheduler.xml. Contributed by Peter Bacsko	2021-01-08 13:40:33 +01:00
Szilard Nemeth	580a6a75a3	YARN-10528. maxAMShare should only be accepted for leaf queues, not parent queues. Contributed by Siddharth Ahuja	2021-01-08 12:38:43 +01:00
srinivasst	1b1791075a	YARN-10538: Add RECOMMISSIONING nodes to the list of updated nodes returned to the AM (#2564 ) Contributed by Srinivas S T	2021-01-08 10:52:52 +05:30
Akira Ajisaka	ae4945fb2c	YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2. (#2592 )	2021-01-06 12:12:46 +09:00
Masatake Iwasaki	5abeecd33f	YARN-10558. Fix failure of TestDistributedShell#testDSShellWithOpportunisticContainers. (#2586 )	2021-01-05 13:38:34 +09:00
Neil	fa4cf91b57	YARN-10541. capture the performance metrics of ZKRMStateStore (#2568 )	2020-12-30 22:20:12 -08:00
Ahmed Hussein	513f1995ad	YARN-10334. Close clients in TestDistributedShell (#2571 )	2020-12-28 11:09:10 -08:00
Eric Badger	4c5d88e230	YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes. Contributed by Jim Brennan.	2020-12-21 23:10:31 +00:00
zhuqi	bb528e3239	YARN-10463: For Federation, we should support getApplicationAttemptReport. (#2563 ) Qi Zhu via Zhankun Tang	2020-12-21 10:04:16 +08:00
Ahmed Hussein	7a88f45366	YARN-10536. Client in distributedShell swallows interrupt exceptions (#2554 )	2020-12-17 15:13:28 -08:00
Akira Ajisaka	df7f1e5199	YARN-10499. TestRouterWebServiceREST fails (#2490 ). Contributed by Akira Ajisaka	2020-12-16 17:26:15 +01:00
Szilard Nemeth	df35c7f519	YARN-10526. RMAppManager CS Placement ignores parent path. Contributed by Gergely Pollak	2020-12-15 17:30:14 +01:00
adamantal	3234e5eaf3	YARN-10031. Create a general purpose log request with additional query parameters. Contributed by Andras Gyori	2020-12-12 09:42:22 +01:00
adamantal	0a45bd034e	YARN-10520. Deprecated the residual nested class for the LCEResourceHandler (#2527 ). Contributed by Wanqiang Ji	2020-12-09 12:55:24 +01:00
zhuqi	d67ccd03e3	YARN-10380: Import logic of multi-node allocation in CapacityScheduler (#2494 ) Contributed by Qi Zhu.	2020-12-09 19:48:39 +08:00
Eric Badger	7d3c8ef606	YARN-10495. make the rpath of container-executor configurable. Contributed by angerszhu.	2020-12-08 00:53:24 +00:00
Andrea Scarpino	9170eb566b	YARN-10511. Update yarn.nodemanager.env-whitelist value in docs (#2512 ) Reviewed-by: Adam Antal <adamantal@apache.org> Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2020-12-04 00:14:04 +09:00
adamantal	9969745343	YARN-9883. Reshape SchedulerHealth class. Contributed by D M Murali Krishna Reddy	2020-12-03 09:55:06 +01:00
Eric Payne	fa773a8326	YARN-10278: CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework. Contributed by Szilard Nemeth (snemeth)	2020-12-01 22:06:47 +00:00
kevinzhao1661	4d2ae5b398	YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484 ) Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2020-11-30 11:17:10 +09:00
Akira Ajisaka	2ce2198287	HADOOP-17394. [JDK 11] Fix error in mvn package -Pdocs (#2488 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>	2020-11-26 11:32:10 +09:00
Akira Ajisaka	65002c9486	Revert "[JDK 11] Fix error in mvn package -Pdocs (#2488 )" because JIRA ID is missing. This reverts commit `ce5b3d7975`.	2020-11-26 11:30:53 +09:00
Akira Ajisaka	ce5b3d7975	[JDK 11] Fix error in mvn package -Pdocs (#2488 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>	2020-11-26 11:27:30 +09:00
Ankit Kumar	08b2e285db	YARN-10488. Several typos in package: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair (#2485 ) Co-authored-by: akumar <akumar@cloudera.com> Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2020-11-25 13:00:09 +09:00
Ahmed Hussein	569b20e31c	YARN-10468. Fix TestNodeStatusUpdater timeouts and broken conditions (#2461 )	2020-11-24 11:09:30 -08:00
Akira Ajisaka	c4ba0ab7df	YARN-10470. When building new web ui with root user, the bower install should support it. Contributed by zhuqi.	2020-11-24 15:22:07 +09:00
Szilard Nemeth	0d3155a687	YARN-10457. Add a configuration switch to change between legacy and JSON placement rule format. Contributed by Gergely Pollak	2020-11-18 17:02:47 +01:00
Szilard Nemeth	5ff70a59c4	YARN-10486. FS-CS converter: handle case when weight=0 and allow more lenient capacity checks in Capacity Scheduler. Contributed by Peter Bacsko	2020-11-18 16:51:42 +01:00
Ahmed Hussein	0b2510ee1f	YARN-10485. TimelineConnector swallows InterruptedException (#2450 ). Contributed by Ahmed Hussein	2020-11-16 14:55:40 -06:00
Peter Bacsko	567600fd80	YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak.	2020-11-11 17:10:15 +01:00
Peter Bacsko	cd0490e8c6	Revert "YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak." This reverts commit `b0ab222a6c`.	2020-11-11 17:06:12 +01:00
Peter Bacsko	b0ab222a6c	YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak.	2020-11-11 17:03:16 +01:00
Gabriel Medeiros Coelho	61f8c5767e	YARN-10480. replace href tags with ng-href (#2426 )	2020-11-10 10:54:28 -08:00
Akira Ajisaka	4331c88352	HADOOP-17344. Harmonize guava version and shade guava in yarn-csi. (#2434 ) Reviewed-by: Ayush Saxena <ayushsaxena@apache.org> Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-11-10 15:19:40 +09:00
Eric E Payne	55339c2bdd	YARN-10479. RMProxy should retry on SocketTimeout Exceptions. Contributed by Jim Brennan (Jim_Brennan)	2020-11-05 21:50:46 +00:00
Eric E Payne	0461a07c01	YARN-10475: Scale RM-NM heartbeat interval based on node utilization. Contributed by Jim Brennan (Jim_Brennan).	2020-11-02 16:55:06 +00:00
neko	deea5d8f2b	YARN-10469. YARN-UI2 The accuracy of the percentage values in the same chart on the YARN 'Cluster OverView' page are inconsistent (#2401 )	2020-11-02 17:52:58 +08:00
Szilard Nemeth	4890b855ec	YARN-10420. Update CS MappingRule documentation with the new format and features. Contributed by Peter Bacsko	2020-10-31 15:17:01 +01:00
Surendra Singh Lilhore	7169ec4509	YARN-10442. RM should make sure node label file highly available. (#2390 ) * YARN-10442. RM should make sure node label file highly available. * YARN-10442. Corrected method name. Added license header for file TestNodeLabelFileReplication.java Added Test code. * YARN-10442. Changed property name and log. * YARN-10442. Changed default replication for FS store file.	2020-10-30 23:05:16 +05:30
Wangda Tan	c47c9fd65d	YARN-10458. Hive On Tez queries fails upon submission to dynamically created pools. (Peter Bacsko via wangda) Change-Id: I518dc925187ce55e9d35a37ba20878c0f4e37e5c	2020-10-30 09:49:48 -07:00
Jim Brennan	8ee6bc2518	YARN-10471. Prevent logs for any container from becoming larger than a configurable size. Contributed by Eric Payne	2020-10-29 19:49:18 +00:00
Jonathan Hung	bab5bf9743	YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen	2020-10-28 10:33:18 -07:00
Sunil G	7435604a91	YARN-10453. Add partition resource info to get-node-labels and label-mappings api responses. Contributed Akhil PB.	2020-10-22 12:52:58 +05:30
Szilard Nemeth	31477a224f	YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko	2020-10-19 13:17:51 +02:00
Ayush Saxena	1e3a6efcef	HADOOP-17288. Use shaded guava from thirdparty. (#2342 ). Contributed by Ayush Saxena.	2020-10-17 12:01:18 +05:30
Eric Badger	8b8c672780	YARN-10450. Add cpu and memory utilization per node and cluster-wide metrics. Contributed by Jim Brennan.	2020-10-15 17:57:43 +00:00
Szilard Nemeth	42358169f6	YARN-10454: Add applicationName policy. Contributed by Peter Bacsko	2020-10-13 10:11:10 +02:00
Jim Brennan	deb35a32ba	YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by Ahmed Hussein	2020-10-08 18:46:47 +00:00
Jim Brennan	b361f29dda	YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne	2020-10-06 18:18:08 +00:00
Adam Antal	a1f7e760df	YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan	2020-10-05 10:09:14 +02:00
Fei Hui	d0d10f7e8f	HADOOP-17276. Extend CallerContext to make it include many items (#2327 )	2020-10-04 02:02:28 +09:00
Adam Antal	bb8446e80c	YARN-10447. TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing. Contributed by Peter Bacsko	2020-10-01 11:42:04 +02:00
Szilard Nemeth	a7a1f1541a	YARN-10413. Change fs2cs to generate mapping rules in the new format. Contributed by Peter Bacsko	2020-09-29 20:18:26 +02:00
Aryan Gupta	bbbfa7d415	HADOOP-17286. Updated jQuery in yarn-common. (#2340 )	2020-09-27 03:18:01 +09:00
Shubham Gupta	e3cd627069	YARN-10438. Handle null containerId in ClientRMService#getContainerReport() (#2313 ) Co-authored-by: Shubham Gupta <gshubham@microsoft.com>	2020-09-25 12:20:39 +05:30
Aryan Gupta	486ddb73f9	HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2020-09-24 21:15:55 +09:00
Adam Antal	7ec16ac7f6	YARN-10443. Document options of logs CLI. Contributed by Ankit Kumar. Fixes #2325	2020-09-23 17:42:56 +02:00
tkuramoto33	18f9d77c46	YARN-6754. Fair scheduler docs should explain meaning of weight=0 for a queue. (#2300 )	2020-09-23 14:50:28 +09:00
Hui Fei	474fa80bfb	HADOOP-17277. Correct spelling errors for separator (#2322 ) Contributed by Hui Fei.	2020-09-22 17:10:33 +01:00
Szilard Nemeth	eacbe07b56	YARN-9333. TestFairSchedulerPreemption.testRelaxLocalityPreemptionWithNoLessAMInRemainingNodes fails intermittently. Contributed by Peter Bacsko	2020-09-17 19:57:19 +02:00
Jim Brennan	90894ea641	YARN-10430. Log improvements in NodeStatusUpdaterImpl. Contributed by Bilwa S T.	2020-09-14 21:19:14 +00:00
Eric E Payne	9afec2ed17	YARN-10390: LeafQueue: retain user limits cache across assignContainers() calls. Contributed by Samir Khan (samkhan).	2020-09-11 13:29:26 +00:00
Szilard Nemeth	773ac799c6	YARN-10387. Implement logic which returns MappingRule objects based on mapping rules. Contributed by Peter Bacsko	2020-09-09 15:12:58 +02:00
Brahma Reddy Battula	43572fc7f8	YARN-10397. SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints. Contributed by Bilwa S T.	2020-09-09 17:08:13 +05:30
Peter Bacsko	aba4a506d6	YARN-10424. Adapt existing AppName and UserGroupMapping unittests to ensure backwards compatibility. Contributed by Benjamin Teke.	2020-09-09 09:41:11 +02:00
Peter Bacsko	c4fb4044b2	YARN-10415. Create a group matcher which checks ALL groups of the user. Contributed by Gergely Pollak.	2020-09-08 10:57:00 +02:00
Adam Antal	34fe74da0e	YARN-10332. RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. Contributed by yehuanhuan	2020-09-07 11:44:09 +02:00
Peter Bacsko	360bbcd3bc	YARN-10411. Create an allowCreate flag for MappingRuleAction. Contributed by Gergely Pollak.	2020-09-07 11:39:03 +02:00
Adam Antal	dd6c66556e	YARN-9136. getNMResourceInfo NodeManager REST API method is not documented	2020-09-07 10:29:20 +02:00
Peter Bacsko	1841a5bb03	YARN-10376. Create a class that covers the functionality of UserGroupMappingPlacementRule and AppNameMappingPlacementRule using the new mapping rules. Contributed by Gergely Pollak.	2020-09-05 10:00:33 +02:00
Szilard Nemeth	696e4fe50e	Typos in YarnScheduler#allocate method's doc comment. Contributed by Siddharth Ahuja	2020-09-04 09:59:54 +02:00
Peter Bacsko	5c15815773	YARN-10375. CS Mapping rule config parser should return MappingRule objects. Contributed by Gergely Pollak.	2020-09-03 18:54:22 +02:00
Peter Bacsko	288dab7563	YARN-10372. Create MappingRule class to represent each CS mapping rule. Contributed by Gergely Pollak	2020-09-03 15:02:48 +02:00
Masatake Iwasaki	5e12dc51f0	YARN-10419. Javadoc error in hadoop-yarn-server-common module. (#2271 )	2020-09-03 16:17:52 +09:00
Peter Bacsko	c7dab2b23e	YARN-10374. Create Actions for CS mapping rules. Contributed by Gergely Pollak.	2020-09-01 17:38:49 +02:00
Adam Antal	73a0d149e2	YARN-10386. Addendum patch: Fixing ASF Warnings. Contributed by Peter Bacsko.	2020-09-01 14:39:24 +02:00
Szilard Nemeth	4ffe26f9b8	[UI1] Provide a way to hide Tools section in Web UIv1. Contributed by Andras Gyori	2020-08-31 16:00:39 +02:00
Szilard Nemeth	56a5c360a1	YARN-10373. Create Matchers for CS mapping rules. Contributed by Gergely Pollak	2020-08-29 21:34:55 +02:00
Szilard Nemeth	f4f872b778	YARN-10371. Create variable context class for CS queue mapping rules. Contributed by Gergely Pollak	2020-08-29 17:31:48 +02:00
Szilard Nemeth	a888d580d8	YARN-10409. Improve MockQueueHierarchyBuilder to detect queue ambiguity. Contributed by Gergely Pollak	2020-08-29 15:24:50 +02:00
Szilard Nemeth	44542863f4	YARN-10408. Extract MockQueueHierarchyBuilder to a separate class. Contributed by Gergely Pollak	2020-08-28 17:35:45 +02:00
Szilard Nemeth	6e3d7050a9	YARN-10386. Create new JSON schema for Placement Rules. Contributed by Peter Bacsko and Benjamin Teke	2020-08-28 17:09:31 +02:00
Prabhu Joseph	75db5526b5	YARN-1806. Add ThreadDump Option in YARN UI2 to fetch for running containers Contributed by Siddharth Ahuja. Reviewed by Akhil PB.	2020-08-26 13:08:17 +05:30
Adam Antal	82a7505646	YARN-10304. Create an endpoint for remote application log directory path query. Contributed by Andras Gyori	2020-08-25 13:29:12 +02:00
Adam Antal	c049296235	YARN-10106. Yarn logs CLI filtering by application attempt. Contributed by Hudáky Márton Gyula	2020-08-25 09:53:39 +02:00
Prabhu Joseph	5c927eb550	YARN-10360. Support Multi Node Placement in SingleConstraintAppPlacementAllocator Reviewed by Sunil G.	2020-08-24 15:00:02 +05:30
Zhengbo Li	9b9f7ea16a	YARN-10399 Refactor NodeQueueLoadMonitor class to make it extendable (#2228 ) Refactor NodeQueueLoadMonitor class to make it extendable	2020-08-19 10:14:50 -07:00
Sunil G	82ec28f442	YARN-10396. Max applications calculation per queue disregards queue level settings in absolute mode. Contributed by Benjamin Teke.	2020-08-19 11:54:48 +05:30
Jim Brennan	4a7deae478	YARN-10391. --module-gpu functionality is broken in container-executor. Contributed by Eric Badger	2020-08-17 16:41:15 +00:00
hemanthboyina	86bbd38c8d	YARN-10336. RM page should throw exception when command injected in RM REST API to get applications. Contributed by Bilwa S T.	2020-08-13 23:44:42 +05:30
Prabhu Joseph	6c2ce3d56b	YARN-10389. Option to override RMWebServices with custom WebService class Contributed by Tanu Ajmera. Reviewed by Bilwa ST and Sunil G.	2020-08-11 15:46:47 +05:30
Prabhu Joseph	5e0f879779	YARN-10364. Fix logic of queue capacity is absolute resource or percentage. Contributed by Bilwa ST. Reviewed by Sunil G.	2020-08-08 22:33:58 +05:30
Jonathan Hung	64753addba	YARN-10251. Show extended resources on legacy RM UI. Contributed by Eric Payne	2020-08-07 17:43:01 -07:00
bibinchundatt	81da221c75	YARN-10388. RMNode updatedCapability flag not set while RecommissionNodeTransition. Contributed by Pranjal Protim Borah	2020-08-07 08:36:52 +05:30
Prabhu Joseph	c7e71a6c0b	YARN-10361. Make custom DAO classes configurable into RMWebApp#JAXBContextResolver. Contributed by Bilwa ST.	2020-08-06 13:19:52 +05:30
Eric Badger	5edd8b925e	YARN-4575. ApplicationResourceUsageReport should return ALL reserved resource. Contributed by Bibin Chundatt and Eric Payne.	2020-08-05 18:47:49 +00:00
Prabhu Joseph	0277856738	YARN-10377. Fix filter index to show apps while clicking on queue in RM UI Contributed by Tarun Parimi.	2020-08-04 23:10:45 +05:30
Prabhu Joseph	aa5afa72c5	YARN-10381. Add application attempt state in AppAttempts RM REST API Contributed by Siddharth Ahuja. Reviewed by Bilwa ST.	2020-08-04 23:07:28 +05:30
kevinzhao1661	ab2b3df2de	YARN-10383. YarnCommands.md is inconsistent with the source code (#2177 )	2020-08-04 12:02:16 +09:00
Brahma Reddy Battula	eac558380f	YARN-10229. [Federation] Client should be able to submit application to RM directly using normal client conf. Contributed by Bilwa S T.	2020-08-03 12:54:36 +05:30
bibinchundatt	5323e83edf	YARN-10359. Log container report only if list is not empty. Contributed by Bilwa S T.	2020-08-01 13:03:46 +05:30
Eric E Payne	e0c9653166	YARN-1529: Add Localization overhead metrics to NM. Contributed by Jim_Brennan.	2020-07-30 15:30:22 +00:00
bibinchundatt	5d8600e80a	YARN-10369. Make NMTokenSecretManagerInRM sending NMToken for nodeId DEBUG. Contributed by Jim Brennan.	2020-07-29 23:26:08 +05:30
Jonathan Hung	3eaf62726f	YARN-10343. Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. Contributed by Eric Payne	2020-07-28 13:43:19 -07:00
bibinchundatt	5dadf963d3	YARN-10208. Add capacityScheduler metric for NODE_UPDATE interval. Contributed by Pranjal Protim Borah.	2020-07-28 11:55:47 +05:30
Prabhu Joseph	60a254621a	YARN-10366. Fix Yarn rmadmin help message shows two labels for one node for --replaceLabelsOnNode. Contributed by Tanu Ajmera.	2020-07-27 12:23:21 +05:30
Akira Ajisaka	d02be17a26	YARN-10362. Javadoc for TimelineReaderAuthenticationFilterInitializer is broken. Contributed by Xieming Li.	2020-07-27 01:55:04 +09:00
Eric Badger	ac5f21dbef	YARN-4771. Some containers can be skipped during log aggregation after NM restart. Contributed by Jason Lowe and Jim Brennan.	2020-07-24 22:35:16 +00:00
Prabhu Joseph	247eb0979b	YARN-10319. Record Last N Scheduler Activities from ActivitiesManager Reviewed by Tao Yang and Adam Antal.	2020-07-24 11:43:31 +05:30
bibinchundatt	bfcd775381	YARN-10315. Avoid sending RMNodeResourceupdate event if resource is same. Contributed by Sushil Ks.	2020-07-23 16:41:35 +05:30
Masatake Iwasaki	1b29c9bfee	HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. (#2155 )	2020-07-22 13:40:20 +09:00
Eric Badger	736bed6d6d	[YARN-10353] Log vcores used and cumulative cpu in containers monitor. Contributed by Jim Brennan	2020-07-20 18:08:27 +00:00
Ayush Saxena	6bcb24d269	HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.	2020-07-18 14:33:43 +05:30
Prabhu Joseph	cc71d50b21	YARN-10339. Fix TimelineClient in NodeManager failing when Simple Http Auth used in Secure Cluster Contributed by Tarun Parimi.	2020-07-16 23:36:49 +05:30
Ahmed Hussein	1f71c4ae71	HADOOP-17099. Replace Guava Predicate with Java8+ Predicate Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-07-15 11:39:06 -05:00
Ahmed Hussein	98fcffe93f	HADOOP-17101. Replace Guava Function with Java8+ Function Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-07-15 09:53:18 -05:00
Bilwa S T	8980c23fc2	YARN-10350. TestUserGroupMappingPlacementRule fails Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2020-07-15 14:31:34 +09:00
Eric Badger	48f90115b5	YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by Jim Brennan	2020-07-13 23:10:39 +00:00
Eric E Payne	0427100b75	YARN-10297. TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently. Contributed by Jim Brennan (Jim_Brennan)	2020-07-13 18:57:50 +00:00
Xiaoyu Yao	f91a8ad88b	HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. (#2085 )	2020-07-09 11:33:37 -07:00
Sunil G	5dd270e208	YARN-10333. YarnClient obtain Delegation Token for Log Aggregation Path. Contributed by Prabhu Joseph.	2020-07-09 12:50:25 +05:30
Brahma Reddy Battula	dfe60392c9	YARN-10341. Yarn Service Container Completed event doesn't get processed. Contributed by Bilwa S T.	2020-07-09 12:34:52 +05:30
Akira Ajisaka	10d218934c	YARN-10344. Sync netty versions in hadoop-yarn-csi. (#2126 )	2020-07-09 10:29:47 +05:30
Prabhu Joseph	3a4d05b850	YARN-8047. RMWebApp make external class pluggable. Contributed by Bilwa S T.	2020-07-08 12:43:20 +05:30
Prabhu Joseph	2bbd00dff4	YARN-10337. Fix failing testcase TestRMHATimelineCollectors. Contributed by Bilwa S T.	2020-07-07 17:10:59 +05:30
Akira Ajisaka	639acb6d89	HADOOP-17111. Replace Guava Optional with Java8+ Optional. Contributed by Ahmed Hussein.	2020-07-06 16:08:36 +09:00
Szilard Nemeth	04abd0eb17	YARN-10330. Add missing test scenarios to TestUserGroupMappingPlacementRule and TestAppNameMappingPlacementRule. Contributed by Peter Bacsko	2020-07-01 14:10:55 +02:00
Szilard Nemeth	9b5557a9e8	YARN-10325. Document max-parallel-apps for Capacity Scheduler. Contributed by Peter Bacsko	2020-07-01 13:41:30 +02:00
Eric Yang	e8dc862d38	YARN-9809. Added node manager health status to resource manager registration call. Contributed by Eric Badger via eyang	2020-06-30 11:39:16 -07:00
Akira Ajisaka	cd188ea9f0	YARN-10331. Upgrade node.js to 10.21.0. (#2106 )	2020-06-30 16:52:57 +09:00
Eric E Payne	74fc13cf91	YARN-9903: Support reservations continue looking for Node Labels. Contributed by Jim Brennan (Jim_Brennan).	2020-06-29 18:39:53 +00:00
Eric Yang	0be26811f3	YARN-10328. Fixed ZK Curator NodeExists exception in YARN service AM logs Contributed by Bilwa S T via eyang	2020-06-29 09:21:24 -07:00
Brahma Reddy Battula	2c03524fa4	YARN-6526. Refactoring SQLFederationStateStore by avoiding to recreate a connection at every call. COntributed by Bilwa S T.	2020-06-26 20:43:27 +05:30
Szilard Nemeth	e0c1d8a969	YARN-10327. Remove duplication of checking for invalid application ID in TestLogsCLI. Contributed by Marton Hudaky	2020-06-26 13:21:55 +02:00
Szilard Nemeth	6459cc73e6	YARN-10318. ApplicationHistory Web UI incorrect column indexing. Contributed by Andras Gyori	2020-06-26 13:15:14 +02:00
Szilard Nemeth	33d3df7ee2	YARN-10277. CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy. Contributed by Szilard Nemeth	2020-06-26 11:14:24 +02:00
Szilard Nemeth	6a8fd73b27	YARN-10279. Avoid unnecessary QueueMappingEntity creations. Contributed by Marton Hudaky	2020-06-25 17:27:58 +02:00
Szilard Nemeth	03f855e3e7	YARN-10316. FS-CS converter: convert maxAppsDefault, maxRunningApps settings. Contributed by Peter Bacsko	2020-06-23 12:13:04 +02:00
Prabhu Joseph	ce1008fe61	YARN-10321. Break down TestUserGroupMappingPlacementRule#testMapping into test scenarios Contributed by Szilard Nemeth. Reviewed by Gergely Pollak.	2020-06-22 12:49:14 +05:30
Surendra Singh Lilhore	b2facc84a1	YARN-9460. QueueACLsManager and ReservationsACLManager should not use instanceof checks. Contributed by Bilwa S T.	2020-06-20 20:09:31 +05:30
Szilard Nemeth	469841446f	YARN-9930. Support max running app logic for CapacityScheduler. Contributed by Peter Bacsko	2020-06-19 14:50:24 +02:00
Eric Yang	89689c52c3	YARN-10308. Update javadoc and variable names for YARN service. Contributed by Bilwa S T via eyang	2020-06-17 09:04:26 -07:00
Szilard Nemeth	5b1a56f9f1	YARN-10281. Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule. Contributed by Gergely Pollak	2020-06-17 14:34:40 +02:00
Vinayakumar B	e154084770	HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026 )	2020-06-12 23:16:33 +05:30
Tao Yang	7c4de59fc1	YARN-10293. Reserved Containers not allocated from available space of other nodes in CandidateNodeSet for MultiNodePlacement. Contributed by Prabhu Joseph.	2020-06-12 22:10:25 +08:00
Eric Badger	fed6fecd3a	YARN-10312. Add support for yarn logs -logFile to retain backward compatibility. Contributed by Jim Brennan.	2020-06-11 21:02:41 +00:00
Eric E Payne	56247db302	YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger).	2020-06-09 18:43:16 +00:00
Szilard Nemeth	8abff5151a	YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke	2020-06-05 15:00:39 +02:00
Szilard Nemeth	8b146c17b3	YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak	2020-06-05 11:38:32 +02:00
Szilard Nemeth	a4835db95a	YARN-10296. Make ContainerPBImpl#getId/setId synchronized. Contributed by Benjamin Teke	2020-06-05 10:11:40 +02:00
Szilard Nemeth	b5efdea4fd	YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak	2020-06-02 18:29:15 +02:00
Szilard Nemeth	aa6d13455b	YARN-10284. Add lazy initialization of LogAggregationFileControllerFactory in LogServlet. Contributed by Adam Antal	2020-06-02 15:53:48 +02:00
Szilard Nemeth	e0a0741ac8	YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori	2020-06-02 15:42:39 +02:00
Szilard Nemeth	593af878c0	YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak	2020-05-27 13:05:52 +02:00
Szilard Nemeth	fb030f3239	Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak" This reverts commit `ac4540dd8e`. Reverting because of missing Jira-ID in the commit message.	2020-05-27 13:04:48 +02:00
Jonathan Hung	c30c23cb66	YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R	2020-05-26 16:29:42 -07:00
Szilard Nemeth	ac4540dd8e	FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak	2020-05-21 13:11:44 +02:00
Eric Yang	726b8e324b	YARN-10228. Relax restriction of file path character in yarn.service.am.java.opts. Contributed by Bilwa S T via eyang	2020-05-20 09:23:56 -07:00
Prabhu Joseph	cef0756929	YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. Contributed by Bilwa S T. Reviewed by Akira Ajisaka.	2020-05-20 11:36:52 +05:30
Akira Ajisaka	b65815d691	Revert "YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient." This reverts commit `7836bc4c35`.	2020-05-18 16:29:07 +09:00
Prabhu Joseph	7836bc4c35	YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. Contributed by Bilwa S T. Reviewed by Brahma Reddy Battula.	2020-05-14 19:40:42 +05:30
Prabhu Joseph	6ce295b787	YARN-10259. Fix reservation logic in Multi Node Placement. Reviewed by Wangda Tan.	2020-05-14 16:52:11 +05:30
Surendra Singh Lilhore	1958cb7c2b	YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng	2020-05-14 15:09:34 +05:30
Ayush Saxena	0918433b4d	YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.	2020-05-14 00:36:20 +05:30
Inigo Goiri	108ecf992f	YARN-8942. PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value. Contributed by Bilwa S T.	2020-05-13 10:04:12 -07:00
Prabhu Joseph	450e5aa9dd	YARN-10154. Addendum Patch which fixes below bugs 1. RM fails to start when LeafQueueTemplate max capacity is not specified. 2. Job stuck in ACCEPTED state with DominantResourceCalculator as Queue Capacity is set to NaN during RM startup with clusterResource is zero. Reviewed by Sunil G and Manikandan R.	2020-05-13 14:35:37 +05:30
Jonathan Hung	fff1d2c122	YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T	2020-05-12 10:48:54 -07:00
Elixir Kook	a3f945fb84	HADOOP-17035. fixed typos (timeout, interruped) (#2007 ) Co-authored-by: Sungpeo Kook <elixir.kook@kakaocorp.com>	2020-05-12 10:50:04 -05:00
Inigo Goiri	35010120fb	YARN-9017. PlacementRule order is not maintained in CS. Contributed by Bilwa S T.	2020-05-06 13:22:54 -07:00
Ahmed Hussein	92e3ebb401	YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-05-06 12:27:17 -05:00
Szilard Nemeth	cb6399c109	YARN-10257. FS-CS converter: skip increment properties for mem/vcores and fix DRF check. Contributed by Peter Bacsko	2020-05-05 16:00:43 +02:00
Szilard Nemeth	0debe55d6c	YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph	2020-05-05 13:11:28 +02:00
Ahmed Hussein	f5081a9a5d	YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles) Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-05-04 10:13:51 -05:00
Surendra Singh Lilhore	257b04ce4e	YARN-6553. Replace MockResourceManagerFacade with MockRM for AMRMProxy/Router tests. Contributed by Bilwa S T.	2020-05-02 12:43:20 +05:30
Inigo Goiri	d125d39108	YARN-6973. Adding RM Cluster Id in ApplicationReport. Contributed by Bilwa S T.	2020-04-29 11:23:58 -07:00
Szilard Nemeth	410c605aec	YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G	2020-04-29 15:53:30 +02:00
Szilard Nemeth	ab36429559	YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori	2020-04-28 18:53:28 +02:00
Akira Ajisaka	6ef01646ba	YARN-9848. Revert YARN-4946. Contributed by Steven Rand.	2020-04-28 05:35:36 +09:00
Prabhu Joseph	9224568b0e	YARN-10156. Destroy Jersey Client in TimelineConnector. Contributed by Tanu Ajmera.	2020-04-27 18:49:15 +05:30
Szilard Nemeth	f91e21ac10	YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph	2020-04-27 08:26:11 +02:00
Szilard Nemeth	d4874585f4	YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph	2020-04-24 17:04:47 +02:00
Akira Ajisaka	9827ff2961	YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. (#1939 )	2020-04-24 09:32:25 +09:00
Prabhu Joseph	60fa15366e	YARN-10240. Prevent Fatal CancelledException in TimelineV2Client when stopping Contributed by Tarun Parimi.	2020-04-21 12:01:33 +05:30
Szilard Nemeth	92b28e38b7	YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja	2020-04-18 10:13:37 +02:00
Szilard Nemeth	76900b4f5b	YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke	2020-04-18 09:57:27 +02:00
Jonathan Hung	3601054822	YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T	2020-04-17 10:34:44 -07:00
Sunil G	2fe122e322	YARN-10154. CS Dynamic Queues cannot be configured with absolute resources. Contributed by Manikandan R.	2020-04-17 18:51:19 +05:30
Szilard Nemeth	4db598e0e6	YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke	2020-04-15 13:18:44 +02:00
Prabhu Joseph	94f7470c71	YARN-10233. Fix YARN UI2 Daemon Logs Contributed by Akhil PB.	2020-04-15 13:39:37 +05:30
Szilard Nemeth	55fcbcb5c2	YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko	2020-04-15 07:24:04 +02:00
Prabhu Joseph	c791b0e90e	YARN-10219. Fix YARN Native Service Placement Constraints with Node Attributes. Contributed by Eric Yang.	2020-04-14 12:34:53 +05:30
Jonathan Hung	23481ad378	YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T	2020-04-09 11:06:11 -07:00
Szilard Nemeth	c04a484aed	YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T.	2020-04-09 15:46:25 +02:00
Sunil G	6fb29d0f17	YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko.	2020-04-09 18:51:42 +05:30
Szilard Nemeth	8e4517f251	YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja	2020-04-09 12:31:01 +02:00
Akira Ajisaka	e53d472bb0	HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 (#1933 )	2020-04-09 01:47:31 +09:00
Wilfred Spiegelenburg	2214005c0f	YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja	2020-04-08 11:30:03 +10:00
Szilard Nemeth	bffb43b00e	YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja	2020-04-07 17:03:17 +02:00
Szilard Nemeth	45362a9f4c	YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja	2020-04-07 15:51:55 +02:00
Prabhu Joseph	7a39da3537	YARN-10120. Amendment fix for Java Doc. Contributed by Bilwa S T.	2020-04-03 13:07:41 +05:30
Prabhu Joseph	25361b077b	YARN-10120. Https Support in Router WebServiceClient. Contributed by Bilwa S T.	2020-04-02 18:11:43 +05:30
Akira Ajisaka	c162648aff	YARN-10202. Fix documentation about NodeAttributes. Contributed by Sen Zhao.	2020-04-01 16:05:47 +09:00
Brahma Reddy Battula	8914cf9167	Preparing for 3.4.0 development	2020-03-29 23:24:25 +05:30
RogPodge	50f7f6dfd1	YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover.	2020-03-26 08:14:30 -07:00
Szilard Nemeth	3d5ade1839	YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R	2020-03-26 13:23:43 +01:00
Szilard Nemeth	9a297ff31d	YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko	2020-03-26 12:35:10 +01:00
Szilard Nemeth	348685dcb9	YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke	2020-03-26 12:18:15 +01:00
Jonathan Hung	6ce189c621	YARN-10200. Add number of containers to RMAppManager summary	2020-03-25 10:41:59 -07:00
RogPodge	2d294bd575	HADOOP-16938. Make non-HA proxy providers pluggable	2020-03-25 08:06:58 -07:00
Sunil G	cdb2107066	YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak.	2020-03-25 16:50:19 +05:30
Prabhu Joseph	d312991ead	YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue Contributed by Peter Bacsko. Reviewed by Manikandan R.	2020-03-23 18:01:14 +05:30
Prabhu Joseph	f2d3ac2a3f	YARN-10034. Remove Allocation Tags from released container from Decommission node Contributed by Kyungwan Nam. Reviewed by Adam Antal.	2020-03-19 12:55:14 +05:30
Eric Badger	1975479285	YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed Hussein.	2020-03-16 22:24:02 +00:00
Weiwei Yang	b9d825f178	YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang.	2020-03-13 14:39:36 -07:00
Weiwei Yang	d273f6a2b4	YARN-9538. Document scheduler/app activities and REST APIs. Contributed by Tao Yang.	2020-03-13 14:36:52 -07:00
Bilwa	c83644deac	YARN-10110. Adding RouterPolicyProvider for RM Federation. Contributed by Bilwa S T.	2020-03-13 16:56:44 +05:30
Szilard Nemeth	5ead9c15ca	YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori	2020-03-12 12:29:03 +01:00
Szilard Nemeth	38d87883b6	YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko	2020-03-12 12:18:37 +01:00
Sunil G	0fd8bf5f6b	YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko.	2020-03-12 15:03:12 +05:30
Szilard Nemeth	cf9cf83a43	YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori	2020-03-10 16:44:48 +01:00
Szilard Nemeth	61f4cf3055	YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke	2020-03-10 16:35:04 +01:00
Szilard Nemeth	9314ef947f	YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko	2020-03-10 16:07:46 +01:00
Eric E Payne	ede05b19d1	YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein)	2020-03-10 14:17:45 +00:00
Szilard Nemeth	44afe1154d	YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori	2020-03-09 16:08:24 +01:00
bibinchundatt	3859fa76d0	YARN-6924. Metrics for Federation AMRMProxy. Contributed by Young Chen	2020-03-07 09:34:42 +05:30
Szilard Nemeth	ea0444851d	YARN.10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke	2020-03-06 17:49:45 +01:00
Szilard Nemeth	004e955348	YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. Contributed by Peter Bacsko	2020-03-05 09:56:42 +01:00
Eric Badger	2649f8b327	YARN-10173. Make pid file generation timeout configurable in case of reacquired container. Contributed by Adam Antal.	2020-03-04 23:31:57 +00:00
Szilard Nemeth	c0d084247c	YARN-10175. FS-CS converter: only convert placement rules if a cmd line switch is defined. Contributed by Peter Bacsko	2020-03-03 09:46:32 +01:00
Szilard Nemeth	e9eecedf69	YARN-10148. addendum: Fix method call parameter order of setAdminAndSubmitACL in TestCapacitySchedulerQueueACLs. Contributed by Kinga Marton	2020-03-02 16:25:14 +01:00
Inigo Goiri	a43510e21d	YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan.	2020-02-27 13:18:30 -08:00
Inigo Goiri	b420ddeada	YARN-10155. TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk. Contributed by Manikandan R.	2020-02-27 12:53:20 -08:00
Szilard Nemeth	10461e0193	YARN-10148. Add Unit test for queue ACL for both FS and CS. Contributed by Kinga Marton	2020-02-27 20:10:32 +01:00
Akira Ajisaka	7dfa37e8f0	YARN-10152. Fix findbugs warnings in hadoop-yarn-applications-mawo-core module (#1852 )	2020-02-27 10:36:49 +09:00
Brahma Reddy Battula	3a9ccf7f6d	YARN-10141.Interceptor in FederationInterceptorREST doesnt update on RM switchover. Contributed by D M Murali Krishna Reddy.	2020-02-26 23:24:00 +05:30
tanu.ajmera	b5df1da3a1	YARN-9593. Support Comma in the value of Scheduler Configuration Mutation API.	2020-02-26 15:04:29 +05:30
Elixir Kook	d608e94f92	YARN-10156. Fix typo 'complaint' which means quite different in Federation.md (#1856 )	2020-02-26 17:29:26 +09:00
Szilard Nemeth	d68616b045	YARN-10130. FS-CS converter: Do not allow output dir to be the same as input dir. Contributed by Adam Antal	2020-02-25 21:30:04 +01:00
Wei-Chiu Chuang	dda00d3ff5	YARN-10074. Update netty to 4.1.42Final in yarn-csi. Contributed by Wei-Chiu Chuang. (#1807 )	2020-02-25 13:47:52 +09:00
Szilard Nemeth	72fa9c7f1b	YARN-10135. FS-CS converter tool: issue warning on dynamic auto-create mapping rules. Contributed by Peter Bacsko	2020-02-24 21:54:07 +01:00
Szilard Nemeth	34cf63c7d0	YARN-10157. FS-CS converter: initPropertyActions() is not called without rules file. Contributed by Peter Bacsko	2020-02-24 21:45:49 +01:00
Prabhu Joseph	181e6d033e	YARN-10119. Option to reset AM failure count for YARN Service Contributed by kyungwan nam.	2020-02-20 20:34:06 +05:30
Szilard Nemeth	4af2556b48	YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal	2020-02-20 15:27:15 +01:00

... 9 10 11 12 13 ...

6861 Commits