Peter Bacsko
ceb75e1e2a
YARN-10674. fs2cs should generate auto-created queue deletion properties. Contributed by Qi Zhu.
2021-03-24 08:15:06 +01:00
Jim Brennan
174f3a96b1
YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T.
2021-03-23 18:21:45 +00:00
Cyrus Jackson
cd44e917d0
YARN-10476. Queue metrics for Unmanaged applications ( #2674 ). Contributed by Cyrus Jackson
2021-03-19 15:49:05 +05:30
Peter Bacsko
ce6bfd5718
YARN-10641. Refactor the max app related update, and fix maxApllications update error when add new queues. Contributed by Qi Zhu.
2021-03-18 13:40:16 +01:00
Szilard Nemeth
a5745711dd
YARN-10659. Improve CS MappingRule %secondary_group evaluation. Contributed by Gergely Pollak
2021-03-18 12:43:01 +01:00
Peter Bacsko
d7eeca4d0c
YARN-10685. Fix typos in AbstractCSQueue. Contributed by Qi Zhu.
2021-03-18 11:49:16 +01:00
Eric Badger
49f89f1d3d
YARN-10688. ClusterMetrics should support GPU capacity related metrics.. Contributed by Qi Zhu.
2021-03-17 18:11:37 +00:00
Peter Bacsko
3e58d5611d
YARN-10497. Fix an issue in CapacityScheduler which fails to delete queues. Contributed by Wangda Tan and Qi Zhu.
2021-03-17 13:38:20 +01:00
Wilfred Spiegelenburg
f276f1af80
YARN-10652. Capacity Scheduler fails to handle user weights for a user that has a "." (dot) in it
...
This only fixes the user name resolution for weights in the queues. It
does not add generic support for user names with dots in all use cases
in the capacity scheduler.
Contributed by: Siddharth Ahuja
2021-03-17 10:55:05 +11:00
Peter Bacsko
b80588b688
YARN-10682. The scheduler monitor policies conf should trim values separated by comma. Contributed by Qi Zhu.
2021-03-16 15:23:27 +01:00
zhuqi
e9c98548e9
YARN-10689. Fix the finding bugs in extractFloatValueFromWeightConfig. ( #2760 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-03-16 13:50:29 +09:00
Eric Payne
aa4c17b9d7
YARN-10588. Percentage of queue and cluster is zero in WebUI . Contributed by Bilwa S T
2021-03-15 19:09:40 +00:00
Szilard Nemeth
5db4c0bf70
YARN-10412. Move CS placement rule related changes to a separate package. Contributed by Gergely Pollak
2021-03-12 14:10:16 +01:00
Peter Bacsko
d5e035dbe1
YARN-9615. Add dispatcher metrics to RM. Contributed by Jonathan Hung and Qi Zhu.
2021-03-09 14:33:14 +01:00
Peter Bacsko
3851994cd6
Revert "YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu."
...
This reverts commit 369f75b7a7
.
2021-03-09 14:32:02 +01:00
Peter Bacsko
369f75b7a7
YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu.
2021-03-09 14:28:23 +01:00
Peter Bacsko
20416bc62d
YARN-10675. Consolidate YARN-10672 and YARN-10447. Contributed by Szilard Nemeth.
2021-03-09 11:34:37 +01:00
Peter Bacsko
04cd3115ba
YARN-10658. CapacityScheduler QueueInfo add queue path field to avoid ambiguous QueueName. Contributed by Qi Zhu.
2021-03-08 12:29:44 +01:00
Peter Bacsko
e82e7c597a
YARN-10640. Adjust the queue Configured capacity to Configured weight number for weight mode in UI. Contributed by Qi Zhu.
2021-03-05 15:56:56 +01:00
Peter Bacsko
e19c00925f
YARN-10639. Queueinfo related capacity, should adjusted to weight mode. Contributed by Qi Zhu.
2021-03-05 13:18:06 +01:00
Peter Bacsko
6699198b54
YARN-10532. Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used. Contributed by Qi Zhu.
2021-03-04 17:18:35 +01:00
Peter Bacsko
a85aeee876
YARN-10623. Capacity scheduler should support refresh queue automatically by a thread policy. Contributed by Qi Zhu.
2021-03-04 12:23:11 +01:00
Neil
d615e2d3bd
YARN-10649. Fix RMNodeImpl.updateExistContainers leak ( #2719 ). Contributed by Max Xie
2021-03-04 14:52:58 +05:30
Peter Bacsko
7e8040e6ad
YARN-10655. Limit queue creation depth relative to its first static parent. Contributed by Andras Gyori.
2021-03-03 17:44:30 +01:00
Peter Bacsko
947b50489d
YARN-10627. Extend logging to give more information about weight mode. Contributed by Benjamin Teke.
2021-02-26 21:57:54 +01:00
Jonathan Hung
ac37ca0b67
YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen
2021-02-25 16:01:18 -08:00
Jim Brennan
c373da9f88
[YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne
2021-02-25 16:46:19 +00:00
Peter Bacsko
d28b6f90c8
YARN-10636. CS Auto Queue creation should reject submissions with empty path parts. Contributed by Gergely Pollak.
2021-02-19 13:57:30 +01:00
Peter Bacsko
4383726d19
YARN-10635. CSMapping rule can return paths with empty parts. Contributed by Gergely Pollak.
2021-02-19 12:01:31 +01:00
Szilard Nemeth
9882a22365
YARN-10625. FairScheduler: add global flag to disable AM-preemption. Contributed by Peter Bacsko
2021-02-16 18:06:57 +01:00
Szilard Nemeth
bad6038a48
YARN-10622. Fix preemption policy to exclude childless ParentQueues. Contributed by Andras Gyori
2021-02-15 14:48:42 +01:00
Szilard Nemeth
852aac34f2
Support max queues limit configuration in new auto created queue, consistent with old auto created. Contributed by Qi Zhu
2021-02-15 14:20:50 +01:00
Masatake Iwasaki
f9a073c6c1
YARN-10500. TestDelegationTokenRenewer fails intermittently. ( #2619 )
2021-02-11 13:55:04 -06:00
Szilard Nemeth
a8bd516e39
YARN-10620. fs2cs: parentQueue for certain placement rules are not set during conversion. Contributed by Peter Bacsko
2021-02-10 11:15:53 +01:00
Szilard Nemeth
fc5f241535
YARN-10619. CS Mapping Rule %specified rule catches default submissions. Contributed by Gergely Pollak
2021-02-09 18:16:42 +01:00
Amogh Rajesh Desai
0b05dd8802
YARN-9650. Set thread names for CapacityScheduler AsyncScheduleThread ( #2665 )
...
Amogh Rajesh Desai via Zhankun Tang
2021-02-09 11:05:14 +08:00
Szilard Nemeth
5aa9866ec2
YARN-10615. Fix Auto Queue Creation hierarchy construction to use queue path instead of short queue name. Contributed by Andras Gyori
2021-02-05 17:43:01 +01:00
Szilard Nemeth
c19326c051
YARN-10610. Add queuePath to RESTful API for CapacityScheduler consistent with FairScheduler queuePath. Contributed by Qi Zhu
2021-02-05 17:34:45 +01:00
Szilard Nemeth
79a46599f7
YARN-10428. Zombie applications in the YARN queue using FAIR + sizebasedweight. Contributed by Guang Yang, Andras Gyori
2021-02-05 17:09:08 +01:00
Szilard Nemeth
c17414111b
YARN-10612. Fix findbugs issue introduced in YARN-10585. Contributed by Gergely Pollak
2021-02-04 08:12:13 +01:00
bibinchundatt
6fc26ad539
YARN-10352 Skip schedule on not heartbeated nodes in Multi Node Placement. Contributed by Prabhu Joseph and Qi Zhu
2021-02-03 08:50:45 +05:30
Szilard Nemeth
9bf2ac07bb
YARN-10605. Add queue-mappings-override.enable property in FS2CS conversions. Contributed by Andras Gyori
2021-02-02 09:13:46 +01:00
Szilard Nemeth
7d88953768
YARN-10604. Support auto queue creation without mapping rules. Contributed by Andras Gyori
2021-02-02 09:07:25 +01:00
Peter Bacsko
fa15594ae6
YARN-10600. Convert root queue in fs2cs weight mode conversion. Contributed by Benjamin Teke.
2021-01-28 20:58:51 +01:00
Szilard Nemeth
9628aa87bf
YARN-10598. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to extend the creation type with additional information. Contributed by Benjamin Teke
2021-01-27 18:14:06 +01:00
Szilard Nemeth
7c4ef42837
YARN-10599. fs2cs should generate new 'auto-queue-creation-v2.enabled' properties for all parents. Contributed by Peter Bacsko
2021-01-27 15:49:26 +01:00
Szilard Nemeth
e2a7008d50
YARN-10585. Create a class which can convert from legacy mapping rule format to the new JSON format. Contributed by Gergely Pollak
2021-01-26 18:31:39 +01:00
Wangda Tan
4f008153ef
YARN-10587. Fix AutoCreateLeafQueueCreation cap related caculation when in absolute mode. (Qi Zhu via wangda)
...
Change-Id: I8889c393a9f08b633c72a2e873c6f45ca12cc82b
2021-01-26 08:24:29 -08:00
Wangda Tan
b7384a8d02
YARN-10531. Be able to disable user limit factor for CapacityScheduler Leaf Queue. (Qi Zhu via wangda)
...
Change-Id: I670e5525619b320745254609c48e7e1afb084835
2021-01-26 08:24:28 -08:00
Szilard Nemeth
f1766e5bb4
YARN-10596. Allow static definition of childless ParentQueues with auto-queue-creation-v2 enabled. Contributed by Andras Gyori
2021-01-26 16:22:35 +01:00