big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-19051: Highlight Capacity Scheduler new features in release for the release 3.4.0 (#6500) Contributed by Benjamin Teke.

Browse Source 
Signed-off-by: Shilun Fan <slfan1989@apache.org>
This commit is contained in:
Benjamin Teke 2024-01-26 06:33:55 +01:00 committed by GitHub
parent da34ecdb83
commit 897f446d54
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 21 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
hadoop-project/src/site/markdown/index.md.vm
View File

@ -57,6 +57,27 @@ We have enhanced the YARN Federation functionality for improved usability. The e
6. The page function of the router has been enhanced.
7. A set of commands has been added to the Router side for operating on SubClusters and Policies.
YARN Capacity Scheduler improvements
----------------------------------------
[YARN-10496](https://issues.apache.org/jira/browse/YARN-10496) Support Flexible Auto Queue Creation in Capacity Scheduler
Capacity Scheduler resource distribution mode was extended with a new allocation mode called weight mode.
Defining queue capacities with weights allows the users to use the newly added flexible queue auto creation mode.
Flexible mode now supports the dynamic creation of both **parent queues** and **leaf queues**, enabling the creation of
complex queue hierarchies application submission time.
[YARN-10888](https://issues.apache.org/jira/browse/YARN-10888) New capacity modes for Capacity Scheduler
Capacity Scheduler's resource distribution was completely refactored to be more flexible and extensible. There is a new concept
called Capacity Vectors, which allows the users to mix various resource types in the hierarchy, and also in a single queue. With
this optionally enabled feature it is now possible to define different resources with different units, like memory with GBs, vcores with
percentage values, and GPUs/FPGAs with weights, all in the same queue.
[YARN-10889](https://issues.apache.org/jira/browse/YARN-10889) Queue Creation in Capacity Scheduler - Various improvements
In addition to the two new features above, there were a number of commits for improvements and bug fixes in Capacity Scheduler.
HDFS RBF: Code Enhancements, New Features, and Bug Fixes
----------------------------------------