Skip to content

tas: preserve grouped leader placement capacity - #13383

Merged
kubernetes-prow[bot] merged 4 commits into
kubernetes-sigs:mainfrom
YQ-Wang:feat/7446-podset-group-leader-placement
Aug 3, 2026
Merged

kubernetes-prow[bot] merged 4 commits into
kubernetes-sigs:mainfrom
YQ-Wang:feat/7446-podset-group-leader-placement

Conversation

@YQ-Wang

@YQ-Wang YQ-Wang commented Jul 23, 2026 •

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind bug

What this PR does / why we need it:

TAS summarizes grouped PodSet capacity using the smallest leader-placement penalty among the available domains. During downward assignment, however, the leader domain was selected from the existing worker-capacity order without checking that its penalty fit within the available worker slack. This could consume more capacity than the summary accounted for, leave worker pods unassigned, and return an empty topology assignment even though a feasible placement existed.

This change preserves that summarized invariant during domain selection and hierarchy descent. It prioritizes a leader-capable domain only when its capacity penalty fits within the available slack, applies the same constraint to BestFit and LeastFreeCapacity selection, and uses one consistent partial-consumption path for sliced and non-sliced assignments.

The regression scenario has three hosts with worker capacities 2, 4, and 3, one 1-CPU leader, and four 2-CPU workers. The leader is placed on the 3-capacity host, allowing the workers to be assigned 1/2/1 across the hosts.

AI assistance was used to help implement and review this change. The author reviewed and tested the resulting code and takes responsibility for it.

Which issue(s) this PR fixes:

Fixes #7446

Special notes for your reviewer:

Validation completed:

  • gofmt check on all changed Go files
  • git diff --check HEAD
  • go test -race ./pkg/cache/scheduler -run 'TestFindTopologyAssignments|TestFindLevelWithFitDomainsPreservesFeasibleGroupedPlacements|TestUpdateCountsToMinimumGenericPreservesFeasibleGroupedPlacements' -count=1
  • go test ./pkg/cache/... ./pkg/scheduler/...
  • go vet ./pkg/cache/... ./pkg/scheduler/...
  • Local kind A/B validation on Kubernetes v1.36.1 with isolated 2/4/3-capacity worker nodes: the clean base repeatedly failed with remainingCount: 1, while the patched controller admitted the workload in one scheduling cycle and reserved all nine CPU slots. Restoring the patched image after the base reproduction admitted the workload again.

make verify could not be run locally because this macOS environment does not have GNU sed; the directly relevant formatting, unit, race, and vet checks above all passed.

Does this PR introduce a user-facing change?

TAS: Fixed a bug that could prevent admission of otherwise feasible LeaderWorkerSet workloads when the selected leader domain reduced the capacity available to workers. For example, a 1-CPU leader and four 2-CPU workers can now be placed across 2-, 4-, and 3-CPU hosts in one rack by assigning the leader to the 3-CPU host, leaving capacity for all four workers.

Summary by CodeRabbit

  • Bug Fixes
    • Improved grouped workload placement so leader and worker workloads respect available capacity across topology domains.
    • Corrected placement decisions for preferred, unconstrained, and slice-based topology configurations.
    • Improved handling of leader resource usage when selecting domains and calculating remaining capacity.
    • Ensured workloads are distributed across appropriate domains while preserving placement constraints.

@kubernetes-prow kubernetes-prow Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. labels Jul 23, 2026
@netlify

netlify Bot commented Jul 23, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name Link
🔨 Latest commit 0d5f099
🔍 Latest deploy log https://app.netlify.com/projects/kubernetes-sigs-kueue/deploys/6a6324f5289f7400086d7e61

@coderabbitai

coderabbitai Bot commented Jul 23, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

TAS grouped PodSet placement now incorporates leader capacity during domain selection, topology descent, and capacity consumption. Five table-driven scenarios validate leader and worker assignments across hostname, slice, preferred, mixed-profile, and below-slice constraints.

Changes

TAS grouped placement

Layer / File(s) Summary
Leader-aware domain selection
pkg/cache/scheduler/tas_flavor_snapshot.go
Best-fit and least-free-capacity selection filter domains by leader capacity and use leader-aware slice capacity calculations.
Leader-prioritized capacity consumption
pkg/cache/scheduler/tas_flavor_snapshot.go
Domain descent prioritizes suitable leader domains, while leader and worker capacity counters are consumed consistently.
Grouped placement validation
pkg/cache/scheduler/tas_cache_test.go
Five grouped-podset scenarios validate capacity preservation and expected assignments across topology and TAS profile variations.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant findTopologyAssignment
  participant findBestFitDomainBy
  participant prioritizeLeaderDomain
  participant consumeWithLeadersGeneric
  findTopologyAssignment->>findBestFitDomainBy: select domain with sufficient leader capacity
  findTopologyAssignment->>prioritizeLeaderDomain: order fitting domains for leader placement
  prioritizeLeaderDomain->>consumeWithLeadersGeneric: consume leader-adjusted capacity
Loading

Possibly related PRs

Suggested labels: area/tas

Suggested reviewers: amy, mbobrovskyi, tenzen-y, kshalot

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The code and tests address #7446 by making leader-aware domain selection preserve worker capacity during grouped PodSet descent.
Out of Scope Changes check ✅ Passed Changes stay focused on TAS placement logic and regression tests for grouped leader/worker assignment.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the PR's main change: preserving capacity during grouped leader placement.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kubernetes-prow kubernetes-prow Bot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 23, 2026
@kubernetes-prow

Copy link
Copy Markdown

Hi @YQ-Wang. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@kubernetes-prow
kubernetes-prow Bot requested review from kshalot and tenzen-y July 23, 2026 04:43
@kubernetes-prow kubernetes-prow Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jul 23, 2026
@YQ-Wang
YQ-Wang marked this pull request as ready for review July 23, 2026 05:09
@kubernetes-prow kubernetes-prow Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 23, 2026
@kubernetes-prow
kubernetes-prow Bot requested review from amy and mbobrovskyi July 23, 2026 05:09
@tenzen-y

Copy link
Copy Markdown
Member

/ok-to-test

@kubernetes-prow kubernetes-prow Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 23, 2026
@YQ-Wang

YQ-Wang commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/cache/scheduler/tas_flavor_snapshot.go`:
- Around line 1362-1386: The fallback in findBestFitDomainBy must not select a
domain with domain.leaderState below leaderCount when no candidate meets the
state requirement. Preserve the already-selected leader-feasible domain from the
caller or otherwise pass it into this selection path, and return it instead of
candidates[0] whenever the affinity-tier candidates exclude all leader-feasible
domains.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 79aeb670-33e7-45a5-b0fb-5748d8916a0c

📥 Commits

Reviewing files that changed from the base of the PR and between a7afef6 and 1e6f436.

📒 Files selected for processing (3)
  • pkg/cache/scheduler/tas_cache_test.go
  • pkg/cache/scheduler/tas_flavor_snapshot.go
  • pkg/cache/scheduler/tas_flavor_snapshot_grouped_placement_test.go

Comment thread pkg/cache/scheduler/tas_flavor_snapshot.go
@mimowo

mimowo commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

cc @pajakd ptal

Comment thread pkg/cache/scheduler/tas_flavor_snapshot.go
Comment thread pkg/cache/scheduler/tas_flavor_snapshot.go
Comment thread pkg/cache/scheduler/tas_flavor_snapshot_grouped_placement_test.go Outdated
@kubernetes-prow kubernetes-prow Bot removed the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jul 24, 2026
@kubernetes-prow kubernetes-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 24, 2026
@YQ-Wang
YQ-Wang requested a review from mimowo July 27, 2026 18:20
@mimowo

mimowo commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

/area tas

@kubernetes-prow kubernetes-prow Bot added the area/tas Topology-Aware Scheduling label Aug 3, 2026
@mimowo

mimowo commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

/release-note-edit

TAS: Fixed a bug that could prevent admission of otherwise feasible LeaderWorkerSet workloads when the selected leader domain reduced the capacity available to workers. For example, a 1-CPU leader and four 2-CPU workers can now be placed across 2-, 4-, and 3-CPU hosts in one rack by assigning the leader to the 3-CPU host, leaving capacity for all four workers.

@mimowo

mimowo commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

/lgtm
/approve
/cherrypick release-0.19
/cherrypick release-0.18
Thank you for fixing this epic TAS bug for PodGroups 👍

@k8s-infra-cherrypick-robot

Copy link
Copy Markdown
Contributor

@mimowo: once the present PR merges, I will cherry-pick it on top of release-0.18, release-0.19 in new PRs and assign them to you.

Details

In response to this:

/lgtm
/approve
/cherrypick release-0.19
/cherrypick release-0.18
Thank you for fixing this epic TAS bug for PodGroups 👍

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@kubernetes-prow kubernetes-prow Bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 3, 2026
@kubernetes-prow

Copy link
Copy Markdown

LGTM label has been added.

DetailsGit tree hash: 8590618300c96866ce84dcc5e811a1be880bec54

@kubernetes-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mimowo, YQ-Wang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 3, 2026
@kubernetes-prow
kubernetes-prow Bot merged commit f66249e into kubernetes-sigs:main Aug 3, 2026
55 checks passed
@kubernetes-prow kubernetes-prow Bot added this to the v0.20 milestone Aug 3, 2026
@k8s-infra-cherrypick-robot

Copy link
Copy Markdown
Contributor

@mimowo: new pull request created: #13766

Details

In response to this:

/lgtm
/approve
/cherrypick release-0.19
/cherrypick release-0.18
Thank you for fixing this epic TAS bug for PodGroups 👍

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-infra-cherrypick-robot

Copy link
Copy Markdown
Contributor

@mimowo: new pull request created: #13767

Details

In response to this:

/lgtm
/approve
/cherrypick release-0.19
/cherrypick release-0.18
Thank you for fixing this epic TAS bug for PodGroups 👍

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/tas Topology-Aware Scheduling cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inconsistency in finding place for leader in PodSetGrouping

5 participants