Skip to content

[release-0.20] Hold scheduling until the initial cache matches the apiserver - #17455

Merged
kubernetes-prow[bot] merged 2 commits into
kubernetes-sigs:release-0.20from
k8s-infra-cherrypick-robot:cherry-pick-16630-to-release-0.20
Oct 19, 2026
Merged

[release-0.20] Hold scheduling until the initial cache matches the apiserver#17455
kubernetes-prow[bot] merged 2 commits into
kubernetes-sigs:release-0.20from
k8s-infra-cherrypick-robot:cherry-pick-16630-to-release-0.20

Conversation

@k8s-infra-cherrypick-robot

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot commented Oct 19, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

This is an automated cherry-pick of #16630

/assign mimowo

TAS: Fixed a bug where a cold-started scheduler could admit workloads onto occupied nodes before existing ClusterQueues and their admitted Workloads were loaded into its cache. Kueue now waits for cache-filling handlers to process their initial data before scheduling.

/kind bug

@dundysm k8s-infra-cherrypick-robot
Wait for cache event handlers before the first scheduling cycle
74570a7 
After a cold leader start, Scheduler.Start begins as soon as the lease is
held. Informer stores are already synced, but ClusterQueue create handlers
can still be walking the list. A victim whose flavor is loaded becomes
active while occupant ClusterQueues are missing, so Topology Aware
Scheduling treats nodes held by those occupants as free. The same window
applies to cohort usage.

The scheduler cache wraps the controller-runtime sources whose handlers
fill it (ClusterQueue, Cohort, ResourceFlavor, Workload, LocalQueue, and
Topology plus the TAS ResourceFlavor when TAS runs). Scheduler.Start
blocks until each Kind source reports synced, which happens only after its
handler has processed the informer's initial list, and then starts the
scheduling loop. No polling, no lists, no copies, no timeout. A source that
never syncs already stops the manager via CacheSyncTimeout.

Builds on kubernetes-sigs#17346, which keeps Cohorts in the cache on every replica.

Tests: unit coverage for the sync tracker, and integration coverage for
scheduler startup and a TAS cold start.

Integration suites now add the scheduler to the manager instead of
calling Start during setup, the same way cmd/kueue does, since Start
blocks until the manager has started the tracked sources.

Signed-off-by: Dundy Pasupuleti <dundysm@gmail.com>
@dundysm k8s-infra-cherrypick-robot
Use a ClusterQueue watcher instead of a test hook in the cold start test
f2e4763 
Drop the package level create hook from the ClusterQueue reconciler. The
cold start test now registers a ClusterQueueUpdateWatcher through
SetupControllersOpts, and the Consistently checks use ConsistentDuration.

Signed-off-by: Dundy Pasupuleti <dundysm@gmail.com>
@kubernetes-prow kubernetes-prow Bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Oct 19, 2026
@kubernetes-prow kubernetes-prow Bot added this to the v0.20 milestone Oct 19, 2026
@kubernetes-prow kubernetes-prow Bot added the kind/bug Categorizes issue or PR as related to a bug. label Oct 19, 2026
@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Oct 19, 2026
Merged
@netlify

netlify Bot commented Oct 19, 2026 •
edited
Loading

Copy link
Copy Markdown

✅ Deploy Preview for kubernetes-sigs-kueue ready!

Name Link
🔨 Latest commit f2e4763
🔍 Latest deploy log https://app.netlify.com/projects/kubernetes-sigs-kueue/deploys/6ad685bc2dd4fd8e39a6cbdc
😎 Deploy Preview https://deploy-preview-17455--kubernetes-sigs-kueue.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Oct 19, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 1
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: kubernetes-sigs/kueue/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fd8cf799-d6c7-4766-94fd-bca9ac2f459e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@kubernetes-prow kubernetes-prow Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 19, 2026
@kubernetes-prow
kubernetes-prow Bot requested review from kshalot and tenzen-y October 19, 2026 21:04
@mimowo

mimowo commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@kubernetes-prow kubernetes-prow Bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 19, 2026
@kubernetes-prow

kubernetes-prow Bot commented Oct 19, 2026

Copy link
Copy Markdown

LGTM label has been added.

DetailsGit tree hash: 814b59fb9b125b0ff869ffca9adc650f3aefa271

@kubernetes-prow

kubernetes-prow Bot commented Oct 19, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: k8s-infra-cherrypick-robot, mimowo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 19, 2026
@kubernetes-prow
kubernetes-prow Bot merged commit f700479 into kubernetes-sigs:release-0.20 Oct 19, 2026
62 checks passed
@anguszzzz anguszzzz mentioned this pull request Oct 21, 2026
Merged
@mbobrovskyi

mbobrovskyi commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

/set-release-note
TAS: Fixed a bug where a cold-started scheduler could admit workloads onto occupied nodes before existing ClusterQueues and their admitted Workloads were loaded into its cache. Kueue now waits for cache-filling handlers to process their initial data before scheduling.

@github-actions

github-actions Bot commented Oct 22, 2026

Copy link
Copy Markdown

✅ The release note was updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kshalot kshalot Awaiting requested review from kshalot
@tenzen-y tenzen-y Awaiting requested review from tenzen-y

Assignees

@mimowo mimowo

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Milestone

v0.20

Development

Successfully merging this pull request may close these issues.

4 participants

@k8s-infra-cherrypick-robot @mimowo @mbobrovskyi @dundysm