Repository navigation
Require HPA coverage before treating the evidence as conclusive - #1755
Conversation
âĶnclusive A synced, non-nil HPA informer answers an empty list for every namespace it does not watch, and it answers without an error. Both rightsizing paths read that as "this workload has no autoscaler" and set HPAEvidenceAvailable, which is not a caveat but a safety gate: a reduction is withheld when the evidence is missing, so a false positive here lets one through on a workload whose HPA was never visible. Radar falls back to namespace-scoped informers whenever cluster-wide list is denied, so this is reachable in ordinary operation. Availability now follows the informer's own coverage, per workload, since a cluster-wide scan can span a watched and an unwatched namespace at once. Full access is unchanged by construction: a cache with no scope restrictions normalises to Enabled with an empty namespace, for which KindCoversNamespace is true everywhere, so both gates are no-ops. Coverage is consulted rather than cluster-wide-ness, which keeps a multi-namespace informer authoritative for every namespace it actually watches. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125QDX24TNuod7FKLwTDhvW
|
â Your Qodo trial ends soon. Ask your workspace admin to set up billing to keep reviews running after the trial. Manage billing |
PR Summary by QodoRequire HPA namespace coverage for conclusive rightsizing evidence
AI Description
Diagram
High-Level Assessment
Files changed (3)
|
Code Review by Qodo
1.
|
âĶange The comment explained the test by what the code did beforehand, which a reader cannot check. What matters is why the case holds: an unrestricted cache watches every namespace, so no workload's empty HPA result is a gap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125QDX24TNuod7FKLwTDhvW
Closes #1749.
HPAEvidenceAvailableis not a caveat â it is a safety gate. Rightsizing withholds a reduction whenFit == FitOversized && !HPAEvidenceAvailable, because shrinking a request an autoscaler is scaling on is the one mistake that matters here. A false positive therefore does not merely mislead; it lets the reduction through.A synced, non-nil HPA informer answers an empty list for every namespace it does not watch, and it answers without an error. Both rightsizing paths read that as "this workload has no autoscaler" and set the flag. Radar falls back to namespace-scoped informers whenever cluster-wide list is denied, so this is reachable in ordinary multi-tenant operation rather than a corner case.
What changed
Availability now follows the informer's own coverage, decided per workload â a cluster-wide scan can span a watched and an unwatched namespace at once, so a single answer for the whole scan would be wrong either way.
Coverage is consulted via
KindCoversNamespace, deliberately rather than asking whether the informer is cluster-wide. A multi-namespace informer is authoritative for every namespace it actually watches, and testing cluster-wide-ness would have suppressed valid recommendations in all of them.Full access is unchanged by construction
NewResourceCachenormalises an unrestricted cache toEnabled: truewith an empty namespace, for whichKindCoversNamespacereturns true for every namespace. Both new gates are no-ops there. The reverse â that this cannot newly suppress a valid recommendation â holds for the same reason: coverage is derived from the sameresourceScopeNamespacesused to build the informer, so a namespace that is watched is a namespace that is covered.Testing
Go tests pin both directions: full access marks every workload available, and an HPA informer scoped to one namespace marks only that namespace's workloads â driven through
enrichScanHPAwith a real utilization-based HPA, so target matching and thehpaManagedflag are exercised rather than just the marking helper.go build ./...,go vet,go test ./internal/prometheus/(including-race), and the separatepkg/k8scoremodule tests pass.Live, against GKE with real Prometheus and kube-state-metrics. Binaries from this branch and from
mainwere run against the same cluster under two identities.Full access â the case that must not change:
mainDeployment staging/koala-backendhpaEvidenceAvailable=true, same fit and reasonhpaEvidenceAvailabletrue on 52Repeat scans differed only in measured values (observed, sample counts), and those differences moved between runs and workloads â live drift, not behaviour.
HPA readable in
stagingonly â the condition the bug needs. A throwaway ServiceAccount was granted everything cluster-wide excepthorizontalpodautoscalers, plus a Role granting HPA reads instagingalone, which drives Radar to a namespace-scoped HPA informer (Using namespace-scoped informers for namespace(s): ["staging"]).Deployment dev/finops(HPAs not readable indev)mainhpaEvidenceAvailabletruefalserecommendationReasonhpa_evidence_unavailable100m, memory256Mimainsuggests shrinking a workload whose autoscaler it could never have seen. In the same run,Deployment staging/koala-backendâ where the informer does watch the namespace â is identical on both builds, recommendations included, so the gate suppresses only what it cannot see.The ServiceAccount, Role, RoleBinding, ClusterRole, ClusterRoleBinding and namespace were deleted afterwards and verified gone.
Follow-up
Both paths still reject HPA evidence while any deferred informer is warming or has failed, even when the HPA informer itself is fully synced and cluster-wide â a false negative that suppresses valid recommendations. It predates this change and fixing it here would alter full-access behaviour, which this PR deliberately does not do.
ðĪ Generated with Claude Code
https://claude.ai/code/session_0125QDX24TNuod7FKLwTDhvW
Note
Medium Risk
Changes the safety gate that blocks downsize recommendations when HPA inventory may be incomplete; behavior is intentionally stricter for namespace-scoped informers but full-access paths are designed to stay equivalent.
Overview
Fixes a safety bug where namespace-scoped HPA informers return an empty list (with no error) for namespaces they do not watch, and rightsizing treated that as proof there was no autoscalerâallowing CPU/memory reduction recommendations that should be blocked when HPA evidence is unknown.
Both the single-workload path (
loadHPAManagedResources) and the cluster scan path (markScanHPAAvailable) now callKindCoversNamespaceper workload namespace before treating an HPA list as authoritative. Workloads outside the informerâs scope no longer gethpaEvidenceAvailable/hpaAvailable, so oversized fits stay withheld withhpa_evidence_unavailableinstead of a shrink. Unrestricted caches are unchanged because full coverage still marks every namespace as readable.Tests cover scoped vs full access, the marking helper, and end-to-end
enrichScanHPAwith a real utilization-based HPA.Reviewed by Cursor Bugbot for commit f64b5fd. Bugbot is set up for automated code reviews on this repo. Configure here.