Repository navigation
Diagnose a workload's HPA on the workload itself - #1685
Conversation
PR Summary by QodoSurface RBAC-gated HPA diagnosis on workload findings
AI Description
Diagram
High-Level Assessment
Files changed (18)
|
Code Review by Qodo
1.
|
|
Review round 1 (Cursor Bugbot + Qodo, head c036885):
Visual check: the section was rendered in a second Radar instance pointed at a copy of the local investigation store with |
|
Review round (Cursor Bugbot, head d4c2cc7):
|
Radar already diagnoses an HPA when the fetched object is the HPA; a workload scaled by one only got a scaledBy ref. Each scaledBy entry that is an HPA the caller can read now carries the same HPASummary under hpaSummary, looked up from the topology provider only after the ref passed the RBAC gate. KEDA scalers keep the bare ref; the JSON for existing entries is unchanged. The Findings pane renders a "Scaled by" section on the resource card with the HPA's state, replica bounds, summary and reasons. An HPA that cannot act (no metrics, cannot read its target) or is pinned at maxReplicas lifts the card to supporting/warning and is named in the card summary; ordinary scale-ups and scale-downs stay as detail. The display snapshot includes the scaler diagnosis so an HPA-only change is not folded into the previous observation. Both request-scoped RBAC checkers resolved builtin namespaced kinds through discovery and fell open to the unknown-kind passthrough while discovery was cold or partial. They now resolve builtins from the static catalogue first, so the HPA diagnosis (and every other builtin ref) is always gated by a SAR.
Topology refs carry an empty group for non-core builtins while discovery is cold, so a checker that resolved the plural from the static catalogue but kept the ref's group asked the apiserver about a core-group resource that does not exist. Both request-scoped checkers now use the group the catalogue or discovery resolved. The Scaled by reason list uses list markers instead of a hand-built dot.
…s reading
A KEDA-created HPA is called keda-hpa-<name> and says nothing about who
owns it. Each scaledBy HPA entry now carries managedBy, resolved from the
ScaledObject owner reference or the scaledobject.keda.sh/name label and
gated like every other ref, and the Scaled by section shows "managed by
KEDA ScaledObject <name>", linked when Radar also listed the ScaledObject.
HPA condition reasons led with the controller's sentence ("the desired
replica count is less than the minimum replica count"), which names
neither the bound nor the number. They now lead with Radar's reading ("HPA
is held at minReplicas=2") and keep the controller's text as detail; the
summary helpers that parsed that text read it from detail.
…detection hpadiag now leads each condition reason with Radar's reading and keeps the controller's text as detail, so the missing-request classifier has to look at the detail too or it misreads a missing CPU request as an absent metrics API. The maxed-HPA expectation follows the new wording.
The reason that named the HPA's state restated the summary in different
words, and the controller's condition text under it read as a
contradiction ("holding at minReplicas=2" next to "the desired replica
count is less than the minimum"). The state's reason now contributes
only the controller's sentence, quoted and attributed to its condition,
and the min-held summary says the HPA wants fewer replicas when the
controller reported the clamp. The HPA page applies the same rule for
every state instead of only max-limited.
The workload card attributes the controller's sentence to its condition, but the reason summary only copied id, message and detail, so a live diagnose result rendered a bare quote.
Two ways the HPA card stated more than the controller had established. An HPA whose status carries no observedGeneration skipped the staleness check entirely and fell through to OK, so a freshly created one the controller has never touched rendered as "Stable" over 0/0 replicas. It is now stale when the controller has recorded nothing at all — an unset observedGeneration together with no conditions. Some controllers reconcile without ever setting the field, so an HPA that does carry conditions is still read as live; a real cluster HPA with no observedGeneration and three real conditions is what that guard is for. A ScalingLimited condition observed against the previous spec was interpreted with the bounds in front of us. Raising maxReplicas from 5 to 10 produced "HPA is capped at maxReplicas=10" from a condition recorded when the cap was 5, sending an operator to change a limit that had already been changed. A stale condition now states the fact without the numbers it cannot vouch for. Both found by Codex in the merge sign-off review.
75ae0c6 to
a9a1aeb
Compare
|
Merge sign-off on the rebased branch (Claude + Codex). No blockers, one deferral, verified live. Rebase. Replayed over #1681 and #1684 with two conflicts, both additions on either side in test files. Codex independently confirmed integrity: both PR merge commits are ancestors and the conflict files retain all mainline content with only additions. Codex's one blocker, skipped with evidence. It flagged that Verified against the live GKE cluster, which is what this PR's own staleness work needed:
Verification: tsc, 1,009 web tests, 3,603 k8s-ui tests, both Go modules, |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a9a1aeb. Configure here.
Leading the HPA reason with Radar's reading moved the controller's own sentence out of the issue message. A guard downstream recognises the missing-resource- requests case by that sentence — "missing request for cpu" — so that it is not attributed to a down metrics.k8s.io APIService, which does not cause it. With the sentence gone, an operator whose pods simply lack CPU requests was told the cluster's metrics API was down: the wrong layer entirely, and the exact misattribution the guard's comment says it exists to prevent. The guard now recognises both spellings, and reads Cause as well, which is where Radar's reading of the condition lands. The existing test kept passing because its fixture carries the controller sentence in the message, which is the shape this branch stopped producing. The new one uses the shape the detector now emits, and fails without the fix. Found by Cursor Bugbot. The same hazard was already handled a few lines away in detect_workload.go; this was its sibling site.

Summary
Radar already diagnoses an HPA when you open the HPA. A workload scaled by one only got "scaled by X". Now each
scaledByentry that is an HPA the caller can read carries the sameHPASummaryunderhpaSummary, and the investigation Findings pane shows it on the workload's resource card. Track D3 of the Findings next layer.What changed
pkg/resourcecontext:ScaledByis[]ScalerRef(aContextRefplus optionalhpaSummary). The HPA object is looked up from the topology provider only after the ref passedfilterRefs, so a denied HPA is omitted with the existingscaledBy rbac_deniedentry and nothing about it leaks. KEDA scalers keep the bare ref; JSON for existing entries is byte-identical.current/desired replicas · bounds min-max, summary, reasons). A scaler that cannot act (unable_to_scale,metrics_unavailable) or is pinned atmaxReplicaslifts the card to supporting/warning and is named in the card summary (5/5 replicas ready · HPA: Maxed); ordinary scale-ups and scale-downs stay as detail. The display snapshot includes the scaler diagnosis so an HPA-only change is not folded into the previous observation.RefAccessCheckers resolved builtin namespaced kinds through discovery and fell open to the unknown-kind passthrough while discovery was cold or partial. They now consult the existingk8s.NamespacedBuiltinGVRcatalogue first (exported for this), so every builtin ref, and now the HPA diagnosis, is always gated by a SAR. One MCP diagnose test relied on that fail-open for its ConfigMap/Secret reads and now grants them explicitly.scaledobject.keda.sh/namelabel) carriesmanagedByon itsscaledByentry, gated like every other ref, and the section reads "managed by KEDA ScaledObject ", linked when Radar also listed the ScaledObject.hpadiagcondition reasons lead with Radar's reading ("HPA is held at minReplicas=2") and keep the controller's sentence as detail, so the card bullet and the HPA drawer say the same thing the summary does.docs/mcp.md:get_resourcerow mentionsscaledBy[].hpaSummary.The field reaches the agent unchanged: MCP
get_resourceanddiagnoseattachresourceContextbeside the minified object, and the AI minifier never touches it.Testing
make tsc: passcd web && npm test: 83/84 files pass; the one failing file issrc/api/client.authRedirect.test.ts(6 tests time out at 5s) and it fails identically on the untouchedmaincheckout, unrelated to this diffcd packages/k8s-ui && npm test: 172 files, 3549 passed, 1 skippedgo test ./...(root module): pass;cd pkg && go test ./...: passmake build: passgit diff --check: cleanscaledBy[0].hpaSummarywith statemetrics_unavailable, the realFailedGetResourceMetricreason, and groupautoscalingon the ref.scaledBy; KEDA ScaledObject serializes exactly as before; both checkers resolve builtins without discovery and reject a same-kind CRD in another group. Frontend — pane renders the section and stays quiet for undiagnosed scalers; tier/tone/summary for unhealthy, maxed and merely scaling HPAs; display snapshot changes on an HPA-only state change.Scope and tradeoffs
hpaStateLevel === unhealthyorlimited_max.metrics_incompleteand the transient scaling states stay as detail; widening that is a one-line change.Note
Medium Risk
Changes RBAC gating for resource-context refs and embeds live HPA state on workloads; incorrect SAR resolution or diagnosis leakage would affect authorization-sensitive MCP/API responses.
Overview
Workloads scaled by an HPA now surface the same
hpaSummarydiagnosis onresourceContext.scaledBy(after RBAC), soget_resource/diagnoseanswer “why did the autoscaler do that?” without a second read. KEDA-owned HPAs can include a gatedmanagedByScaledObject ref; undiagnosed scalers stay ref-only.HPA diagnosis quality:
hpadiagleads with Radar phrasing and keeps controller text indetail; unreconciled or generation-lagged HPAs avoid misleading limit messages. Issue linking no longer blames a metrics API outage when missing CPU requests show up in Radar’sCausewording.RBAC: Server and MCP ref checkers resolve builtin namespaced kinds via
NamespacedBuiltinGVRand use the resolved API group in SARs, closing fail-open while discovery is cold.UI: Findings resource cards add a Scaled by section; unhealthy or maxed HPAs lift card tone/summary. Docs updated for
scaledBy/hpaSummary.Reviewed by Cursor Bugbot for commit f9c5847. Bugbot is set up for automated code reviews on this repo. Configure here.