Skip to content

Diagnose a workload's HPA on the workload itself - #1685

Merged
nadaverell merged 8 commits into
mainfrom
feat/hpa-diagnosis-workloads
Sep 9, 2026
Merged

nadaverell merged 8 commits into
mainfrom
feat/hpa-diagnosis-workloads

Conversation

@nadaverell

@nadaverell nadaverell commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Radar already diagnoses an HPA when you open the HPA. A workload scaled by one only got "scaled by X". Now each scaledBy entry that is an HPA the caller can read carries the same HPASummary under hpaSummary, and the investigation Findings pane shows it on the workload's resource card. Track D3 of the Findings next layer.

What changed

  • pkg/resourcecontext: ScaledBy is []ScalerRef (a ContextRef plus optional hpaSummary). The HPA object is looked up from the topology provider only after the ref passed filterRefs, so a denied HPA is omitted with the existing scaledBy rbac_denied entry and nothing about it leaks. KEDA scalers keep the bare ref; JSON for existing entries is byte-identical.
  • Findings pane: a "Scaled by" section on resource cards (kind + name, state via the shared HPA state label/level helpers, current/desired replicas · bounds min-max, summary, reasons). A scaler that cannot act (unable_to_scale, metrics_unavailable) or is pinned at maxReplicas lifts the card to supporting/warning and is named in the card summary (5/5 replicas ready · HPA: Maxed); ordinary scale-ups and scale-downs stay as detail. The display snapshot includes the scaler diagnosis so an HPA-only change is not folded into the previous observation.
  • RBAC hardening (from cross-model review): both request-scoped RefAccessCheckers resolved builtin namespaced kinds through discovery and fell open to the unknown-kind passthrough while discovery was cold or partial. They now consult the existing k8s.NamespacedBuiltinGVR catalogue first (exported for this), so every builtin ref, and now the HPA diagnosis, is always gated by a SAR. One MCP diagnose test relied on that fail-open for its ConfigMap/Secret reads and now grants them explicitly.
  • KEDA attribution: an HPA that KEDA created (ScaledObject owner reference or scaledobject.keda.sh/name label) carries managedBy on its scaledBy entry, gated like every other ref, and the section reads "managed by KEDA ScaledObject ", linked when Radar also listed the ScaledObject.
  • hpadiag condition reasons lead with Radar's reading ("HPA is held at minReplicas=2") and keep the controller's sentence as detail, so the card bullet and the HPA drawer say the same thing the summary does.
  • docs/mcp.md: get_resource row mentions scaledBy[].hpaSummary.

The field reaches the agent unchanged: MCP get_resource and diagnose attach resourceContext beside the minified object, and the AI minifier never touches it.

Testing

  • make tsc: pass
  • cd web && npm test: 83/84 files pass; the one failing file is src/api/client.authRedirect.test.ts (6 tests time out at 5s) and it fails identically on the untouched main checkout, unrelated to this diff
  • cd packages/k8s-ui && npm test: 172 files, 3549 passed, 1 skipped
  • go test ./... (root module): pass; cd pkg && go test ./...: pass
  • make build: pass
  • git diff --check: clean
  • Live check: built binary against a kind cluster with a seeded Deployment + HPA (no metrics-server) returned scaledBy[0].hpaSummary with state metrics_unavailable, the real FailedGetResourceMetric reason, and group autoscaling on the ref.
  • New tests: Go — Deployment scaled by HPA attaches the summary; denied HPA omitted with no leak of state/reason/name; unscaled workload has no scaledBy; KEDA ScaledObject serializes exactly as before; both checkers resolve builtins without discovery and reject a same-kind CRD in another group. Frontend — pane renders the section and stays quiet for undiagnosed scalers; tier/tone/summary for unhealthy, maxed and merely scaling HPAs; display snapshot changes on an HPA-only state change.
  • Visual: screenshots to follow in a comment (fixture-store capture of the section in light and dark).

Scope and tradeoffs

  • The adverse-state threshold is hpaStateLevel === unhealthy or limited_max. metrics_incomplete and the transient scaling states stay as detail; widening that is a one-line change.
  • The HPA lookup is a linear scan of the provider's HPA list per workload, done once per Build only when a scaler ref is an HPA.

Note

Medium Risk
Changes RBAC gating for resource-context refs and embeds live HPA state on workloads; incorrect SAR resolution or diagnosis leakage would affect authorization-sensitive MCP/API responses.

Overview
Workloads scaled by an HPA now surface the same hpaSummary diagnosis on resourceContext.scaledBy (after RBAC), so get_resource / diagnose answer “why did the autoscaler do that?” without a second read. KEDA-owned HPAs can include a gated managedBy ScaledObject ref; undiagnosed scalers stay ref-only.

HPA diagnosis quality: hpadiag leads with Radar phrasing and keeps controller text in detail; unreconciled or generation-lagged HPAs avoid misleading limit messages. Issue linking no longer blames a metrics API outage when missing CPU requests show up in Radar’s Cause wording.

RBAC: Server and MCP ref checkers resolve builtin namespaced kinds via NamespacedBuiltinGVR and use the resolved API group in SARs, closing fail-open while discovery is cold.

UI: Findings resource cards add a Scaled by section; unhealthy or maxed HPAs lift card tone/summary. Docs updated for scaledBy / hpaSummary.

Reviewed by Cursor Bugbot for commit f9c5847. Bugbot is set up for automated code reviews on this repo. Configure here.

@nadaverell
nadaverell requested a review from hisco as a code owner September 8, 2026 06:46
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Surface RBAC-gated HPA diagnosis on workload findings

✨ Enhancement 🐞 Bug fix 🧪 Tests 📝 Documentation 🕐 40+ Minutes

Grey Divider

AI Description

• Attach readable HPA diagnoses to workload scaledBy context without leaking denied scalers.
• Surface adverse autoscaler states in Findings cards and observation snapshots.
• Resolve builtin namespaced resources statically to enforce SAR checks before discovery.
Diagram

sequenceDiagram
    actor Caller
    participant Context as Context Builder
    participant RBAC as RBAC Checker
    participant Catalog as GVR Catalog
    participant Provider as Topology Provider
    participant Findings as Findings UI
    Caller->>Context: Diagnose workload
    Context->>RBAC: Filter scaler refs
    RBAC->>Catalog: Resolve builtin GVR
    Catalog-->>RBAC: SAR resource
    RBAC-->>Context: Allowed HPA refs
    Context->>Provider: Load readable HPAs
    Provider-->>Context: HPA objects
    Context-->>Findings: ScalerRef and summary
    Findings-->>Caller: Autoscaler findings
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Fetch each HPA from the client
  • ➕ Keeps workload resource context smaller when scaler details are not displayed.
  • ➕ Separates workload and HPA retrieval lifecycles.
  • ➖ Adds request latency and client-side orchestration.
  • ➖ Duplicates authorization-sensitive lookup behavior across consumers.
  • ➖ Prevents MCP and other resource-context consumers from receiving the diagnosis directly.
2. Return a separate HPA summary map
  • ➕ Keeps scaler references structurally unchanged.
  • ➕ Could deduplicate summaries if multiple relationships reference one HPA.
  • ➖ Requires consumers to join summaries back to scaler references.
  • ➖ Introduces a parallel resource-context structure with weaker relationship locality.
  • ➖ Provides little benefit because each scaler entry naturally owns its diagnosis.

Recommendation: Keep the PR's nested ScalerRef.hpaSummary approach. It preserves existing JSON for undiagnosed scalers, keeps diagnosis beside its relationship, serves UI and agent consumers without extra reads, and correctly performs authorization before accessing HPA details.

Files changed (18) +604 / -18

Enhancement (5) +200 / -8
build.goAttach authorized HPA diagnosis to scaledBy entries +43/-3

Attach authorized HPA diagnosis to scaledBy entries

• Builds 'ScalerRef' entries by filtering references before loading HPA objects, preventing denied scaler details from leaking. Authorized HPA references receive the existing HPA summary, while unsupported scalers remain bare references.

pkg/resourcecontext/build.go

types.goIntroduce enriched ScalerRef context type +10/-1

Introduce enriched ScalerRef context type

• Changes 'ResourceContext.ScaledBy' to '[]ScalerRef'. The new type embeds the existing reference fields and optionally carries an HPA summary.

pkg/resourcecontext/types.go

InvestigationEvidencePane.tsxRender diagnosed scalers on workload cards +86/-1

Render diagnosed scalers on workload cards

• Adds a Scaled by section for entries carrying HPA summaries. It uses shared state helpers and displays status, replica bounds, summary text, and detailed reasons.

web/src/components/diagnose/InvestigationEvidencePane.tsx

diagnoseEvidenceTypes.tsModel HPA summaries on diagnosis scaler references +31/-1

Model HPA summaries on diagnosis scaler references

• Defines frontend types for HPA state, bounds, metrics, reasons, and targets. Updates diagnosis resource context so 'scaledBy' entries may carry an optional HPA summary.

web/src/components/diagnose/diagnoseEvidenceTypes.ts

investigationEvidence.tsPromote adverse HPA states in Findings +30/-2

Promote adverse HPA states in Findings

• Adds adverse scaler state detection using shared HPA health helpers. Workload summaries now name blocked or maxed HPAs, while ordinary scaling states remain informational details.

web/src/components/diagnose/investigationEvidence.ts

Bug fix (4) +16 / -3
change_rbac.goExport static namespaced builtin GVR resolution +3/-3

Export static namespaced builtin GVR resolution

• Exports 'NamespacedBuiltinGVR' so request-scoped authorization checkers can resolve builtin namespaced kinds without discovery. Existing change-resource resolution now uses the exported helper.

internal/k8s/change_rbac.go

rc_rbac.goGate MCP builtin references through static GVR resolution +6/-0

Gate MCP builtin references through static GVR resolution

• Resolves namespaced builtin kinds from the static catalog before consulting discovery. This prevents builtin references from entering the unknown-kind fail-open path while discovery is unavailable or incomplete.

internal/mcp/rc_rbac.go

rc_rbac.goGate server builtin references through static GVR resolution +6/-0

Gate server builtin references through static GVR resolution

• Uses the static namespaced builtin catalog before dynamic discovery in the server access checker. Builtin references therefore consistently require a SubjectAccessReview.

internal/server/rc_rbac.go

investigationEvidencePresentation.tsInclude scaler diagnosis in display snapshots +1/-0

Include scaler diagnosis in display snapshots

• Adds 'scaledBy' context to resource evidence snapshots so HPA-only changes are not folded into earlier observations.

web/src/components/diagnose/investigationEvidencePresentation.ts

Tests (8) +387 / -6
rc_rbac_test.goTest discovery-independent MCP RBAC resolution +23/-0

Test discovery-independent MCP RBAC resolution

• Adds coverage for namespaced and cluster-scoped builtin resolution without discovery. It also verifies group-colliding CRDs and KEDA kinds do not inherit builtin authorization resources.

internal/mcp/rc_rbac_test.go

tools_diagnose_test.goGrant referenced resources explicitly in diagnose test +6/-0

Grant referenced resources explicitly in diagnose test

• Explicitly grants ConfigMap and Secret reads needed by the deployment diagnosis fixture. The test no longer depends on the former discovery-cold fail-open behavior.

internal/mcp/tools_diagnose_test.go

rc_rbac_test.goTest server builtin RBAC resolution without discovery +26/-0

Test server builtin RBAC resolution without discovery

• Covers static resolution for HPA, Secret, Deployment, and Node kinds. Confirms mismatched groups and non-builtin scaler kinds remain unresolved rather than borrowing builtin permissions.

internal/server/rc_rbac_test.go

build_test.goCover HPA scaler enrichment and RBAC isolation +165/-1

Cover HPA scaler enrichment and RBAC isolation

• Extends the mock provider with HPAs and tests attached summaries, denied-reference omission, unscaled workloads, and KEDA compatibility. Serialization assertions verify both the new HPA shape and unchanged bare scaler JSON.

pkg/resourcecontext/build_test.go

types_test.goValidate ScalerRef ordering and round trips +13/-5

Validate ScalerRef ordering and round trips

• Updates field-ordering fixtures for 'ScalerRef' and adds an HPA summary to resource-context serialization round-trip coverage.

pkg/resourcecontext/types_test.go

InvestigationEvidencePane.test.tsxTest workload card autoscaler details +71/-0

Test workload card autoscaler details

• Verifies Findings renders the HPA name, state, bounds, summary, and reasons on a workload card. Confirms scalers without an attached diagnosis do not produce an empty section.

web/src/components/diagnose/InvestigationEvidencePane.test.tsx

investigationEvidence.test.tsTest autoscaler-driven evidence severity +52/-0

Test autoscaler-driven evidence severity

• Verifies unavailable metrics and maximum-replica limits promote healthy workload cards to supporting warnings. Confirms normal scale activity remains neutral context.

web/src/components/diagnose/investigationEvidence.test.ts

investigationEvidencePresentation.test.tsTest HPA-sensitive evidence snapshots +31/-0

Test HPA-sensitive evidence snapshots

• Ensures otherwise identical resource observations produce different display snapshots when the attached HPA state changes.

web/src/components/diagnose/investigationEvidencePresentation.test.ts

Documentation (1) +1 / -1
mcp.mdDocument workload HPA summaries in get_resource +1/-1

Document workload HPA summaries in get_resource

• Documents that 'scaledBy' may contain an authorized HPA diagnosis, including state, bounds, metrics, and reasons. Clarifies that consumers can explain autoscaling behavior without a second resource read.

docs/mcp.md

@qodo-code-review

qodo-code-review Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. HPA details can bypass access controls ✓ Resolved 🐞 Bug ⛨ Security
Description
lookupResourceName resolves a group-less HorizontalPodAutoscaler reference to its plural
resource but discards the canonical autoscaling group, so CanRead submits the original empty
group to authorization. Because topology-generated HPA IDs omit apiVersion, legitimate HPA readers
lose the scaledBy/hpaSummary context, while callers with matching core-group wildcard
permissions can receive HPA state, bounds, metrics, and reasons through both REST and MCP.
Code

internal/server/rc_rbac.go[R88-89]

+	if g, r, ok := k8s.NamespacedBuiltinGVR(kind); ok && (group == "" || group == g) {
+		return r
Relevance

●●● Strong

Recent RBAC precedents accept fixes for incorrect API-group authorization and access-control gaps.

PR-#1425
PR-#719
PR-#1313

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
HPA topology node IDs contain only kind, namespace, and name, and relationship conversion derives
the reference group solely from apiVersion, leaving ResourceRef.Group empty. The scaler builder
passes that reference through filterRefs and checkRef, while both REST and MCP request-scoped
permission implementations resolve only the plural resource from the static catalogue and use the
unchanged empty group as the exact SubjectAccessReview or cache key. Although the catalogue
identifies the canonical group as autoscaling, that value is never propagated to authorization.

pkg/topology/builder.go[3623-3638]
pkg/topology/relationships.go[1063-1098]
pkg/resourcecontext/build.go[1203-1223]
pkg/resourcecontext/build.go[1701-1708]
internal/k8s/change_rbac.go[31-45]
internal/server/rc_rbac.go[50-64]
internal/mcp/rc_rbac.go[48-65]
internal/server/rc_rbac.go[85-90]
pkg/topology/builder.go[3634-3647]
pkg/topology/relationships.go[317-326]
pkg/topology/neighborhood.go[596-608]
internal/k8s/change_rbac.go[37-45]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
HPA references created from topology have an empty API group because their node IDs omit `apiVersion`. The static lookup resolves `HorizontalPodAutoscaler` to `horizontalpodautoscalers` but discards the canonical `autoscaling` group, causing authorization to check the core group instead of `autoscaling/horizontalpodautoscalers`.

## Issue Context
This affects both REST and MCP request-scoped resource-context access checkers. It can reject callers who legitimately have permission to read HPAs, omitting the newly attached `scaledBy`/`hpaSummary` workload context, and can expose HPA state, bounds, metrics, and reasons when unrelated core-group wildcard permissions satisfy the mismatched check. Preserve the collision guard for explicitly supplied noncanonical groups, but when static or discovery resolution selects a canonical resource mapping, propagate and use its canonical group in the authorization request and cache key. Add tests proving that an empty-group HPA topology reference is checked against the `autoscaling` group in both request-scoped checkers.

## Fix Focus Areas
- internal/server/rc_rbac.go[50-64]
- internal/server/rc_rbac.go[71-96]
- internal/mcp/rc_rbac.go[48-65]
- internal/mcp/rc_rbac.go[73-99]
- internal/server/rc_rbac_test.go[8-26]
- internal/mcp/rc_rbac_test.go[7-23]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Autoscaler warnings look inconsistent ✗ Dismissed 📘 Rule violation ⚙ Maintainability
Description
ScaledBySection renders HPA state and reasons with a custom StatusDot, spans, and list rather
than AlertBanner, ProblemAlerts, or ConditionsSection. When an HPA is unhealthy or capped,
this bespoke layout surfaces the warning and its condition reasons outside the shared problem
presentation used elsewhere.
Code

web/src/components/diagnose/InvestigationEvidencePane.tsx[R1454-1457]

+                <span className="inline-flex items-center gap-1.5 text-theme-text-secondary">
+                  <StatusDot
+                    tone={mapHealthToTone(hpaStateLevel(summary.state))}
+                    className="shrink-0"
Relevance

●●● Strong

Recent frontend precedent accepts replacing bespoke status styling with shared presentation
components.

PR-#1585

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 3036704 requires user-facing problems and conditions to use the shared alert,
problem, or condition components. The added scaled-by UI instead constructs the HPA status and
reason list from StatusDot, spans, and list elements.

Rule 3036704: Use standard problem and label selector components in renderers
web/src/components/diagnose/InvestigationEvidencePane.tsx[1435-1497]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The scaled-by section hand-builds its warning status and reason presentation instead of using the standard problem or condition components.

## Issue Context
HPA states such as unavailable metrics or maximum-replica limits are surfaced as adverse workload findings. Preserve the scaler identity and bounds while routing the warning summary and reasons through `AlertBanner`, `ProblemAlerts`, or `ConditionsSection` as appropriate.

## Fix Focus Areas
- web/src/components/diagnose/InvestigationEvidencePane.tsx[1435-1497]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Reason bullets bypass theme backgrounds ✓ Resolved 📘 Rule violation ⚙ Maintainability
Description
ScaledBySection colors each reason bullet with bg-theme-text-tertiary, which is not one of the
approved background utility tokens. Whenever HPA reasons are rendered, that marker uses a text-color
token as a background and carries the nonstandard styling into every affected workload card.
Code

web/src/components/diagnose/InvestigationEvidencePane.tsx[1478]

+                        <span className="mt-1.5 h-1 w-1 shrink-0 rounded-full bg-theme-text-tertiary" />
Relevance

●●● Strong

Theme-token compliance findings are deterministic style fixes, and recent frontend convention
findings were accepted.

PR-#1585

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 3036653 allows only the designated theme background utilities for background colors.
The newly added reason marker applies bg-theme-text-tertiary, which is outside that allowed set.

Rule 3036653: Use theme background tokens instead of hardcoded utility color classes
web/src/components/diagnose/InvestigationEvidencePane.tsx[1478-1478]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The HPA reason marker uses `bg-theme-text-tertiary`, but background utilities are restricted to the approved theme background tokens.

## Issue Context
Use an approved background token or render the marker as a text glyph with a theme text utility so no text token is repurposed as a background color.

## Fix Focus Areas
- web/src/components/diagnose/InvestigationEvidencePane.tsx[1474-1485]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 41 rules
✅ Cross-repo context — repo relationships
  Explored: repo: skyhook-dev/radar-hub (sha: 34e22e7a)
  Explored: repo: skyhook-dev/radar-hub-web (sha: e5bc6761)
Review mode: 🧠 Deep: This is a broad, behavior-changing cross-layer feature spanning RBAC/security gating, resource-context construction, API types, UI projection, and multiple independent logic paths, with substantial scope for subtle defects.

Grey Divider

Tip of the day
💡 Did you know, you can commit Qodo's fix in one click with committable suggestions (GitHub & GitLab)

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread web/src/components/diagnose/InvestigationEvidencePane.tsx
Comment thread web/src/components/diagnose/InvestigationEvidencePane.tsx Outdated
Comment thread internal/server/rc_rbac.go Outdated

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread internal/server/rc_rbac.go
@nadaverell

Copy link
Copy Markdown
Contributor Author

Review round 1 (Cursor Bugbot + Qodo, head c036885):

# Source Issue Verdict Reasoning Action
1 Bugbot, Qodo #3 Checkers resolve the plural statically but SAR with the ref's group, which topology leaves empty for non-core builtins while discovery is cold Fix Real: a SAR on core-group horizontalpodautoscalers answers the wrong question. Both checkers now SAR with the group the catalogue or discovery resolved; tests assert the resolved group. b4558c9
2 Qodo #2 Reason bullets used bg-theme-text-tertiary Fix Not an approved background token. Reasons now use list-disc with a tertiary marker. b4558c9
3 Qodo #1 Scaled by section should use AlertBanner / ProblemAlerts Skip Those are drawer-renderer primitives. Inside a Findings card the established pattern is StatusDot + text (the Conditions rows on the same card); a banner there would style a Radar fact like an alert and break the card's register.

Visual check: the section was rendered in a second Radar instance pointed at a copy of the local investigation store with scaledBy[].hpaSummary injected into a saved run's diagnose result. Verified in light and dark: the card summary reads 0/1 replicas ready · HPA: Maxed for the adverse state, the section shows kind, name, state, 4/4 replicas · bounds 2-4, the summary and reasons, and a later healthy observation (Stable) keeps the earlier maxed one under Previous observations rather than folding it away.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread web/src/components/diagnose/InvestigationEvidencePane.tsx
@nadaverell

Copy link
Copy Markdown
Contributor Author

Review round (Cursor Bugbot, head d4c2cc7):

# Source Issue Verdict Reasoning Action
1 Bugbot Condition fields dropped from HPA summary Fix HPAReasonSummary copied only id, message and detail, so a live card rendered a bare quote conditionType and conditionReason now carried through pkg/resourcecontext

Radar already diagnoses an HPA when the fetched object is the HPA; a
workload scaled by one only got a scaledBy ref. Each scaledBy entry that is
an HPA the caller can read now carries the same HPASummary under hpaSummary,
looked up from the topology provider only after the ref passed the RBAC
gate. KEDA scalers keep the bare ref; the JSON for existing entries is
unchanged.

The Findings pane renders a "Scaled by" section on the resource card with
the HPA's state, replica bounds, summary and reasons. An HPA that cannot act
(no metrics, cannot read its target) or is pinned at maxReplicas lifts the
card to supporting/warning and is named in the card summary; ordinary
scale-ups and scale-downs stay as detail. The display snapshot includes the
scaler diagnosis so an HPA-only change is not folded into the previous
observation.

Both request-scoped RBAC checkers resolved builtin namespaced kinds through
discovery and fell open to the unknown-kind passthrough while discovery was
cold or partial. They now resolve builtins from the static catalogue first,
so the HPA diagnosis (and every other builtin ref) is always gated by a SAR.
Topology refs carry an empty group for non-core builtins while discovery is
cold, so a checker that resolved the plural from the static catalogue but
kept the ref's group asked the apiserver about a core-group resource that
does not exist. Both request-scoped checkers now use the group the catalogue
or discovery resolved. The Scaled by reason list uses list markers instead
of a hand-built dot.
…s reading

A KEDA-created HPA is called keda-hpa-<name> and says nothing about who
owns it. Each scaledBy HPA entry now carries managedBy, resolved from the
ScaledObject owner reference or the scaledobject.keda.sh/name label and
gated like every other ref, and the Scaled by section shows "managed by
KEDA ScaledObject <name>", linked when Radar also listed the ScaledObject.

HPA condition reasons led with the controller's sentence ("the desired
replica count is less than the minimum replica count"), which names
neither the bound nor the number. They now lead with Radar's reading ("HPA
is held at minReplicas=2") and keep the controller's text as detail; the
summary helpers that parsed that text read it from detail.
…detection

hpadiag now leads each condition reason with Radar's reading and keeps the
controller's text as detail, so the missing-request classifier has to look
at the detail too or it misreads a missing CPU request as an absent
metrics API. The maxed-HPA expectation follows the new wording.
The reason that named the HPA's state restated the summary in different
words, and the controller's condition text under it read as a
contradiction ("holding at minReplicas=2" next to "the desired replica
count is less than the minimum"). The state's reason now contributes
only the controller's sentence, quoted and attributed to its condition,
and the min-held summary says the HPA wants fewer replicas when the
controller reported the clamp. The HPA page applies the same rule for
every state instead of only max-limited.
The workload card attributes the controller's sentence to its condition,
but the reason summary only copied id, message and detail, so a live
diagnose result rendered a bare quote.
Two ways the HPA card stated more than the controller had established.

An HPA whose status carries no observedGeneration skipped the staleness check
entirely and fell through to OK, so a freshly created one the controller has
never touched rendered as "Stable" over 0/0 replicas. It is now stale when the
controller has recorded nothing at all — an unset observedGeneration together
with no conditions. Some controllers reconcile without ever setting the field,
so an HPA that does carry conditions is still read as live; a real cluster HPA
with no observedGeneration and three real conditions is what that guard is for.

A ScalingLimited condition observed against the previous spec was interpreted
with the bounds in front of us. Raising maxReplicas from 5 to 10 produced "HPA
is capped at maxReplicas=10" from a condition recorded when the cap was 5,
sending an operator to change a limit that had already been changed. A stale
condition now states the fact without the numbers it cannot vouch for.

Both found by Codex in the merge sign-off review.
@nadaverell
nadaverell force-pushed the feat/hpa-diagnosis-workloads branch from 75ae0c6 to a9a1aeb Compare September 9, 2026 21:05
@nadaverell

Copy link
Copy Markdown
Contributor Author

Merge sign-off on the rebased branch (Claude + Codex). No blockers, one deferral, verified live.

Rebase. Replayed over #1681 and #1684 with two conflicts, both additions on either side in test files. Codex independently confirmed integrity: both PR merge commits are ancestors and the conflict files retain all mainline content with only additions.

Codex's one blocker, skipped with evidence. It flagged that chooseState gives stale status the lowest precedence, so a prior-generation condition can report "Unable to scale" as current instead of "Stale". The precedence is not this PR's: pkg/hpadiag already exists on main with that exact ordering, and this PR only modifies it (+53/−9). It is also a deliberate, named decision — TestAnalyzePrefersScalingOverStaleStatus pins it. Whether a stale HPA's headline state overstates is a fair question and it is recorded for the next round; it is not a change to make here.

Verified against the live GKE cluster, which is what this PR's own staleness work needed:

  • A KEDA-managed workload renders the ScaledObject and the HPA behind it — scaledobject/koala-backend, hpa/keda-hpa-koala-backend — with "At min · HPA wants fewer replicas but is held at minReplicas=2". That matches the cluster (0/2 (avg), min 2, 2 replicas).
  • A plain HPA renders "Stable · HPA is within configured bounds" and does not report stale. That is the regression check for the broadened staleness detection: every HPA on this cluster carries real conditions while omitting observedGeneration, which is exactly the shape the guard has to leave alone. It does.

Verification: tsc, 1,009 web tests, 3,603 k8s-ui tests, both Go modules, go build.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a9a1aeb. Configure here.

Comment thread pkg/hpadiag/diagnosis.go
Leading the HPA reason with Radar's reading moved the controller's own sentence
out of the issue message. A guard downstream recognises the missing-resource-
requests case by that sentence — "missing request for cpu" — so that it is not
attributed to a down metrics.k8s.io APIService, which does not cause it. With
the sentence gone, an operator whose pods simply lack CPU requests was told the
cluster's metrics API was down: the wrong layer entirely, and the exact
misattribution the guard's comment says it exists to prevent.

The guard now recognises both spellings, and reads Cause as well, which is
where Radar's reading of the condition lands.

The existing test kept passing because its fixture carries the controller
sentence in the message, which is the shape this branch stopped producing. The
new one uses the shape the detector now emits, and fails without the fix.

Found by Cursor Bugbot. The same hazard was already handled a few lines away in
detect_workload.go; this was its sibling site.
@nadaverell
nadaverell merged commit cc95f0d into main Sep 9, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant