Skip to content

fix(upgrade): attribute incomplete evidence precisely - #1513

Merged
nadaverell merged 2 commits into
mainfrom
fix/upgrade-readiness-evidence-guidance
Aug 28, 2026
Merged

nadaverell merged 2 commits into
mainfrom
fix/upgrade-readiness-evidence-guidance

Conversation

@nadaverell

@nadaverell nadaverell commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #1481: make Kubernetes upgrade impact honest and actionable when Radar cannot collect all of the live evidence needed for the 1.37 checks. Operators now see the difference between no evidence and partial evidence, explicit authorization denials and transient collection failures, and timeless upgrade-path policy versus release-gated findings.

This matters for Kubernetes 1.37 because several checks depend on kubelet metrics/effective configuration and admission-plane resources that the default in-cluster identity deliberately cannot read. A missing permission must not look like a compatibility failure, a clean bill of health, or an unexplained Unknown.

Kubernetes 1.37 context

Radar's 1.37 review covers the operationally relevant removals and transitions documented by Kubernetes, including removed or locked feature gates, kubeadm configuration changes, kubelet eventRecordQPS, removed cAdvisor flags/metrics, the scheduling v1alpha2 removal, SELinux mount behavior, webhook readiness, and the required one-minor control-plane upgrade sequence.

Authoritative references:

What changed

Precise evidence attribution

  • Authorization decisions now preserve whether the result was authoritative. A cached or successful SAR denial can produce permission guidance; a nil client, timeout, or SAR error remains a generic evidence gap.
  • The same decision model is used by the HTTP and MCP upgrade-readiness surfaces.
  • Actual Forbidden responses from kubelet node-proxy requests and webhook list requests remain authoritative even when a permissive no-auth preflight allowed the attempt.
  • Webhook collection separately records unavailable kinds and the denied subset, so transient API failures do not tell operators to request RBAC they may already have.
  • Non-authorization kubelet collection failures are logged once per scan as an aggregate instead of up to two lines per node.

Safe node-runtime access

  • Adds rbac.viewNodeRuntime, default false, granting only get on nodes/proxy when explicitly enabled.
  • Keeps the permission off by default because kubelet metrics and effective configuration expose node-level runtime details to the audience of a no-auth Radar instance.
  • Authenticated Radar requests continue to impersonate the user, so those users need nodes/proxy on their own Kubernetes identity; enabling the ServiceAccount grant does not bypass per-user RBAC.
  • Documents the toggle and trust boundary in the chart README, values, and in-cluster installation guide. Helm tests pin both the default absence and exact opt-in rule.

Honest upgrade-impact communication

  • Rows with an Unknown result, a caveat, and zero inspected objects show No evidence; rows that inspected some evidence show Partial evidence.
  • Summary copy uses incomplete evidence rather than implying every gap is partial.
  • Webhook coverage copy stays deployment-neutral in the reusable scanner/UI, while exact identity-level remediation comes from the backend evidence.
  • Upgrade-path findings no longer claim an Applies from release: the one-minor sequencing policy is timeless, including a live 1.37 → 1.39 path.

Validation

Live Kubernetes validation

  • Shared GKE nonprod, read-only: Kubernetes v1.35.6-gke.1710000, 9/9 nodes Ready. Radar evaluated 1.35 → 1.37 with reviewedThrough=1.37; the UI rendered the required 1.35 → 1.36 → 1.37 sequence and 73 grounded findings with no console errors. Independent EndpointSlice reads confirmed the unavailable webhook backends cited by fail-closed blockers and fail-open warnings. The access path rejected mutating HTTP verbs and CONNECT; node count was unchanged, while one already-running Pod became NotReady during the observation window and was treated as ambient nonprod drift.
  • Controlled kind 1.36.1 → 1.37: injected live feature-gate, component-flag, kubeadm, kubelet eventRecordQPS, and Prometheus metric-consumer evidence. The API values matched the injected settings, and the expected blocker/warning/review/N/A classifications rendered with zero console errors. The cluster was restored and deleted after capture.
  • Official Kubernetes 1.37.0 runtime: disposable kind v0.33.0 cluster using kindest/node:v1.37.0@sha256:a1ed56cfb0e7b93589bdf97c8cd566405a265939e3620fc4f5de89adff580ae5; API server and Ready node both reported v1.37.0 with containerd 2.3.4. Published Radar 1.12.1 (ghcr.io/skyhook-io/radar@sha256:a95ec5d9a50e7a874ddc85ec9fecd216eaebb95d90be4e3d36500436273c2e35) was Ready with zero restarts; health, cluster-info, resource-counts, upgrade-readiness, and the Overview UI passed. The follow-up branch also ran under the default chart ServiceAccount and rendered at 1280×800 and 1920×1080 with zero console errors.
  • GKE 1.37 availability: Cloud SDK 578.0.0 and a clean 582.0.0 both observed 1.37.0-gke.1173000+preview in Rapid preview versions, but the documented alpha create request returned HTTP 400 and listed Rapid create support only through 1.36.3. No cluster, VM, disk, forwarding rule, address, or firewall was created. This is recorded only as an advertised-preview/create-API availability mismatch; no provider root cause is inferred.

The final authoritative-SAR distinction, observed-403 attribution, and zero-evidence wire-shape correction were made after the live browser captures. They are covered by HTTP-level collector, HTTP/MCP authorizer, scanner, and frontend tests rather than represented as additional live-cluster proof.

Local QA

  • go test ./...
  • cd pkg && go test ./...
  • make tsc
  • npx vitest run src/components/audit/UpgradeReadinessView.test.ts — 14 passed
  • helm unittest -f 'tests/upgrade_impact_rbac_test.yaml' deploy/helm/radar — 3 passed
  • ./scripts/test-chart.sh
  • make build — frontend build, embed, and Go binary
  • git diff --check

Security and residual risk

The only new permission is explicit, narrowly scoped, and default-off. The main failure mode is incomplete evidence, not an incorrect compatibility verdict: unavailable sources keep affected checks Unknown/incomplete and explain the evidence boundary. Final copy/authorization corrections are test-backed but were not re-captured on the deleted disposable cluster.


Note

Medium Risk
Touches authorization for upgrade evidence and adds an opt-in nodes/proxy grant; misconfiguration could expose node runtime details on no-auth installs, but defaults stay off and denied/missing evidence is surfaced as incomplete rather than false compatibility verdicts.

Overview
Upgrade impact now separates authoritative RBAC denials from non-authoritative auth check failures (missing client, SAR errors, timeouts). HTTP and MCP upgrade authorizers expose EvidenceAuthorizationDecision (Allowed + Authoritative); webhook collection tracks denied kinds separately from transient list failures; kubelet nodes/proxy fetches record observed 403 so checks can cite permission gaps instead of generic “missing evidence.”

Adds opt-in Helm value rbac.viewNodeRuntime (default false) granting only get on nodes/proxy for no-auth in-cluster Radar, with README/docs and a unittest for the exact ClusterRole rule.

Scanner and UI propagate NodeProxyForbidden and AdmissionWebhookDeniedKinds into caveats with actionable remediation; node-runtime gaps no longer blame disabled configz when proxy was forbidden. Upgrade-path blockers drop misleading AppliesFrom on timeless sequencing policy. Upgrade Readiness UI labels rows No evidence vs Partial evidence, says “incomplete evidence” in summaries, and avoids guessing Helm RBAC keys in coverage banners.

Reviewed by Cursor Bugbot for commit d77f1ca. Bugbot is set up for automated code reviews on this repo. Configure here.

@nadaverell
nadaverell requested a review from hisco as a code owner August 27, 2026 11:50

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f74b36f. Configure here.

Comment thread pkg/upgradereadiness/checks_137.go Outdated
Comment thread pkg/upgradereadiness/checks_evidenced.go
@nadaverell
nadaverell merged commit 2caec60 into main Aug 28, 2026
9 checks passed
@nadaverell
nadaverell deleted the fix/upgrade-readiness-evidence-guidance branch August 28, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant