Skip to content

feat: Applications backend + package overlay data - #823

Closed
nadaverell wants to merge 6 commits into
mainfrom
feat/addons-overlay
Closed

nadaverell wants to merge 6 commits into
mainfrom
feat/addons-overlay

Conversation

@nadaverell

@nadaverell nadaverell commented May 29, 2026 •

Copy link
Copy Markdown
Contributor

Adds the per-cluster Applications backend surface on top of the shared subject/app-overlay model.

Current scope:

  • Exposes /api/applications, grouping concrete workloads into deployable app/release rows using topology structural roots plus the pkg/subject Tier-2 overlay.
  • Rows carry provenance tier/confidence, image-tag versions, health, warning events, relationships, and workload-scoped debug inputs.
  • Includes workload_class (service, worker, job, unknown) on rows and workloads.
  • Treats Jobs and CronJobs as first-class application/release units.
  • Treats add-on as classification evidence, not identity: mixed app/add-on groups report category=mixed instead of forcing the whole app to addon.
  • Keeps system-namespace workloads visible; classification/UI can fold or badge them, but the raw row remains present.
  • Threads app-overlay metadata through /api/packages as data-layer support for Applications and future package/add-on UI iteration.

This PR does not add or require package UI changes.

Grouping-accuracy fixes

Two over-merge fixes from pressure-testing the model against the full topology edge inventory:

  • structuralRoot stops at GitOps managers. The EdgeManages walk had no kind discrimination, so it climbed past a Flux Kustomization through the GitRepository→Kustomization source-ref edge (also an EdgeManages edge) to the GitRepository — collapsing a whole Flux mono-repo (one GitRepository, N Kustomizations) into one app via the structural union-find atom. It now halts at the first in-cluster manager (Application/Kustomization/HelmRelease); ownerRef chains + operator CRs still climb to the topmost owner. Regression test added.
  • app.kubernetes.io/instance added as overlay tier 6 (medium confidence). It's Argo CD's default tracking label and Helm's install-identity label — its absence meant default-config Argo apps fell through to the semantic name/part-of tiers and two releases of one chart (identical app.kubernetes.io/name) over-merged. Placed above name/part-of, below the explicit Helm-release annotation (tiers renumbered to 9). Env-specific, so it stays a single-cluster overlay signal and is excluded as a cross-cluster join key in the Hub.

Lockstep: the tier renumber is a wire contract — the SPA's TIER_PROVENANCE / sourceOf consumers are updated in radar-hub-web #92; merge/deploy together.

Consumed by radar-hub #55 and radar-hub-web #92.

@nadaverell
nadaverell requested a review from hisco as a code owner May 29, 2026 22:30
Comment thread internal/server/packages.go
@nadaverell nadaverell changed the title packages: emit Tier-2 app-overlay on PackageRow (Applications grouping) feat: Applications backend — /api/applications + PackageRow app-overlay May 30, 2026
Comment thread internal/server/applications.go Outdated
Comment thread internal/server/applications.go Outdated
}
continue
}
ae := &appEvent{Type: e.Type, Reason: e.Reason, Message: e.Message, Count: c, Object: e.InvolvedObject.Kind + "/" + e.InvolvedObject.Name, LastSeen: e.LastTimestamp.Format(time.RFC3339)}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

App events ignore EventTime

Medium Severity

podsEvents formats and compares only LastTimestamp for warning events. When LastTimestamp is unset but EventTime is populated (common on newer clusters), lastSeen becomes the zero RFC3339 time, so the broken-event feed can sort incorrectly and show misleading timestamps.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 14e84f1. Configure here.

if ns == "" {
items, _ = ssLister.List(labels.Everything())
} else {
items, _ = ssLister.StatefulSets(ns).List(labels.Everything())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applications ignores lister errors

Medium Severity

collectAppWorkloads discards errors from Deployment, DaemonSet, and StatefulSet list calls. A partial RBAC failure or informer error returns HTTP 200 with an incomplete application list and no signal that workloads were skipped, unlike collectWorkloadInputs for packages.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 14e84f1. Configure here.

Comment thread internal/server/applications.go Outdated
Comment thread pkg/packages/aggregate.go
nadaverell added a commit that referenced this pull request May 30, 2026
…x stale contracts

- lessIssue sorts by first_seen (onset) not last_seen, matching the UI
  comparator — last_seen bumps to compose-time every poll and would reshuffle
  the /api/issues + MCP queue on each refetch.
- SourceCondition classification narrows the broad fluxcd/argoproj group-
  substring match to exact sync kinds (Argo Application, Flux Kustomization/
  HelmRelease); Flux source CRDs (GitRepository et al.) and Argo control-plane
  CRDs fall to operator_condition_failed instead of inheriting the sync category.
- Issue severity contract doc corrected to 2-tier (critical/warning); info is
  dropped at compose, not emitted.
- subject.go package doc no longer claims pkg/topology consumes it — topology
  migration is staged for #823, not wired here; until then it's the shared
  vocabulary, not a second live truth.
- Drop the dead pre-classifier id fallback in IssuesView (id is required).
nadaverell added a commit that referenced this pull request May 30, 2026
…x stale contracts

- lessIssue sorts by first_seen (onset) not last_seen, matching the UI
  comparator — last_seen bumps to compose-time every poll and would reshuffle
  the /api/issues + MCP queue on each refetch.
- SourceCondition classification narrows the broad fluxcd/argoproj group-
  substring match to exact sync kinds (Argo Application, Flux Kustomization/
  HelmRelease); Flux source CRDs (GitRepository et al.) and Argo control-plane
  CRDs fall to operator_condition_failed instead of inheriting the sync category.
- Issue severity contract doc corrected to 2-tier (critical/warning); info is
  dropped at compose, not emitted.
- subject.go package doc no longer claims pkg/topology consumes it — topology
  migration is staged for #823, not wired here; until then it's the shared
  vocabulary, not a second live truth.
- Drop the dead pre-classifier id fallback in IssuesView (id is required).
Comment thread internal/server/applications.go
Comment thread pkg/packages/addons.go
nadaverell added a commit that referenced this pull request May 31, 2026
…c cycles, rename heuristic resolver

pkg/subject is the canonical identity primitive #823 will build on, so its
contract must be exact before that wiring lands. Review-driven:

- OwnerResolver doc now states the contract explicitly: Kubernetes CONTROLLER
  ownership only (ownerReferences[].controller chain), NOT declarative/app
  management. Removes the trap that pointed the topology adapter at
  walkTopmostOwner — which follows every EdgeManages edge, and EdgeManages is
  overloaded to include GitOps/Helm management (Argo App→resource,
  GitRepo→Kustomization). Wrapping it would collapse a Deployment's Subject up
  to its Application, erasing the Tier-1/Tier-2 boundary. #823 must resolve from
  controllerReferences (k8s.topOwnerForPodResolved is the reference impl).
  TestResolveSubject_StopsAtController pins it: Pod→RS→Deployment yields
  Deployment even when an Application manages it.
- Ownership cycles now resolve to a deterministic, start-independent
  representative (min refKey) instead of the last-hop-before-revisit (which gave
  a→b but b→a). A canonical identity can't depend on traversal start.
  TestResolveSubject_CycleIsDeterministic pins it.
- PodOwnerResolver → HeuristicPodOwnerResolver: it fabricates a Deployment from
  any ReplicaSet by hash-stripping (wrong for Rollouts/custom controllers). It
  has zero production callers (test-only); the name now carries the caveat so it
  can't be mistaken for canonical. Dropped the doc's false claim that issues
  passes it (issues uses only ScopeForKind + StableID).
nadaverell added a commit that referenced this pull request May 31, 2026
…umented

Round 2 of the canonical-primitive review — close the paths back to
'management edge as identity':

- HeuristicPodOwnerResolver no longer falls back to refs[0] when there's no
  controller ownerRef. The contract says controller-only; the fallback
  collapsed a pod under an arbitrary NON-controller owner. Now a pod with only
  non-controller refs resolves to itself (bare). New test pins both arms.
- Resolve doc corrected: obj feeds ONLY the Tier-2 overlay; ownership ALWAYS
  comes from the injected resolver. Resolve(ref, obj, nil, …) yields a bare
  Subject even if obj has ownerRefs — the previous 'obj supplies BOTH' framing
  was misleading. Spelled out that a pod owner-walk needs an injected resolver.
- Scrubbed EdgeManages / walkTopmostOwner from Tier-1 language in the Subject
  doc and OwnerLookup (operatorroots.go) — those comments are the #823
  integration guide, and 'derivable from the EdgeManages chain' reintroduced
  the exact ambiguity the contract fix removes. Now: controller ownerReferences
  / controllerRef-derived edges only.
@nadaverell
nadaverell force-pushed the feat/addons-overlay branch from a00168c to 54b1da2 Compare May 31, 2026 14:56
@nadaverell
nadaverell changed the base branch from feat/pkg-subject to main May 31, 2026 14:56
Attaches the app-overlay {key, tier, confidence} to each PackageRow so the
fleet Applications surface can group by logical app instead of by chart, using
the unified pkg/subject resolver (no fourth divergent cascade):
- Helm-labeled workloads → subject.ResolveOverlay on their metadata.
- GitOps declarations → derived from their own identity (Argo App tier 3,
  Flux HelmRelease tier 1, Kustomization tier 2), mirroring subject's key
  format so a workload + its managing declaration collapse to one app.
- Aggregate keeps the highest-confidence overlay per row (lowest tier wins).

pkg/packages stays pure (plain Overlay{key,tier,confidence}; the server maps
subject.AppOverlay). Serializes on /api/packages via the existing PackageRow
JSON. Stacked on feat/pkg-subject.

go build/vet + pkg/packages + internal/server tests green.
…erlay

Workload-centric twin of /api/packages (which is chart-centric, the Add-ons
surface). Enumerates Deployments/StatefulSets/DaemonSets, groups them into
logical apps by the pkg/subject Tier-2 app-overlay key, and anchors the version
on the running container image:tag (chart versions are empty for git-based
apps). Per workload it carries: image, ready/desired replicas, pod restart
count + last-terminated reason (the crashloop signal), and correlated recent
Warning events (BackOff / ImagePullBackOff / FailedScheduling / FailedMount —
the failures restarts alone miss). System namespaces + catalog-matched add-on
workloads are excluded; raw-always keeps unlabeled services visible.

Consumed by the hub /api/fleet/applications fan-out. Stacked on feat/pkg-subject.
go build + internal/server tests green.
One add-on catalog (crdGroupToChart canonical chart names + a curated extras
list) with ClassifyAddon(chart, app, partOf, name). Replaces the per-surface
allowlists so 'is this an add-on?' has a single home; a new CRD group joins the
catalog for free (pinned by test).
… add-on classifier

Group workloads into logical apps via union-find over structural-root, overlay,
and canonical-Argo atoms (cross-namespace Argo declarations collapse with their
workloads). Carry per-app structural relationships (Services/Ingress/Routes/
config/scalers/PDBs), O(pods) namespace-bucketed restart+event joins, and
add-on classification via pkg/packages.ClassifyAddon (drops the private allowlist).
@nadaverell
nadaverell force-pushed the feat/addons-overlay branch from 54b1da2 to 41ddc01 Compare May 31, 2026 20:51
Comment thread internal/server/applications.go Outdated
agg.Routes = append(agg.Routes, rel.Routes...)
agg.Configs += rel.Configs
agg.Scalers += rel.Scalers
agg.PDBs += rel.PDBs

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shared configs counted twice

Low Severity

When merging grouped workloads, mergeRelationships adds each workload’s Configs, Scalers, and PDBs counts. Shared ConfigMaps, HPAs, or PDBs linked from several workloads in the same app are counted once per workload, while Services/Ingresses/Routes are deduped in finalizeRelationships.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 41ddc01. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 6 total unresolved issues (including 4 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 8b416be. Configure here.


func cronJobHealth(cj *batchv1.CronJob) packages.Health {
return packages.HealthHealthy
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CronJob health always healthy

Medium Severity

cronJobHealth always returns healthy and never inspects the CronJob or its Jobs. Failed or stuck scheduled runs still surface as a healthy app workload and can keep the parent app row health optimistic when a CronJob is the only or dominant member.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 8b416be. Configure here.

m = map[string][]*corev1.Event{}
out[e.Namespace] = m
}
m[e.InvolvedObject.Name] = append(m[e.InvolvedObject.Name], e)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning events keyed by name

Medium Severity

Warning events are indexed only by InvolvedObject.Name. In a namespace, a Deployment and Service (or other kinds) with the same name share one bucket, so eventsForWorkload can attach another kind’s warnings to the wrong workload and bubble them into the app’s event feed.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 8b416be. Configure here.

@nadaverell nadaverell changed the title feat: Applications backend — /api/applications + PackageRow app-overlay feat: Applications backend + package overlay data Jun 2, 2026
…tes.io/instance tier

Two grouping-accuracy fixes from pressure-testing the model against the full
topology edge inventory.

structuralRoot walked incoming EdgeManages edges with no kind discrimination,
so it climbed past a Flux Kustomization through the GitRepository->Kustomization
source-ref edge (also an EdgeManages edge) to the GitRepository. A Flux mono-repo
(one GitRepository sourcing N Kustomizations) then resolved every workload to the
same structural root and union-find merged all installations into one app. The
walk now stops AT the first in-cluster GitOps manager (Application / Kustomization
/ HelmRelease); ownerRef chains and operator CRs still climb to the topmost owner.

app.kubernetes.io/instance was absent from the overlay tiers. It is Argo CD's
default resource-tracking label and Helm's installation-identity label, so
default-config Argo apps fell through to the semantic name/part-of tiers and two
releases of one chart (identical app.kubernetes.io/name) over-merged. Added as
tier 6 (medium confidence) above name/part-of and below the explicit Helm-release
annotation; tiers renumbered to 9. It is env-specific, so it stays a single-cluster
overlay signal and must not become a cross-cluster join key in the Hub.

Regression tests: two Kustomizations under one GitRepository -> two app rows; the
instance tier disambiguates installations and is outranked by the Helm annotation.
@nadaverell

Copy link
Copy Markdown
Contributor Author

Superseded. The Applications backend + pkg/subject Tier-2 app overlay landed on main through #875 (app topology and identities) and #958 (declared-only app identity), with later health refactors (#1024/#1032) touching the same files. The app.kubernetes.io/instance overlay tier shipped as TierInstance Tier = 6 (kept at 6 rather than the renumber-to-9 proposed here), and workload_class / the /api/applications surface are all present on main. This branch is stale and conflicting; closing.

@nadaverell nadaverell closed this Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant