Skip to content

Fast dev loop: make test-changed, with the full suite once at the end - #2371

Merged
nadaverell merged 2 commits into
mainfrom
chore/fast-dev-loop
Oct 27, 2026
Merged

nadaverell merged 2 commits into
mainfrom
chore/fast-dev-loop

Conversation

@nadaverell

@nadaverell nadaverell commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Agents and people iterating on Radar have been running the full suites after every fix: about 20 minutes per branch (internal/server alone takes ~4.5 minutes, and the two vitest suites are about 7,700 tests). A focused run of the same change takes about 2 minutes. This makes the fast tier the default and says when the full run is needed.

What changed

  • make test-changed (scripts/test-changed.sh [base], default origin/main). It takes the files changed since the merge base, plus staged, unstaged and untracked files, and runs:
    • go vet + go test on the changed Go packages only, in both the root and pkg/ modules. GO_TEST_RUN=<regexp> narrows a big package such as internal/server.
    • A deleted Go file still selects its package; a testdata/ file selects the Go package that owns it.
    • Each project's npm run tsc (TypeScript 7, as in CI) when frontend files changed, then vitest related --run on the changed files, including shared testdata/ fixtures the frontend tests import.
    • go.mod/go.sum changes build and vet the whole module. A frontend config change (package.json, tsconfig, vite/vitest config) runs both type-checks and prints that one full vitest run is needed.
    • It sets NODE_OPTIONS=--no-experimental-webstorage on Node 25 (its localStorage global breaks jsdom), and runs on the bash 3.2 macOS ships.
  • .claude/commands/qa.md: verification has two tiers.
    • Iteration: make test-changed, once per batch of fixes.
    • Completion: once, after the final batch. CI on the pushed head is that run. Run the full local suite only for a branch CI won't test, such as a PR stacked on another branch.
    • It also says not to run suites alongside other heavy work, and to rerun a failure in an untouched file on its own before debugging it. It names the timing-sensitive suites we hit (ResourcesView.kind-view, TestUpgradeReadinessEndpointLargeMetrics).
  • CLAUDE.md: one paragraph with the same rule next to the day-to-day commands.

Testing

scripts/test-changed.sh HEAD~1 on this branch picked the 7 changed frontend files of #2348, ran tsc, and ran vitest related (54 files, 404 tests, all pass). The same selection on a branch with Go changes (only pkg/gitops, pkg/gitops/diagnose and pkg/issuesapi, plus 24 related vitest files) took about 2 minutes, against about 20 for the full run. Timing depends heavily on machine load. No CI or product code changes.

â€Ķuidance

scripts/test-changed.sh vets and tests only the changed Go packages, type-checks the frontend and runs vitest related on changed frontend files. qa.md and CLAUDE.md make that the iteration tier and the full suite a once-at-the-end run, with CI on the pushed head counting as it.
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Add change-scoped tests for a faster development loop

âœĻ Enhancement 📝 Documentation 🕐 10-20 Minutes

Grey Divider

AI Description

â€Ē Add make test-changed to check changed Go packages and related frontend tests during iteration.
â€Ē Document one full completion run, using CI when it tests the pushed branch.
Diagram

graph TD
  M["Make target"] --> S["Git change selection"] --> G["Go packages"] --> V["Vet and test"]
  S --> F["Frontend sources"] --> T["Type check"] --> R["Vitest related"]
  V -->|"later"| C["Completion checks"]
  R -->|"later"| C
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Dependency-aware test selection
  • ➕ Could catch affected consumers and changes to shared fixtures or configuration that direct source-file matching misses.
  • ➖ Adds selection complexity and may lengthen the iteration loop substantially.

Recommendation: Keep direct package and Vitest-related selection for the fast iteration tier; it fits the stated speed goal. Treat it as deliberately incomplete, and ensure completion checks actually run for the branch: CI uses path filters, so a pushed head does not necessarily trigger every suite.

Files changed (4) +100 / -7

Enhancement (1) +59 / -0
test-changed.shRun focused checks from the Git change set +59/-0

Run focused checks from the Git change set

â€Ē Selects committed, working-tree, and untracked files relative to a merge base; vets and tests changed Go packages in both modules, then type-checks both frontend projects and runs related Vitest tests for affected projects. Supports 'GO_TEST_RUN', macOS Bash 3.2, and a Node 25 webstorage workaround.

scripts/test-changed.sh

Documentation (2) +34 / -6
qa.mdDefine iteration and completion verification tiers +32/-6

Define iteration and completion verification tiers

â€Ē Replaces reflexive full-suite guidance with change-scoped checks during iteration and one completion run through CI or, when needed, locally. Adds batching, load-related failure triage, and Node 25 advice while retaining the visual-test guidance.

.claude/commands/qa.md

CLAUDE.mdSurface the fast-loop rule in day-to-day guidance +2/-0

Surface the fast-loop rule in day-to-day guidance

â€Ē Adds a concise rule to run change-scoped checks per batch, defer the full run until completion, and avoid competing heavy work. Points readers to the detailed QA guidance.

CLAUDE.md

Other (1) +7 / -1
MakefileExpose the test-changed target +7/-1

Expose the test-changed target

â€Ē Adds a phony 'test-changed' target that passes an optional 'BASE' to the new script, and lists the target in help.

Makefile

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) ðŸŽĻ UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Deleted Go sources evade package tests ✓ Resolved
Description
test-changed.sh discards every changed path that no longer exists before it determines which Go
package to check. Deleting a source file while leaving its package and tests in place therefore runs
neither go vet nor go test for that package.
Code

scripts/test-changed.sh[24]

+  [ -e "$f" ] || continue
Evidence
Git supplies deleted paths, but the existence check drops them before package selection. The example
package retains a test file that should be checked after a source deletion.

scripts/test-changed.sh[18-29]
scripts/test-changed.sh[43-51]
pkg/argoapi/client_test.go[1-10]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Deleted Go files are discarded before their remaining package can be checked.
## Fix Focus Areas
- scripts/test-changed.sh[19-30]
## Recommended Fix
Use deleted paths to identify affected Go packages, and run checks when the package still exists; distinguish packages removed entirely.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Fixture edits receive no related tests ✓ Resolved
Description
test-changed.sh selects only Go and TypeScript extensions, so a changed JSON fixture adds nothing
to either test list. An edit to the shared changes fixture, for example, skips both its importing
frontend test and the Go test that checks the same fixture.
Code

scripts/test-changed.sh[R26-29]

+    pkg/*.go) pkg_pkgs="$pkg_pkgs ./$(dirname "${f#pkg/}")" ;;
+    *.go) root_pkgs="$root_pkgs ./$(dirname "$f")" ;;
+    packages/k8s-ui/*.ts|packages/k8s-ui/*.tsx) k8sui="$k8sui ${f#packages/k8s-ui/}" ;;
+    web/*.ts|web/*.tsx) web="$web ${f#web/}" ;;
Evidence
The classifier has no JSON case and both test blocks require a populated list. The cited frontend
test imports the fixture, while the cited Go test names that same fixture as its shared baseline.

scripts/test-changed.sh[23-30]
scripts/test-changed.sh[43-57]
web/src/components/diagnose/investigationEvidence/adapters/changes.test.ts[1-11]
internal/meaningfulchanges/compact_creates_test.go[252-258]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Fixture-only edits do not select the tests that consume those files.
## Fix Focus Areas
- scripts/test-changed.sh[23-30]
- scripts/test-changed.sh[43-57]
## Recommended Fix
Recognize changed test fixtures and select their dependent Go packages and frontend tests, using a broader check when a dependency cannot be mapped.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Configuration edits skip all fast checks ✓ Resolved
Description
test-changed.sh has no selection path for module manifests or frontend compiler configurations.
When go.mod, pkg/go.mod, or either frontend tsconfig.json changes without a source-file edit,
the script exits without testing the affected module or type-checking the affected project.
Code

scripts/test-changed.sh[R26-29]

+    pkg/*.go) pkg_pkgs="$pkg_pkgs ./$(dirname "${f#pkg/}")" ;;
+    *.go) root_pkgs="$root_pkgs ./$(dirname "$f")" ;;
+    packages/k8s-ui/*.ts|packages/k8s-ui/*.tsx) k8sui="$k8sui ${f#packages/k8s-ui/}" ;;
+    web/*.ts|web/*.tsx) web="$web ${f#web/}" ;;
Evidence
Both Go modules declare dependencies in their manifests, and the frontend projects define compiler
settings in JSON. None of those paths matches the script's selection cases, so the guarded commands
do not run.

scripts/test-changed.sh[23-30]
scripts/test-changed.sh[43-54]
go.mod[1-10]
pkg/go.mod[1-12]
web/tsconfig.json[1-26]
packages/k8s-ui/tsconfig.json[1-24]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Configuration-only edits bypass the checks whose behavior they change.
## Fix Focus Areas
- scripts/test-changed.sh[23-30]
- scripts/test-changed.sh[43-57]
## Recommended Fix
Recognize Go module manifests and frontend compiler configurations; run the affected module's Go checks or project's type-check when one changes.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View medium (2)
4. Test names with spaces break focused runs ✓ Resolved
Description
GO_TEST_RUN is assembled into run_flag and expanded without quotes in each go test invocation.
Selecting the existing matching plain request subtest splits its regexp into extra arguments, so
the focused test command fails instead of passing the complete expression to -run.
Code

scripts/test-changed.sh[50]

+    go vet $root_pkgs && go test -count=1 $run_flag $root_pkgs
Evidence
The regexp is stored alongside the flag and expanded unquoted in both Go test commands. The
repository contains a subtest whose name includes spaces, providing a concrete affected expression.

scripts/test-changed.sh[34-34]
scripts/test-changed.sh[44-50]
internal/server/localterm_context_unix_test.go[132-133]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Unquoted expansion splits or glob-expands valid focused-test expressions.
## Fix Focus Areas
- scripts/test-changed.sh[34-34]
- scripts/test-changed.sh[44-50]
## Recommended Fix
Pass `-run "$GO_TEST_RUN"` as two quoted arguments only when the variable is set, preserving spaces and glob metacharacters in the regexp.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Fast type-checks use a different compiler ✓ Resolved
Description
test-changed.sh runs npx tsc, which selects the repository's TypeScript 6 executable rather than
the TypeScript 7 executable used by each project's npm run tsc. Frontend iteration checks
consequently do not exercise the same compiler as the established type-check commands and CI.
Code

scripts/test-changed.sh[54]

+    (cd web && npx tsc --noEmit -p .) && (cd packages/k8s-ui && npx tsc --noEmit -p .)
Evidence
The lockfile installs TypeScript 6 as typescript and TypeScript 7 separately as typescript-7;
both package scripts explicitly invoke the latter. CI runs the web package script.

scripts/test-changed.sh[52-54]
package-lock.json[6008-6030]
web/package.json[19-27]
packages/k8s-ui/package.json[65-68]
.github/workflows/ci.yml[273-280]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The fast tier invokes a different TypeScript compiler from the project and CI checks.
## Fix Focus Areas
- scripts/test-changed.sh[52-54]
## Recommended Fix
Invoke `npm run tsc` in each frontend project instead of `npx tsc --noEmit -p .`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
ðŸ’Ą Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread scripts/test-changed.sh Outdated
Comment thread scripts/test-changed.sh Outdated
Comment thread scripts/test-changed.sh Outdated
Comment thread scripts/test-changed.sh Outdated
Comment thread scripts/test-changed.sh Outdated
â€Ķnges, quoted -run, project tsc

A deleted Go file still selects its package; a testdata file selects its Go package and is passed to vitest related; go.mod changes build and vet the module; frontend config changes type-check and ask for one full vitest run; GO_TEST_RUN is passed as one argument; type-checks use each project's npm run tsc (TypeScript 7).
@nadaverell

Copy link
Copy Markdown
Contributor Author

All five fixed in 07729ea:

  1. Deleted Go files still select their package (when it still has Go files).
  2. A testdata/ file selects the Go package that owns it and is passed to vitest related in both projects, so the frontend tests that import shared fixtures run.
  3. go.mod/go.sum changes build and vet the module; frontend config changes (package.json, tsconfig, vite/vitest config) run both type-checks and print that one full vitest run is needed.
  4. GO_TEST_RUN is passed to -run as a single quoted argument.
  5. Type-checks use each project's npm run tsc (TypeScript 7), as CI does.

@nadaverell
nadaverell merged commit 145a691 into main Oct 27, 2026
11 checks passed
@nadaverell
nadaverell deleted the chore/fast-dev-loop branch October 27, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant