Skip to content

docs(wiki): drop the zod-compiler benchmark - #6505

Merged
colinhacks merged 2 commits into
mainfrom
drop-zod-compiler-bench
Aug 29, 2026
Merged

colinhacks merged 2 commits into
mainfrom
drop-zod-compiler-bench

Conversation

@colinhacks

Copy link
Copy Markdown
Owner

Removes the zod-compiler comparison from wiki/compile.md, the bench that produced it (added in #6499), and the zod-compiler devDependency it pulled into packages/bench.

The two mentions that credit zod-compiler for the lookahead-free email grammar and the function-splitting technique stay, and the ecosystem entry from #6122 is untouched.

Removes the comparison section from wiki/compile.md and the bench that produced it, along with the zod-compiler devDependency. The two remaining mentions credit the email regex form and the function-splitting technique, and stay.
Copilot AI lite review requested due to automatic review settings August 29, 2026 18:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — a pure removal of the zod-compiler comparison, spanning the bench, its dependency, the lockfile and the wiki section it produced.

  • packages/bench/compile-vs-zod-compiler.ts deleted — the 591-line bench added in #6499; it was the only harness running z.compile() against zod-compiler's jit() entry. Every fixture in it was file-local, and nothing imported from it.
  • zod-compiler@1.28.0 dropped from packages/bench/package.json — the last consumer went with the bench.
  • pnpm-lock.yaml regenerated — removes zod-compiler, unplugin@3.3.0, webpack-virtual-modules@0.6.2 and es-module-lexer@2.3.2, and flips jiti@2.7.0 to optional: true now that only vite's optional edge reaches it. Machine-generated shape, and pnpm install --frozen-lockfile --lockfile-only succeeds against it.
  • wiki/compile.md loses ### Against zod-compiler — plus two sentences reworded to stop pointing at it. The ### Against arktype → ## Output construction seam reads cleanly and no anchor links to the removed heading survive.

Repo-wide grep confirms the PR body's claims: the only remaining zod-compiler mentions are the email-grammar credit and the function-splitting credit in the Open section, and the ecosystem entry from #6122. Nothing else referenced the deleted bench — no CI workflow, no packages/bench/index.ts entry, no README index. No packages/zod/src code is touched, so no axis measurements apply.

ℹ️ The z.url() 0.57x figure outlives the only bench that produced it

wiki/compile.md keeps the 0.57x on invalid input number in two places — the exception clause under "Why the failure path is not worth compiling" and the "Hoisted validators run twice on invalid input" bullet — and the deleted bench was the only one that timed invalid input compiled-vs-runtime. compile-matrix.ts has a z.url() case but its Case interface carries no invalid variant at all, so nothing left in packages/bench reproduces it. Not a reason to hold the PR; flagging it since the figure is load-bearing for the decision not to compile the failure path.

The 243-leaf figures are unaffected — the failure-path table has its own measurements for that fixture, so it isn't orphaned by this deletion.

Technical details
# `z.url()` invalid-input ratio has no remaining reproduction

## Affected sites
- `wiki/compile.md:134` — "0.57x against the runtime, the one place the run-twice bound has a price", reworded by this PR to drop the zod-compiler attribution but keeping the number.
- `wiki/compile.md:146` — "for `z.url()` that is a second `new URL()` — 0.57x on invalid input".
- `packages/bench/compile-matrix.ts` — nearest surviving bench; its `Case` interface is `{ group, name, schema, input }` with no invalid-input path, and its `z.url()` row uses `"https://example.com/a/b?c=1"`.

## Required outcome
- Either the 0.57x claim is reproducible from a bench in the repo, or it is marked as a historical one-off measurement so a future reader doesn't go looking for the harness.

## Suggested approach (optional)
- Add an optional invalid-input variant to `compile-matrix.ts`'s `Case` and populate it for the hoisted-format rows, which also covers the "every other hoisted format is cheap enough not to register" half of the claim.

## Open questions for the human
- Is `wiki/` held to "every figure has a runnable bench", or is it acceptable for design notes to cite measurements whose harness has since been removed?

Pullfrog  | View workflow run | Using Claude Opus | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — one commit since the last review, a docs-prose follow-up that resolves the ℹ️ raised against 1c285b86.

  • z.url() 0.57x figure marked as a one-off measurement — wiki/compile.md:134 now reads "0.57x against the runtime in a one-off measurement", and the Open bullet at :146 adds "in a one-off measurement; no bench in the repo reproduces it". This takes the second of the two options the prior review offered (mark it historical) rather than adding an invalid-input variant to compile-matrix.ts, which is the right call for a design note — the figure's job is to justify not compiling the failure path, not to be re-runnable.

The added claim checks out. The only surviving z.url() benches are compile-matrix.ts:42, whose Case interface carries no invalid-input field at all, and compile-helper-scope.ts:8, whose URL_DATA is a valid padded URL; bench-invalid.ts is a check-method comparison that touches no URL schema. Nothing else in packages/bench times a URL rejection.

No behavioral surface in this delta — wiki/compile.md is the only file touched, so the three axes still don't apply.

Pullfrog  | View workflow run | Using Claude Opus | 𝕏

@colinhacks
colinhacks merged commit 93186ca into main Aug 29, 2026
9 checks passed
@colinhacks
colinhacks deleted the drop-zod-compiler-bench branch August 29, 2026 18:53
@husayt

husayt commented Sep 11, 2026

Copy link
Copy Markdown

@colinhacks why was it removed? Anything wrong with zod-compiler?

This branch was successfully deployed

1 active deployment
Preview – zod-v4 — b54e9181 Deployed Aug 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants