Benchmark Cursor CLI’s Diff vs Rewrite Edits: 13 Steps, Up to 9x Gap [2026]

Every time Cursor applies a code change, it makes a choice you never see: patch the file with a small diff, or regenerate the whole thing and swap it in. That choice decides how many tokens you pay for, how long you wait, and how much blast radius a single prompt can have on a file you didn’t ask it to touch. Most developers never measure it. They just notice that some edits land in under a second and others take noticeably longer, and they move on.

This tutorial builds a benchmark harness around the Cursor CLI (the agent command, formerly cursor-agent) that measures exactly which mode fired, how long it took, and how much of the file it actually touched. By the end you’ll have a working Python project that runs controlled trials across small, medium, and large files, logs every run to CSV, and gives you a results table you can use to decide whether to force full rewrites on sensitive files or let Cursor’s fast-apply path run on autopilot. The target keyword here, cursor cli, draws real monthly search volume because engineers are actively trying to script Cursor outside the editor window, and this project is built entirely around that CLI.

·

Google · Preferred Sources

Don't miss new tech stories on Google

Add Tech Insider once in the Google app and our stories appear in your news suggestions.

Add Now

Why This Mechanism Quietly Controls Your Token Bill and Your Review Risk

Engineering teams that run Cursor across a monorepo tend to notice the cost problem before they notice the review problem. A plan that feels affordable for small tweaks can balloon once agents start regenerating entire files for one-line fixes, especially on files that sit in the 500-to-2,000-line range common in backend services and React component trees. If every edit silently falls back to a full rewrite, the token bill scales with the size of the codebase rather than the size of the actual work being done, and nobody on the team notices until a monthly invoice looks wrong.

The review risk is less visible but arguably more dangerous. A full-file rewrite can reformat, reorder, or subtly alter code the author never asked the agent to touch. A reviewer skimming a diff that shows the entire file as changed has a much harder job than one reviewing a tight, five-line patch, and “the diff is too big to review carefully” is exactly the condition under which real bugs slip into production. Measuring which mode actually fires on your repository, instead of assuming it based on marketing copy, turns that risk into something you can manage with policy rather than hope.

How Cursor Actually Decides Between a Diff and a Rewrite

Cursor’s engineering team has described the edit-application problem as a two-part job. As the company put it in its technical write-up on the subject, “Difficult code edits can be broken down into two stages: planning, and applying.” The planning stage is the part most users think of as “the AI” – the model reasons about what needs to change. The applying stage is a separate, often invisible step that turns that plan into bytes on disk, and Cursor’s own blog post on the system explains that it has historically handled this in two different ways.

The first and simplest path is a full rewrite. Cursor’s team describes it plainly: “By default, we have language models generate the fully rewritten file conditioned on the current file, the conversation history, and the current code block.” Every line comes back out of the model, whether it changed or not. That’s reliable, but it means cost and latency scale with file size rather than with the size of the actual edit. A one-line fix in a 2,000-line file still costs roughly what a 2,000-line file costs to generate.

The second path is what Cursor calls speculative edits, a custom decoding approach built specifically for code. In the company’s words, “Our biggest win comes from our custom speculative decoding algorithm called ‘speculative edits’.” Rather than generating every line from scratch, the model treats the existing file as a near-perfect draft, accepts unchanged spans speculatively, and only generates the material that actually changed. Cursor’s own framing of the result is direct: “It is equivalent to a full-file-rewrite, while being up to 9x faster.” A separate mechanism described on the same page skips line-numbered diffs entirely in favor of a different format, with Cursor noting “Instead of a standard diff format, the model proposes diff hunks as search/replace blocks.” That detail matters for anyone trying to detect which mode fired by reading raw model output, since there’s no single canonical diff syntax to grep for.

Cursor reports its fast-apply system runs on a 70-billion-parameter model at roughly 1,000 tokens per second, or about 3,500 characters per second, a figure a third-party technical comparison on dreaming.press repeats while cautioning that Cursor, Morph, and Relace each publish their own throughput numbers on different benchmarks, so none of the figures are directly comparable across vendors. That caveat is exactly why benchmarking your own repository matters more than quoting any single company’s number.

Characteristic	Diff / Patch Mode (Speculative Edits)	Full-File Rewrite Mode
Cost scaling	Scales with the size of the edit	Scales with the size of the whole file
Reported throughput	~1,000 tokens/sec on Cursor’s 70B apply model	Limited by the generating model’s normal token rate
Reported speed delta	Up to 9x faster than full rewrite, per Cursor	Baseline
Failure mode	Patch can fail to apply cleanly against a changed file	Can silently drop unrelated code or reformat untouched lines
Best suited for	Small, localized changes in large files	Large structural rewrites or new files

None of this is configurable through a single documented switch inside Cursor. The model picks a strategy based on the edit, the file, and internal heuristics Cursor hasn’t published as a fixed threshold. That’s the gap this tutorial fills: instead of guessing, you measure.

It also helps to be clear about what “Cursor” means in this context, since the product bundles several moving parts under one name. The apply mechanism described above is the layer that turns a plan into an actual file write. Sitting on top of it is Composer, Cursor’s own family of agentic models, which went through several generations over the course of 2026: Composer 1 shipped on October 29, 2025, Composer 1.5 followed on February 9, 2026 with adaptive thinking and self-summarization, Composer 2 landed on March 19, 2026 with a new base model and a reported fourfold increase in reinforcement-learning compute, and Composer 2.5 became the default model in Cursor’s picker on May 18, 2026. Each generation can reason differently about when a small patch versus a full regeneration is the right call, which is one more reason a one-time benchmark goes stale and a repeatable harness, wired into the CI job built in Step 12, earns its keep over time.

Prerequisites and Versions

• A Cursor account with CLI access and an API key (Pro or Business tier works, and the CLI authenticates independently of the desktop app login)
• Cursor CLI build 2026.10.14 or newer – check with agent --version after install
• Python 3.11 or newer
• Git 2.40 or newer
• macOS, Linux, or Windows via WSL (the installer targets POSIX shells)
• jq (optional, useful for inspecting JSON output in the terminal)
• About 90 minutes for the full setup, a first benchmark pass, and the CI wiring

Step 1: Install the Cursor CLI

Cursor ships its CLI as a single installer script. It installs an executable called agent (with a cursor-agent symlink kept for backward compatibility) under ~/.local/bin.

# macOS, Linux, or WSL
curl https://cursor.com/install -fsS | bash

# confirm it's on your PATH
agent --version

# if "command not found", add the install directory
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

Run agent --version again after fixing your PATH. You want a build dated October 2026 or later so the --output-format and --force flags used later in this project behave as documented.

Step 2: Authenticate the CLI for Headless Runs

Interactive login works for manual testing, but a benchmark harness needs to run unattended. Cursor’s CLI reads an API key from the CURSOR_API_KEY environment variable, which keeps credentials out of your shell history and out of the scripts themselves.

# generate a key from your Cursor account settings, then export it
export CURSOR_API_KEY="your_api_key_here"

# sanity check: a single headless prompt against an empty directory
mkdir -p ~/cursor-bench/sanity && cd ~/cursor-bench/sanity
git init -q
echo "print('hello')" > hello.py
git add . && git commit -q -m "init"
agent -p --force --trust --output-format json "Add a docstring to hello.py"

If that returns a JSON object instead of hanging on a permission prompt, your headless setup is working. --force (aliased as --yolo) auto-approves file edits and shell commands, and --trust skips the interactive workspace-trust check that otherwise blocks CI runs.

Step 3: Build an Isolated Benchmark Repository

Never benchmark against a real working repository. Cursor’s CLI with --force enabled will happily run shell commands and overwrite files, and a benchmark loop runs the same prompt dozens of times. Give it a throwaway sandbox instead.

mkdir -p ~/cursor-bench/sandbox && cd ~/cursor-bench/sandbox
git init -q
git config user.email "[email protected]"
git config user.name "Cursor Bench"
mkdir fixtures results
echo "fixtures/*.bak" >> .gitignore
git add . && git commit -q -m "bootstrap benchmark repo"

Step 4: Generate Small, Medium, and Large Test Fixtures

The whole point of this harness is to see whether file size changes which apply mode Cursor reaches for. Generate three Python files of roughly 50, 500, and 2,000 lines, each with the same repeating, syntactically valid pattern so the edit you request later is comparable across all three.

# fixtures/generate_fixtures.py
import pathlib

TEMPLATE = '''def handler_{n}(payload):
    """Process payload {n} and return a normalized result."""
    value = payload.get("value", 0)
    return {"id": {n}, "value": value * 2}

'''

SIZES = {"small": 12, "medium": 120, "large": 480}  # ~4 lines per function

def build(name, count):
    out = pathlib.Path(f"fixtures/{name}.py")
    body = "".join(TEMPLATE.format(n=i) for i in range(count))
    out.write_text(body)
    print(f"{name}: {len(body.splitlines())} lines written to {out}")

if __name__ == "__main__":
    for name, count in SIZES.items():
        build(name, count)

Run it once, commit the fixtures, and treat that commit as the reset point for every trial:

python3 fixtures/generate_fixtures.py
git add fixtures/*.py
git commit -q -m "add small/medium/large fixtures"
git tag fixtures-baseline

Step 5: Design a Fixed Prompt Matrix

Hold the edit constant across file sizes so file size is the only variable that changes. Use a small, well-defined, mechanical change: rename every function’s return key from value to normalized_value and add a type hint to the payload parameter. That’s small enough to be a plausible diff candidate in every file, and specific enough that you can verify correctness automatically afterward with a simple grep.

• Prompt text stays identical across small, medium, and large runs
• Model stays fixed for a given trial set (don’t mix models mid-benchmark)
• 30 to 50 repetitions per file-size bucket, matching the sample size that separates real signal from model-to-model variance

Step 6: Write the Benchmark Driver Script

This is the core of the project. It resets the fixture to baseline, times a single headless agent call, captures the JSON output, and hands the result off to a classifier you’ll write in the next step.

# bench/driver.py
import json
import subprocess
import time

PROMPT = (
    "In this file, rename the 'value' key in every returned dict to "
    "'normalized_value', and add a type hint 'dict' to the payload parameter "
    "of every function. Make no other changes."
)

def reset_fixture():
    subprocess.run(["git", "checkout", "fixtures-baseline", "--", "fixtures"], check=True)

def run_trial(fixture_path, model="composer-2.5"):
    reset_fixture()
    start = time.monotonic()
    proc = subprocess.run(
        [
            "agent", "-p", "--force", "--trust",
            "--model", model,
            "--output-format", "json",
            f"In {fixture_path}: {PROMPT}",
        ],
        capture_output=True, text=True, timeout=180,
    )
    elapsed_ms = round((time.monotonic() - start) * 1000, 1)
    try:
        payload = json.loads(proc.stdout.strip().splitlines()[-1])
    except (json.JSONDecodeError, IndexError):
        payload = {"raw": proc.stdout[-500:]}
    return {"elapsed_ms": elapsed_ms, "returncode": proc.returncode, "payload": payload}

Note the git checkout fixtures-baseline -- fixtures reset at the top of every trial. Skipping that step is the single most common way to corrupt a benchmark run, since the second trial would start from an already-edited file instead of the clean baseline.

Step 7: Classify Diff-Style vs Rewrite-Style Edits From Git

Cursor doesn’t expose a flag telling you which apply path fired. You infer it from the shape of the resulting diff. A speculative-edit / diff-style change touches a small, contiguous fraction of the file’s lines. A full rewrite tends to touch most or all of them, because the model regenerated the whole thing even if large stretches ended up byte-identical to the original.

# bench/classify.py
import subprocess

def rewrite_ratio(fixture_path):
    """Return the fraction of lines git considers changed, 0.0 to 1.0."""
    diff = subprocess.run(
        ["git", "diff", "--numstat", "fixtures-baseline", "--", fixture_path],
        capture_output=True, text=True, check=True,
    ).stdout.strip()
    if not diff:
        return 0.0
    added, removed, _ = diff.split("\t")
    total_lines = int(
        subprocess.run(["wc", "-l", fixture_path], capture_output=True, text=True)
        .stdout.split()[0]
    )
    touched = max(int(added), int(removed))
    return round(min(touched / max(total_lines, 1), 1.0), 3)

def classify(ratio, threshold=0.6):
    return "rewrite-like" if ratio >= threshold else "diff-like"

The 0.6 threshold is a starting heuristic, not a Cursor-documented constant – nobody outside Cursor has published the model’s real internal cutoff. Treat it as a tunable parameter and adjust it against what you actually observe once real data comes back from Step 9.

Step 8: Log Every Run to CSV

Wire the driver and classifier together and append each trial to a CSV file so you can analyze the full matrix afterward without re-running anything.

# bench/run_matrix.py
import csv
import pathlib
from driver import run_trial
from classify import rewrite_ratio, classify

BUCKETS = ["small", "medium", "large"]
REPS = 30
OUT = pathlib.Path("results/runs.csv")

def main():
    OUT.parent.mkdir(exist_ok=True)
    is_new = not OUT.exists()
    with OUT.open("a", newline="") as f:
        writer = csv.writer(f)
        if is_new:
            writer.writerow(["bucket", "rep", "elapsed_ms", "rewrite_ratio", "mode_guess"])
        for bucket in BUCKETS:
            fixture = f"fixtures/{bucket}.py"
            for rep in range(1, REPS + 1):
                result = run_trial(fixture)
                ratio = rewrite_ratio(fixture)
                mode = classify(ratio)
                writer.writerow([bucket, rep, result["elapsed_ms"], ratio, mode])
                print(f"{bucket} rep {rep}/{REPS}: {result['elapsed_ms']}ms, ratio={ratio}, mode={mode}")

if __name__ == "__main__":
    main()

Step 9: Run the Full Benchmark Matrix

With 30 reps across three buckets, expect 90 headless agent calls. On a typical broadband connection this takes somewhere between 20 and 45 minutes depending on the model and server load, so kick it off and let it run in the background rather than babysitting the terminal.

cd ~/cursor-bench/sandbox
python3 bench/run_matrix.py | tee results/run_log.txt

A sample of what the live output looks like while it’s running:

small rep 1/30: 842.3ms, ratio=0.91, mode=rewrite-like
small rep 2/30: 790.1ms, ratio=0.88, mode=rewrite-like
medium rep 1/30: 1340.7ms, ratio=0.22, mode=diff-like
medium rep 2/30: 1298.4ms, ratio=0.19, mode=diff-like
large rep 1/30: 2210.6ms, ratio=0.11, mode=diff-like
large rep 2/30: 2340.9ms, ratio=0.09, mode=diff-like

This is illustrative sample output from a single test machine, not a published Cursor benchmark, and your own numbers will move with network latency, account tier, and the model you select. What you’re watching for is the pattern, not the absolute milliseconds: small files tend to come back rewritten wholesale, while larger files are more likely to trigger a narrower, diff-style patch because regenerating the entire file stops being the cheap option.

Step 10: Read the Results Table

Aggregate the CSV with a short pandas script that reports median and 95th-percentile latency per bucket alongside the average rewrite ratio.

# bench/aggregate.py
import pandas as pd

df = pd.read_csv("results/runs.csv")
summary = df.groupby("bucket").agg(
    median_ms=("elapsed_ms", "median"),
    p95_ms=("elapsed_ms", lambda s: s.quantile(0.95)),
    avg_rewrite_ratio=("rewrite_ratio", "mean"),
    rewrite_like_pct=("mode_guess", lambda s: round((s == "rewrite-like").mean() * 100, 1)),
).reset_index()
print(summary.to_string(index=False))

Bucket	Lines	Median Latency	P95 Latency	Avg. Rewrite Ratio	Rewrite-Like Runs
Small	~50	810ms	1,120ms	0.89	93%
Medium	~500	1,310ms	1,980ms	0.24	17%
Large	~2,000	2,280ms	3,640ms	0.10	7%

Again, treat these as example output from one test run, not a universal Cursor spec sheet. What the table is designed to surface for your own codebase is the inflection point: the file size where Cursor stops reliably rewriting the whole thing and starts favoring a narrower patch. That inflection point is exactly what lets you decide, with your own data, where to tighten review on large diffs and where a quick full-file rewrite is cheap enough not to worry about.

Step 11: Block Accidental Full-File Rewrites With a Pre-Commit Guard

Once you know your classifier works, reuse it outside the benchmark. A pre-commit hook can flag any staged change where the rewrite ratio looks suspiciously high relative to the size of the requested edit, a cheap early warning that an agent rewrote a large file when a small patch would have done.

#!/usr/bin/env bash
# .git/hooks/pre-commit
THRESHOLD=0.6
for file in $(git diff --cached --name-only --diff-filter=M -- "*.py"); do
  stats=$(git diff --cached --numstat -- "$file")
  added=$(echo "$stats" | awk '{print $1}')
  total=$(wc -l < "$file")
  ratio=$(python3 -c "print(round(min($added/max($total,1),1.0),3))")
  if (( $(echo "$ratio >= $THRESHOLD" | bc -l) )); then
    echo "WARNING: $file looks fully rewritten (ratio=$ratio). Review before committing."
  fi
done
exit 0

Make it executable with chmod +x .git/hooks/pre-commit. It only warns by default, so change the final exit 0 to exit 1 once you’re confident the threshold isn’t producing false positives on legitimate large refactors.

Step 12: Wire the Benchmark Into GitHub Actions

Running the benchmark once tells you where things stand today. Running it on a schedule tells you when Cursor changes its apply behavior underneath you, which matters since the company updates its models without announcing that the diff/rewrite balance has shifted.

# .github/workflows/cursor-bench.yml
name: cursor-apply-benchmark
on:
  schedule:
    - cron: "0 6 * * 1"   # weekly, Monday 06:00 UTC
  workflow_dispatch: {}

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Install Cursor CLI
        run: curl https://cursor.com/install -fsS | bash
      - name: Run benchmark matrix
        env:
          CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }}
        run: |
          export PATH="$HOME/.local/bin:$PATH"
          pip install pandas
          python3 bench/run_matrix.py
          python3 bench/aggregate.py
      - uses: actions/upload-artifact@v4
        with:
          name: cursor-bench-results
          path: results/runs.csv

Store CURSOR_API_KEY as a repository secret, never as a plaintext value in the workflow file. The weekly cadence catches model-behavior drift without burning a full budget of API calls every day.

Step 13: Extend the Harness to Compare Models

The driver script already takes a model parameter. Loop over a short list of models available in your Cursor plan, such as a Composer variant and a frontier third-party model, and add a model column to the CSV. That turns this from a single-model latency check into a genuine comparison of apply behavior across the models you actually pay for, which is the question most teams really want answered: not “is Cursor fast,” but “which model, on my codebase, gives me the best cost-to-correctness tradeoff.”

Comparing CLI Behavior to the Desktop Composer Panel

Everything built in this tutorial talks to Cursor through the headless CLI, not the desktop editor’s Composer panel, and that distinction matters more than it looks. The CLI and the IDE both route requests through the same underlying apply infrastructure, but the surrounding context differs: the desktop app carries open-tab state, cursor position, and a running conversation history that can influence how a model frames its edit, while a CLI invocation starts fresh from a single prompt and a file on disk. Two developers debating whether Cursor “always rewrites small files” might both be right, if one is testing from the desktop panel mid-conversation and the other is testing from a clean CLI call.

If your team does most of its actual work in the desktop app, treat the CLI benchmark from this project as a reasonable proxy rather than an exact stand-in. It tells you how the underlying apply model behaves given a clean prompt and a known file, which is still useful for setting expectations and for the CI-based drift detection in Step 12. For a closer match to in-editor behavior, you’d need to replay representative multi-turn conversations rather than single-shot prompts, which is a reasonable next step once the basic harness is working and you want to go further than this tutorial’s scope.

Reading False Positives Out of the Classifier

The rewrite-ratio heuristic built in Step 7 is deliberately simple, and simple heuristics misfire in predictable ways. Before trusting a week of CI results, spend ten minutes checking a handful of runs by hand. Open a “rewrite-like” result and a “diff-like” result side by side in a real diff viewer and confirm the classification matches what you’d call it intuitively.

Two situations commonly produce a false “rewrite-like” verdict even when the underlying apply mechanism used a narrow patch. The first is a model that reindents or reflows code style as a side effect of an edit, which git counts as changed lines even though the logic didn’t move. The second is a fixture where the requested change happens to touch a line near the top and a line near the bottom of the file, which can push the numstat ratio high purely because of where the two edits landed, not because the model rewrote everything in between. Neither case means the harness is broken. It means the single-number ratio is a useful first-pass signal, not a courtroom-grade proof of which code path Cursor took internally.

Common Pitfalls When Benchmarking Diff vs Rewrite

• Forgetting to reset the fixture between runs. If trial 2 starts from the output of trial 1, your file sizes drift and every subsequent measurement is contaminated. Always git checkout back to the baseline tag first.
• Mixing models mid-benchmark. Different models have different apply heuristics layered on top of Cursor’s own system. Comparing a run on one model against a run on another and calling the difference “diff vs rewrite behavior” conflates two separate variables.
• Running too few repetitions. A single run per bucket can land on either side of the classifier threshold by chance. Treat anything under 10 reps per bucket as anecdote, not data.
• Ignoring whitespace normalization. Some models reformat untouched lines, like trailing whitespace or quote style, without changing their meaning, which inflates the rewrite ratio and produces false “rewrite-like” classifications. Strip or normalize whitespace before diffing if your fixtures are sensitive to it.
• Treating the CLI result as proof of UI behavior. The Cursor CLI and the Cursor desktop editor don’t necessarily route every request through identical apply logic. Numbers from this harness describe the CLI specifically, not a guarantee about what happens inside the IDE’s Composer panel.
• Leaving --force enabled against a real repository. Auto-approval mode is built for sandboxes. Running it against your actual codebase means Cursor can execute shell commands and apply file edits with nobody reviewing them first.

Troubleshooting Guide

• “agent: command not found” after install. The installer put the binary in ~/.local/bin, which isn’t always on PATH by default. Add it to your shell profile and re-source it.
• The CLI hangs waiting for input in CI. You’re missing -p / --print. Without it, the CLI assumes an interactive terminal session.
• Every run times out at 180 seconds. Large fixtures combined with a slow model can legitimately exceed a short timeout. Raise the timeout value in subprocess.run before assuming something is broken.
• JSON parsing fails on the last line of stdout. Some CLI versions print log lines before the final JSON payload. Filter stdout for lines starting with a brace before parsing, or switch to --output-format stream-json and parse the final event explicitly.
• Workspace-trust prompt blocks a headless run. Add --trust alongside --force. They control two different gates, approval versus workspace trust, and both need to be disabled for a clean automated run.
• Rewrite ratio comes back as exactly 0.0 on every run. This usually means the git diff --numstat command is being run against the wrong baseline ref. Confirm the fixtures-baseline tag actually points at the pre-edit commit.
• Results look inconsistent run-to-run on the same file. Some variance is expected since the underlying model isn’t deterministic. If variance is extreme, check whether you’re accidentally running against a shared, already-modified fixture instead of a freshly reset one.
• GitHub Actions run fails with an authentication error. Confirm the secret name in the workflow file matches exactly what’s stored in the repository settings, including case, and that the key hasn’t expired.
• The pre-commit hook never triggers. Hooks placed directly in .git/hooks/ aren’t tracked by git and don’t survive a fresh clone. For team-wide enforcement, use a hook manager like pre-commit or husky and commit the configuration instead.

Extending the Harness to Multi-File Agentic Runs

This tutorial’s benchmark deliberately keeps the scope to one prompt touching one file, because isolating variables is the entire point of a controlled experiment. Real work rarely stays that contained. A typical Cursor agent session touches three, five, or a dozen files in a single run, renaming a function in one place and updating every call site across the repository, and the diff-versus-rewrite question gets more interesting once multiple files are in play at once.

Extending the harness for this case means changing the fixture layout rather than the core driver logic. Build a small synthetic package with a handful of files that reference each other, point the same kind of mechanical, verifiable prompt at the whole package instead of a single path, and run rewrite_ratio() from Step 7 against every file that appears in the resulting git diff --name-only output, not just the one you expected to change. Two patterns tend to show up once you do this: a primary target file that gets a tight, diff-like patch, and secondary call-site files that get smaller, surgical one-line updates. Whether those secondary files ever trigger a full rewrite is itself useful data, since an agent treating a dependent file as “safe to regenerate” is exactly the kind of blast radius a sensitive-paths policy, like the one described in the advanced tips below, is meant to catch.

Advanced Tips for Production Use

Once the basic harness works, a few refinements make it genuinely useful for engineering decisions rather than a one-off curiosity.

Track rewrite ratio against real pull request diffs, not just synthetic fixtures. Pull the git history from a handful of past Cursor-assisted commits in a non-production branch, measure the actual ratio of lines changed to lines in the file, and compare that distribution to your synthetic benchmark. Synthetic fixtures are useful for isolating variables, but real-world prompts are messier, and your production risk profile should be calibrated against real diffs eventually.

Separate apply latency from end-to-end latency. The timing captured in this harness measures the full round trip, including planning time. If you want to isolate just the apply step the way Cursor’s own engineering benchmarks do, you’d need an API that exposes a planning/applying boundary, which the public CLI doesn’t currently expose. Treat this harness’s numbers as end-to-end UX latency, not a pure apply-model benchmark.

Use the rewrite-ratio signal to drive policy, not just observation. A mature version of this project tags file paths, say anything under src/billing/ or src/auth/, as high sensitivity, and pipes the pre-commit guard’s warning into a required review step rather than a soft console message when those paths see a rewrite-like change. That turns a passive benchmark into an actual guardrail for the parts of the codebase where an unreviewed full-file regeneration is the most expensive mistake.

Keep an eye on vendor-published numbers for context, not as ground truth for your own repo. Independent apply-model vendors like Morph and Relace publish their own throughput and accuracy claims for competing fast-apply systems, and Relace’s own cost illustration shows that rewriting a roughly 1,000-line file with a frontier model can take more than 100 seconds and cost at least $0.18 in token spend, a useful anchor for roughly how expensive the “always rewrite” fallback can get at scale. None of those figures describe Cursor’s system directly, and mixing vendor marketing numbers into an internal engineering decision is how teams end up optimizing for the wrong thing.

Finally, re-run the whole matrix after any Cursor CLI update. Release notes tracked by third-party trackers like thismonthinai.com show Cursor ships CLI and model changes frequently, and a model swap under the hood can shift your bucket thresholds without any announcement calling out the diff/rewrite balance specifically.

Frequently Asked Questions

Does Cursor let you force diff mode or rewrite mode manually?
Not through a single documented toggle in the CLI or the editor. The model selects an apply strategy based on the edit and the file, and Cursor hasn’t published a fixed control for overriding that choice run by run.

Is the Cursor CLI the same thing as cursor-agent?
Yes. agent is the current primary command, and cursor-agent is kept as a compatibility symlink installed by the same script.

Why does my rewrite ratio classifier disagree with what I expected?
The 0.6 threshold used in this tutorial is a starting heuristic, not something Cursor publishes. Run your own matrix first, look at the actual distribution of ratios in your results CSV, and adjust the threshold to match a real gap in your data rather than trusting a default number.

Can I run this benchmark against a private, non-Python codebase?
Yes. Swap the fixture generator for representative files in your own language and adjust the prompt to a mechanical, verifiable change in that language. The driver, classifier, and logging scripts don’t depend on Python being the target language.

How many trials do I actually need before I trust the results?
Thirty to fifty repetitions per file-size bucket is a reasonable floor for separating real signal from normal model variance. Fewer than ten per bucket should be treated as a rough first look, not a decision-making dataset.

Does a higher rewrite ratio always mean worse performance?
No. A full rewrite on a small file can be perfectly fine, and the benchmark results above show small files landing in rewrite-like territory far more often than large ones. The ratio is a signal for risk and cost, not a correctness judgment on its own.

Will this benchmark work with models other than Cursor’s own Composer line?
Yes, as long as the model is available through your Cursor plan and can be passed to the --model flag. Step 13 covers extending the harness to compare multiple models directly.

Is it safe to run this benchmark in GitHub Actions with –force enabled?
It’s safe specifically because the workflow operates on a disposable checkout of a sandboxed fixtures directory, not a live production path. Never point a --force-enabled headless run at a branch or directory containing code you haven’t already backed up elsewhere.