Every edit the Cursor AI code editor makes ends the same way: a model reply has to turn into a changed file on disk. The format of that reply, whether a full rewrite, a unified diff or search/replace blocks, decides how many output tokens you pay for and how the edit fails when the model slips. In the fuzzer you build below, one clean edit to a 4,000-line file costs far more output tokens as a whole-file rewrite than as a unified diff.
Cursor’s own engineers have described the trade-off. In a 2024 engineering post, the Cursor team wrote: âMotivated by the diff format from Aider, we eliminate the line number problem. Instead of a standard diff format, the model proposes diff hunks as search/replace blocks.â (Cursor engineering team, Editing Files at 1000 Tokens per Second). That post predates every model named in this tutorial, and the sources we could verify today do not document how the editor applies edits now. So this guide does not claim to reverse-engineer Cursor. It builds the test bench instead: an offline fuzzer that shows which slips each edit format survives.
You will build the project in 13 steps and about 50 minutes, with no API key and no network calls. If you want to inspect what your editor really changed after the fact, pair it with the Cursor edit tracer. The fuzzer answers the opposite question: how an edit format behaves before any damage reaches your repo.
Don't miss new tech stories on Google
Add Tech Insider once in the Google app and our stories appear in your news suggestions.
What the Cursor-Style Edit Fuzzer Measures (and What It Does Not)
The fuzzer takes one small, realistic change, such as swapping a log call inside a single function, and expresses it in three formats. It then corrupts the model’s reply in ways that real models are known to corrupt replies, feeds the result to an applier and scores what lands on disk. Every attempt ends in one of three outcomes.
- exact: the file after applying equals the intended target, byte for byte.
- rejected: the applier refused the edit with an error. This is a loud failure, and a human or a retry loop can act on it.
- corrupted: the applier reported success but the file differs from the target. This is a silent failure, and it is the number that matters most.
Silent corruption deserves the spotlight because it passes the checks people lean on. The patch applied, the file parses and most unit suites never touch the changed branch. A wrong-occurrence edit looks identical to a right one until someone reads the diff.
The fuzzer also tracks cost. Output tokens are the expensive side of an edit, and since October 12, 2026 new Cursor plans bill third-party models such as Claude and GPT at API list prices rather than from the bundled allowance, according to a Cursor pricing breakdown. A format that emits far fewer tokens is no longer a stylistic preference. It is a line item.
Be clear about the limits before you trust any percentage. The corpus is synthetic and repetitive on purpose. There is one edit shape. Token counts are estimates at four characters per token. The faults are injected by hand, not sampled from a live model. So the rates describe how a format and an applier respond to a given slip. They do not tell you how often any particular model slips.
Prerequisites and Versions for the Edit-Format Fuzzer
The project uses only the Python standard library, so setup takes minutes. Any editor works, including Cursor itself (if you are new to it, the walkthrough in How to Master Cursor AI covers installation and the basics).
| Requirement | Version | Notes |
|---|---|---|
| Python | Not listed | Built and run on 3.12.3. The code uses f-strings, dataclasses and list type hints only. |
| git | Not listed | Run on 2.43.0. Used to commit your baseline results in step 13. |
| Shell | bash, zsh or PowerShell | Commands below use bash. On Windows, replace python3 with py -3. |
| Third-party packages | None | The tests use unittest from the standard library, so there is nothing to pip install. |
| Disk and time | Under 1 MB, about 50 minutes | Not listed |
| API keys | None | Everything runs offline. The optional live-model hook is covered in the advanced tips. |
Confirm your versions before you start.
python3 --version
git --versionExpected output on the machine used for this article:
Python 3.12.3
git version 2.43.0Three Edit Formats a Cursor-Style Editor Can Use
A coding assistant such as the Cursor AI code editor has to bridge two worlds. The model produces text, and the editor needs a precise change to a file. Three formats dominate that bridge, and they sit at different points on the cost-versus-fragility curve.
A whole-file rewrite asks the model to output the entire updated file. There is no anchor to miss, so location errors cannot happen. The price is that the model regenerates every unchanged line, which costs tokens and gives it a chance to alter code it was never asked to touch.
A unified diff is the format git diff prints. It carries a header with line numbers and counts, a few lines of context and the removed and added lines. It is compact, but models are poor at counting lines, which is exactly the problem the Cursor quote above says its team designed around.
Search/replace blocks drop the line numbers. The model quotes the exact text to find and the text to put in its place. They are compact and easy for a model to write, but they depend on the quoted text matching the file exactly and matching only once.
| Property | Whole-file rewrite | Unified diff | Search/replace |
|---|---|---|---|
| What the model emits | The full updated file | Header, context lines, removed and added lines | A SEARCH block and a REPLACE block |
| Output grows with | File size | Edit size plus 3 context lines per side | Edit size plus enough context to be unique |
| How the editor finds the change | It does not need to | Line numbers in the hunk header | Exact text match in the file |
| Loud failure | Almost never | Hunk does not apply, bad counts | No match, ambiguous match |
| Silent failure | Dropped code, copy drift, changed line endings | Wrong location when matching is fuzzy | Wrong occurrence when the first match wins |
If you want a live-editor view of the same split, the Cursor CLI diff versus rewrite benchmark times real runs. The settings that nudge the editor toward one style are covered in the Cursor rules guide for diff and rewrite edits. This tutorial sits between the two. It holds the model constant (it is a script) so that the only variable is the format.
Steps 1 and 2: Project Skeleton and a Corpus Built to Break Anchors
Step 1: Create the project layout. Make a folder, a package directory and a tests directory.
mkdir editfuzz && cd editfuzz
mkdir editfuzz tests
touch editfuzz/__init__.py
lsYou should see two directories side by side:
editfuzz testsStep 2: Write the corpus generator. The corpus must punish lazy anchors, so every generated function repeats the same three-line shape. A search block that quotes only the log line will match every function in the file. Create editfuzz/corpus.py.
import random
from dataclasses import dataclass
TEMPLATE = [
"def handler_{i}(payload):",
" if payload is None:",
" return None",
" value = payload.get('key_{k}')",
" if value is None:",
" log.warning('missing value')",
" return None",
" return value * {m}",
"",
]
HEADER = ["import logging", "", "log = logging.getLogger(__name__)", ""]
@dataclass
class Case:
name: str
original: list
target: list
start: int
old: list
new: list
@property
def original_text(self):
return "".join(l + "\n" for l in self.original)
@property
def target_text(self):
return "".join(l + "\n" for l in self.target)
def make_source(n_lines, seed):
rng = random.Random(seed)
lines = list(HEADER)
i = 0
while len(lines) + len(TEMPLATE) <= n_lines:
lines += [t.format(i=i, k=rng.randint(0, 9), m=rng.randint(2, 9)) for t in TEMPLATE]
i += 1
return lines, i
def make_case(n_lines, position, seed):
lines, count = make_source(n_lines, seed)
idx = {"start": 0, "middle": count // 2, "end": count - 1}[position]
start = len(HEADER) + len(TEMPLATE) * idx + 5
old = [lines[start]]
new = [f" log.error('missing value in handler_{idx}')",
" metrics.incr('missing_value')"]
target = lines[:start] + new + lines[start + 1:]
return Case(f"{n_lines}L-{position}-s{seed}", lines, target, start, old, new)Two details deserve attention. First, make_case picks the target function by position (first, middle or last), which lets the fuzzer test edits at the start, middle and end of a file. Second, the edit replaces one line with two, so line counts change and the diff header has to be right.
A 40-line file holds 4 handler functions, a 400-line file holds 44 and a 4,000-line file holds 444. Those counts show up later as the number of matches an under-specified search block hits.
Step 3: Emit the Same Edit in Three Formats
The emitters play the part of a careful model. Each one produces the ideal reply for a case. Create editfuzz/formats.py.
def emit_whole(case):
return case.target_text
def emit_udiff(case, n=3):
s, e = case.start, case.start + len(case.old)
lo, hi = max(0, s - n), min(len(case.original), e + n)
body = ([" " + l for l in case.original[lo:s]] + ["-" + l for l in case.old]
+ ["+" + l for l in case.new] + [" " + l for l in case.original[e:hi]])
new_len = hi - lo - len(case.old) + len(case.new)
head = f"@@ -{lo + 1},{hi - lo} +{lo + 1},{new_len} @@"
return ("--- a/service.py\n+++ b/service.py\n" + head + "\n"
+ "".join(l + "\n" for l in body))
def sr_block(search, replace):
return ("<<<<<<< SEARCH\n" + "".join(l + "\n" for l in search)
+ "=======\n" + "".join(l + "\n" for l in replace)
+ ">>>>>>> REPLACE\n")
def count_block(lines, block):
n = len(block)
return sum(1 for i in range(len(lines) - n + 1) if lines[i:i + n] == block)
def emit_sr(case):
s, e = case.start, case.start + len(case.old)
lo = s
while count_block(case.original, case.original[lo:e]) != 1 and lo > 0:
lo -= 1
return sr_block(case.original[lo:e], case.original[lo:s] + case.new)
EMITTERS = {"whole": emit_whole, "udiff": emit_udiff, "sr": emit_sr}The search/replace emitter extends its quoted context upward one line at a time until the block matches exactly once, which is what a careful model does when it notices a repeated pattern. The diff emitter writes three lines of context on each side, matching the default of git diff -U3. If you want a production-grade take on applying these formats, the fast-apply patch engine build goes deeper on the speed side.
Sanity-check the emitters on the smallest case:
python3 - <<'EOF'
from editfuzz.corpus import make_case
from editfuzz.formats import emit_udiff, emit_sr
c = make_case(40, 'start', 0)
print(emit_udiff(c))
print(emit_sr(c))
EOFThe diff and the search block come out like this:
--- a/service.py
+++ b/service.py
@@ -7,7 +7,8 @@
return None
value = payload.get('key_6')
if value is None:
- log.warning('missing value')
+ log.error('missing value in handler_0')
+ metrics.incr('missing_value')
return None
return value * 4
<<<<<<< SEARCH
value = payload.get('key_6')
if value is None:
log.warning('missing value')
=======
value = payload.get('key_6')
if value is None:
log.error('missing value in handler_0')
metrics.incr('missing_value')
>>>>>>> REPLACENotice the blank context line at the bottom of the diff. It is a single space after the diff prefix, and it will cause trouble in the next step.
Step 4: Fault Injectors That Copy Real Model Slips
Now the fun part. Each mutation takes a clean reply and damages it in one specific, realistic way. Create editfuzz/mutate.py.
import re
from .formats import sr_block
def clean(text, case):
return text
def crlf(text, case):
return text.replace("\n", "\r\n")
def elide_tail(text, case):
lines = text.splitlines(keepends=True)
keep = case.start + len(case.new) + 10
if keep >= len(lines):
return text
return "".join(lines[:keep]) + "# ... rest of file unchanged ...\n"
def drop_final_newline(text, case):
return text.rstrip("\n")
def copy_drift(text, case):
lines = text.splitlines(keepends=True)
far = [i for i, l in enumerate(lines)
if "payload.get('key_" in l and abs(i - case.start) > 9]
if not far:
return text
i = far[len(far) // 2]
lines[i] = lines[i].replace("key_", "keys_", 1)
return "".join(lines)
def shift_hunk_start(text, case, by=3):
def bump(m):
return f"@@ -{int(m[1]) + by}{m[2] or ''} +{int(m[3]) + by}{m[4] or ''} @@"
return re.sub(r"^@@ -(\d+)(,\d+)? \+(\d+)(,\d+)? @@", bump, text, flags=re.M)
def blank_context_unpadded(text, case):
return "\n".join("" if l == " " else l for l in text.split("\n"))
def wrong_counts(text, case):
return re.sub(r"^@@ -(\d+),(\d+) ",
lambda m: f"@@ -{m[1]},{int(m[2]) + 1} ", text, flags=re.M)
def minimal_anchor(text, case):
return sr_block(case.old, case.new)
def _edit_search(text, fn):
head, tail = text.split("=======\n", 1)
return fn(head) + "=======\n" + tail
def strip_indent(text, case):
def fn(head):
first, *rest = head.split("\n")
return "\n".join([first] + [l.lstrip() for l in rest])
return _edit_search(text, fn)
def paraphrase_search(text, case):
return _edit_search(text, lambda h: h.replace("missing value", "missing val"))
MUTATIONS = {
"whole": [clean, elide_tail, drop_final_newline, copy_drift, crlf],
"udiff": [clean, shift_hunk_start, blank_context_unpadded, wrong_counts, crlf],
"sr": [clean, minimal_anchor, strip_indent, paraphrase_search, crlf],
}The catalog below ties each injector to the slip it imitates. The crlf injector is the one most teams discover late. If your repository mixes line endings, the gitattributes setup for Cursor diffs shows how to stop AI edits from rewriting every line.
| Format | Mutation | Slip it simulates |
|---|---|---|
| whole | elide_tail | The model writes a placeholder such as a rest-of-file-unchanged comment and drops the remaining code |
| whole | drop_final_newline | The reply ends without the trailing newline the file had |
| whole | copy_drift | A character changes in a line far from the edit, the classic copy error in a long rewrite |
| whole | crlf | The reply uses Windows line endings and the file used Unix ones |
| udiff | shift_hunk_start | The model miscounts and the hunk header points 3 lines too low |
| udiff | blank_context_unpadded | Blank context lines lose their single leading space |
| udiff | wrong_counts | The old-line count in the hunk header is off by one |
| udiff | crlf | The diff arrives with Windows line endings |
| sr | minimal_anchor | The SEARCH block quotes only the changed line, which appears in every function |
| sr | strip_indent | The SEARCH block loses its leading indentation |
| sr | paraphrase_search | The SEARCH block quotes a slightly reworded line |
| sr | crlf | The block arrives with Windows line endings |
Every format also keeps a clean baseline, so each run proves the happy path before it measures the sad ones.
Step 5: Strict and Lenient Appliers
An applier is where the policy lives. The same broken reply can be rejected by a strict applier and quietly accepted by a lenient one, and that difference is the whole point of the experiment. Create editfuzz/apply.py.
import re
class ApplyError(Exception):
pass
def _join(lines):
return "".join(l + "\n" for l in lines)
def apply_whole(original, text, policy="raw"):
if policy == "guarded":
if re.search(r"\.\.\.\s*(rest|remaining|unchanged|existing|same)", text, re.I):
raise ApplyError("elision marker in rewrite")
if not text.endswith("\n"):
raise ApplyError("missing final newline")
if len(text.splitlines()) < 0.7 * len(original.splitlines()):
raise ApplyError("rewrite shrank the file by more than 30%")
return text
SR_RE = re.compile(r"<<<<<<< SEARCH\r?\n(.*?)=======\r?\n(.*?)>>>>>>> REPLACE", re.S)
def _find(lines, block, key=lambda s: s):
n, kb = len(block), [key(b) for b in block]
return [i for i in range(len(lines) - n + 1) if [key(x) for x in lines[i:i + n]] == kb]
def apply_sr(original, text, policy="strict"):
lines = original.splitlines()
blocks = SR_RE.findall(text)
if not blocks:
raise ApplyError("no SEARCH/REPLACE block found")
for search, replace in blocks:
s, r = search.splitlines(), replace.splitlines()
hits = _find(lines, s)
if not hits and policy == "lenient":
hits = _find(lines, s, key=str.strip)
if not hits:
raise ApplyError("SEARCH block not found")
if len(hits) > 1 and policy == "strict":
raise ApplyError(f"SEARCH block ambiguous ({len(hits)} matches)")
at = hits[0]
lines[at:at + len(s)] = r
return _join(lines)
HUNK_RE = re.compile(r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@")
def _parse_hunks(text, policy):
hunks, cur = [], None
for raw in text.splitlines():
m = HUNK_RE.match(raw)
if m:
cur = {"at": int(m[1]) - 1, "n_old": int(m[2] or 1), "n_new": int(m[4] or 1),
"old": [], "new": []}
hunks.append(cur)
elif cur is None:
continue
elif raw.startswith("\\"):
continue
else:
tag, body = (raw[:1], raw[1:]) if raw else (" ", "")
if raw == "" and policy == "strict":
raise ApplyError("empty line inside hunk (blank context lost its leading space)")
if tag in " -":
cur["old"].append(body)
if tag in " +":
cur["new"].append(body)
return hunks
def apply_udiff(original, text, policy="strict"):
lines = original.splitlines()
hunks = _parse_hunks(text, policy)
if not hunks:
raise ApplyError("no hunks found")
delta = 0
for h in hunks:
if policy == "strict" and (len(h["old"]) != h["n_old"] or len(h["new"]) != h["n_new"]):
raise ApplyError("hunk header counts do not match the body")
at = h["at"] + delta
if lines[at:at + len(h["old"])] != h["old"]:
if policy == "strict":
raise ApplyError(f"hunk does not apply at line {at + 1}")
hits = _find(lines, h["old"], key=str.rstrip)
if not hits:
raise ApplyError("hunk context not found anywhere")
at = min(hits, key=lambda i: abs(i - at))
lines[at:at + len(h["old"])] = h["new"]
delta += len(h["new"]) - len(h["old"])
return _join(lines)
APPLIERS = {"whole": apply_whole, "udiff": apply_udiff, "sr": apply_sr}
POLICIES = {"whole": ["raw", "guarded"], "udiff": ["strict", "lenient"], "sr": ["strict", "lenient"]}Read the policies as a ladder of trust:
- whole / raw writes whatever the model sent. whole / guarded rejects an elision marker, a missing final newline or a file that shrank by more than 30%.
- udiff / strict requires the header counts to match the body and the old lines to sit exactly at the stated line number. udiff / lenient ignores the counts, treats an empty line as blank context and searches for the old lines, picking the match nearest the stated line.
- sr / strict demands exactly one exact match. sr / lenient falls back to a whitespace-insensitive match and, if several matches remain, takes the first. That last choice is the one to watch.
Steps 6 to 8: Round-Trip Tests, Scoring and the Command Line
Step 6: Prove the clean path works. Before fuzzing anything, confirm that every clean reply applies to the exact target under every policy. Create tests/test_roundtrip.py.
import unittest
from editfuzz.apply import APPLIERS, POLICIES, ApplyError
from editfuzz.corpus import make_case
from editfuzz.formats import EMITTERS, count_block
from editfuzz.mutate import minimal_anchor
class RoundTrip(unittest.TestCase):
def test_clean_edits_apply_exactly(self):
for size in (40, 400):
for pos in ("start", "middle", "end"):
case = make_case(size, pos, 1)
for fmt, emit in EMITTERS.items():
for policy in POLICIES[fmt]:
out = APPLIERS[fmt](case.original_text, emit(case), policy)
self.assertEqual(out, case.target_text, (size, pos, fmt, policy))
def test_minimal_anchor_is_ambiguous(self):
case = make_case(400, "middle", 1)
self.assertGreater(count_block(case.original, case.old), 1)
patch = minimal_anchor(EMITTERS["sr"](case), case)
with self.assertRaises(ApplyError):
APPLIERS["sr"](case.original_text, patch, "strict")
wrong = APPLIERS["sr"](case.original_text, patch, "lenient")
self.assertNotEqual(wrong, case.target_text)
if __name__ == "__main__":
unittest.main()The second test asserts something subtle. A minimal anchor must be rejected by the strict policy and must apply to the wrong place under the lenient one. If that test ever fails, your fuzzer has stopped modeling the failure it exists to catch.
Step 7: Write the scoring module. It runs every combination of size, position, seed, format, mutation and policy, and attaches a token estimate. Create editfuzz/score.py.
import math
from collections import defaultdict
from .apply import APPLIERS, POLICIES, ApplyError
from .corpus import make_case
from .formats import EMITTERS
from .mutate import MUTATIONS
SIZES = (40, 400, 4000)
POSITIONS = ("start", "middle", "end")
# USD per 1M output tokens. Replace with the prices you pay today.
PRICES = {"claude-haiku-5.5": 0.50, "composer-2.5": 2.50, "grok-4.7": 6.00}
def est_tokens(text):
return math.ceil(len(text) / 4)
def run_one(case, fmt, mutation, policy):
text = mutation(EMITTERS[fmt](case), case)
row = {"case": case.name, "fmt": fmt, "mutation": mutation.__name__,
"policy": policy, "tokens": est_tokens(text), "reason": ""}
try:
result = APPLIERS[fmt](case.original_text, text, policy)
row["outcome"] = "exact" if result == case.target_text else "corrupted"
except ApplyError as exc:
row["outcome"], row["reason"] = "rejected", str(exc)
return row
def run_all(seeds=5):
rows = []
for size in SIZES:
for pos in POSITIONS:
for seed in range(seeds):
case = make_case(size, pos, seed)
for fmt, mutations in MUTATIONS.items():
for mut in mutations:
for policy in POLICIES[fmt]:
r = run_one(case, fmt, mut, policy)
r["size"], r["pos"] = size, pos
rows.append(r)
return rows
def pct(rows, outcome):
return 100 * sum(r["outcome"] == outcome for r in rows) / len(rows) if rows else 0.0
def summarize(rows):
groups = defaultdict(list)
for r in rows:
groups[(r["fmt"], r["policy"])].append(r)
return groupsThe PRICES dictionary holds output prices in dollars per million tokens. The three values are Claude Haiku 5.5 at $0.50, Composer 2.5 at $2.50 and Grok 4.7 at $6.00, taken from each vendor’s published pricing as of October 2026. Prices change, so treat them as placeholders and edit them before you quote a cost to anyone.
Step 8: Write the report and command line. Create editfuzz/cli.py and a two-line editfuzz/__main__.py.
import argparse
import json
import sys
from collections import defaultdict
from pathlib import Path
from .apply import APPLIERS
from .score import PRICES, SIZES, pct, run_all, summarize
def md(headers, body):
out = ["| " + " | ".join(headers) + " |", "|" + "---|" * len(headers)]
out += ["| " + " | ".join(str(c) for c in row) + " |" for row in body]
return "\n".join(out)
def report(rows):
parts = []
clean = [r for r in rows if r["mutation"] == "clean"]
by = defaultdict(list)
for r in clean:
by[(r["fmt"], r["size"])].append(r["tokens"])
body = []
for fmt in ("whole", "udiff", "sr"):
cells = [fmt]
for size in SIZES:
v = by[(fmt, size)]
cells.append(round(sum(v) / len(v)))
body.append(cells)
parts.append("### Output tokens per clean edit (estimate: chars / 4)\n" +
md(["format"] + [f"{s} lines" for s in SIZES], body))
body = []
for fmt in ("whole", "udiff", "sr"):
v = by[(fmt, 4000)]
avg = sum(v) / len(v)
body.append([fmt] + [f"${avg * p / 1e6 * 1000:,.2f}" for p in PRICES.values()])
parts.append("### Output cost per 1,000 edits on a 4,000-line file\n" +
md(["format"] + list(PRICES), body))
body = []
for (fmt, policy), g in summarize(rows).items():
c = [r for r in g if r["mutation"] == "clean"]
m = [r for r in g if r["mutation"] != "clean"]
body.append([fmt, policy, f"{pct(c, 'exact'):.0f}%", f"{pct(m, 'exact'):.1f}%",
f"{pct(m, 'rejected'):.1f}%", f"{pct(m, 'corrupted'):.1f}%"])
parts.append("### Outcomes under fuzzing (mutated outputs only, except column 3)\n" +
md(["format", "policy", "clean exact", "exact", "rejected (loud)",
"corrupted (silent)"], body))
cell = defaultdict(list)
for r in rows:
if r["mutation"] != "clean":
cell[(r["fmt"], r["mutation"], r["policy"])].append(r)
body = [[f, mu, po, f"{pct(g, 'exact'):.0f}%", f"{pct(g, 'rejected'):.0f}%",
f"{pct(g, 'corrupted'):.0f}%"] for (f, mu, po), g in cell.items()]
parts.append("### Per-mutation breakdown\n" +
md(["format", "mutation", "policy", "exact", "rejected", "corrupted"], body))
return "\n\n".join(parts)
def main(argv=None):
ap = argparse.ArgumentParser(prog="editfuzz")
sub = ap.add_subparsers(dest="cmd", required=True)
run = sub.add_parser("run")
run.add_argument("--seeds", type=int, default=5)
run.add_argument("--out", default="out")
app = sub.add_parser("apply")
app.add_argument("--format", choices=sorted(APPLIERS), required=True)
app.add_argument("--policy", default="strict")
app.add_argument("--file", required=True)
app.add_argument("--patch", required=True)
args = ap.parse_args(argv)
if args.cmd == "run":
rows = run_all(args.seeds)
out = Path(args.out)
out.mkdir(exist_ok=True)
(out / "results.json").write_text(json.dumps(rows, indent=1))
text = report(rows)
(out / "report.md").write_text(text + "\n")
print(text)
print(f"\n{len(rows)} attempts written to {out}/")
else:
original = Path(args.file).read_text()
patch = Path(args.patch).read_text()
sys.stdout.write(APPLIERS[args.format](original, patch, args.policy))
if __name__ == "__main__":
main()from .cli import main
main()Run the tests:
python3 -m unittest discover -s tests -vExpect green results:
test_clean_edits_apply_exactly (test_roundtrip.RoundTrip.test_clean_edits_apply_exactly) ... ok
test_minimal_anchor_is_ambiguous (test_roundtrip.RoundTrip.test_minimal_anchor_is_ambiguous) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.010s
OKSteps 9 and 10: Run the Fuzzer and Read Its Output
Step 9: Run the full matrix. Run five seeds across three file sizes and three positions, for every format-and-mutation pair under both policies, and log each attempt.
python3 -m editfuzz run --seeds 5Step 10: Read the tables. The command prints four tables and writes them to out/report.md, with every raw attempt in out/results.json. The first two cover cost.
Token cost: the gap grows with file size
| format | 40 lines | 400 lines | 4000 lines |
|---|---|---|---|
| whole | Not listed | Not listed | Not listed |
| udiff | 76 | 76 | 76 |
| sr | 77 | 103 | 103 |
A whole-file rewrite costs many times a diff, and the gap widens as the file grows from 40 to 400 to 4,000 lines. The diff stays flat because the edit is the same size no matter how large the file grows. Search/replace sits slightly higher for large files because it quotes extra context to stay unique.
| format | claude-haiku-5.5 | composer-2.5 | grok-4.7 |
|---|---|---|---|
| whole | Not listed | Not listed | Not listed |
| udiff | $0.04 | Not listed | Not listed |
| sr | $0.05 | $0.26 | Not listed |
In the PRICES table, a thousand whole-file edits on a 4,000-line file cost far more than the same thousand edits as diffs, with Haiku 5.5 and with Grok 4.7 alike. This comparison counts output tokens only and ignores input, caching and retries.
Silent versus loud failures
| format | policy | clean exact | exact | rejected (loud) | corrupted (silent) |
|---|---|---|---|---|---|
| whole | raw | 100% | Not listed | 0.0% | Not listed |
| whole | guarded | 100% | Not listed | Not listed | 50.0% |
| udiff | strict | 100% | 25.0% | 75.0% | 0.0% |
| udiff | lenient | 100% | 100.0% | 0.0% | 0.0% |
| sr | strict | 100% | 25.0% | 75.0% | 0.0% |
| sr | lenient | 100% | Not listed | 25.0% | Not listed |
Four rows tell the story. Raw whole-file rewrite corrupted a large share of the mutated attempts without raising a single error. Adding the guard turned some of them into rejections, but many still slipped through, because a guard that checks for placeholders and newlines cannot see a changed character or a flipped line ending. Strict diff and strict search/replace never corrupted anything. They paid for that with a rejection rate of 75.0% on mutated replies.
The lenient rows carry the trade-off. Lenient diff recovered every mutated reply in this corpus. Lenient search/replace recovered only some replies exactly, rejected some and silently corrupted some. The corrupted share is the cost of letting the first match win.
Where lenient mode bites
| format | mutation | policy | exact | rejected | corrupted |
|---|---|---|---|---|---|
| whole | elide_tail | raw | 33% | 0% | Not listed |
| whole | elide_tail | guarded | 33% | Not listed | 0% |
| whole | drop_final_newline | raw | 0% | 0% | 100% |
| whole | drop_final_newline | guarded | 0% | 100% | 0% |
| whole | copy_drift | raw | 0% | 0% | 100% |
| whole | copy_drift | guarded | 0% | 0% | 100% |
| whole | crlf | raw | 0% | 0% | 100% |
| whole | crlf | guarded | 0% | 0% | 100% |
| udiff | shift_hunk_start | strict | 0% | 100% | 0% |
| udiff | shift_hunk_start | lenient | 100% | 0% | 0% |
| udiff | blank_context_unpadded | strict | 0% | 100% | 0% |
| udiff | blank_context_unpadded | lenient | 100% | 0% | 0% |
| udiff | wrong_counts | strict | 0% | 100% | 0% |
| udiff | wrong_counts | lenient | 100% | 0% | 0% |
| udiff | crlf | strict | 100% | 0% | 0% |
| udiff | crlf | lenient | 100% | 0% | 0% |
| sr | minimal_anchor | strict | 0% | 100% | 0% |
| sr | minimal_anchor | lenient | 33% | 0% | Not listed |
| sr | strip_indent | strict | 0% | 100% | 0% |
| sr | strip_indent | lenient | 100% | 0% | 0% |
| sr | paraphrase_search | strict | 0% | 100% | 0% |
| sr | paraphrase_search | lenient | 0% | 100% | 0% |
| sr | crlf | strict | 100% | 0% | 0% |
| sr | crlf | lenient | 100% | 0% | 0% |
Look at minimal_anchor under the lenient policy: only some replies were exact, and the rest were corrupted. The exact ones are the cases where the edit sat in the first function, so the first match happened to be the right one. Everywhere else the edit landed in the wrong function and nothing complained. In the middle and end of the file the applier picked a different handler than the one the model meant.
Note two more patterns. copy_drift and crlf corrupt 100% of whole-file attempts under both whole-file policies, so the guard does not help against them. For catching drift in live sessions rather than in a fuzzer, the Cursor Plan Mode drift checker is the better tool. And the crlf injector is harmless to both patch formats, because the appliers split lines themselves and rebuild the file with the original’s endings. A rewrite inherits whatever endings the model chose.
Steps 11 to 13: Replay a Real Reply, Set Your Prices and Gate the Result
Step 11: Replay a pasted reply with the apply command. The fuzzer is more useful when you can feed it something real. Copy a search/replace reply from any assistant into reply.txt, save the target file as service.py and call apply. The example below reproduces the wrong-occurrence case from the corpus: an edit meant for handler_22 with a one-line anchor.
python3 - <<'EOF'
from editfuzz.corpus import make_case
from editfuzz.formats import EMITTERS
from editfuzz.mutate import minimal_anchor
c = make_case(400, "middle", 1)
open("service.py", "w").write(c.original_text)
open("reply.txt", "w").write(minimal_anchor(EMITTERS["sr"](c), c))
EOF
cat reply.txt<<<<<<< SEARCH
log.warning('missing value')
=======
log.error('missing value in handler_22')
metrics.incr('missing_value')
>>>>>>> REPLACEApply it under the strict policy first:
python3 -m editfuzz apply --format sr --policy strict --file service.py --patch reply.txteditfuzz.apply.ApplyError: SEARCH block ambiguous (44 matches)The strict applier refuses and tells you why. Now apply the same reply leniently and diff the result against the original:
python3 -m editfuzz apply --format sr --policy lenient --file service.py --patch reply.txt > patched.py
diff service.py patched.py10c10,11
< log.warning('missing value')
---
> log.error('missing value in handler_22')
> metrics.incr('missing_value')Line 10 belongs to handler_0. The new log message says handler_22. The applier reported success, the file still parses and the edit is wrong. That is silent corruption in four lines of output, and it is exactly what the table above counted.
Step 12: Set the prices you actually pay. Open editfuzz/score.py and edit PRICES. Use the output price per million tokens for each model you run, and keep the units consistent. As a reference point, Anthropic lists Claude Haiku 5.5 at $0.50 per million output tokens for prompts up to 100,000 tokens, per the Claude Haiku 5.5 announcement. Cursor lists Composer 2.5 at $0.50 per million input and $2.50 per million output tokens, and xAI lists Grok 4.7 at $6 per million output tokens for prompts under 200,000 tokens.
Step 13: Add a gate and commit a baseline. A fuzzer only protects you if it runs. Create tests/test_gate.py so that any change to an applier that lets a strict policy corrupt a file fails the build.
import unittest
from editfuzz.score import pct, run_all, summarize
class Gate(unittest.TestCase):
def test_strict_policies_never_corrupt(self):
for (fmt, policy), rows in summarize(run_all(seeds=2)).items():
if policy == "strict":
self.assertEqual(pct(rows, "corrupted"), 0.0, (fmt, policy))
def test_guard_beats_raw_on_whole_file(self):
groups = summarize(run_all(seeds=2))
self.assertLess(pct(groups[("whole", "guarded")], "corrupted"),
pct(groups[("whole", "raw")], "corrupted"))
if __name__ == "__main__":
unittest.main()python3 -m unittest discover -s tests
git init && git add -A && git commit -m "Edit-format fuzzer baseline"Expect four passing tests. When you tune an applier later, rerun python3 -m editfuzz run --seeds 5, commit out/report.md and let the diff of that file show what your change did to the corrupted column.
What the Numbers Say About Rewrite vs Diff Edits
For anyone who leans on the Cursor AI code editor every day, the fuzzer supports four conclusions, and each carries a caveat from the limits listed earlier.
Cost favors patches on anything but tiny files. The 3-times gap at 40 lines is small enough to ignore. The gap at 4,000 lines is not. If your assistant rewrites large files wholesale, you pay for every unchanged line, and the bill scales with file size rather than with the size of your request.
A rewrite never misses an anchor, and that is its real strength. There is no location to get wrong, so whole-file replies never produced a loud failure in the clean runs. The same property is the weakness: with nothing to reject, every slip lands on disk. A reviewer reading only the changed lines in the editor’s diff view will catch a dropped tail. A flipped character twenty lines away is far easier to miss.
Line numbers are a useful tiebreaker. Lenient diff scored 100% here partly because the stated line number picks the nearest of several identical sequences. The Cursor quote above describes dropping line numbers to avoid miscounting, and this fuzzer shows the flip side: dropping them removes the tiebreaker, so search/replace has to earn uniqueness from its quoted text.
Strict by default, lenient by exception. The corrupted column is zero for both strict patch policies, so strictness converts silent failures into retries. A retry costs one more reply, which at Haiku 5.5 output prices is a fraction of a cent for a diff. A silent wrong-occurrence edit costs a review cycle. If you want to review rewrites after the fact, the Cursor Composer rewrite audit walks through it, and the software category collects the rest of our editor and tooling guides.
One more point on scope. Cursor’s current lineup, per the editor’s own documentation, includes Grok 4.7 as its flagship, Composer 2.5 and the opt-in Claude Haiku 5.5. None of those pages document which edit format the editor uses behind the scenes, so no number in this article describes Cursor’s internals. They describe the formats.
Common Pitfalls When You Benchmark AI Edit Formats
These traps catch most teams the first time they compare edit formats, whether the target is the Cursor AI code editor or any similar tool.
- Treating a clean apply as a correct apply. The lenient search/replace rows applied without errors and still corrupted the file. Always compare the result with the intended target, not just the exit code.
- Testing only small files. At 40 lines the rewrite costs 3 times a diff. At 4,000 lines it costs far more. A benchmark on a toy file hides the cost curve that decides real bills.
- Using unique anchors in your corpus. If every quoted line appears once, search/replace looks flawless. Real code repeats guard clauses, imports and error handling, so build repetition in on purpose, as the corpus here does.
- Trusting the character-based token estimate for billing. Dividing characters by four is fine for ratios. It will not match a vendor’s tokenizer, and code often tokenizes differently from prose. Swap in the real tokenizer before you publish dollar figures.
- Reading injected-fault rates as model quality. A 100% corruption rate for
copy_driftmeans the format has no defense against that slip. It does not mean any model makes the slip 100% of the time. - Ignoring line endings. A mixed-endings repository turns a harmless-looking rewrite into a diff that touches every line. See the Cursor fast-apply safety net for a review workflow that catches this before commit.
- Making the lenient policy the default. Recovery is attractive because it turns rejections into successes. Check the corrupted column before you switch it on.
Troubleshooting the Edit-Format Fuzzer
| Symptom | Likely cause | Fix |
|---|---|---|
| ModuleNotFoundError: No module named ‘editfuzz’ | You ran the command from the wrong directory | Run from the project root and use python3 -m editfuzz, not python3 editfuzz/cli.py |
| SEARCH block not found on a clean reply | The file has tabs, trailing spaces or CRLF endings that the quoted block lacks | Normalize the file first, or use the lenient policy to match on stripped lines |
| SEARCH block ambiguous (44 matches) | The anchor is too small and the pattern repeats | Add a unique line, such as the def line, above the change |
| hunk does not apply at line N | The hunk header’s line number is wrong | Regenerate the diff with git diff, or use the lenient policy to search for the old lines |
| hunk header counts do not match the body | The model miscounted lines in the header | Recompute the counts from the body, or use the lenient policy |
| empty line inside hunk | Blank context lines lost their leading space | Repair the diff by padding empty lines with a space before applying |
| elision marker in rewrite | The model wrote a rest-of-file placeholder | Reject the reply and re-prompt for a diff or search/replace format |
| missing final newline | The reply ended without a trailing newline | Append a newline, or reject if you want strict fidelity |
| Percentages differ from this article | A different –seeds value, or an edited corpus or price table | Run python3 -m editfuzz run –seeds 5 on the unmodified code |
| Token counts look low next to your tokenizer | est_tokens divides characters by four | Replace est_tokens with a call to the tokenizer for the model you pay for |
If a failure does not fit the table, open out/results.json and filter by the reason field. Every rejection records the exact message the applier raised, so a quick count of reasons shows which slip dominates. In the reference run, the most common reason was a missing SEARCH block, followed by four families of 45 each: a missing final newline, an empty line inside a hunk, mismatched header counts and ambiguous search blocks.
Advanced Tips: Extend the Fuzzer Toward Real Cursor AI Code Editor Workloads
- Compose faults. Real replies often carry two slips at once, such as stripped indentation plus a reworded line. Loop over pairs of mutations from the same format and see which combinations break the lenient policies.
- Add more edit shapes. The corpus changes one line into two. Add pure deletions, multi-hunk changes, renames across the file and edits near the very first or last line, where context windows get clipped.
- Tighten the lenient search policy. Instead of taking the first match, rank candidates by distance to a hint such as the nearest function name. Re-run and watch the corrupted figure fall.
- Hook up a live model. Replace the emitters in
formats.pywith a function that sends the original file and an instruction to the model you use, then pass the reply through the same mutators and appliers. Keep the seeds fixed so runs compare cleanly. - Review the result structurally. A line diff will not tell you whether a rewrite moved code around. Try Difftastic on Cursor AI edits, and add a linter pass as a final check on whatever the appliers accept.
Finish by committing a fresh out/report.md every time you change an applier or add a mutation. The report is only about 50 lines, and the git history of that file becomes a record of how your edit pipeline’s safety changed over time.
FAQ: Cursor AI Code Editor Edit Formats and the Fuzzer
Does this tell me which edit format Cursor uses today?
No. The fuzzer tests generic formats against an offline applier. The sources verified for this article do not document the editor’s current apply behavior, so nothing here describes Cursor’s internals. A 2024 Cursor engineering post described search/replace blocks in its design, and that is the extent of what the evidence supports.
Is a whole-file rewrite always worse than a diff?
No. It cannot miss an anchor. Its weakness grows with file size, and with its inability to reject anything it should reject.
Diff hunks carry a line number, so the applier can choose the closest of several identical sequences. Search/replace has no such hint, so with several matches the applier falls back to the first. Larger line-number errors than the 3-line shift injected here would narrow the gap.
How many seeds do I need?
The default of 5 seeds covers the full matrix. Raise the seed count if you add mutations or edit shapes.
Can I run the fuzzer against a live model?
Yes. Swap the emitters for calls to your model, or paste a single reply into reply.txt and use the apply command. Keep prompts and seeds fixed, and report both token counts and the corrupted rate, not just the success rate.
Why do the patch formats ignore CRLF but the rewrite does not?
The appliers split the file and the patch into lines and rebuild the result with the original’s line endings. A whole-file reply is written as sent, so any ending the model chooses becomes the new ending.
How accurate are the dollar figures?
They are estimates. Tokens are counted at four characters each, only output tokens are priced, and the prices are the October 2026 list prices for three models. Replace both the estimator and the price table before using the figures in a budget.
