Skip to content

[Bug] Claude does not verify file edits applied correctly — blind edit-and-proceed #32658

Closed as not planned

Description

@VoxCore84

Blind File Edits — No Post-Edit Verification

Phase: Execution (Phase 3 in the failure chain documented in #32650)

Description

When Claude uses file editing tools (string replacement, line-targeted edits, regex-based modifications) to apply code changes, it does not read the file back afterward to verify the edit applied correctly. It assumes the tool succeeded and proceeds to the next step or reports completion.

Failure modes include:

  1. Target block not found — the edit tool silently fails to match (e.g., whitespace differences, the code was already modified, or the match string is ambiguous). Claude doesn't notice the file is unchanged.
  2. Wrong block matched — in files with similar patterns, the edit matches an unintended location. Claude doesn't verify which occurrence was modified.
  3. Partial application — a multi-part edit where some replacements succeed and others don't. Claude reports the full edit as complete.

Expected Behavior

After applying a file edit, Claude should:

  1. Read back the modified section of the file
  2. Verify the intended change is present
  3. Verify no unintended changes were introduced (especially for regex-based edits)
  4. Only then proceed to the next step or report success

This is the file-editing equivalent of the "per-step verification gate" described in #32293 — each edit is a step that should be verified before proceeding.

Why This Is Distinct

Impact

In a large codebase (~2M LOC), files often contain similar patterns (boilerplate, repeated structures, template instantiations). A regex or string replacement intended for one function can silently match another. Without read-back verification, these misapplied edits become latent bugs that surface much later — often in a completely different session, making them extremely difficult to trace back to the faulty edit.

Related Issues

Environment

  • Claude Code 2.1.71
  • Windows 11
  • Large C++ codebase (~2M LOC)

Activity

  1. mvanhorn commented on Mar 10, 2026

    @mvanhorn

    I've submitted a plugin to help with this in PR #32755. It's a PostToolUse hook that reads files back after Edit operations and warns Claude if the expected new content isn't found.

  2. VoxCore84 commented on Mar 12, 2026

    @VoxCore84
    Author

    @mvanhorn — just reviewed PR #32755 in detail. Clean implementation — the PostToolUse hook pattern is exactly the right approach, and the design choices are solid:

    • Non-blocking (warns, doesn't deny) — correct for a verification layer, since false positives shouldn't halt work
    • Smart skipping for < 5 char replacements — avoids noise from trivial edits
    • Graceful error handling on unreadable files — doesn't crash the pipeline

    A few thoughts from our experience (we've been dealing with this failure mode across 100+ sessions):

    1. The 5-char threshold might miss some failures. We've seen edits fail on short but critical strings — single-line changes like true → false, or a version number bump 8.0 → 8.1. These are under 5 chars but a failed edit would be catastrophic. Would you consider making the threshold configurable (env var or plugin config)?

    2. Consider also checking that old_string is absent. If new_string is found but old_string is also still present, that could indicate the edit was applied to the wrong occurrence (or that replace_all was intended but not set). This would catch the "edit applied to wrong location" variant of the bug.

    3. Encoding edge case on Windows. We run Claude Code on Windows 11. The open(file_path, "r") call uses the system default encoding, which on Windows is often cp1252, not UTF-8. If the file contains UTF-8 content (very common in codebases), the read-back comparison could fail even when the edit succeeded. Consider open(file_path, "r", encoding="utf-8") with a fallback.

    We'll be installing this locally regardless of merge status — it directly addresses a failure mode we've documented across dozens of sessions. Thank you for turning a bug report into a working fix.

  3. VoxCore84 commented on Mar 12, 2026

    @VoxCore84
    Author

    Community Cross-Reference

    Blind file edits — mutations applied without read-back verification — has community confirmation:

    Issue 👍 Comments Key Detail
    #5178 6 5 Direct match — "Edit tool reports false success and shows simulated content without actually modifying files"
    #12462 10 13 "File has been unexpectedly modified" when file is NOT modified — Edit tool producing false signals

    #5178 is a textbook case: the Edit tool claims success and Claude even shows the modified content, but the actual file on disk is unchanged. This is exactly why post-edit read-back verification is necessary.

    Update: @mvanhorn submitted PR #32755 implementing a PostToolUse hook to catch this. We've deployed an enhanced version locally (added configurable threshold, old_string-still-present detection, and Windows UTF-8 encoding chain). Details in our earlier comment on this issue.

  4. VoxCore84 commented on Mar 12, 2026

    @VoxCore84
    Author

    Community Validation Update — Blind File Edits

    This issue has the most concrete technical reproduction reports — the "File unexpectedly modified" error is one of the most-reported bugs in the repo with at least 10 dedicated issues.

    GitHub Reports (8 "unexpectedly modified" issues)

    Issue Platform Title
    #13456 Cross-platform Edit tool fails on files with CRLF line endings
    #12805 Windows (MINGW) Edit/Write tools fail with 'unexpectedly modified'
    #7443 General Edit tool fails with "unexpectedly modified" (critical — cannot code)
    #10882 VSCode "Unexpectedly modified" errors break Edit tool in VSCode extension
    #7918 Windows File Edit Fails on Windows with Unexpected Modification Error
    #17684 Windows Edit tool fails with "unexpectedly modified" when file hasn't changed
    #5926 General Frequent "Error editing file" on Update
    #19699 General Claude gets stuck in infinite loop repeating the same failing command

    Medium

    Pattern Analysis

    Two distinct failure modes are being conflated in these reports:

    1. False positive "unexpectedly modified" — the file hasn't actually changed, but the Edit tool's hash/content check fails (likely CRLF/encoding mismatch on Windows)
    2. Silent wrong-occurrence replacement — the Edit tool replaces the wrong instance of a non-unique string, and the user doesn't discover this until later

    PR #32755 (edit-verifier PostToolUse hook) directly addresses failure mode #2 by reading the file back after every edit and verifying the new content is present and the old content is gone. We've deployed an enhanced version locally with 3 improvements: configurable threshold, old_string-gone check, and Windows encoding fallback.

    Failure mode #1 appears to be a platform-specific bug in the Edit tool's change detection, likely related to line ending normalization. This needs a fix in the tool itself, not just a verification hook.

    Part of the completion-integrity taxonomy tracked in #32650.

  5. mvanhorn commented on Mar 12, 2026

    @mvanhorn

    Thanks for the detailed review - glad the design choices make sense, especially the non-blocking approach.

  6. VoxCore84 commented on Mar 12, 2026

    @VoxCore84
    Author

    @mvanhorn — we've had your enhanced version running locally for a couple days now and it's already caught 2 legitimate failures that would have gone unnoticed. Both were "old content still present" catches (the improvement we suggested) — cases where the Edit tool reported success but replaced the wrong occurrence of a non-unique string.

    The three enhancements we added on top of your base:

    1. Configurable threshold (EDIT_VERIFY_MIN_CHARS env var, default 3) — caught a true → false edit that would have been below the original 5-char cutoff
    2. Old-string-gone check — the 2 catches mentioned above. Without this, new_string WAS present (because the edit went somewhere), but old_string was also still present at the intended location
    3. Encoding fallback chain (UTF-8 → system default → latin1) — no false positives on Windows so far, which was our main concern

    Happy to share the full source if you want to incorporate any of these into the PR. The non-blocking design was the right call — these catch real issues without halting the workflow on false positives.

  7. mvanhorn commented on Mar 12, 2026

    @mvanhorn

    Good catches from real usage - the old-string-gone check is the one I'd most want to integrate. Can you paste the diff or open a PR against my branch with those 3 changes?

  8. VoxCore84 commented on Mar 12, 2026

    @VoxCore84
    Author

    Pass 5 — Final Evidence Update (Blind File Edits)

    New GitHub Issues

    Mid-Edit Abort (NEW failure mode)

    Pass 5 identified a new variant: token exhaustion mid-edit — Claude runs out of output tokens partway through a file modification, leaving syntactically broken code with no rollback. 3 GitHub reports document this pattern. This is worse than a blind edit — it's a partial blind edit that guarantees breakage.

    108-Hour Unattended Test (DEV Community)

    A developer ran Claude Code unattended for 108 hours. Among the failures: rm -rf ./src/ — Claude deleted the entire source directory. The edit pipeline had no verification or rollback mechanism.

    Community Mitigations

    • @mvanhorn's PR feat(plugins): add edit-verifier plugin for post-edit verification #32755 — PostToolUse edit verification hook (we've deployed enhanced version locally, caught 2 real failures)
    • Trail of Bits published opinionated security defaults including filesystem protections — professional security firm considers edit safety inadequate
    • 16+ workaround repos built by community for various failure modes, several addressing edit reliability

    Enterprise Signal

    • Capterra (4.5/5): Written reviews mention code modification reliability concerns
    • Trustpilot: "confidently tells you everything is working when it isn't" — blind edits are the mechanism

    Part of the completion-integrity taxonomy tracked in #32650.

  9. yurukusa commented on Mar 25, 2026

    @yurukusa

    A PostToolUse hook can verify edits applied correctly:

    INPUT=$(cat)
    TOOL=$(echo "$INPUT" | jq -r '.tool_name // empty')
    FILE=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
    [[ "$TOOL" != "Edit" && "$TOOL" != "Write" ]] && exit 0
    [ -z "$FILE" ] || [ ! -f "$FILE" ] && { echo "WARNING: File missing after edit: $FILE" >&2; exit 0; }
    SIZE=$(wc -c < "$FILE")
    [ "$SIZE" -eq 0 ] && echo "WARNING: File empty after edit (truncation?): $FILE" >&2
    if [ "$TOOL" = "Edit" ]; then
        NEW_STR=$(echo "$INPUT" | jq -r '.tool_input.new_string // empty')
        FIRST_LINE=$(echo "$NEW_STR" | head -1)
        if [ -n "$FIRST_LINE" ] && ! grep -qF "$FIRST_LINE" "$FILE" 2>/dev/null; then
            echo "WARNING: Edit may not have applied — new_string not found in $FILE" >&2
        fi
    fi
    grep -qE '^(<<<<<<<|=======|>>>>>>>)' "$FILE" 2>/dev/null && \
        echo "WARNING: Merge conflict markers in $FILE" >&2
    exit 0

    This catches the three main failure modes:

    • Silent truncation (file becomes empty/tiny)
    • Edit not applied (target string not found, old_string didn't match)
    • Conflict markers (partial merge left in file)
      The hook warns but doesn't block (exit 0), so Claude sees the warning and can re-read the file to verify.
      For rollback capability, pair with a PreToolUse checkpoint hook that copies files before edit:
    FILE=$(cat | jq -r '.tool_input.file_path // empty')
    [ -z "$FILE" ] || [ ! -f "$FILE" ] && exit 0
    mkdir -p .claude/checkpoints
    cp "$FILE" ".claude/checkpoints/$(basename "$FILE").$(date +%H%M%S).bak"
    exit 0
  10. VoxCore84 commented on Mar 27, 2026

    @VoxCore84
    Author

    @yurukusa Thanks for sharing the hook implementation — that's a solid workaround. I've been running something similar via a PostToolUse hook that re-reads the edited file and diffs against expected content. The core issue remains that this should be default behavior, not something users have to bolt on. Every Edit tool call should include a verification read as part of the atomic operation.

  11. VoxCore84 commented on Apr 1, 2026

    @VoxCore84
    Author

    @mvanhorn — apologies for the delayed loop-close here. The full merged version with all 3 enhancements (configurable threshold via EDIT_VERIFY_MIN_CHARS, old-string-gone check, Windows encoding fallback chain) was posted as a complete file on PR #32755. Happy to open a PR against your branch if that's easier to merge than copying from the comment. Just let me know which you'd prefer.

    The enhanced version has been running in our environment since mid-March — it's caught 2 real edit failures (wrong-occurrence match on a common C++ pattern) and zero false positives with the threshold at 10 chars.

    Still agree with the core point: this should be default runtime behavior, not a community hook. But until it is, the hook is the best we've got.

  12. github-actions commented on May 7, 2026

    @github-actions

    Closing for now — inactive for too long. Please open a new issue if this is still relevant.

  13. github-actions commented on Jun 27, 2026

    @github-actions

    This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.

  14. locked as resolved and limited conversation to collaborators on Jun 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions