Coder Legion - Recent posts tagged custody

The Slop Gap: Why Rising Model Capability Hasn't Solved Code Quality

Tue, 28 Jul 2026 02:03:56 +0000

The Slop Gap: Why Rising Model Capability Hasn't Solved Code Quality And what happens when you turn “slop” into a falsifiable engineering hypothesis

The gap nobody's benchmark shows

Model capability on coding tasks has risen sharply. The useful question is not whether those scores are high; it is what they actually measure.

On February 23, 2026, OpenAI disclosed flaws in SWE-bench Verified, including task-specific information that let some models reproduce gold-patch behavior. Its audit found material test or issue-description problems in at least 59.4 percent of the 138 tasks it reviewed.

The contemporaneous top-line movement, 74.9 to 80.9 percent, is therefore not a clean measure of general autonomous engineering ability. It is evidence that benchmark numbers need provenance, split, scaffold, and budget attached before they can carry much interpretive weight. 1

Contamination-resistant evaluations such as SWE-bench Pro report materially lower performance, but aggregating their results into a single 23-to-59-percent range would hide the conditions that matter: the model, public or private split, agent scaffold, and reasoning budget.

That contamination-resistant label is now itself in question: days before this piece went out, OpenAI ran the same kind of audit on SWE-bench Pro and found roughly 30 percent of its tasks broken in ways that misrepresent model capability, retracting its own recommendation that the field treat Pro as the leading coding eval. This is the second major coding benchmark OpenAI has walked back in five months. 2

A related distinction is even more important for this note. GBQA asks an agent to discover a bug rather than repair an already named issue; its best reported setting found only 48.39 percent of verified bugs. Repairing a known defect and finding an unnamed one are different capabilities. 3
That is the gap this note is about. Generation capability and detection capability are not the same curve, and right now they are not even close.

Why slop keeps shipping while models keep improving

The persistence of slop is not evidence that models are simply weak. It is evidence that generation and verification scale differently.

Review is the bottleneck.
Faros AI's observational study of more than 10,000 developers across 1,255 teams associated high AI adoption with more completed work and merged PRs, but also more review time, larger PRs, and more bugs per developer. Faster generation can move the queue into review rather than create trustworthy throughput. 4
Local success is not architectural judgment.
Current coding agents are generally more reliable on local, well-scoped edits than on cross-cutting decisions involving dependencies, historical conventions, and architectural intent (Imai, 2022). 5
Repository context is more than file volume.
Even when a repository fits within a large context window, an agent may not recover its implicit conventions and task-relevant dependencies reliably. A locally correct change can still violate a poorly represented boundary. 4
Supply-chain plausibility is still risky.
In a 2026 package-hallucination replication, 53 of 127 package names hallucinated by all five tested models remained registerable. That supports separate existence checks; it does not validate every dependency-quality metric. 6

The result is familiar to maintainers: code can look plausible enough to pass a hurried review while still creating structural work for the next engineer.

What "slop" means here

The term has a broader social meaning than this tool can measure. "Why Slop Matters" describes a family resemblance including superficial competence, asymmetric effort, and mass producibility. 7

For this detector, we operationalize only the first: the mismatch between superficial competence and underlying substance. The other two remain central to the ecosystem-level maintainer problem, but are outside static analysis.

That distinction matters. Open-source maintainers have reported AI-generated contributions consuming review capacity without producing valid fixes 8, while developer-LLM studies document automation bias and related review risks. 9 Those are reasons to improve inspection; they are not evidence that one score has measured all of “AI slop.”

Scope: a review-risk hypothesis, not a definition of AI slop

The detector does not measure AI authorship, developer intent, or software quality as a whole. It operationalizes a narrower hypothesis: some observable structural mismatches are useful, auditable risk signals for review prioritization.

The indicators are adjusted through repository-local operational feedback. That can reduce recurring false positives and align review priorities with local conventions, but it does not independently establish construct validity or predictive validity. A change in an engineer's behavior is not a quality label. It can reflect attention, urgency, convention, or avoidance as well as a correct diagnosis.
This distinction matters because the model can flag poor human-written code and miss polished AI-written code. It is not a detector of provenance. It is a deterministic, inspectable hypothesis about structural review risk.

So we operationalized a structural-risk hypothesis

Our answer was not to define slop as a binary label. It was to operationalize four candidate observables for structural review risk. Each is inspectable, deterministic, and contestable; none is a validated measure of software quality by itself.

Logic Density Ratio (LDR): logic_lines / total_lines. It is a proxy for files whose apparent size overstates executable substance. It can be wrong for schemas, adapters, declarative configuration, type contracts, and test fixtures, so role classification and explicit skips matter.
Inflation: effective jargon density adjusted by local complexity. It is a proxy for rhetorical or documentary claims that outpace the visible implementation. It is especially vulnerable to domain vocabulary, which is why a jargon score is a review prompt rather than a verdict.
Dependency Usage Ratio (DDC): used_runtime_imports / runtime_imports. It measures whether declared runtime imports participate in executable code, excluding annotation-only imports, TYPE_CHECKING branches, and intentional re-exports. Package existence and registry validity are evaluated separately through phantom-import rules; DDC alone cannot detect an invented package that is imported and used.
Critical-pattern burden: purity = exp(-0.5 * n_critical_patterns). This gives critical findings a continuous effect inside the composite. The current implementation also applies a severity-weighted additive pattern_penalty over the same issue list, including critical findings. That is explicit severity amplification, not an independent fifth measurement, and it is a design choice that requires empirical evaluation rather than rhetorical justification.

The score therefore has four structural proxies plus an additive policy term. The output retains a deficit_breakdown, fired rules, and a suppression ledger so that a reviewer can inspect the reason for a score instead of treating it as an oracle.
The final composite is not an arithmetic mean. It is a weighted geometric mean because a weak dimension should retain more influence than it would under an arithmetic average. This is the production method in src/slop_detector/core.py:

weights = self.config.get_weights()
w_ldr = weights.get("ldr", 0.40)
w_inf = weights.get("inflation", 0.30)
w_ddc = weights.get("ddc", 0.20)
w_pur = weights.get("purity", 0.10)

gqg = exp((
    w_ldr * log(max(1e-4, ldr.ldr_score))
    + w_inf * log(max(1e-4, 1.0 - inflation_normalized))
    + w_ddc * log(max(1e-4, ddc.usage_ratio))
    + w_pur * log(max(1e-4, purity))
) / (w_ldr + w_inf + w_ddc + w_pur))

base_deficit_score = 100 * (1 - gqg)
deficit_score = min(base_deficit_score + pattern_penalty, 100.0)

The floor at 1e-4 is not cosmetic. It keeps the log transform defined while retaining strong sensitivity to weak dimensions. A weighted geometric mean is more weak-link-sensitive than an arithmetic average, while still allowing limited compensation across signals. The initial 0.40 / 0.30 / 0.20 / 0.10 weights are design defaults, not externally estimated parameters.

What the model does in practice

The detector runs a bounded operational loop: scan a file or repository, decompose the score into LDR, inflation, DDC, and pattern pressure, inspect the dominant signal, patch or narrow the finding, then rescan. The difference between consecutive runs is an operational signal for review, not evidence that quality improved.
Project-scoped history records repeated scans. A changed file whose score falls past the configured threshold is recorded in code as an improvement label; in this article it is an improvement candidate. A stable flagged file becomes an fp_candidate. Neither is an independent quality label.

How adjudicated findings can inform the next review cycle

A structured report can direct attention to imports, empty paths, role boundaries, critical patterns, and explicit suppressions that ordinary generation may miss. But a finding does not become useful merely because the detector emitted it.
It becomes useful after a reviewer accepts, rejects, or narrows it. An accepted finding may become a patch or test. A rejected finding may become a role rule, suppression rationale, or explicit repository exception. In the current implementation, those decisions can be preserved through project history, configuration, and suppression records. They are not yet guaranteed to be retrieved automatically into every future agent context.

A finding became useful only after it was adjudicated, recorded, and made available as a repository boundary for the next review cycle.
Adjacent work such as Reflexion 10, Self-Refine 11, and RepoCoder 12 suggests that explicit feedback and retrievable local context can influence later agent behavior without weight updates. That supports an evaluation hypothesis, not a result already demonstrated by this detector.

The appropriate test would compare a baseline coding workflow with a report-and-adjudication workflow, then measure recurring boundary violations, human-accepted changes, review burden, and false-positive handling. A lower detector score alone would not count as evidence.

What first-party dogfooding did, and did not, show

We exercised the detector across internal codebases spanning deterministic governance workflows, mathematical computing, BioAI pipelines, and finance-facing backend work. The useful observation was that the same static signal can mean different things in different roles: orchestration can be sparse but load-bearing; scientific notation can be dense but necessary; defensive validation can be repetitive but justified.
That is why the implementation has role rules, exclusions, domain anchors, and a suppression ledger. These mechanisms make the boundary of a finding explicit: not simply “this pattern is bad,” but “this pattern is suspicious unless this file role, domain convention, or documented exception applies.”

It is not an external experiment. The codebases share one project ecosystem, with no blinded labels, independent reviewers, published precision/recall, or baseline comparison. The experience shows that false-positive boundaries can be revised in varied internal work; it does not show that the score measures a unified latent condition, predicts defects, or generalizes.

The next useful artifact is a public case record: detector warning, human acceptance criterion, reason the proxy failed, boundary change, rescan result, trade-off, and residual uncertainty. Until such evidence exists, this remains an operational account rather than a validation study.

The calibration algorithm

The weighted geometric mean is only the scoring surface. The operational-adjustment mechanism in src/slop_detector/ml/self_calibrator.py is repository-scoped and history-driven.

Every recorded run carries a project_id, so local history is not mixed by default.
The automatic milestone is based on files scanned more than once, not raw scan count, because only repeated files can create a before-and-after record.
The engine requires at least five improvement events and five fp_candidate events before searching for a new profile.
Search is constrained around the domain anchor, limiting arbitrary drift from the current configuration.
The code field is named confidence_gap, but it is not a statistical confidence interval. It is the objective separation between the best and second-best weight candidates; if the separation is too small, the engine returns insufficient_data.
Git metadata filters obvious measurement noise on both labels. Neither is an independent ground-truth signal: a score drop can be gamed, and an unchanged file can just as easily reflect a lower-priority task as a genuine false positive.
Those claims are also enforced in code. The calibration path first refuses thin evidence, then compares candidate profiles by error and a tie-break, and finally refuses to choose when the apparent winner is not meaningfully separated:

improvements = [e for e in events if e.label == "improvement"]
fp_candidates = [e for e in events if e.label == "fp_candidate"]

if len(improvements) < min_imp or len(fp_candidates) < min_fp:
    return CalibrationResult(status="insufficient_data")

candidates.sort(key=lambda c: (c.combined_score, c.tiebreak_score))
winner = candidates[0]

if len(candidates) >= 2:
    runner_up = candidates[1]
    primary_gap = runner_up.combined_score - winner.combined_score
    if abs(primary_gap) < 0.0001:
        confidence_gap = (runner_up.tiebreak_score - winner.tiebreak_score) / max(
            1.0, abs(winner.tiebreak_score)
        )
    else:
        confidence_gap = primary_gap
else:
    confidence_gap = 1.0

if confidence_gap < CONFIDENCE_GAP:
    return CalibrationResult(status="insufficient_data")

The production path additionally rejects a score drop inside the same Git commit as measurement noise, and rejects a stable file whose surrounding commit changed as ambiguous. These are useful safeguards for operational adjustment. They do not convert behavioral history into an external quality label.
The practical result is narrower than a self-improving quality model. A team can start with a default profile, run the detector against real repository changes, and use recurring local behavior to review whether a signal is over- or under-sensitive.
That is the part that turns a formula into an inspectable operational artifact rather than a one-size-fits-all lint rule. It does not prove the formula is valid. It makes the formula, its thresholds, its suppressions, and its revisions available for challenge.

The implementation and its limits are open source

The logic described above, the proxy definitions, the weighted geometric composite, the disclosed additive severity term, and the operational-adjustment path are released as open source in the AI-SLOP Detector repository. The validation boundary is released with them. Open source does not validate a metric; it makes the implementation, its assumptions, and its failure modes available for inspection and independent testing.

Why operationalize a concern at all?

A soft concern in prose is difficult to compare, challenge, or reproduce. An explicit heuristic gives a reviewer observable inputs, a decision rule, an evidence trail, and a specific term to dispute. That is useful for triage, not a substitute for tests, formal verification, change control, dual approval, or human review.
The engineering advantage is auditability, not automatic truth. A scalar score can prioritize inspection; a deterministic gate can prohibit an unsafe transition; a human may remain the final authority. An explicit heuristic can still be consistently wrong, but its assumptions are visible enough to test, revise, or reject.

Conclusion

Capability is rising, but benchmark scores do not establish that a model can find unseen faults or preserve architectural quality. Review remains the scarce resource, and plausible code can consume it without earning trust.
This project does not claim to have mathematically solved AI slop. It makes a narrower claim: structural review concerns can be expressed as deterministic, auditable proxy signals. In this implementation, LDR, inflation, DDC, and critical-pattern burden are combined with a weak-link-sensitive geometric composite and an explicitly disclosed additive severity term that may overlap with the critical-pattern signal. Repository-local history can adjust review sensitivity, while explicit data floors, Git noise filters, and objective separation between candidate profiles can refuse weak recommendations.
That is operational calibration, not independent validation. The tool has not yet shown that its score measures a unified latent condition, predicts defects or maintenance cost, outperforms simpler baselines, or generalizes beyond its development environment. Those are empirical questions requiring blinded labels, external reviewers, out-of-sample repositories, and published failures.

The contribution for now is an inspectable engineering hypothesis: a way to make a vague concern explicit enough to compute, audit, dispute, suppress with a record, and test against stronger evidence. A number is not a quality bar by itself. It becomes useful only when its assumptions remain visible and challengeable.

References

OpenAI, "Why SWE-bench Verified No Longer Measures Frontier Coding Capabilities," February 23, 2026
OpenAI, "Separating Signal From Noise in Coding Evaluations," July 2026 (SWE-bench Pro audit and retraction)
"GBQA: A Game Benchmark for Evaluating LLMs as Quality Assurance Engineers," arXiv:2604.02648
Farrag, "The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software Development," arXiv:2605.01160 (May 2026)
Imai, S., "Is GitHub Copilot a Substitute for Human Pair-Programming? An Empirical Study," ICSE '22 Companion Proceedings, 319–321 (2022)
Churilov, "The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort," arXiv:2605.17062
Kommers, Duede, Gordon, Holtzman, McNulty, Stewart, Thomas, So, and Long, "Why Slop Matters," ACM AI Letters, Vol. 1 (2026), arXiv:2601.06060
"An Endless Stream of AI Slop: The Growing Burden of AI-Assisted Software Development," arXiv:2603.27249 (March 2026)
Zhou, Saghi, Sabouri, Pandita, McGuire, and Chattopadhyay, "Cognitive Biases in LLM-Assisted Software Development," arXiv:2601.08045 (2026)
Shinn et al., "Reflexion: Language Agents with Verbal Reinforcement Learning," arXiv:2303.11366
Madaan et al., "Self-Refine: Iterative Refinement with Self-Feedback," arXiv:2303.17651
Zhang et al., "RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation," arXiv:2303.12570
Zechner and Ronacher, interview on "vibe slop," The Wall Street Journal, May 23, 2026, as reported by InfoWorld
Ronacher, "Pi: The Minimal Agent Within OpenClaw," lucumr.pocoo.org, January 31, 2026
Harding and Kloster, "Coding on Copilot," GitClear, 2024/2025 longitudinal analysis

We Dug Up Our Own Fake-Physics Artifact — and Rebuilt It Into Something You Can Check

Sat, 20 Jun 2026 05:12:06 +0000

Why there is a second post

About six months ago we built and published the original QSOT artifact. The name — Quantum State Over Time — promised a deeper link between quantum evolution and temporal structure than the thing underneath could actually support.

Then last week we published a post-mortem with an uncomfortable title: "We Made a High-Formality, Fake Physics Slop Artifact." A quiet admission that we had been wrong, and that the original claims did not survive scrutiny.

It was not a marketing move and not self-flagellation. It was a confession: we had shipped a repository with a DOI, two LaTeX papers, a project page, release notes, plots, and tests — the full visual grammar of serious science — around a core claim that was hollow underneath.

That post diagnosed the failure. It did not repair it. This is the repair.

If we made the mess, we clean it up.

The repair is called QSOT2. It is smaller, narrower, and far less ambitious than the original — and that is the point: the goal is no longer to look impressive, only to keep the claims we can actually justify.

You do not need to have read the first post. I will rebuild the context and show what changed, using evidence rather than adjectives.

For readers who never met QSOT

QSOT stands for Quantum State Over Time — a real idea from the physics literature (Lie & Fullwood) that treats time inside quantum mechanics as a first-class object rather than an external dial.

Our implementation of that idea — QSOT Compiler v1 — tried to map curved-spacetime backgrounds (Schwarzschild, de Sitter, AdS, Gödel) into single-qubit noise channels, then report whether the channels stayed physically valid, whether curvature showed up as decoherence, and whether quantum-information signals changed between backgrounds.

That is an ambitious sentence. It was ambitious in the wrong order.

The foundations were real — standard quantum-channel mathematics, Kraus operators, CPTP conditions. Those are not in dispute. The problem was everything we stacked on top of them.

What was actually wrong

The original problem was not that the repository looked broken. It was the opposite: it looked credible. The code ran, the paper compiled, the plots looked scientific, and the project carried the surface markers of serious work.

But the chain from claim to evidence was not disciplined enough.

The clearest example was the project's theoretical heart: Kirkwood-Dirac negativity. In the actual run, that number was a placeholder. So was the memory-depth signal. The curved-spacetime metrics were named in the papers, but not executed in the way the prose implied.

The form had outrun the evidence. QSOT2 begins by refusing to let that happen again.

Why dig up our own grave six months later

Two reasons, one of them not flattering.

The honorable one: by 2026 we had built executable verification, reproducibility surfaces, hash-based provenance, and a public ledger. The old artifact stopped feeling like an embarrassment and started feeling like an obligation — we made the mess, we finally had the tools to understand it, so cleaning it was our job.

The other: curiosity. Criticising a bad artifact after the fact is easy. Taking the same domain and producing something tighter, smaller, and able to survive its own tooling is not. We wanted to know whether the newer skill was real. QSOT2 was the test.

What QSOT2 changed — and it was not a new theorem

The most important change in QSOT2 is not a result. It is a boundary.

Boundary first. QSOT2 is not a quantum-gravity result. It is not a first-principles derivation. It is a bounded mathematical-consistency verifier for one implemented phenomenological model.

QSOT v1 tried to be a new-physics paper, a quantum-gravity gesture, a governance platform, and a math verifier at once. QSOT2 refuses all of that and asks one question:

Does the implemented model stay internally consistent — as quantum mechanics and as software — and are its numbers honest about what they are?

Smaller question. Much better question.

1. The math matured by subtraction, not ambition

QSOT2 did not make the model bolder. It made its mathematical status clean, by removing every place the old version could pretend:

the curvature-to-noise map is stated plainly — p = 1 - exp(-alpha * ||Riemann||_F), with alpha a free, dimensionless calibration parameter, not a derived constant;
the curvature inputs are disclosed as non-derived toy proxies, so the product alpha * ||Riemann||_F carries no hidden physical-unit claim;
Kirkwood-Dirac is demoted from "contextuality witness" to a bounded, flat-relative diagnostic;
the memory proxy is demoted from "full reconstruction" to a self-tested one.

The curvature tensors are not smuggled in as derived physics: they are explicitly labelled toy proxies in the artifact (a per-background riemann_is_toy flag), so the model cannot silently upgrade itself into a physical claim.

This is what strengthening looks like when the problem was epistemic inflation, not numerical instability. You do not fix inflation by computing harder. You fix it by removing the claims the math could never support.

2. The code stopped being "simulation" and became the witness

In QSOT2 the code's job is not to simulate. It is constraint enforcement — it fixes what may be claimed, what must be measured, what can be re-run, what gets hashed, and what is allowed to fail.

A paper can write "the axioms hold" for free. QSOT2 instead emits, per run: linearity deviation 1.1e-16, CPTP completeness 1.6e-16, trace preservation 4.4e-16 — all at machine epsilon — across 35 deterministic checks (34 pass / 1 degraded / 0 fail). The rule behind it:

Do not let prose carry a claim that the code cannot witness.

There is an architectural version too: the math layer is forbidden from importing the reporting layer — the separation that was missing in v1 is now enforced by the code, not promised by prose.

3. The negative result is advertised, not hidden

!6.png

The overall verdict is DEGRADED_PASS, on purpose: the flat-baseline KD optimizer does not converge inside its fixed 50-step budget — meaning it was still improving when the budget ran out, not that it diverged, so the reported value remains interpretable within the fixed-budget run.

Instead of rounding up, the run says so — in the verdict, the schema, the README, and the paper. The difference between an artifact that wants to impress and one that wants to be trusted often lives in exactly this footnote.

4. The workflow became a claim-custody pipeline

The workflow ties it together: every claim travels a gated path —

claim → code check → artifact → canonical-LF hash → paper line → DOI.

One command runs the experiment and emits one machine-readable result under a fixed schema with a four-field claim boundary; one command runs 50 tests behind a ≥ 90% coverage gate (currently ~96%); the line is DOI-archived on Zenodo (QSOT2 v0.1.2). None of this makes the model physically true — it makes the artifact auditable.

One detail matters here: the reproducibility claim is not hand-waved. The result is anchored to a canonical-LF SHA-256; the known non-byte-stable field is generated_at, and that exception is disclosed rather than hidden. The sensitivity numbers live in a separate committed sweep artifact, and the paper maps each claim back to a named file and hash.

The paper is the formal boundary record

The paper is not decoration. In the original artifact, formal writing made the work look stronger than the evidence underneath. In QSOT2, the paper has the opposite job: it records the boundary of the claim.

Its title says the scope plainly: A Mathematical-Consistency Verifier for a Phenomenological Curved-Spacetime Quantum-Channel Model. The paper does not present QSOT2 as a quantum-gravity result, a first-principles derivation, or an externally validated physical model. It presents it as an executable internal-consistency check for one implemented ansatz.

The structure follows that boundary. It starts with scope and background, then moves through temporal axiom checks, curvature-noise baselines, the flat-relative KD diagnostic, the TTM-inspired memory proxy, consolidated results, explicit non-claims, and reproducibility.

That order matters. The paper does not ask the reader to trust a polished argument. It asks the reader to follow a constrained chain: what the model claims, what the code checks, what the result reports, what the artifact refuses to claim, and how the run can be reproduced.

The abstract already carries the important limits: the curvature-to-channel mapping is a conceptual ansatz with a free calibration parameter; the curvature norms are non-derived toy proxies; the product alpha * ||Riemann||_F carries no physical-unit claim; and the reference run ends in DEGRADED_PASS for one documented reason, the flat-baseline KD optimizer’s fixed-budget non-convergence.

So the paper is not where QSOT2 becomes more impressive. It is where QSOT2 becomes harder to overclaim.

If the repository is the executable surface, and the public ledger is the receipt surface, the paper is the formal boundary surface: the place where the claim is written down in a form that can be checked against the code, the result artifact, and the hash trail.

The receipt: every v1 failure → a measurable

The honest way to prove a repair is to put the old defect and the new measurement side by side. The three that mattered:

1. Placeholders → bounded diagnostics

v1: KD negativity and non-Markovianity were both exactly 0.0.
QSOT2: kd_delta = +0.1230; self-test nm = 1.41e-4 (must fire on injected backflow) vs. model nm = 0.
Means: reproducible, model-relative comparisons — not proofs of contextuality or curvature-induced decoherence.

2. Named spacetime labels → explicit model inputs

v1: the curved-spacetime metrics were named in the papers, absent from the run.
QSOT2: curvature drives the channel across six backgrounds; purities verified — Schwarzschild 0.99943, de Sitter 0.63607, AdS5 0.67764, Eguchi-Hanson 0.99887.
Means: the behaviour of a phenomenological ansatz — not a derived spacetime result.

3. Asserted axioms → verified axioms

v1: temporal and axiomatic properties were stated, not checked.
QSOT2: five temporal-state axioms verified as algebraic invariants, deviations ≤ 4.4e-16.
Means: the model's quantum mechanics is internally consistent under its assumptions. That is the whole claim.

Read this section as the remediation map. The next question is where that map can be inspected outside this essay.

The public ledger is the receipt surface

The public Flamehaven Verification Ledger is where this repair becomes inspectable outside the essay.

The QSOT2 entry is recorded as EQA-TEST-0058, a verification run dated 2026-06-17: "QSOT2: Mathematical-Consistency Verifier (re-scoped honest successor of the 0057 slop)."

That entry states the result directly: DEGRADED_PASS (34 / 0 / 0 / 1). It also states why: the KD flat-baseline optimizer did not converge within its fixed budget. The limitation is surfaced, not hidden.

The ledger record makes the repair checkable in four directions:

JSON — the raw machine-readable result;
Report — the human-readable verification note;
Repo — the source implementation;
Paper — the formal boundary record.

In about three minutes, a reader can move from the ledger card to the raw JSON, the report, the repository, and the paper, and see the same claim boundary repeated across all of them: model-output consistency only, not external physical validity.

That is the point of the ledger. It is not a marketing page. It is a public verification record for Flamehaven's governance reviews, physics verification runs, and biomedical audit scans. Each published run is meant to be deterministic, locally executed, and inspectable from repository state.

The ledger answers one question: where can the evidence be inspected? The next question is narrower and more important: what exactly does that evidence support?

What QSOT2 can verify — and what it refuses to claim

Stated plainly, because vagueness is where slop hides.

QSOT2 verifies, and proves per run:

the implemented single-qubit channels are CPTP (completeness error at machine epsilon);
evolved states stay valid density matrices (Hermitian, unit-trace, positive semidefinite);
five temporal-state axioms hold to ≤ 4.4e-16;
the reported purities, entropies, and KD-delta are deterministic, reproducible, hash-anchored outputs of the implemented model;
the non-Markovianity detector actually fires on injected backflow before it is trusted to report the model's 0.

QSOT2 does not claim, and says so in the artifact:

that the curvature-to-channel map is physically real;
any first-principles derivation;
any quantum-gravity or external-physical result;
that KD negativity is a contextuality proof.

Those refusals are not a closing disclaimer. They are serialized as a claim_boundary block in every result file the system writes.

Why this is not just "nicer slop"

Here is the test, and you can run it yourself.

Slop is the condition where, if you delete the rhetoric, nothing load-bearing remains. In QSOT v1, if you stripped the LaTeX and the project page, the core claim — KD negativity — collapsed to a placeholder. The formality was load-bearing for the deception.

Apply the same deletion to QSOT2. Strip the prose, the README, this newsletter. What is left:

a machine-readable result with 35 named checks and their pass/degrade status,
numerical deviations at machine epsilon you can recompute,
a four-field boundary that states the scope in the data itself,
byte-deterministic outputs with a SHA-256 you can regenerate,
a DOI you can resolve,
a test suite you can run,
an import boundary the interpreter will enforce whether you read the docs or not.

In other words: remove the rhetoric and QSOT2 is still standing; remove the rhetoric and QSOT v1 was a placeholder. That is the difference between persuasion and verification, and it is the only argument here I actually trust — because it does not depend on trusting me.

The cleanest single proof is the one most products would bury: QSOT2 ships its own failing check in plain sight. Slop hides its failures behind formality. QSOT2 prints its one degraded result on the front of the box.

What this actually felt like

Unpleasant first, then clarifying. We had worked hard on QSOT v1 — that was never in question. The hard part was accepting that effort and legitimacy are not the same thing. You can write clean code around a weak claim and a polished paper around a boundary you never defended, and if AI is accelerating your workflow, it accelerates the self-deception too.

That is why the confession had to be public — and why the confession was not enough on its own. Anyone can say "we got this wrong." The harder move is to say: here is what was wrong, here is how we constrained it, here is the artifact that now withstands inspection. The repair matters more than the confession.

The broader lesson

The lesson is not "don't use AI" — we used it heavily for both artifacts. It is sharper:

When AI lets you produce high-formality output in minutes, the scarce resource is no longer generation. It is claim custody.

What is being claimed? What file carries it? What check fails if it drifts? What survives when you delete the rhetoric? QSOT v1 had abundant generation and almost no custody. QSOT2 inverts that ratio in one narrow domain.

Closing

Six months ago we had something that looked like a scientific success and wasn't. Today we have something smaller, less glamorous, and much harder to fool — including harder for us to fool ourselves with. We trust the second one more. That is the progress.

QSOT v1 was the cautionary tale. QSOT2 is what happened when we treated that failure not as embarrassment, but as engineering input.

QSOT2 — a mathematical-consistency verifier, Open source under MIT (Flamehaven Labs). If you disagree with any of it, use the path the artifact was built to survive: clone it, re-run it, and check the hash.