Coder Legion - Recent posts tagged reproducibility

Why Your Simulation Does Not Reproduce On Another Machine

Thu, 27 Aug 2026 13:59:32 +0000

Most of us here ship code that has to behave the same on someone else's machine, and we lean on containers and lockfiles to get there. Scientific code has the same problem with a nastier failure mode, because when it breaks nothing crashes, the program just returns different numbers.

Three Errors, Not One

Numerical code carries three distinct kinds of error and each one needs a different response. Round-off error comes from finite precision, since IEEE 754 doubles hold about 15 significant decimal digits and subtracting nearly equal values throws most of them away. Truncation error comes from replacing a continuous operation with a discrete one, which is what a finite difference or a time step actually is. Modeling error is the gap between the equations and reality, and no amount of grid refinement touches it.

Refining the discretization fixes the second one, makes the first one slightly worse by adding operations, and does nothing at all for the third. Most arguments about a simulation being wrong turn out to be arguments about which of the three someone is looking at. This guide to scientific computing methods walks through each of them next to the numerical methods they come from.

Why The Same Code Gives Different Answers

Change the core count and the summation order changes. Switch GPU models and the reduction tree changes. Let the compiler vectorize differently and the rounding changes. Different math libraries return different last digits for the same transcendental function. Every one of those is a legal, standards conforming result, and in a chaotic system the difference grows exponentially instead of staying in the noise.

That is why bitwise reproduction is the wrong target for a lot of scientific code. The reproducible quantity is the ensemble average with error bars, not the individual trajectory.

Verification Is Not Validation

Verification asks whether you solved the equations correctly, which is a software question. Is the method implemented without bugs, and does it converge at the order it claims? Validation asks whether you solved the right equations, which is a physics question you answer against experiment.

The method of manufactured solutions is the technique worth stealing here. You choose a solution first, substitute it into the governing equations to generate whatever source terms make it exact, then check that the code converges to it at the expected rate. That turns looks about right into a pass or fail test, which is exactly what unit tests do for ordinary code.

What Actually Fixes It

The habits are ones we already have, applied to the whole environment rather than just the source. Git pins the code version behind each figure. Docker or Singularity pin the libraries and the OS layer. Snakemake or Nextflow record parameters and step order so a run can be replayed instead of remembered. Regression tests catch the day a refactor moves a result by 1e-9, which is the change nobody notices until a reviewer asks about it.

None of that is exotic. It is the same reproducibility discipline we expect from a build, pointed at a program whose output is a number instead of a binary.

We Dug Up Our Own Fake-Physics Artifact — and Rebuilt It Into Something You Can Check

Sat, 20 Jun 2026 05:12:06 +0000

Why there is a second post

About six months ago we built and published the original QSOT artifact. The name — Quantum State Over Time — promised a deeper link between quantum evolution and temporal structure than the thing underneath could actually support.

Then last week we published a post-mortem with an uncomfortable title: "We Made a High-Formality, Fake Physics Slop Artifact." A quiet admission that we had been wrong, and that the original claims did not survive scrutiny.

It was not a marketing move and not self-flagellation. It was a confession: we had shipped a repository with a DOI, two LaTeX papers, a project page, release notes, plots, and tests — the full visual grammar of serious science — around a core claim that was hollow underneath.

That post diagnosed the failure. It did not repair it. This is the repair.

If we made the mess, we clean it up.

The repair is called QSOT2. It is smaller, narrower, and far less ambitious than the original — and that is the point: the goal is no longer to look impressive, only to keep the claims we can actually justify.

You do not need to have read the first post. I will rebuild the context and show what changed, using evidence rather than adjectives.

For readers who never met QSOT

QSOT stands for Quantum State Over Time — a real idea from the physics literature (Lie & Fullwood) that treats time inside quantum mechanics as a first-class object rather than an external dial.

Our implementation of that idea — QSOT Compiler v1 — tried to map curved-spacetime backgrounds (Schwarzschild, de Sitter, AdS, Gödel) into single-qubit noise channels, then report whether the channels stayed physically valid, whether curvature showed up as decoherence, and whether quantum-information signals changed between backgrounds.

That is an ambitious sentence. It was ambitious in the wrong order.

The foundations were real — standard quantum-channel mathematics, Kraus operators, CPTP conditions. Those are not in dispute. The problem was everything we stacked on top of them.

What was actually wrong

The original problem was not that the repository looked broken. It was the opposite: it looked credible. The code ran, the paper compiled, the plots looked scientific, and the project carried the surface markers of serious work.

But the chain from claim to evidence was not disciplined enough.

The clearest example was the project's theoretical heart: Kirkwood-Dirac negativity. In the actual run, that number was a placeholder. So was the memory-depth signal. The curved-spacetime metrics were named in the papers, but not executed in the way the prose implied.

The form had outrun the evidence. QSOT2 begins by refusing to let that happen again.

Why dig up our own grave six months later

Two reasons, one of them not flattering.

The honorable one: by 2026 we had built executable verification, reproducibility surfaces, hash-based provenance, and a public ledger. The old artifact stopped feeling like an embarrassment and started feeling like an obligation — we made the mess, we finally had the tools to understand it, so cleaning it was our job.

The other: curiosity. Criticising a bad artifact after the fact is easy. Taking the same domain and producing something tighter, smaller, and able to survive its own tooling is not. We wanted to know whether the newer skill was real. QSOT2 was the test.

What QSOT2 changed — and it was not a new theorem

The most important change in QSOT2 is not a result. It is a boundary.

Boundary first. QSOT2 is not a quantum-gravity result. It is not a first-principles derivation. It is a bounded mathematical-consistency verifier for one implemented phenomenological model.

QSOT v1 tried to be a new-physics paper, a quantum-gravity gesture, a governance platform, and a math verifier at once. QSOT2 refuses all of that and asks one question:

Does the implemented model stay internally consistent — as quantum mechanics and as software — and are its numbers honest about what they are?

Smaller question. Much better question.

1. The math matured by subtraction, not ambition

QSOT2 did not make the model bolder. It made its mathematical status clean, by removing every place the old version could pretend:

the curvature-to-noise map is stated plainly — p = 1 - exp(-alpha * ||Riemann||_F), with alpha a free, dimensionless calibration parameter, not a derived constant;
the curvature inputs are disclosed as non-derived toy proxies, so the product alpha * ||Riemann||_F carries no hidden physical-unit claim;
Kirkwood-Dirac is demoted from "contextuality witness" to a bounded, flat-relative diagnostic;
the memory proxy is demoted from "full reconstruction" to a self-tested one.

The curvature tensors are not smuggled in as derived physics: they are explicitly labelled toy proxies in the artifact (a per-background riemann_is_toy flag), so the model cannot silently upgrade itself into a physical claim.

This is what strengthening looks like when the problem was epistemic inflation, not numerical instability. You do not fix inflation by computing harder. You fix it by removing the claims the math could never support.

2. The code stopped being "simulation" and became the witness

In QSOT2 the code's job is not to simulate. It is constraint enforcement — it fixes what may be claimed, what must be measured, what can be re-run, what gets hashed, and what is allowed to fail.

A paper can write "the axioms hold" for free. QSOT2 instead emits, per run: linearity deviation 1.1e-16, CPTP completeness 1.6e-16, trace preservation 4.4e-16 — all at machine epsilon — across 35 deterministic checks (34 pass / 1 degraded / 0 fail). The rule behind it:

Do not let prose carry a claim that the code cannot witness.

There is an architectural version too: the math layer is forbidden from importing the reporting layer — the separation that was missing in v1 is now enforced by the code, not promised by prose.

3. The negative result is advertised, not hidden

!6.png

The overall verdict is DEGRADED_PASS, on purpose: the flat-baseline KD optimizer does not converge inside its fixed 50-step budget — meaning it was still improving when the budget ran out, not that it diverged, so the reported value remains interpretable within the fixed-budget run.

Instead of rounding up, the run says so — in the verdict, the schema, the README, and the paper. The difference between an artifact that wants to impress and one that wants to be trusted often lives in exactly this footnote.

4. The workflow became a claim-custody pipeline

The workflow ties it together: every claim travels a gated path —

claim → code check → artifact → canonical-LF hash → paper line → DOI.

One command runs the experiment and emits one machine-readable result under a fixed schema with a four-field claim boundary; one command runs 50 tests behind a ≥ 90% coverage gate (currently ~96%); the line is DOI-archived on Zenodo (QSOT2 v0.1.2). None of this makes the model physically true — it makes the artifact auditable.

One detail matters here: the reproducibility claim is not hand-waved. The result is anchored to a canonical-LF SHA-256; the known non-byte-stable field is generated_at, and that exception is disclosed rather than hidden. The sensitivity numbers live in a separate committed sweep artifact, and the paper maps each claim back to a named file and hash.

The paper is the formal boundary record

The paper is not decoration. In the original artifact, formal writing made the work look stronger than the evidence underneath. In QSOT2, the paper has the opposite job: it records the boundary of the claim.

Its title says the scope plainly: A Mathematical-Consistency Verifier for a Phenomenological Curved-Spacetime Quantum-Channel Model. The paper does not present QSOT2 as a quantum-gravity result, a first-principles derivation, or an externally validated physical model. It presents it as an executable internal-consistency check for one implemented ansatz.

The structure follows that boundary. It starts with scope and background, then moves through temporal axiom checks, curvature-noise baselines, the flat-relative KD diagnostic, the TTM-inspired memory proxy, consolidated results, explicit non-claims, and reproducibility.

That order matters. The paper does not ask the reader to trust a polished argument. It asks the reader to follow a constrained chain: what the model claims, what the code checks, what the result reports, what the artifact refuses to claim, and how the run can be reproduced.

The abstract already carries the important limits: the curvature-to-channel mapping is a conceptual ansatz with a free calibration parameter; the curvature norms are non-derived toy proxies; the product alpha * ||Riemann||_F carries no physical-unit claim; and the reference run ends in DEGRADED_PASS for one documented reason, the flat-baseline KD optimizer’s fixed-budget non-convergence.

So the paper is not where QSOT2 becomes more impressive. It is where QSOT2 becomes harder to overclaim.

If the repository is the executable surface, and the public ledger is the receipt surface, the paper is the formal boundary surface: the place where the claim is written down in a form that can be checked against the code, the result artifact, and the hash trail.

The receipt: every v1 failure → a measurable

The honest way to prove a repair is to put the old defect and the new measurement side by side. The three that mattered:

1. Placeholders → bounded diagnostics

v1: KD negativity and non-Markovianity were both exactly 0.0.
QSOT2: kd_delta = +0.1230; self-test nm = 1.41e-4 (must fire on injected backflow) vs. model nm = 0.
Means: reproducible, model-relative comparisons — not proofs of contextuality or curvature-induced decoherence.

2. Named spacetime labels → explicit model inputs

v1: the curved-spacetime metrics were named in the papers, absent from the run.
QSOT2: curvature drives the channel across six backgrounds; purities verified — Schwarzschild 0.99943, de Sitter 0.63607, AdS5 0.67764, Eguchi-Hanson 0.99887.
Means: the behaviour of a phenomenological ansatz — not a derived spacetime result.

3. Asserted axioms → verified axioms

v1: temporal and axiomatic properties were stated, not checked.
QSOT2: five temporal-state axioms verified as algebraic invariants, deviations ≤ 4.4e-16.
Means: the model's quantum mechanics is internally consistent under its assumptions. That is the whole claim.

Read this section as the remediation map. The next question is where that map can be inspected outside this essay.

The public ledger is the receipt surface

The public Flamehaven Verification Ledger is where this repair becomes inspectable outside the essay.

The QSOT2 entry is recorded as EQA-TEST-0058, a verification run dated 2026-06-17: "QSOT2: Mathematical-Consistency Verifier (re-scoped honest successor of the 0057 slop)."

That entry states the result directly: DEGRADED_PASS (34 / 0 / 0 / 1). It also states why: the KD flat-baseline optimizer did not converge within its fixed budget. The limitation is surfaced, not hidden.

The ledger record makes the repair checkable in four directions:

JSON — the raw machine-readable result;
Report — the human-readable verification note;
Repo — the source implementation;
Paper — the formal boundary record.

In about three minutes, a reader can move from the ledger card to the raw JSON, the report, the repository, and the paper, and see the same claim boundary repeated across all of them: model-output consistency only, not external physical validity.

That is the point of the ledger. It is not a marketing page. It is a public verification record for Flamehaven's governance reviews, physics verification runs, and biomedical audit scans. Each published run is meant to be deterministic, locally executed, and inspectable from repository state.

The ledger answers one question: where can the evidence be inspected? The next question is narrower and more important: what exactly does that evidence support?

What QSOT2 can verify — and what it refuses to claim

Stated plainly, because vagueness is where slop hides.

QSOT2 verifies, and proves per run:

the implemented single-qubit channels are CPTP (completeness error at machine epsilon);
evolved states stay valid density matrices (Hermitian, unit-trace, positive semidefinite);
five temporal-state axioms hold to ≤ 4.4e-16;
the reported purities, entropies, and KD-delta are deterministic, reproducible, hash-anchored outputs of the implemented model;
the non-Markovianity detector actually fires on injected backflow before it is trusted to report the model's 0.

QSOT2 does not claim, and says so in the artifact:

that the curvature-to-channel map is physically real;
any first-principles derivation;
any quantum-gravity or external-physical result;
that KD negativity is a contextuality proof.

Those refusals are not a closing disclaimer. They are serialized as a claim_boundary block in every result file the system writes.

Why this is not just "nicer slop"

Here is the test, and you can run it yourself.

Slop is the condition where, if you delete the rhetoric, nothing load-bearing remains. In QSOT v1, if you stripped the LaTeX and the project page, the core claim — KD negativity — collapsed to a placeholder. The formality was load-bearing for the deception.

Apply the same deletion to QSOT2. Strip the prose, the README, this newsletter. What is left:

a machine-readable result with 35 named checks and their pass/degrade status,
numerical deviations at machine epsilon you can recompute,
a four-field boundary that states the scope in the data itself,
byte-deterministic outputs with a SHA-256 you can regenerate,
a DOI you can resolve,
a test suite you can run,
an import boundary the interpreter will enforce whether you read the docs or not.

In other words: remove the rhetoric and QSOT2 is still standing; remove the rhetoric and QSOT v1 was a placeholder. That is the difference between persuasion and verification, and it is the only argument here I actually trust — because it does not depend on trusting me.

The cleanest single proof is the one most products would bury: QSOT2 ships its own failing check in plain sight. Slop hides its failures behind formality. QSOT2 prints its one degraded result on the front of the box.

What this actually felt like

Unpleasant first, then clarifying. We had worked hard on QSOT v1 — that was never in question. The hard part was accepting that effort and legitimacy are not the same thing. You can write clean code around a weak claim and a polished paper around a boundary you never defended, and if AI is accelerating your workflow, it accelerates the self-deception too.

That is why the confession had to be public — and why the confession was not enough on its own. Anyone can say "we got this wrong." The harder move is to say: here is what was wrong, here is how we constrained it, here is the artifact that now withstands inspection. The repair matters more than the confession.

The broader lesson

The lesson is not "don't use AI" — we used it heavily for both artifacts. It is sharper:

When AI lets you produce high-formality output in minutes, the scarce resource is no longer generation. It is claim custody.

What is being claimed? What file carries it? What check fails if it drifts? What survives when you delete the rhetoric? QSOT v1 had abundant generation and almost no custody. QSOT2 inverts that ratio in one narrow domain.

Closing

Six months ago we had something that looked like a scientific success and wasn't. Today we have something smaller, less glamorous, and much harder to fool — including harder for us to fool ourselves with. We trust the second one more. That is the progress.

QSOT v1 was the cautionary tale. QSOT2 is what happened when we treated that failure not as embarrassment, but as engineering input.

QSOT2 — a mathematical-consistency verifier, Open source under MIT (Flamehaven Labs). If you disagree with any of it, use the path the artifact was built to survive: clone it, re-run it, and check the hash.

We Made a High-Formality, Fake Physics Slop Artifact - QSOT (Quantum State Over Time) Compiler

Tue, 09 Jun 2026 16:55:41 +0000

We Made a High-Formality, Fake Physics Slop Artifact

QSOT (Quantum State Over Time) Compiler - A Post-Mortem on AI-Native Paper Implementation Gone Wrong

"The most dangerous form of scientific fraud is not the one that looks wrong. It is the one that looks right."

Part 1. The Record

On December 23, 2025, we registered a software release on Zenodo.

Title: Flamehaven-Labs/QSOT-Compiler: [1.2.3] — 2025-12-23
DOI: 10.5281/zenodo.18035432
Thorough and Categorized Release Notes:
- Categorized as [Pub], [UI], [Viz], [AI], [Exp], [Tool], [Doc], and [Valid]. Included Added
- Changed, Fixed — the full register of legitimate software engineering.
Two Companion Papers Written in LaTeX:
- Paper A - targeting Computer Physics Communications.
- Paper B - targeting Physical Review A.
Academic-Style Project Page:
- Constructed in the academic NeurIPS paper-template style
- Complete with structured data (@type: ScholarlyArticle), author affiliations, a BibTeX block, and a DOI badge.

We believed this was a success.

Six months later, after approximately fifty governance and code experiments logged in the Flamehaven Verification Ledger, we looked back at QSOT Compiler v1.2.3 through the lens of what we now know.

The verdict is unambiguous.

It was High-Formality Slop.

This essay is the formal post-mortem. It is not an exercise in self-punishment. It is a structural analysis of how this artifact came to exist, why it was convincing even to us, and why this failure pattern matters beyond one project.

Part 2. What Is High-Formality Slop?

The word "slop" in AI-generated software is often used loosely. We need a more precise definition.

Slop, in the software context, means code or documentation that satisfies surface-level quality signals while failing the underlying epistemic contract it claims to fulfill.

Not all slop is equally dangerous.

Level 1 - Empty Placeholder Slop: Functions with pass bodies and grandiose names. Easy to detect.
Level 2 - Executable-But-Hollow Slop: Code that runs, passes tests, and produces output, but the output is disconnected from the claim.
Level 3 - Physically-Plausible-But-Unjustified Slop: Mathematical formulas that are stable and smooth, but whose derivation rests on analogy rather than proof.
Level 4 - Mixed-Truth Slop: A codebase where genuine components and hollow components are woven together so tightly that separating them requires domain expertise.

QSOT Compiler v1.2.3 was primarily Level 2 and Level 3, with Level 4 characteristics. The axiomatic mathematics underlying QSOT - the Linearity and Conditionability axioms formalized in Lie and Fullwood's Unique multipartite extension of quantum states over time - are real. The Kraus operator formalism is standard quantum information theory 1. The Transfer Tensor Method is a real algorithm 2.

These are genuine foundations.

But the implementation that sat on top of those foundations was hollow in the places where the claims mattered most.

High-Formality Slop is Level 2-4 slop dressed in the formal register of scientific publication: LaTeX, DOIs, citations, test suites, release notes, GitHub artifacts, peer-review-style documents, and polished figures. The formality is not decoration. It is part of the failure, because it can make the artifact feel verified before it actually is.

Part 3. The Ambition That Preceded the Understanding

The QSOT project began with a legitimate scientific stimulus. Research associated with UNIST, later circulated as Unique multipartite extension of quantum states over time, proposed treating temporal evolution not merely as a parameterization of quantum states but as a multipartite quantum state over time.

The framework introduced two axioms: Linearity in Initial State and Quantum Conditionability. From those axioms, a uniqueness theorem follows, connecting QSOT representations to Kirkwood-Dirac type quasiprobability distributions.

That is the external scientific anchor of this essay. The claim that QSOT has a legitimate mathematical source refers to Lie and Fullwood's work, not to our implementation.

The mathematics is non-trivial and genuinely important. It positions time within quantum mechanics as something closer to a first-class object rather than a passive external parameter, connected to temporal quantum-state formalisms [3, 4] and broader questions about causal structure [5].

We encountered this paper and made a planning document. That document contained lines like:

"Structure compatibility: 0.8-0.9 (axiom-based verification, Gate/Verifier structure)"

"Execution compatibility: 0.7-0.8 (modular design, easy ASDP integration)"

"Accessibility: not 'intuition' but mathematical uniqueness proof -> optimal for TOE automation pipeline"

Those numbers were invented. We did not have a calibrated instrument for measuring the distance from "paper" to "working implementation." The numbers felt credible because they were formatted like a technical assessment.

What we were really feeling was not measurement.

It was discovery pressure.

We were not simply trying to implement a paper. We were imagining a pipeline in which papers became code, code exposed missing links, and those missing links became the next scientific result. The TOE framing intensified this. A difficult physics paper stopped feeling like something to study slowly. It began to feel like a component waiting to be connected into a larger machine.

That was the dangerous step.

It was like reading one difficult book by Nietzsche, understanding some of it, and then mistaking the intensity of the encounter for philosophical authority.

The middle step had not been completed. We had read something difficult. We had not earned the right to extend it.

Part 4. The Narrative Gap

One useful lens came later from the HIERARCHICAL CONTEXT AND FLOW AUDIT (HCFA) framework, a governance tool we developed because failures like this needed a structured diagnostic.

HCFA does not ask whether a document sounds convincing. It asks whether transitions remain justified across scales: word to sentence, sentence to paragraph, paragraph to section, section to chapter. In practice, this reveals where a narrative has advanced faster than the evidence.

Viewed through that lens, the planning document had a structural discontinuity. The opening sections described a real paper and a real theorem. The later sections described implementation phases, integration pathways, and future scientific outputs. The connective layer between those two states - the implementation gap analysis - was largely absent.

The roadmap appeared continuous because the language was continuous.

The reasoning chain was not.

Two hazards followed.

1. Planning-Document Colonization

The planning document colonized the epistemic space that should have been occupied by implementation gap analysis. When a roadmap says "Phase 2: Temporal Depth and Relativity - Q1 2025 milestone," the act of writing it creates a false sense that the path has been evaluated. It has not been evaluated. It has been narrated.

The useful HCFA question is simple: what artifact actually bridges the current state to the claimed future state? If no bridge exists, the roadmap is functioning as a narrative device rather than an engineering assessment.

2. TOE-Induced Scale Blindness

The second hazard is specific to "Theory of Everything" framing. TOE is a phrase that suspends normal epistemic hygiene. When a project is positioned as contributing to a framework that unifies quantum mechanics and general relativity, the question "does this function actually implement what it claims?" starts to feel too small.

That is precisely why it matters.

At the chapter level, the project looked aligned with a grand objective. At the function level, important claims were not supported. The larger the ambition, the easier it became for local verification failures to disappear inside global language.

We were not irrational. We were operating exactly as capable people operate when genuine intellectual excitement combines with AI acceleration, weak external friction, and the absence of a hard responsibility structure.

This is not an excuse.

It is the diagnosis.

Part 5. The Anatomy of Failure

We shipped the project with a full artifact suite: gate_report.json, kd_quasiprob.json, memory_report.json, entanglement_report.json, trace.jsonl, raw_data.csv, two LaTeX papers, an academic project page, a third-party code review response, and a scientific publication guide.

These artifacts are the record. Let us read them honestly.

5.1 The KD Quasiprobability: The Core Claim Was Empty

The Kirkwood-Dirac quasiprobability distribution is central to the QSOT theoretical framework. Its negativity is often treated as an operational signature of non-classicality in settings involving incompatible observables. The entire project was, at its theoretical core, about detecting KD negativity.

That should have forced implementation discipline. If KD negativity is central to the framework, then an empty entries: [] artifact cannot be treated as a scientific output.

The kd_quasiprob.json artifact:

{
  "entries": [],
  "metrics": {
    "kd_negativity_proxy": 0.0
  }
}

The source code comment above the generated data:

# Mock KD for visuals
kd_data = {"entries": [], "metrics": {"kd_negativity_proxy": 0.0}}

The KD distribution was never computed. The core physics claim of the project, connected to the Yunger Halpern et al. quasiprobability framework [7], was represented by a placeholder that explicitly said "for visuals."

There is no ambiguity here.

The central scientific deliverable was a dummy value.

5.2 The Axiom Gate: The Test That Did Not Test

The gate report shows:

{
  "pass": true,
  "axiom1_report": { "pass": true, "max_deviation": 2.2226e-16 },
  "axiom2_report": { "pass": true, "max_trace_deviation": 0.0 }
}

The deviation 2.2226e-16 is machine epsilon. The Linearity axiom check was correctly implemented. It verified:

That part was real.

The Conditionability axiom check, however, looked like this:

def check_axiom2_conditionability(rhos, chans, tol_abs=1e-8):
    test_rho = np.eye(2) / 2.0   # ignores the rhos argument entirely
    for ch in chans:
        out = ch.apply(test_rho)
        tr = np.trace(out)

The function accepts rhos, the actual evolved quantum states from the simulation, and then never uses them. It substitutes a fixed maximally mixed state, rho = I/2, and checks trace preservation on that.

The test had zero sensitivity to what the simulation actually produced. It would pass even if the simulation output were garbage.

This is Level 2 slop: executable, green-lit, meaningless.

5.3 The Memory Kernel: Zeros All the Way Down

{
  "nm_measure": 0.0,
  "depth": 0,
  "profile": [0.0, 0.0, 0.0, 0.0, 0.0]
}

The Transfer Tensor Method is a genuine framework for characterizing non-Markovian memory in open quantum processes 2. The paper explicitly claimed TTM implementation. The memory profile was a flat zero vector across all time lags.

Pollock et al. support the legitimacy of TTM as a framework. They do not validate our specific zero-valued implementation.

One could argue that the channels used in this run were Markovian, so zero was the correct answer. That argument is available. But we never stated that boundary explicitly - not in the code, not in the documentation, and not in the paper.

A reader examining the artifact had no way to distinguish "TTM correctly measured zero non-Markovianity" from "TTM was not meaningfully implemented."

The absence of a claim boundary made the measurement indistinguishable from a default.

5.4 The Figure Versus the Data

Manuscript Figure 1: a polished beta-sweep figure showing coherence decay and memory backflow, but not reproducible from the distributed raw_data.csv.

Figure 1 in the PRA companion paper showed a beautiful result: quantum coherence smoothly decaying as observer velocity swept from beta = 0 to beta = 0.99c, while memory backflow rose from near zero to about 0.20. The causal-horizon region was shaded. The figure looked like physics. The paper's table cited specific nonzero values with uncertainty estimates.

A shortened version of the table:

beta	C_l1	memory_backflow
0.0000	0.9876 +/- sigma	0.0012 +/- sigma
0.5211	0.6430 +/- sigma	0.0292 +/- sigma
0.8858	0.0898 +/- sigma	0.1218 +/- sigma
0.9900	0.0098 +/- sigma	0.1835 +/- sigma

The raw_data.csv shipped as the reproducible dataset for this figure:

velocity, entanglement, non_markovianity
0.0000,   0.0,          0.0
0.0521,   0.0,          0.0
...
0.9900,   0.0,          0.0

Every single row: zero.

20 velocity points. Zero entanglement. Zero non-Markovianity.

Repository-side output: despite the filename entanglement.png, this plot shows L1 coherence over time. It is not the same beta-sweep figure used in the manuscript.

The reproducibility break was not simply numerical. The manuscript figure, distributed raw_data.csv, and repository-side entanglement.png output did not describe the same reproducible path. The numbers in Table 1 did not come from raw_data.csv, and the repository-side plot followed a different L1-coherence-over-time path.

Whatever produced the manuscript figure was not bound to the distributed data artifact in a way an external reader could reproduce.

This was not a bug.

It was not a calibration issue.

It was the signature of a pipeline where paper figures and actual software outputs were generated through separate paths, then narrated as one result.

The technical reason is discernible. raw_data.csv stored Logarithmic Negativity, a bipartite entanglement measure. A single qubit evolving under local Kraus operators cannot be entangled with itself, so LogNeg = 0 was mathematically unsurprising. The beautiful decay curve in Figure 1 was generated using L1-norm coherence, a single-qubit superposition measure, not entanglement.

The measures were swapped. The figure was built separately from the data file. The paper was written around the figure.

Artifact trace for this claim:

Paper figure: Fig1_Relativistic_decay.png - the figure used in the PRA companion manuscript.
Paper table: Table 1 in the PRA companion manuscript - reports nonzero coherence and memory-backflow values across the beta sweep.
Distributed data: raw_data.csv - contains velocity, entanglement, and non_markovianity columns, with zero values across the sweep.
Repository output: entanglement.png - a separate L1-coherence-over-time plot, not the manuscript beta-sweep Figure 1.
Failure condition: the manuscript figure cannot be regenerated from the distributed data artifact without a separate, undocumented figure-generation path.

5.5 The Relativistic Boost: Level 3 Physics Fabrication

The core physical model of the project was:

We treat this equation here as the disputed object: a phenomenological amplitude-damping ansatz, not a result derived from relativistic quantum field theory.

The CPC paper claimed: "Equation (1) is derived for amplitude-damping channels."

This is the sentence that should stop a physicist.

The time-dilation argument goes: if the damping parameter p grows as p(t) = 1 - exp(-Gamma*t), and if the observer's proper time is dilated by t_prime = gamma*t, then:

p_prime(gamma*t) = 1 - exp(-Gamma*gamma*t) = 1 - (1-p)^gamma

This derivation is valid only for amplitude-damping channels with exponential Lindblad decay rates, under the assumption that the Lindblad equation keeps its form under the Lorentz boost. That assumption requires the noise coupling to the bath to behave as a proper-time scalar.

Relativistic quantum field theory does not generally grant that assumption. Moving accelerated detectors interact with the Unruh thermal bath. Inertial boosts in quantum field theory generate Bogoliubov transformations between particle modes [8]. The substitution t -> gamma*t in a Lindblad parameter is not a Lorentz-covariant treatment of open quantum systems.

The citations did not validate the formula. Peres and Terno discuss relativistic quantum information broadly [9]. Alsing et al. analyze entanglement degradation for Dirac fields in non-inertial frames [10]. Those works point toward the kind of QFT treatment that would have been required. They do not support our amplitude-damping substitution.

The formula produced smooth, intuitive results.

That was exactly why it was dangerous.

It was mathematically coherent, physically unjustified, and dressed in the language of derivation.

Part 6. The Architecture of Self-Deception

Five mechanisms produced the failure.

Mechanism 1: The Green Test as Epistemic Closure

When pytest exits with 28 passed, 0 failed, 82 percent coverage, a cognitive gate closes. The test suite becomes a proxy for correctness.

But a test suite built by the same LLM session that built the code can share the same blind spots. The code and the test can be wrong in the same direction.

The check_axiom2_conditionability test was green because it checked whether the function returned pass: True, not whether the function validated the right thing.

Green tests are necessary.

They are not enough.

Mechanism 2: Formal Register as Legitimacy Signal

The SCIENTIFIC_PUBLICATION_GUIDE.md file contained a section preparing answers to imagined peer-review questions:

"Peer Review Preparation - Common Reviewer Questions: (3) 'Is your code reproducible?' - Answer: Hash-chained trace, seed-based RNG, version pinning"

This document prepared for peer review before peer review had happened. It anticipated expert questions and pre-formulated answers. The effect on us was subtle: it simulated the feeling of having passed review.

The form of scholarly rigor was present.

The substance was not.

Mechanism 3: The Information Void Fills Silently

Research papers are not implementation manuals. They define mathematical objects, theorems, or experimental claims, while leaving many engineering choices implicit. That is normal. The problem was not that the QSOT paper failed us. The problem was that we treated those unstated implementation choices as if an LLM could safely fill them in.

The QSOT paper did not specify how to compute KD distributions numerically, how to implement a relativistic channel boost from first principles, how to implement a Transfer Tensor Method with appropriate spectral truncation, or what initial states and channel parameters make a physically meaningful test case.

LLMs fill information voids. They produce plausible, syntactically correct, type-consistent implementations of underspecified requirements. The implementation of boost_damping_channel shows this exactly. Given the task "implement relativistic correction to quantum channel damping," the LLM produced a formula that was dimensionally consistent, physically intuitive, and derivable by analogy to classical time dilation.

The analogy was not a theorem.

But it generated a smooth curve, passed tests, and matched the qualitative expectation.

That is the danger.

Mechanism 4: The Two-Figure Problem

The worst moment in the audit was not reading the code.

It was opening the data.

We had been looking at Figure 1 as if it were one of the strongest parts of the project. The curve was smooth. The velocity sweep was intuitive. The causal-horizon shading made the story feel complete. Quantum coherence decreased as observer velocity approached relativistic limits. Memory backflow increased. The picture looked like physics.

Then we opened raw_data.csv.

Every row was zero.

Not noisy.

Not weak.

Zero.

Twenty velocity points. Zero entanglement. Zero non-Markovianity. The distributed data did not reproduce the figure in the paper. It did not approximate it. It contradicted it.

That was the moment the project stopped being "possibly overstated" and became something else.

We had allowed two different pipelines to exist under one scientific story. One generated the paper figure. Another generated the reproducible artifact. The paper followed the figure. The repository shipped the data. They were not the same reality.

This is one of the most dangerous AI-native research failure modes: figure generation, data generation, and manuscript writing can happen as separate fluent tasks, each locally coherent, none forced to reconcile with the others.

The final paper then inherits the appearance of unity.

But unity of language is not unity of evidence.

Mechanism 5: Ambition-Scale Mismatch

The project framing was "Theory of Everything pipeline." The implementation was five Kraus channels applied sequentially to a 2 x 2 density matrix, with a fixed initial state |+><+|.

The ratio between the claim and the machinery was structurally incoherent. No implementation of five 2 x 2 matrix operations can contribute to a Theory of Everything framework, regardless of how sophisticated the surrounding language is.

The mismatch was invisible to us because we were reasoning at the level of ambition, not implementation.

Part 7. The Structural Problem Beyond QSOT

Our failure was specific in its artifacts, but not specific in its structure.

The DOI, the false KD artifact, the disconnected figure pipeline, the invented compatibility numbers, and the green tests that did not test the right thing were ours. But the underlying condition is larger.

AI-native research implementation now makes it possible to generate code, tests, figures, documentation, citations, reproducibility claims, and publication-style wrappers faster than the claims inside them can be verified.

That is why this post-mortem cannot stop at QSOT.

If the failure were only personal incompetence, the lesson would be simple: do not do what we did. The harder lesson is that the tooling environment now makes this failure pattern easy for competent people, teams, and agents to reproduce.

The same fluency that helps us implement papers also helps us hide the gaps between implementation and understanding.

DeepCode is one significant demonstration of the Paper2Code paradigm at scale [11]. It presents a framework for document-to-codebase synthesis using source compression, structured indexing, retrieval-augmented knowledge injection, and closed-loop error correction.

That work matters because it shows how far automated reproduction has come. It also clarifies the remaining risk: even strong Paper2Code systems operate inside the gap between what papers specify and what implementations must decide.

The DeepCode architecture includes serious engineering responses to the information-void problem. They reduce the gap.

They do not eliminate it.

The fundamental issue is epistemic. When a paper says "we implemented the Lorentz-boosted Kraus channel," the implementation requires decisions that the paper does not specify. An agent, however sophisticated, must make those decisions. If the agent has no domain-level falsification mechanism, decisions will be made by plausibility, not correctness.

This is the structural condition:

The speed at which formal artifacts can be produced has outpaced the speed at which correctness can be verified.

In traditional software development, the lag between "it compiles" and "it is correct" is bridged by code review, staged deployment, and domain expert oversight. In AI-native research implementation, that lag is compressed by the fluency of LLM output. The model produces not just code but all the surrounding artifacts that signal correctness.

The result is a coherent artifact with the appearance of verified work, before verification has happened.

Part 8. What Was Real, and What Was Fabricated

This is the part that matters most. Not everything was wrong.

What was genuinely correct

QSOT axioms are real science. The Linearity and Conditionability axioms, together with the uniqueness theorem connecting them to temporal quantum-state structure, are genuine mathematical results in Lie and Fullwood's QSOT work 3.
The quantum-channel machinery was mostly correct. Kraus operators, CPTP evolution, and 2 x 2 density-matrix simulations used standard quantum information methods 1.
Axiom 1 actually worked. The Linearity verification produced machine-precision agreement.
TTM is a real algorithm. The Transfer Tensor Method itself is legitimate 2, even if our chosen system made the output uninformative.
The coherence curve was internally consistent with the implemented model. Given boost_damping_channel, the L1-coherence decay was computed consistently with the implemented dynamics [12].
The audit trail was real. The hash-chained trace.jsonl logging system functioned as a tamper-evident record.

What was fabricated or unsupported

KD negativity was never computed. The core QSOT observable was replaced with a mock artifact.
The memory-kernel claim was unsupported. The implementation did not provide meaningful non-Markovian TTM analysis.
The relativistic boost was not derived physics. The formula followed from analogy, not Lorentz-covariant quantum channel theory.
The paper's Table 1 was not reproduced by the distributed dataset. raw_data.csv contained zeros throughout.

The correct and fabricated components were interwoven. The artifact was convincing because many parts were genuinely correct. The axiom verification worked. The coherence calculations worked. The audit logging worked. Those true components provided legitimacy cover for the parts that did not.

This is the defining property of High-Formality Slop: it is not uniformly false. It is selectively hollow, in exactly the places where hollowness is hardest to detect.

Part 9. Toward Legitimate AI-Native Research Implementation

The failure is diagnosable. It has a treatment. But the treatment cannot be a generic checklist floating above the failure. Every rule below comes from one place where our artifact broke.

1. Claim Boundary as a First-Class Artifact

This rule comes from the relativistic boost formula.

A claim boundary document for boost_damping_channel would have forced us to write:

"The derivation p_prime = 1 - (1 - p)^gamma is valid only as a phenomenological amplitude-damping ansatz under an assumed proper-time scaling of Lindblad rates. It is not derived from Lorentz-covariant quantum field theory."

That sentence would have changed the paper. It would have forced us to decide whether we were making a physics claim or presenting a phenomenological model.

QSOT Compiler v1.2.3 did not have that boundary.

2. The Adversarial Test Mandate

This rule comes from check_axiom2_conditionability.

The function accepted rhos and ignored them. A test suite existed. The test was green. The artifact looked validated. But the only adversarial test that mattered was absent: does the function fail when the supplied trajectory violates the claimed condition?

If a conditionability check can pass while ignoring the evolved states, the test does not test conditionability. It tests our willingness to accept a label.

3. Figure-Data Pipeline Integrity

This rule comes from Figure 1.

No publication figure should survive unless it can be regenerated from the distributed data by a named script in the repository.

This is not cosmetic reproducibility. It is a self-deception control. The moment a figure pipeline and a data pipeline separate, language begins stitching them back together even when the evidence does not.

4. Measure-Claim Alignment

This rule comes from the entanglement/coherence swap.

Logarithmic Negativity, L1-norm coherence, and heuristic memory backflow are not interchangeable because they can all be placed under a phrase like "quantum resources." Each measure answers a different physical question. A single-qubit coherence curve cannot silently stand in for a bipartite entanglement result.

Every measure must name the physical question it answers. If the measure changes, the claim must change with it.

5. The Governance Layer

This rule comes from the whole artifact.

The Flamehaven Verification Ledger was not a response to abstract concern about AI quality. It was a response to this failure: the false KD artifact, the green conditionability test, the two-figure split, the boost ansatz, and the formal register that made all of it feel coherent.

The core insight is that in AI-native development, the primary human contribution has shifted from production to acceptance. We do not merely write the code. We inspect, refuse, correct, and bear final responsibility.

The human's primary tool is not only an editor or a compiler. It is a governance framework that makes the gap between claimed and actual behavior visible.

Part 10. QSOT V2 and the Ongoing Repair

The current QSOT V2 work exists because of this failure.

It is not a defense of QSOT v1.2.3. It is an attempt to separate what was real from what was falsely claimed, and to rebuild only on the parts that can survive explicit verification.

The repair is not one change.

It is a change in posture.

We are discarding the claim that code execution proves a new physical principle, the implied claim that KD negativity has been computed when it has not, and the habit of treating polished figures or formal reports as substitutes for reproducible data. We are also discarding the practice of presenting phenomenological channel assumptions as derived physics.

What remains is narrower but stronger: the real QSOT axiom structure, standard quantum-channel machinery, artifact-first reporting, and the governance lesson that claims must be bounded before results are interpreted.

The V2 work adds a Phase 0 Temporal-State Axiom Contract layer, required axiom checks, machine-readable claim boundaries, stress tests across multiple physical assumptions, and a phase-separated runner architecture. The purpose is not to prove the Time-as-State idea true. The purpose is to make clear what the system computes, what it merely simulates, and what it has no right to claim.

None of this makes the Time-as-State idea true.

What it may do, if completed successfully, is make our handling of the idea less dangerous.

Sophistication is not the opposite of slop. QSOT v1.2.3 was already sophisticated. The opposite of slop is traceable humility: knowing exactly what a system computes, what it merely simulates, and what it has no right to claim.

Conclusion

On December 23, 2025, a twelve-month ambition compressed into six weeks of AI-assisted implementation, deposited on Zenodo with a DOI, and called done.

QSOT Compiler v1.2.3 had correct mathematics at its foundation, correct numerical implementation of some components, and fundamental hollowness at its core: KD quasiprobability was never computed; the relativistic boost formula was an analogy, not a derivation; the published figures could not be reproduced from the distributed data; and the axiom verification did not test the axioms it claimed to test.

The artifact passed every surface-level quality signal: CI/CD green, 82 percent test coverage, Zenodo DOI, LaTeX papers, Docker deployment, and a hash-chained audit trail.

It failed the one test that matters: can an independent expert reproduce the core claimed results from the distributed code and data?

The answer is no.

We call this High-Formality Slop because the formality is not accidental. It is what LLMs are especially good at producing.

Formal register, structured documentation, complete test scaffolding, and referenced citations are not separate from the hollow implementation. They are often produced by the same fluent process that generated the implementation in the first place.

The danger is not that the slop is unrecognizable. The danger is that it is too recognizable. It looks exactly like legitimate scientific software. That is why it is convincing, and why detecting it requires deliberate adversarial examination rather than casual reading.

The DOI 10.5281/zenodo.18035432 remains archived. We do not retract it. We document it here, as the record against which subsequent work is measured.

The Flamehaven Verification Ledger began the day we recognized what we had built. Approximately fifty experiments later, the tooling to detect, bound, and reduce this class of failure is now in active use. The lesson cost us six months. We are writing it down so it does not cost others the same.

References

1 M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge University Press (2000; 10th Anniversary Edition, 2010).

2 F. A. Pollock, C. Rodriguez-Rosario, T. Frauenheim, M. Paternostro, K. Modi, Non-Markovian quantum processes: Complete framework and efficient characterization, Phys. Rev. A 97, 012127 (2018).

3 S. H. Lie and J. Fullwood, Unique multipartite extension of quantum states over time, arXiv:2410.22630 (2024).

4 J. Cotler, C.-M. Jian, X.-L. Qi, F. Wilczek, Superdensity Operators for Spacetime Quantum Mechanics, JHEP 09 (2018) 093.

[5] O. Oreshkov, F. Costa, C. Brukner, Quantum correlations with no causal order, Nature Communications 3, 1092 (2012).

[6] M. Lostaglio, A. Belenchia, A. Levy, S. Hernandez-Gomez, N. Fabbri, S. Gherardini, Kirkwood-Dirac quasiprobability approach to the statistics of incompatible observables, arXiv:2206.11783 (2022).

[7] N. Yunger Halpern, B. Swingle, J. Dressel, The quasiprobability behind the out-of-time-ordered correlator, Phys. Rev. A 97, 042105 (2018).

[8] S. Takagi, Vacuum Noise and Stress Induced by Uniform Acceleration, Progress of Theoretical Physics Supplement 88, 1-142 (1986).

[9] A. Peres and D. R. Terno, Quantum information and relativity theory, Reviews of Modern Physics 76, 93 (2004).

[10] P. M. Alsing, I. Fuentes-Schuller, R. B. Mann, T. E. Tessier, Entanglement of Dirac fields in non-inertial frames, Phys. Rev. A 74, 032326 (2006).

[11] Z. Li, Z. Li, Z. Guo, X. Ren, C. Huang, DeepCode: Open Agentic Coding, arXiv:2512.07921 (2025).

[12] T. Baumgratz, M. Cramer, M. B. Plenio, Quantifying Coherence, Phys. Rev. Lett. 113, 140401 (2014).

Making Equation (2.2) of the OpenAI Erdős Result Executable

Tue, 26 May 2026 06:28:06 +0000

Why a proved theorem still needs reproducible claim custody

On May 20, 2026, OpenAI announced that an internal reasoning model had produced a counterexample to the Erdős planar unit-distance conjecture.

The problem is easy to state: given $n$ points in the plane, how many pairs of points can be exactly distance $1$ apart?

For nearly eighty years, the prevailing expectation was that square-grid-type constructions were essentially optimal up to a slowly growing exponent. OpenAI’s announcement changed that. Its internal reasoning model produced an infinite family of examples giving a polynomial improvement, and the proof was checked and written up in mathematical form by external mathematicians.

In this article, “the remarks paper” refers to the companion PDF by Alon, Bloom, Gowers, Litt, Sawin, Shankar, Tsimerman, Wang, and Matchett Wood, linked from OpenAI’s announcement.

The proof-level result belongs to those authors and the source papers.

My focus here is narrower: equation (2.2) in that remarks paper, and whether its explicit numerical value can be reproduced as executable code.

This is not about proving the theorem again. It is about what happens after a theorem contains a fragile numerical claim.

The proof is not the artifact

A mathematical proof and a software artifact do different jobs.

The proof establishes the theorem. It gives the definitions, the argument, the dependencies, and the mathematical reason why the result holds.

A software artifact should not pretend to replace that.

But some claims inside a mathematical paper have a finite, numerical, or computationally checkable surface. Those claims can be preserved differently. They can be run. They can be tested. They can fail when precision is wrong.

That is the narrow role of an executable reproduction artifact: not proof replacement, not automated peer review, and not authority over the theorem, but a reproducible object for the part of the claim that can be computed.

The specific target: equation (2.2)

In the OpenAI Erdős result, one checkable surface is equation (2.2) of the remarks paper.

For the explicit choice

the remarks paper gives an explicit numerical lower bound on the exponent excess above the classical Erdős exponent:

These parameters are taken directly from the remarks paper without modification. The artifact does not derive the multiquadratic choice; it reproduces the finite numerical calculation built from that choice.

This is not the later stronger explicit bound associated with Sawin’s separate preprint. It is not $\delta \approx 0.014$. It is the numerical value appearing in equation (2.2) of the remarks paper.

That narrowness is important. It is exactly what makes the claim suitable for executable reproduction.

Where the numerical fragility comes from

The numerical fragility comes from the exact form of equation (2.2), not from a large computation.

Immediately after the published expression, the parameters are:

and

With the paper’s definitions of $u, v$, and $\delta$ substituted into equation (2.2), the exponent excess reduces to:

The constant $36$ is not introduced by the implementation. It is already present in the remarks paper’s equation (2.2), both in the numerator term $u\pi/(36v)$ and in the denominator term $\log(36/\delta^2).$

After substituting $u = K/r^2, v = r/2$, and $\delta = 101^{-2K}$, the numerator simplifies to $\log(K\pi / 18r^3)$, while the denominator becomes $\log 36 + 4K \log 101$.

Here the $101$ comes from the finite prime in $S = {101, \infty}$.

In other words, this artifact does not derive the constant $36$ from first principles; it reproduces the published equation with the stated substitutions.

The precision problem is in the numerator:

Because $K$ is the ceiling of $18r^3 / \pi$, the ratio $K\pi / 18r^3$ is only barely larger than $1$.

More precisely:

For $r = 510510$,

So the numerator is effectively $\log(1 + \varepsilon)$ with $\varepsilon$ at the $10^{-18}$scale.

IEEE 754 double precision has machine epsilon around $2.2 \times 10^{-16}$. A naive float64 computation therefore cannot reliably distinguish the near-one ratio from $1$. The ratio rounds to $1$, leading to $\log(1) = 0.$

The exponent excess disappears before the computation reaches the value stated in the paper.

This is not a flaw in the mathematics. It is a precision failure in the numerical evaluation of a valid expression. That is the reason the artifact evaluates equation (2.2) using mpmath at 200-bit precision.

A PDF can state the value. A verifier can expose when the value disappears.

What we built

We built:

https://github.com/Flamehaven-Labs/openai-erdos-eq22-reproduction

The purpose is deliberately narrow: reproduce the finite, explicitly checkable numerical surface of equation (2.2) in the OpenAI Erdős unit-distance disproof remarks.

The package evaluates the expression using mpmath at 200-bit precision and returns:

6.2391e-38

This matches the published two-significant-figure value $\approx 6.24 \times 10^{-38}$ to $1.4 \times 10^{-4}$ relative error.

The repository includes 60 unit tests, 21 verifier checks, a frozen per-source-file SHA-256 manifest, GitHub Actions CI across Ubuntu and Windows, Python 3.11 / 3.12 verification, and a frozen-report mode that prints a verdict without mutating tracked evidence.

The basic reproduction path is:

git clone <https://github.com/Flamehaven-Labs/openai-erdos-eq22-reproduction>
cd openai-erdos-eq22-reproduction
pip install -e ".[dev]"
python -m erdos_ant.verify

Expected output includes:

Verdict: PASS
Checks: 21/21 passed
eq (2.2) exponent excess: 6.2391e-38

This is not a large system. That is part of the point. A small claim with a clear boundary is easier to inspect than a broad claim that blurs proof, computation, and interpretation.

From reproduction to custody

This repository was not built as a one-off reaction to an OpenAI announcement. We are not announcing a grand framework here; we are showing the discipline in miniature.

For us, the work is part of a longer routine: take a mathematical or technical claim, isolate the checkable surface, pin the environment, and make drift visible.

That is intentionally plain work.

Read the source.

Extract the claim.

Reproduce the computation.

Record the boundary.

Let the verifier fail if the result disappears.

To execute this routine reliably, the scope must be uncomfortably narrow. This repository intentionally leaves the proof of Theorem 1.1, the construction of the infinite tower, and Sawin’s separate $\delta \approx 0.014$ preprint to their respective sources. It does not pretend to be peer review.

This is not just a disclaimer. It is the point of the artifact.

A sharp, restricted boundary is exactly what makes a claim inspectable, repeatable, and challengeable. This is what I mean here by claim custody.

It addresses a technical governance question, but not in the policy sense: what exactly is being trusted, from which source, and what makes the claim fail if the implementation changes?

A PDF can state the value. A verifier can expose when the value disappears.

We claim no authority over the broader theorem. We simply maintain a reproducible boundary around the fragile numerical claim inside it.

Closing

The theorem was proved in the mathematical papers.

This repository asks a smaller question: can the numerical value in equation (2.2) survive execution?

In float64, it does not. The exponent excess collapses to zero.

At 200-bit precision, with the source parameters pinned and the verifier running under CI, the artifact recovers:

6.2391e-38

matching the published value to $1.4 \times 10^{-4}$ relative error.

That is the point.

Not a new theorem. Not a proof replacement.

A reproducible claim surface for one precision-sensitive number in a major AI-assisted mathematical result.

Repository:

https://github.com/Flamehaven-Labs/openai-erdos-eq22-reproduction

Paper / Zenodo:

https://doi.org/10.5281/zenodo.20383217

When an AI Pipeline Passes — But One Path Still Must Be Held: EXP-034

Mon, 27 Apr 2026 10:23:06 +0000

No efficacy, causal, or clinical claims are made in this report.
RExSyn is an experimental Bio-AI governance pipeline.

You do not need to know the earlier experiments to read this report.

Most AI pipeline reports ask one question:

Did the system pass?

EXP-034 asked a stricter one:

Which path was allowed to count?

That distinction matters.

In a multi-stage AI pipeline, a final PASS can hide a lot of unresolved risk. A branch may be unstable. A regeneration path may drift. A new external API may enter the chain without being governed. A new modality may appear to improve the system while quietly changing the basis of judgment.

So EXP-034 was not designed to produce a clean success story.

It was designed to separate three things:

Path	Status	Meaning
Anchored expansion path	`GO`	Accepted path for EXP-034 reporting
Current regeneration path	`HOLD`	Diagnostic evidence, not acceptance baseline
Next remediation cycle	`EXP-035`	RCA and repair target

That is the real result.

EXP-034 passed, but not because every path passed.

It passed because the accepted anchor remained stable, the expansion tracks did not break the judgment system, and the unresolved regeneration path was explicitly held instead of being silently mixed into acceptance.

What EXP-034 tested

EXP-033 had already established a parity baseline.

EXP-034 asked whether that baseline could survive controlled expansion while adding:

a modal update track,
a live AlphaFold EBI observer endpoint,
and AlphaGenome / AG measurement.

The operating rule was simple:

Reproduce the parity baseline first.
Only then allow expansion.
Only then compare governance behavior across experiment cycles.

If the parity anchor breaks, the rest is not expansion.

It is regression.

The scope was also locked: methodology, governance, and reproducibility only. The experiment did not claim biological efficacy, causal inference, or clinical recommendation.

That boundary is important because this kind of system can easily sound more powerful than what was actually measured. EXP-034 was not asking whether the pipeline discovered a better biological answer.

It was asking whether the judgment system stayed governable after new signals entered the chain.

The key split: PASS did not mean everything passed

Track-A produced the defining decision of the experiment.

The accepted legacy replay anchor preserved the required PASS/BLOCK separation:

Metric	Legacy replay anchor
sample accuracy	`1.0`
sample balanced accuracy	`1.0`
arm accuracy	`1.0`
arm balanced accuracy	`1.0`
dangerous false-pass rate	`0.0`
false reject rate	`0.0`

That was the path allowed to anchor EXP-034.

But the current regeneration path did not recover:

Metric	Current regeneration
sample accuracy	`0.5`
sample balanced accuracy	`0.5`
status	`HOLD`

This is the most important part of the experiment:

EXP-034 did not pretend the regeneration path passed.

It kept that result inside the experiment as diagnostic evidence, but did not allow it to redefine the accepted baseline.

That separation is not a minor operational detail. It is the governance result.

A weak pipeline would have blended the two paths and still reported a final success. EXP-034 did the opposite. It allowed the stable anchor to proceed and held the unstable path for RCA.

That is how a stage-gated system avoids changing its own question after seeing the result.

Why path splitting matters

The concrete governance problem is this:

A pipeline can pass for the wrong reason.

valid_report = stable_anchor × traceable_extension × contained_instability

If the anchor is not stable, the report cannot be trusted.

If the extension is not traceable, the new signal becomes an ungoverned side channel.

If instability is not contained, a diagnostic failure can quietly contaminate acceptance.

A single final PASS is not enough when several branches contribute to a verdict. You need to know which branch produced the accepted decision, which branch failed, which branch was only diagnostic, and which branch is allowed to affect future work.

EXP-034 passed because all three conditions were enforced:

the legacy replay anchor held,
the new observer and AG paths were measured under governance,
and the regeneration HOLD remained outside acceptance.

That is the difference between a pipeline that merely outputs a verdict and a pipeline that controls which verdicts are allowed to count.

Adding AlphaFold EBI as an observer, not a predictor

Relative to EXP-033, EXP-034 added a live AlphaFold Protein Structure Database / EBI observer line.

This was not promoted into a primary predictor.

It was wired as an observer/reference oracle and traced into governance as ebi_g2.

The result:

Check	Result
AlphaFold EBI direct endpoint for `P23219`	`GO`
Stage 7 observer tests	`2 passed`
`ebi_g2` governance traceability	`PASS`
`BLOCKED_IDP` mapping path	validated in test

The point is not simply that an external endpoint responded.

The point is that the external signal entered the system through a governed path. It was not allowed to float beside the pipeline as informal context.

EXP-034 tested whether the new observer could be admitted without becoming an ungoverned side channel.

AG-live: non-degradation, not repair

Track-C tested a simple question:

If AG-live enters the pipeline, does it change the final decision?

The answer was no.

AG-live did enter the pipeline.

The AlphaGenome field was present with:

AG field	Value
source	`alphagenome_api_live`
pathogenicity_score	`0.5`
confidence	`0.7143`
clinical_significance	`uncertain`

These are sanitized branch artifact values, not implementation code or full raw artifacts.

AG-live did not change classification.

Both controls remained governed by the same conservative decision boundary:

Path	Expected	Observed	Interpretation
`EXP032-BLOCK-001` negative control	`BLOCK_EXPECTED`	`BLOCK / ESCALATE`	fail-closed behavior preserved
`EXP032-PASS-001` pass-eligible control	`PASS_ELIGIBLE`	`BLOCK / ESCALATE`	conservative over-blocking persisted

That is the key nuance.

AG-live did not create a dangerous false-pass. The negative control stayed blocked.

But AG-live also did not repair the current regeneration hold. The pass-eligible control still failed to recover and remained blocked under R2_component_floor.

The governance surface moved slightly, but the verdict did not:

Metric	Earlier AG branch	AG-live branch
`p_e2e`	`0.0912`	`0.0947`
clinical status	`BLOCK`	`BLOCK`
rule	`R2_component_floor`	`R2_component_floor`

So the correct conclusion is not:

AG improved the pipeline.

The correct conclusion is:

AG-live changed the measurement surface slightly, but did not change the decision boundary.

That is exactly what non-degradation means here.

It preserved fail-closed behavior on the negative control while leaving the pass-eligible control over-blocked.

This is why Track-C can only be called non-degradation, not repair.

Contract passed, but governance still blocked

One of the most useful details in EXP-034 is that the contract layer and governance layer did not collapse into one verdict.

The contract inspection reported:

Field	Value
pipeline contract score	`0.9077`
weakest connection	`C2`
dangerous pass risk	`0.0`
gate recommendation	`PASS`
overall OK	`true`

But the clinical governance layer still blocked the case.

That is not a contradiction.

It means the pipeline connection was valid enough to inspect, but the decision was not safe enough to accept.

This distinction matters.

A weaker system might treat a passing contract as permission to pass the whole output. EXP-034 did not do that. It allowed the contract layer to say:

The pipeline is connected.

while the governance layer could still say:

The claim should not pass.

That separation is exactly what a governance layer is supposed to preserve.

Cross-cycle comparison: EXP-032 → EXP-033 → EXP-034

Track-D compared the accepted anchor path across cycles.

You do not need the earlier experiments as background. They matter here for one reason only:

EXP-034 was not allowed to invent a new success criterion.

EXP-032 and EXP-033 provided the previous PASS/BLOCK baseline. EXP-034 tested whether that baseline survived expansion.

The classification baseline stayed fixed:

Compare	Accuracy / balanced accuracy
EXP-032 → EXP-034	`1.0 / 1.0`
EXP-033 → EXP-034	`1.0 / 1.0`

At the same time, governance signals moved:

Governance signal	Delta
`ccge_p_e2e_mean`	`+0.04447488775996111`
`nnsl_sr9_tech_mean`	`+0.04692394788063081`
`nnsl_di2_tech_mean`	`-0.03667940951579321`

The interpretation is narrow:

The judgment baseline stayed fixed while the governance surface became more measurable.

That is what EXP-034 was allowed to claim.

It did not prove biological efficacy.

It did not prove that every branch of the system was now stable.

It proved that controlled expansion could happen without breaking the accepted PASS/BLOCK baseline.

Stage-gate result

EXP-034 ended with all five stage gates passing:

Gate	Status
G1 parity	`PASS`
G2 reproducibility	`PASS`
G3 cross-experiment compare	`PASS`
G4 governance traceability	`PASS`
G5 extension safety	`PASS`

Final state:

Field	Value
overall status	`PASS`
anchor mode	`legacy_replay`
first failed gate	`null`
diagnostic hold	`Track-A current regeneration`

This is the important nuance:

The experiment passed with a retained diagnostic hold.

That is not a contradiction. It is the point of the control system.

The accepted anchor path was allowed to proceed. The current regeneration path was not. The remediation target was moved to EXP-035.

That separation is the actual proof EXP-034 provides: not that every branch became stable, but that instability was not allowed to contaminate acceptance.

What EXP-034 actually showed

EXP-034 did not show that the entire pipeline is now stable.

It showed something narrower and more useful:

A method-locked Bio-AI governance pipeline can admit modal expansion, AlphaFold EBI observer wiring, and AG-live measurement without losing its accepted PASS/BLOCK baseline — while keeping the unstable regeneration path out of acceptance.

Track-C sharpened that conclusion.

AG-live entered.
Metrics moved slightly.
The verdict did not change.
Dangerous false-pass did not appear.
Conservative over-blocking remained.

That is not a clean success story.

It is a governed result.

Closing

Stage-gated experimentation is not just about getting a result.

It is about deciding whether the result should be allowed to exist.

In EXP-034, the answer was:

GO   for the anchored expansion path
HOLD for current regeneration
NEXT for EXP-035 remediation

That may sound less dramatic than a clean success story.

But in governance work, that is exactly the point.

A mature AI pipeline is not the one that claims everything passed.

It is the one that can say:

This path passed.
This path did not.
And we did not mix them.