Coder Legion - Recent posts tagged hallucination

How Synthadoc Handles Hallucination and Why "Don't Hallucinate" Isn't Enough

Fri, 28 Aug 2026 17:18:42 +0000

Ask most AI products how they handle hallucination and you'll get a variant of "we tell the
model not to." That's not wrong — it's just insufficient, and it puts enforcement in the
hands of the exact system doing the generating.

Synthadoc treats hallucination as an architecture problem, not a wording problem. The
distinction matters: a wording fix is hoping the model behaves; an architecture fix is
verifying that it did.

Why Prompt Instructions Aren't Enough

The industry default is a line in a system prompt: don't make things up, answer only from
the provided context, never fabricate. These instructions do nudge model behavior in the
right direction, and we use them too. But they leave enforcement entirely up to the model
doing the generating. A language model producing text token-by-token has no reliable
internal mechanism to audit its own faithfulness to a source.

There's a subtle issue as well. Instructions of the form "don't do X" are easier for a
model to rationalize around than instructions of the form "do only Y." Synthadoc's synthesis
constraint is a positive formulation:

Answer using ONLY these wiki pages. Cite with [[PageTitle]].

That's linguistically harder to rationalize around. But it's still just text in a prompt.
The real work happens at the structural level.

Three Layers, Not One Instruction

Our approach treats hallucination as a systems problem. There are three distinct layers,
each handling a different failure mode:

Scope filtering at ingest time : Wiki content can only be entered if it is on-domain. It is difficult to produce hallucinations about off-domain topics from content that has never been consumed.
Retrieval-augmented grounding at query time : Answers are generated from retrieved
wiki pages, not from the model's weights alone. The synthesis prompt shows only the
relevant pages and instructs the model to answer from them exclusively.
Citation faithfulness audit after the fact : Every claim gets a citation marker
linking it to specific source lines. A separate LLM pass then audits whether each claim
is actually supported by those lines. This is the layer that catches what the other two
miss.

None of these is a complete solution alone. Together they create a system where hallucinations
are detectable and traceable, not just discouraged.

Layer 1: purpose.md as a Content Gate

Every wiki has a wiki/purpose.md file that describes its domain — what it's for, what
topics belong, and what doesn't. When a new source document arrives for ingest, the ingest
agent reads this purpose block before making a decision:

Wiki scope (from purpose.md):
{content of purpose.md}

action="skip" means the source is completely OUTSIDE the wiki's domain
(e.g. spam, medical receipts, unrelated e-commerce).
…

The LLM then decides: skip, create, update, or flag. Out-of-scope content never
becomes a wiki page, which means it can never become the basis for an answer.

Lifecycle isolation for contradicted pages

The scope gate isn't the only place the knowledge base shrinks. When a new source
disagrees with an existing page, the ingest agent issues a flag action and the page
moves to a contradicted lifecycle state, and automatically excluded from retrieval until
the conflict is resolved. This matters because a contradicted page is, by definition,
one where the system can't be confident about what's true. Serving answers from it
would be hallucination with extra steps.

The contradiction resolver in Synthadoc is an interactive agentic workflow: it surfaces
the conflict, shows a diff of the proposed edit, and requires explicit human approval
before writing any change. The approve-before-write gate is unconditional, and the agent
cannot modify a page without a human seeing the diff first.

Layer 2: RAG Grounding at Query Time

When a user asks a question, Synthadoc retrieves the most relevant wiki pages using
hybrid search (BM25 full-text matching with optional vector re-ranking), then builds a
synthesis prompt that includes those pages verbatim:

Answer using ONLY these wiki pages. Cite with [[PageTitle]].
Extract and include all specific facts from the pages — dates, years,
numbers, and names — even when they appear briefly or in passing.
Do not claim a fact is absent unless it is genuinely missing from
every page below.

Question: {question}

Pages:
{retrieved_wiki_pages}

The purpose.md content is prepended to the pages block as ### Wiki Scope, giving the
model a domain reminder during synthesis as well as at ingest time.

Why this isn't enough on its own: A model can still hallucinate within retrieved
content — misattributing which page said what, inventing a specific number close to but
not the same as a number in the source, or synthesizing a conclusion that no single page
actually states. Grounding reduces the search space; it doesn't eliminate drift.

When retrieval confidence falls below threshold, Synthadoc detects the gap and handles
it transparently rather than hallucinating silently. The immediate response answers from
the model's general knowledge while explicitly noting that the wiki lacks dedicated
coverage — the user always knows they're receiving an unverified answer. At the same time,
Synthadoc generates enrichment suggestions: search queries and known URLs scoped to the
domain in purpose.md. Retrieved pages flow through the normal ingest pipeline and stay
in draft state — out of the search corpus — until a human promotes them to active.
Only then do future queries on that topic get answered from verified wiki content rather
than general knowledge.

Layer 3: Citation Faithfulness Audit

This is the layer that makes the system verifiable rather than just hopeful. The citation
faithfulness audit examines each claim in a wiki answer alongside the source lines it cites,
then classifies the relationship:

Verdict	Meaning
`supported`	The claim is directly and accurately supported by the cited source lines.
`drift`	The claim is related to the source but overstated, simplified, or shifted in meaning.
`hallucination`	The source lines do not support the claim, or directly contradict it.

Each audit is done by a separate LLM call with the claim text and raw source lines
side-by-side. The auditor never sees the original question — only the claim and the lines.

Why a separate LLM pass? The same model that wrote the answer is likely to defend
it when asked "is this correct?" Separation of duties matters: the auditor has no stake
in validating the original generation.

What we actually see in practice

In our experience, drift is the most common finding — not outright hallucination.
The model summarizes accurately but slightly overstates a confidence level, rounds a
number, or drops a qualifier ("as of 2023" becomes "currently"). These are small errors
individually but compound across a knowledge base that gets queried repeatedly.

True hallucination , where the cited source lines don't support the claim at all ,
is less common in well-grounded RAG, but it happens: specific dates that don't appear
in the source, an attribution switched between two people mentioned in the same paragraph,
a causal relationship inferred from correlation. Those are the findings that matter most,
and the audit surfaces them per-citation so a human knows exactly which source line to
check.

The audit result is cached per-page with staleness detection. If the page is re-ingested
from updated source material, the cache is invalidated and the next audit reflects the new
content. This surfaces in the web UI as an automatically-loaded faithfulness tab alongside
each wiki page.

What This Doesn't Solve

Honesty matters here too. Two failure modes remain outside what the three layers fully
address:

Synthesis across pages. A conclusion drawn by combining facts from multiple pages may
be accurate at each individual step and still misleading in combination. The citation
faithfulness audit operates per-citation, not across synthesized conclusions, so a
cross-page inference that no single page actually states falls outside its scope.
The practical path: when a synthesized answer matters, the cited pages are explicit and
can be verified directly. If the conclusion is important enough to keep, saving it as a
wiki page subjects it to future source ingestion — a new source that disputes it will
trigger the flag action for human review.

Source quality. When a new source disagrees with an existing page, the flag action
fires and the contradiction is surfaced. The harder case is when all available sources
agree on the same wrong answer: no flag fires, no contradiction is detected, and Synthadoc
answers confidently from incorrect material. Two mechanisms make this less likely: the
adversarial review in lint, which uses the model's own knowledge to flag claims that
clearly contradict well-established facts; and re-ingesting from updated authoritative
sources, which will trigger the flag action against any outdated pages. Neither is a
complete solution, but together they make uniform source error less likely to go
unnoticed indefinitely.

Our View

The hallucination debate tends to frame the problem as a model capability question: will
the next model be more faithful than the last? We think that's the wrong frame for a
production knowledge system. Even a perfectly faithful model can only be as accurate as
the content it's grounded in, and even a slightly drifting model can be made accountable
if the output is systematically audited.

Our architectural bet is that the answer to hallucination isn't a better instruction,
it's a feedback loop. Domain scoping controls what enters. Lifecycle states control what
gets queried. Retrieval grounds generation. Citation audit classifies the output. The
contradiction resolver closes the loop: when a conflict is found, a human resolves it, the
page is corrected, and the knowledge base improves. Errors become visible rather than
accumulating silently.

That's a more defensible position than any number of ALL CAPS instructions.

Try It

Synthadoc is open-source. The citation faithfulness audit, contradiction resolver, and
lifecycle system described here are all in the main branch.

→ github.com/axoviq-ai/synthadoc

Where Is a Hallucination?

Fri, 21 Aug 2026 01:12:45 +0000

GITHUB | WEBSITE | DEMO

Can We Pinpoint the Point of Failure?
We began this inquiry by asking what an idea is, and found no absolute container—only representations, projections, and approximations.

We then asked where an idea lives, discovering that meaning cannot be separated from its position along an evolving chain of custody, governed entirely by context and contrast.

Next, we confronted what a hallucination is: the shadow twin of an idea, possessing all the outward fluency, syntax, and confidence of sound reasoning while completely severed from any grounded referent.

Now, we must confront the ultimate operational and scientific question: Where is a hallucination?

When an enterprise AI system, a legal reasoning engine, or a multi-agent workflow generates an error, traditional evaluation treats the output as a uniform block of text. We label the whole response as "hallucinated" or "unreliable."

Yet, an error does not occur everywhere at once. A twenty-step reasoning trajectory does not fail uniformly across all twenty steps. It begins in solid ground, proceeds along rigorous premises, and then—at one distinct, unheralded moment—it takes a wrong turn. A fracture forms. The trajectory leaps across an unbridged chasm, dragging the rest of the generation into confabulation.

To govern artificial intelligence, eliminate catastrophic failure in enterprise architectures, and build verifiable machine cognition, we must stop treating hallucinations as vague atmospheric fogs. We must locate the geodesic coordinate of failure.

The Philosophical Problem:
Classical philosophy has long observed that deception and error rarely announce themselves at the outset. In formal epistemology and classical logic, a flawed syllogism does not fail because its individual terms are corrupt. It fails because of an illegitimate step: an illicit minor, an undistributed middle, or an unearned inferential leap.

Zeno’s paradoxes and Cartesian skepticism both rely on the same fundamental realization: you can agree with ninety-nine consecutive steps of a journey, but if step one hundred quietly changes the operational rules without declaring the shift, the entire voyage is cast into illusion.

The hallucination is not a state of being. It is an act of departure. It is a discrete, identifiable location in the discourse where coherence abandoned correspondence.

The Psychological Perspective:
Cognitive psychology and neuroscience demonstrate that human confabulation is not a continuous, steady state of confusion. It is a compensatory suture applied across an evidentiary void.

Consider clinical split-brain patients or subjects under deep hypnotic suggestion. When asked why they performed an action initiated by a stimulus they were consciously unaware of, the human brain does not pause or report a missing variable. Instead, it instantly, fluently fabricates an explanation.

Crucially, there is an exact micro-moment when this occurs:

The subject possesses legitimate awareness of their current state
(the Anchor).
The subject encounters an unbridgeable evidentiary gap.
The cognitive "interpreter mechanism" leaps across the gap to maintain narrative continuity (the Drift).

In human cognition, this transition happens beneath conscious awareness. The speaker genuinely believes their own confabulation because the transition was smooth, fast, and syntactically flawless.

When Large Language Models hallucinate, they replicate this exact psychological phenomenon. They do not generate noise; they generate momentum. A hallucination lives at the precise seam where the model runs out of statistical anchoring and substitutes stylistic momentum to bridge the gap.

The Neurological Perspective:
From a neurological standpoint, cognition is the movement of high-dimensional state vectors across an energy landscape sculpted by synaptic weights.

In attractor network theory, healthy reasoning corresponds to a trajectory smoothly navigating deep, grounded attractor basins—valleys of stable, learned associations kept in check by sensory inputs and top-down cognitive constraints.

A hallucination occurs when the neural trajectory reaches a saddle point: an unstable ridge between two basins. If the corrective signal is weak, the state vector bifurcates. It leaves the grounded attractor basin and falls into an unconstrained, runaway drift basin.

Mathematics, Computing, and the Geometry of Meaning:
How do we translate this conceptual trajectory into an empirical measurement inside a live transformer?

In our earlier work with LLM-XRay, we introduced spectral tomography: measuring the global dimensional health of a model’s weight matrices using Hilbert-Schmidt Operator frameworks to see how much of its latent geometry is grounded versus how much is blind.

Now, we zoom in from global capacity to local differential geometry: tracking the minute step-by-step path of a single reasoning chain.

When an LLM produces a step-by-step Chain of Custody (CoC) answer, we can treat each discrete reasoning step as a distinct state in high-dimensional latent space. By projecting these step-embeddings into rank-1 density operators, the reasoning chain becomes a discretized geometric curve—a trajectory traveling across a manifold.

Across this curve, we compute the local geometry of every step-to-step transition:

Manifold Curvature: How sharply does the path bend between
consecutive steps?
Commutator Tension: Do the operator states at step A and step B
smoothly commute, or does their order introduce semantic friction
and non-Euclidean shear?
Fidelity Deficit: How much quantum-inspired state overlap (Uhlmann
fidelity) is lost between the premise and the deduction?

When a model reasons soundly, these geometric signals remain stable, oscillating smoothly along an expected baseline.

But when the model takes an unearned logical leap; when it introduces an undefined variable, confuses a causal relationship, or jumps to an ungrounded conclusion, the transition registers a sharp, violent spike in geometric curvature and entropy production.

The hallucination is no longer invisible. It is mapped as a localised geometric discontinuity.

Isolating the Fault-line:
This is the exact engine we have engineered in the LLM Hallucination Geodesics Auditor:

Phase 1 (Screen): The engine probes the target model across core
cognitive vectors (calculation, tokenization, logic, and factual
memory) to identify its weakest reasoning domain.
Phase 2 (Elicit): It prompts the model to generate an exhaustive,
multi-step Chain-of-Custody answer within that vulnerable domain.
Phase 3 (Locate): The auditor passes the step-by-step latent
embeddings through the Hilbert-Schmidt geometric analyzer,
calculating the exact transition that exhibits maximum curvature,
commutator tension, and fidelity deficit. It isolates the single
transition where the trajectory fractured: Step A (The Anchor)
Phase 4 (Self-Audit Ratification): To prove that this geometric
coordinate represents a genuine hallucination—and not an artefact of
our mathematical framing, the engine extracts that exact Anchor
Phase 5 (Mini-Report): A dedicated evaluation model reviews the real,
empirical measurements and compiles a concise diagnostic summary of
the failure.

The Horizon:
We have moved beyond asking what an idea is, where it sits, and what constitutes its shadow. We now know where a hallucination lives: it is a localised, measurable fracture along the geodesic trajectory of thought.

By isolating the exact point of departure at the hidden-state and weight level, and ratifying that failure through adversarial self-inquiry, we transform AI governance from an exercise in probabilistic guessing into a deterministic, verifiable science.

A hallucination is no longer an invisible ghost in the machine. It is a coordinate. And once a point of failure has a coordinate, it can be audited, proven, and permanently repaired.

Explore the live Geodesic Auditor and inspect real hallucination certificates on Hugging Face Spaces.

Hallucination Is Not One Problem — It's Five Different Architectural Failures

Fri, 03 Jul 2026 21:07:27 +0000

Why your RAG pipeline, guardrails, and prompt engineering each fix a different hallucination — and why none of them fix the others.

The Frustration

You built a RAG system. Added re-ranking. Tuned the temperature. Even threw in a "don't make things up" prompt. And yet — your agent still hallucinates. So you add more retrieval chunks. Then guardrails. Then another LLM call to "verify" the first one. The hallucinations shift shape, but they never disappear.

Here's the hard truth: you're treating five different architectural failures as one "hallucination bug."

Type 1: Confabulation — The LLM Makes Facts Up

What it looks like: "According to a 2023 study by Stanford..." (no such study exists)

Why it happens: The model is a next-token predictor with no ground-truth anchor. It optimizes for plausibility, not accuracy.

The fix — Retrieval Grounding + Fact Verification:

Every claim must cite a retrieved source
Verification LLM checks claim ↔ source alignment
Abstention gate: if no source supports it, output "I don't know"

Prompt engineering won't fix this. The model has no fact-checker in its weights. You need an external verification architecture.

Type 2: Attribution Error — Right Fact, Wrong Source

What it looks like: Cites Document A for a fact that actually came from Document B

Why it happens: Semantic similarity retrieves the wrong chunk; the LLM doesn't verify provenance

The fix — Provenance Tracking + Citation Scoring:

Chunk-level metadata: source ID, section, timestamp
Citation scorer: cross-check generated citation against retrieved chunks
Mismatch → flag or abstain

Type 3: Temporal Drift — The Truth Expired

What it looks like: "The latest React version is 18.2" (it's 19.x now)

Why it happens: Static knowledge base with no freshness signal; retrieval doesn't prioritize recency

The fix — Knowledge Freshness Timestamps + Update Pipelines:

Every chunk tagged with last_updated
Retrieval re-ranks by recency for time-sensitive queries
Scheduled re-indexing pipeline for evolving domains

Type 4: Logical Inconsistency —Contradictory Claims in One Answer

What it looks like: "X is faster than Y" and "Y outperforms X" in the same response

Why it happens: LLM processes tokens locally; no global consistency check across the full output

The fix — Structured Reasoning + Formal Verification:

Chain-of-thought with explicit intermediate claims
Consistency checker: compare all claims pairwise
For critical domains: compile claims to SAT/SMT solver

Type 5: Tool Hallucination — The Agent Calls a Tool That Doesn't Exist

What it looks like: call_tool("get_weather", location="Mars") — tool doesn't exist, or parameter is invalid

Why it happens: The LLM generates tool calls like it generates text — by pattern matching, not by schema validation

The fix — Tool Schema Validation + Execution Sandbox:

Tool registry with JSON schemas
Pre-execution validator: name + params must match schema
Sandbox: tool runs in isolated env, output validated before passing back

The Unified Architecture

Closing

Hallucination is not a bug you patch. It's a symptom of five different architectural gaps. Fix the architecture, and the "hallucination problem" dissolves into five solvable engineering problems.