Coder Legion - Recent posts tagged hilbert-space

What is an Hallucination?

Sun, 16 Aug 2026 16:13:05 +0000

We began by asking what an idea is, and found no final answer, only a representation, a projection, an approximation. We then asked where an idea sits, and discovered that its meaning is inseparable from its position in a chain of custody, from context and contrast. Now we must confront the shadow twin of the idea: the assertion that arrives with every outward property of a true idea, confidently, coherently, fluently, and yet answers to nothing real. This is the hallucination. If an idea is a trajectory through a landscape of meaning, then a hallucination is a trajectory that has come loose from the ground beneath it..

The Philosophical Problem:

Plato's cave already anticipated the hallucination two and a half thousand years before the term existed. The prisoners do not mistake shadows for the fire because the shadows are poorly formed. They mistake them because nothing available to the prisoners distinguishes a well-formed shadow from a true form.

Descartes sharpened this into a formal problem with his evil demon and his dreaming argument. A perception, a memory, an entire chain of reasoning can be internally coherent, can obey every rule of logic and grammar, and still refer to nothing whatsoever outside itself. Coherence, Descartes showed us, is cheap.

This distinction, between coherence theories of truth and correspondence theories of truth, is precisely where the hallucination lives. An idea, however imperfectly it is expressed, still gestures toward something, an experience, an observation, an intention. A hallucination gestures with equal confidence toward nothing. It is not a bad idea. It is fluent reasoning that has been severed from its referent while retaining every syntactic and rhetorical marker of reasoning that has not.

The Psychological Perspective:

Cognitive psychology has long understood that the mind does not passively record the world. It constructs a most probable completion of it. Perception itself is inference: the brain fills gaps, resolves ambiguity, and completes patterns using prior expectation as much as incoming evidence. Under fatigue, sensory deprivation, or simple time pressure/stress, expectation begins to dominate evidence outright, and the result feels exactly like perception because, from the inside, it uses the identical machinery.

Split-brain research gave psychology one of its starkest illustrations of this. When one hemisphere acts on information the other hemisphere never received, the interpreting hemisphere does not report confusion. It confabulates a fluent, confident explanation for behaviour it did not actually cause, and believes its own explanation completely.

But this same construction is not always innocent, and it would be a mistake to treat hallucination as purely a story of information gaps. Hallucination can also be manufactured deliberately, and this is where it shades into deception. A narrative that is confident, coherent, and internally consistent, produced not because evidence ran out but because a corrupted internal state, a motive, an incentive, a concealed intention, required that narrative to be believed, is hallucination aimed with purpose. Deception, in this sense, is hallucination with intent behind it: the same fluency, the same absence of a true anchor, but now directed deliberately at a target rather than arising from an honest gap in evidence. Understanding hallucination psychologically therefore means holding two cases apart. One arises from an honest absence of grounding. The other arises from a corrupt state that manufactures the same absence on purpose, precisely because fluency alone is so easily mistaken for truth.

The Neurological Perspective:

A hallucination, neurologically, is not evidence of some separate, malfunctioning machinery. It is what happens when the same predictive architecture that produces every ordinary perception continues to run while the correcting signal weakens or disappears, through sensory deprivation, altered neurochemistry, extreme fatigue, or the hypnagogic drift at the edge of sleep. The trajectory through neural state space keeps moving with total confidence. Only its tether to a corrective signal is missing.

Mathematics, Computing, and the Geometry of Meaning:

Hilbert–Schmidt Dual-Operator Systems: A Model for Perceptual Dimensional Pooling, gives intuition an actual structure. The core idea is - a model's internal state, at any point in its reasoning, can be treated as an operator occupying a high-dimensional space. Only a portion of that space is genuinely anchored, meaning it is reachable, checkable, and accountable against something verifiable outside the model. The remainder of that space is what the paper calls a blind quotient manifold: directions the system is entirely free to move through, with full internal fluency and consistency, that carry no accountable connection back to any ground truth whatsoever.

A hallucination, in this geometric frame, is simply what happens when a generation trajectory drifts into that unanchored manifold while every surface property of the output, grammar, tone, apparent confidence, register, stays completely intact. Nothing about the output signals the drift. The drift is only visible once you look at the underlying geometry rather than the surface text.

This is exactly the instrument built as my Hugging-Face LLM-XRay. Rather than judging a model's output by its wording alone, it performs a spectral tomography of the model's own weight matrices, measuring how much effective, useable dimensionality the architecture actually carries versus how much collapses under scrutiny. It combines this structural measurement with a live behavioural benchmark: a corpus of verified factual probes, deliberately adversarial questions built on false premises no true answer could satisfy, and paraphrase-stability checks that ask whether the model's internal state stays consistent when the same question is asked a different way. A structural risk score and a behavioural risk score are then combined into a single diagnostic reading, not a verdict of falsehood, but a measured signal of how far, and how often, a model's trajectories are wandering into the unanchored part of its own geometry.

Try It Yourself:

The application is live and open to inspection. Run it against a small model, or point it at a benchmark corpus of your own, and watch the two risk composites move independently: a model can be structurally well-anchored yet behaviourally unreliable under adversarial pressure, or the reverse. Nothing here claims to detect a lie with certainty. It offers something more modest and more useful, a geometry of where confidence has stopped being backed by anchoring.

Hallucinations are a diversion from the norm, not a mystical failure unique to machines, not proof of malice, and not, on their own, evidence of deception, though deception is always a hallucination with intent layered on top of it. A hallucination is a mathematical occurrence: a trajectory that has left its anchored subspace while retaining every outward property of a well-formed idea. This is not speculation about the inner life of models. It is something that can be measured, probed, audited, and shown, directly and repeatably, as demonstrated in LLM-XRay - then finally eradicated at its core.

What is an Idea...?

Sun, 02 Aug 2026 12:22:26 +0000

GITHUB | WEBSITE

Can It Ever Truly Be defined?
Ideas have shaped civilizations, sparked revolutions, inspired masterpieces, and transformed our understanding of reality. Yet despite their immense influence, they remain among the most elusive phenomena we encounter.

We speak of having an idea, sharing an idea, even losing an idea, but what exactly is an idea? Is it a thing? A process? A pattern? And perhaps the most provocative question of all: can an idea ever be objectively measured? The answer, surprisingly, begins long before computers and artificial intelligence.

The Philosophical Problem:
For thousands of years, philosophers have wrestled with the nature of ideas. Plato viewed them as eternal Forms—perfect abstractions existing independently of the physical world. To him, what we perceive are merely imperfect reflections of these ideal truths.

Aristotle disagreed. He argued that ideas emerge through experience, abstraction, and observation rather than existing in some transcendent realm.

Centuries later, John Locke proposed that the mind begins as a tabula rasa, a blank slate upon which experience writes.

The Psychological Perspective:
Psychology shifted the question from What is an idea? to How do humans form ideas Modern cognitive science suggests that ideas are not isolated objects stored inside the brain. Instead, they emerge from enormous networks of interconnected concepts, experiences, emotions, language, and memory.

When someone hears the word apple, they are not merely recalling a dictionary definition. Their brain simultaneously activates colours, tastes, childhood memories, cultural associations, scientific knowledge, emotions, and countless related concepts.

Every idea exists within an immense web of relationships. This immediately presents a problem for measurement.

If every individual's conceptual network is unique, then no two people possess precisely the same idea, even when they use identical words. Context becomes inseparable from meaning. Ideas become dynamic rather than static.

The Neurological Perspective:
Neuroscience pushes the mystery even further. There is no single "idea neuron." Instead, ideas appear to emerge from coordinated activity across vast populations of neurons distributed throughout multiple regions of the brain. Visual cortex, language centres, emotional processing, working memory, and executive reasoning all contribute simultaneously.

An idea is therefore less like a stored object and more like a transient pattern of synchronised activity.

Modern neuroscience increasingly describes cognition using geometry rather than symbols. Neural populations occupy high-dimensional state spaces. Thought becomes movement through those spaces. Learning reshapes the landscape itself. In this view, an idea is not a point—it is a trajectory.

Mathematics, Computing, and the Geometry of Meaning:
Machine learning has quietly embraced a remarkably similar philosophy. Large language models no longer manipulate explicit symbolic definitions. Instead, words, phrases, and documents are transformed into high-dimensional vectors known as embeddings. Concepts that share meaning naturally occupy nearby regions within these mathematical spaces.

This does not imply that the machine understands ideas in the human sense. Rather, it discovers statistical structure within language and represents that structure geometrically. Once ideas become vectors, mathematics offers an astonishing toolkit:

Distances can be measured.
Angles can be compared.
Clusters emerge.
Trajectories can be followed.

Entire conversations become paths through semantic space.

More sophisticated approaches extend this even further by representing ideas not merely as vectors, but as operators, probability distributions, or geometric objects whose relationships can be analysed using techniques from information geometry, Hilbert spaces, spectral analysis, and quantum-inspired mathematics.

Surely we have finally measured an idea?

The Answer Is Still No:
The answer remains a resounding No!

No mathematical framework can claim to have captured an idea itself.

An idea possesses:

Context.
Experience.
Culture.
History.
Emotion.
Purpose.
Intention.

Two people may express identical sentences while meaning entirely different things. Conversely, two completely different sentences may convey precisely the same underlying insight. Any mathematical representation is therefore exactly that:

A representation.
It is a model.
It is a projection.
It is an approximation.

No embedding, neural network, operator, or manifold contains the idea itself. But that does not mean mathematics is powerless.

Measuring the Frame Around an Idea:
While an idea cannot be fully contained, the structure surrounding an idea can often be characterised. This distinction is crucial.

Rather than asking,

Can mathematics understand ideas?

we ask,

Can mathematics compare the structures through which ideas are
expressed?

That question has a far more promising answer. My experimental Hilbert–Schmidt application explores precisely this possibility.

Instead of attempting to declare whether two ideas are "the same," the system constructs mathematical representations that preserve aspects of semantic structure. These representations are converted into density operators and compared using a collection of geometric measures, including Hilbert–Schmidt similarity, Uhlmann fidelity, Bures geodesic distance, spectral projections, intent fidelity, and chain integrity metrics.

Rather than producing a simplistic similarity score, the application examines how meaning appears to evolve across structure.

Where does a chain remain coherent?

Where does it fracture?

Which transition introduces the greatest conceptual drift?

The result is not a judgement of truth, nor a verdict on correctness.

It is a quantitative exploration of the geometry of discourse.

Try It Yourself!
Perhaps the most interesting aspect of this project is that it invites experimentation. Construct a carefully reasoned chain of thought. Then construct another composed of deliberately unrelated ideas. Watch how the geometric relationships evolve. Observe where semantic continuity appears to strengthen—or collapse.

The application makes no claim to have solved the mystery of ideas. Instead, it offers you something arguably more valuable: a framework for exploring OR to dismiss them!

Your ideas/concepts may not reside solely within individual thoughts, semantics, philosophical, psychological, mathematical or computational bounds; but they may reside within a higher geodesic construct that connects them all.

Spectral LLMs and the Koopman Hilbert Space!

Fri, 17 Jul 2026 10:21:12 +0000

Quick question, before you proceed: "Did Your AI, LLM, MCP or Agent Make a Mistake Today?"

Be honest, something unexpected/unexplainable happen - correct?

If you're building AI systems today, something probably DID go wrong.

Maybe your agent called the wrong tool.
Maybe your MCP server returned stale data.
Maybe your RAG retrieved the wrong document.
Maybe your workflow hallucinated because one component quietly failed.

The good news?

If the mistake happened outside your language model, you've probably already fixed it.

You update the tool.
You improve the retrieval.
You patch the API.
You add another guardrail.
You tighten the prompt.

Problem solved.

Or at least... until next time.

But what if the mistake happened inside the model?

That's where things become difficult. Today's LLMs are incredibly capable, but they're also largely opaque. When an internal reasoning error occurs, your options are surprisingly limited.

You can:

retrain the model
fine-tune another checkpoint
realign the model
rebuild your RAG
hard-code another exception
write another prompt workaround

None of these actually tell you what changed inside the model.

There is no audit trail. There is no chain of custody.There is no way to point at a specific internal memory and say:

"This is where the model learned something new."

What if model memory were inspectable?

This is the question behind Spectral LLMs.

Instead of representing memory as an enormous hidden vector that disappears into billions of parameters, Spectral LLMs represent model state as a compact mathematical operator whose evolution can be inspected step by step. Each state transition leaves behind measurable evidence.

Instead of wondering why the model changed, you can observe:

how the internal memory evolved
whether genuinely new information entered the system
whether existing knowledge was preserved
whether multiple memories merged
whether the model is drifting over time

Every update has provenance. Every modification has an explanation. That is a fundamentally different philosophy from today's black-box neural memory.

Traditional fine-tuning changes enormous numbers of parameters simultaneously. Even when the desired change is tiny, the update often spreads across the network.

That creates familiar problems:

catastrophic forgetting
unexpected behavioural changes
difficult debugging
repeated alignment work

Spectral LLMs aim for something much more precise. Because memory exists as explicit operator states rather than hidden activations, new knowledge can be incorporated directly into those states instead of diffusely modifying millions (or billions) of weights.

The long-term goal is directed learning:

learn one thing...

without accidentally changing hundreds of unrelated things.

Surgical tuning instead of model surgery

Imagine updating only the knowledge that actually needs changing. Not the whole model. Not another expensive fine-tune. Just the relevant memory.

That's the vision behind surgical tuning.

Rather than repeatedly rebuilding large portions of a model to correct small mistakes, individual operator memories can be examined, compared, updated and re-integrated while maintaining a complete audit trail.

This doesn't eliminate the hard problems of AI, but it creates a representation where those problems become measurable instead of invisible.

Why the Koopman Hilbert–Schmidt Operator matters

At the heart of Spectral LLMs is the idea of representing persistent model state as a compact Hilbert–Schmidt operator that evolves through Koopman-inspired dynamics.

Rather than treating intelligence as a sequence of hidden vectors that vanish after each context window, the model maintains structured operator states whose evolution can be monitored over time.

Those operator states expose rich spectral properties that act like diagnostics for memory itself. Instead of asking only what the model predicted, we can begin asking how its internal state evolved to reach that prediction.

That opens the door to continuous inspection, targeted updates and reproducible memory management.

Responsible AI starts with accountability

As AI systems become responsible for increasingly important decisions, transparency becomes more than a research goal.

It becomes an engineering requirement. If we cannot explain how an internal memory changed...

we cannot confidently explain why a model behaved differently today than it did yesterday.

Spectral LLMs propose an alternative direction:

inspectable memory
measurable state evolution
complete chain of custody
directed learning
surgical tuning
responsible model updates

The ultimate ambition is not simply to build larger models, but to build models whose internal reasoning and memory can be observed, refined and improved in a controlled and accountable way.

That is an important step toward AI systems that are not only more maintainable, but have the potential to support more structured and reliable reasoning.

Get your Spectral LLMs demo here: https://zenodo.org/records/21410792