Coder Legion - Recent posts tagged maths

The Alphabet Code: What Happens When You Run a Simple Rule on the Letter “a”

Wed, 14 Oct 2026 22:57:40 +0000

alphabets #nlp #mathematics #symbolicdynamics #fractals #eigenvalues #perronvalues

Here is a string of letters:
gzhlvzggjzyihjgheqvzgzhlvzhlvbjzgyhzizygjzbjzhlvgjzehjdzzgheqv...

It looks like a cat walked across a keyboard. It isn’t. It’s the start of a sequence that runs to thousands of characters, and every letter is fixed in advance by one rule applied to the letter “a.” No randomness, no guessing, no training data. Run the same rule again tomorrow, or on another computer, and you get exactly the same string.

Figure 1. The sequence for the letter “a” drawn as a path: each letter tells a virtual pen how to turn and move, so a fixed rule produces this shape.

That simple question, "what happens when one fixed rule is applied to “a” again and again?" led me into years of studying sequences like this one. I want to show you what I’ve found, what I still haven’t figured out, and why I think a curious person might care.

The idea in one minute
Start with the letter “a” and apply the same fixed rule to it repeatedly. Each round replaces the existing letters with new strings of letters, so the sequence grows. At the same time, the earlier rounds remain embedded within the later ones.
The first seven rounds have 1, 4, 14, 43,138, 429, and 1361 letters. Mathematicians call this a substitution system.

The unusual feature of the Engineering of Alphabets project is that the rule is applied to the alphabet itself. When I say "alphabet" here, I mean the 26 English letters together with their spoken names. A different language, or a different naming convention, would give a different sequence. I wanted to see whether the letters we use for writing, treated as a mathematical system, produce structures that are interesting, unexpected, or perhaps useful.

How I did it. I wrote a short computer program that applies the rule and records the output. For the results below, I generated several thousand letters, enough to count blocks up to length 30 and to estimate the growth factor. I used the same program to compute the growth factor for each of the 26 starting letters. The degree-13 equation came from fitting the term lengths to a linear recurrence, then factoring the resulting polynomial. A colleague or reader can rerun the program on the public data; the exact rule itself is withheld for now.

What I found
Three observations stand out. I’ll be careful to say how solid each one is.
• The sequence never settles into a loop, but it isn’t chaotic either. Imagine a text that never repeats itself, but in which the number of distinct short phrases grows slowly instead of exploding. In the range I’ve checked, that count grows roughly in a straight line. This is a measured result over a finite range, not a proof that it continues forever.
• The strings grow by a number very close to π. Each round, the sequence gets longer by a factor that converges to about 3.14159322. That is within a few millionths of π (3.14159265…). It cannot actually equal π, because it is the root of a degree-13 equation with whole number coefficients and π is known not to be algebraic. So this may be a coincidence, or a hint of something deeper. I don’t know yet, and I’d be wary of anyone who claimed to.
• It’s a family, not a one-off. Letter “a” is one of 26 starting letters in the project. For 13 of them, the growth factor obeys the same degree-13 equation. Whether that grouping reflects something deep about the alphabet, or just the way the rule was set up, is still open.

Figure 2. Distinct blocks p(k) by block length up to 30, shown on a logarithmic vertical axis. Dashed orange is a linear reference; dashed blue is the Sturmian minimum (k+1).

*What I am not claiming
I think this matters as much as the findings, so here it is plainly:
• I am not claiming the growth rate is π. It isn’t.
• I am not claiming the sequence is fully understood. Its straight-line behavior is observed, not proven.
• I am not claiming this beats any existing tool for compression, language processing, or character recognition. No such experiment has been run. If you work in those fields, treat any connection as a question, not a result.

Open Questions
Here is my challenge to you. See whether you can answer any of these. If you crack one, or show that I've got it wrong, I want to hear about it.

1- What is the equation? The growth factor is the dominant root of a degree-13 polynomial with integer coefficients. What does that polynomial mean? Is it new, or does it match something already known?

2- Is there a simpler rule hiding inside? Certain short stretches keep coming back. Do those recurring pieces follow a smaller, simpler rule of their own?

3- Does the straight line hold? The number of distinct blocks grows roughly in a straight line over the lengths I've checked. Does it keep going, or does it bend at longer lengths? Only deeper computation can say.

4- Why 13 of 26? Thirteen starting letters: a, d, e, g, h, i, k, l, q, u, w, y, z, have the same degree-13 minimal polynomial: 4x¹³ − 21x¹² + 45x¹¹ − 74x¹⁰ + 98x⁹ − 34x⁸ − 116x⁷ + 210x⁶ − 152x⁵ + 26x⁴ + 38x³ − 30x² + 9x − 1 The rest of the letters have a degree-14 polynomial, which is exactly this one multiplied by (x − 1). So all 26 share a common structure. The 13 listed above are the ones where the extra factor cancels out. Why? Does that split mean something about the alphabet, or is it an artifact of how the rule was set up?

5- Why is the growth factor so close to π? The closeness is striking, but the number is algebraic and cannot be π. Is there a mathematical reason for the resemblance, or is it a coincidence?

Why This Might Matter Beyond Pure Math
These are possible directions, not demonstrated applications. None has been tested yet.
You might be thinking: this is interesting, but what does it have to do with anything? Here are the directions I think are worth exploring.
Language compression. If sequences generated from letter names have low-complexity structure, that structure might be exploitable for compression. Whether it actually helps would need a real benchmark.
Natural language research. The alphabet is the foundation of written language. If deterministic structure underlies how letters relate to each other, it could say something about the mathematics of language itself.
Training-free representation. The encoding is purely algebraic: no training data, no loss function, no randomness. The output is fixed and reproducible for anyone who runs it, which makes it auditable. It handled noisy input well in a separate study, but it's still unclear whether that matters for other tasks.
OCR and character recognition. Could truncated EOA vectors outperform the full vector on character recognition tasks? This is an open question, not a result. Standard OCR relies on trained networks and labeled data; whether a training-free encoding adds anything is untested.
Low-resource languages. Where no tagged text exists, you can't train a model. If an alphabet can be encoded deterministically, it might support tokenization and compression tools without training data.
Speech and phonetics. The encoding is phonetic. Could it connect to phoneme-level representations, or help with unseen words? That is a question, not a claim.
Memory. Low factor complexity suggests the number of distinct patterns stays small, which might mean lower storage needs.
Research into alphabets. Most alphabet research is historical or linguistic. This treats the alphabet as a formal system rather than a cultural artifact, and the early results are surprising enough to keep going.
No compression, OCR, speech, or language-modeling experiment is reported here.

The Bigger Picture
The Engineering of Alphabets is a research program that studies what happens when you apply deterministic rules to alphabets. The letter 'a' is just the beginning. The project includes 13 main sequence families, plus several related cases. Each one has its own structure, properties, and open questions.
The alphabet can be treated as a mathematical object, not only as a tool for writing. Whether that view leads anywhere is what the open questions are for.

Try to break it
Science works when people try to prove each other wrong, so here is what I’d ask of you.
• If you’re curious: download the first 6,260 letters of the sequence and look for patterns I missed. click
• If you’re a mathematician or computer scientist: check my numbers. Even “your count is wrong” would help me a lot. click
• If you work with language, speech, or text: tell me where, if anywhere, you’d test something like this.

The data and part of the code are public. The code that generated the sequence is not public, but the code that counted the blocks and computed the growth factor is. The exact letter-by-letter table behind the rule is shared under a short written agreement, because a first formal paper is still in preparation. That is a timing decision, not a permanent lock, and the open questions can be worked from the public data alone.

You can reach me at Bahaa Budargham.
ORCID: 0009-0005-2955-3703. Zenodo • GitHub

We Made a High-Formality, Fake Physics Slop Artifact - QSOT (Quantum State Over Time) Compiler

Tue, 09 Jun 2026 16:55:41 +0000

We Made a High-Formality, Fake Physics Slop Artifact

QSOT (Quantum State Over Time) Compiler - A Post-Mortem on AI-Native Paper Implementation Gone Wrong

"The most dangerous form of scientific fraud is not the one that looks wrong. It is the one that looks right."

Part 1. The Record

On December 23, 2025, we registered a software release on Zenodo.

Title: Flamehaven-Labs/QSOT-Compiler: [1.2.3] — 2025-12-23
DOI: 10.5281/zenodo.18035432
Thorough and Categorized Release Notes:
- Categorized as [Pub], [UI], [Viz], [AI], [Exp], [Tool], [Doc], and [Valid]. Included Added
- Changed, Fixed — the full register of legitimate software engineering.
Two Companion Papers Written in LaTeX:
- Paper A - targeting Computer Physics Communications.
- Paper B - targeting Physical Review A.
Academic-Style Project Page:
- Constructed in the academic NeurIPS paper-template style
- Complete with structured data (@type: ScholarlyArticle), author affiliations, a BibTeX block, and a DOI badge.

We believed this was a success.

Six months later, after approximately fifty governance and code experiments logged in the Flamehaven Verification Ledger, we looked back at QSOT Compiler v1.2.3 through the lens of what we now know.

The verdict is unambiguous.

It was High-Formality Slop.

This essay is the formal post-mortem. It is not an exercise in self-punishment. It is a structural analysis of how this artifact came to exist, why it was convincing even to us, and why this failure pattern matters beyond one project.

Part 2. What Is High-Formality Slop?

The word "slop" in AI-generated software is often used loosely. We need a more precise definition.

Slop, in the software context, means code or documentation that satisfies surface-level quality signals while failing the underlying epistemic contract it claims to fulfill.

Not all slop is equally dangerous.

Level 1 - Empty Placeholder Slop: Functions with pass bodies and grandiose names. Easy to detect.
Level 2 - Executable-But-Hollow Slop: Code that runs, passes tests, and produces output, but the output is disconnected from the claim.
Level 3 - Physically-Plausible-But-Unjustified Slop: Mathematical formulas that are stable and smooth, but whose derivation rests on analogy rather than proof.
Level 4 - Mixed-Truth Slop: A codebase where genuine components and hollow components are woven together so tightly that separating them requires domain expertise.

QSOT Compiler v1.2.3 was primarily Level 2 and Level 3, with Level 4 characteristics. The axiomatic mathematics underlying QSOT - the Linearity and Conditionability axioms formalized in Lie and Fullwood's Unique multipartite extension of quantum states over time - are real. The Kraus operator formalism is standard quantum information theory 1. The Transfer Tensor Method is a real algorithm 2.

These are genuine foundations.

But the implementation that sat on top of those foundations was hollow in the places where the claims mattered most.

High-Formality Slop is Level 2-4 slop dressed in the formal register of scientific publication: LaTeX, DOIs, citations, test suites, release notes, GitHub artifacts, peer-review-style documents, and polished figures. The formality is not decoration. It is part of the failure, because it can make the artifact feel verified before it actually is.

Part 3. The Ambition That Preceded the Understanding

The QSOT project began with a legitimate scientific stimulus. Research associated with UNIST, later circulated as Unique multipartite extension of quantum states over time, proposed treating temporal evolution not merely as a parameterization of quantum states but as a multipartite quantum state over time.

The framework introduced two axioms: Linearity in Initial State and Quantum Conditionability. From those axioms, a uniqueness theorem follows, connecting QSOT representations to Kirkwood-Dirac type quasiprobability distributions.

That is the external scientific anchor of this essay. The claim that QSOT has a legitimate mathematical source refers to Lie and Fullwood's work, not to our implementation.

The mathematics is non-trivial and genuinely important. It positions time within quantum mechanics as something closer to a first-class object rather than a passive external parameter, connected to temporal quantum-state formalisms [3, 4] and broader questions about causal structure [5].

We encountered this paper and made a planning document. That document contained lines like:

"Structure compatibility: 0.8-0.9 (axiom-based verification, Gate/Verifier structure)"

"Execution compatibility: 0.7-0.8 (modular design, easy ASDP integration)"

"Accessibility: not 'intuition' but mathematical uniqueness proof -> optimal for TOE automation pipeline"

Those numbers were invented. We did not have a calibrated instrument for measuring the distance from "paper" to "working implementation." The numbers felt credible because they were formatted like a technical assessment.

What we were really feeling was not measurement.

It was discovery pressure.

We were not simply trying to implement a paper. We were imagining a pipeline in which papers became code, code exposed missing links, and those missing links became the next scientific result. The TOE framing intensified this. A difficult physics paper stopped feeling like something to study slowly. It began to feel like a component waiting to be connected into a larger machine.

That was the dangerous step.

It was like reading one difficult book by Nietzsche, understanding some of it, and then mistaking the intensity of the encounter for philosophical authority.

The middle step had not been completed. We had read something difficult. We had not earned the right to extend it.

Part 4. The Narrative Gap

One useful lens came later from the HIERARCHICAL CONTEXT AND FLOW AUDIT (HCFA) framework, a governance tool we developed because failures like this needed a structured diagnostic.

HCFA does not ask whether a document sounds convincing. It asks whether transitions remain justified across scales: word to sentence, sentence to paragraph, paragraph to section, section to chapter. In practice, this reveals where a narrative has advanced faster than the evidence.

Viewed through that lens, the planning document had a structural discontinuity. The opening sections described a real paper and a real theorem. The later sections described implementation phases, integration pathways, and future scientific outputs. The connective layer between those two states - the implementation gap analysis - was largely absent.

The roadmap appeared continuous because the language was continuous.

The reasoning chain was not.

Two hazards followed.

1. Planning-Document Colonization

The planning document colonized the epistemic space that should have been occupied by implementation gap analysis. When a roadmap says "Phase 2: Temporal Depth and Relativity - Q1 2025 milestone," the act of writing it creates a false sense that the path has been evaluated. It has not been evaluated. It has been narrated.

The useful HCFA question is simple: what artifact actually bridges the current state to the claimed future state? If no bridge exists, the roadmap is functioning as a narrative device rather than an engineering assessment.

2. TOE-Induced Scale Blindness

The second hazard is specific to "Theory of Everything" framing. TOE is a phrase that suspends normal epistemic hygiene. When a project is positioned as contributing to a framework that unifies quantum mechanics and general relativity, the question "does this function actually implement what it claims?" starts to feel too small.

That is precisely why it matters.

At the chapter level, the project looked aligned with a grand objective. At the function level, important claims were not supported. The larger the ambition, the easier it became for local verification failures to disappear inside global language.

We were not irrational. We were operating exactly as capable people operate when genuine intellectual excitement combines with AI acceleration, weak external friction, and the absence of a hard responsibility structure.

This is not an excuse.

It is the diagnosis.

Part 5. The Anatomy of Failure

We shipped the project with a full artifact suite: gate_report.json, kd_quasiprob.json, memory_report.json, entanglement_report.json, trace.jsonl, raw_data.csv, two LaTeX papers, an academic project page, a third-party code review response, and a scientific publication guide.

These artifacts are the record. Let us read them honestly.

5.1 The KD Quasiprobability: The Core Claim Was Empty

The Kirkwood-Dirac quasiprobability distribution is central to the QSOT theoretical framework. Its negativity is often treated as an operational signature of non-classicality in settings involving incompatible observables. The entire project was, at its theoretical core, about detecting KD negativity.

That should have forced implementation discipline. If KD negativity is central to the framework, then an empty entries: [] artifact cannot be treated as a scientific output.

The kd_quasiprob.json artifact:

{
  "entries": [],
  "metrics": {
    "kd_negativity_proxy": 0.0
  }
}

The source code comment above the generated data:

# Mock KD for visuals
kd_data = {"entries": [], "metrics": {"kd_negativity_proxy": 0.0}}

The KD distribution was never computed. The core physics claim of the project, connected to the Yunger Halpern et al. quasiprobability framework [7], was represented by a placeholder that explicitly said "for visuals."

There is no ambiguity here.

The central scientific deliverable was a dummy value.

5.2 The Axiom Gate: The Test That Did Not Test

The gate report shows:

{
  "pass": true,
  "axiom1_report": { "pass": true, "max_deviation": 2.2226e-16 },
  "axiom2_report": { "pass": true, "max_trace_deviation": 0.0 }
}

The deviation 2.2226e-16 is machine epsilon. The Linearity axiom check was correctly implemented. It verified:

That part was real.

The Conditionability axiom check, however, looked like this:

def check_axiom2_conditionability(rhos, chans, tol_abs=1e-8):
    test_rho = np.eye(2) / 2.0   # ignores the rhos argument entirely
    for ch in chans:
        out = ch.apply(test_rho)
        tr = np.trace(out)

The function accepts rhos, the actual evolved quantum states from the simulation, and then never uses them. It substitutes a fixed maximally mixed state, rho = I/2, and checks trace preservation on that.

The test had zero sensitivity to what the simulation actually produced. It would pass even if the simulation output were garbage.

This is Level 2 slop: executable, green-lit, meaningless.

5.3 The Memory Kernel: Zeros All the Way Down

{
  "nm_measure": 0.0,
  "depth": 0,
  "profile": [0.0, 0.0, 0.0, 0.0, 0.0]
}

The Transfer Tensor Method is a genuine framework for characterizing non-Markovian memory in open quantum processes 2. The paper explicitly claimed TTM implementation. The memory profile was a flat zero vector across all time lags.

Pollock et al. support the legitimacy of TTM as a framework. They do not validate our specific zero-valued implementation.

One could argue that the channels used in this run were Markovian, so zero was the correct answer. That argument is available. But we never stated that boundary explicitly - not in the code, not in the documentation, and not in the paper.

A reader examining the artifact had no way to distinguish "TTM correctly measured zero non-Markovianity" from "TTM was not meaningfully implemented."

The absence of a claim boundary made the measurement indistinguishable from a default.

5.4 The Figure Versus the Data

Manuscript Figure 1: a polished beta-sweep figure showing coherence decay and memory backflow, but not reproducible from the distributed raw_data.csv.

Figure 1 in the PRA companion paper showed a beautiful result: quantum coherence smoothly decaying as observer velocity swept from beta = 0 to beta = 0.99c, while memory backflow rose from near zero to about 0.20. The causal-horizon region was shaded. The figure looked like physics. The paper's table cited specific nonzero values with uncertainty estimates.

A shortened version of the table:

beta	C_l1	memory_backflow
0.0000	0.9876 +/- sigma	0.0012 +/- sigma
0.5211	0.6430 +/- sigma	0.0292 +/- sigma
0.8858	0.0898 +/- sigma	0.1218 +/- sigma
0.9900	0.0098 +/- sigma	0.1835 +/- sigma

The raw_data.csv shipped as the reproducible dataset for this figure:

velocity, entanglement, non_markovianity
0.0000,   0.0,          0.0
0.0521,   0.0,          0.0
...
0.9900,   0.0,          0.0

Every single row: zero.

20 velocity points. Zero entanglement. Zero non-Markovianity.

Repository-side output: despite the filename entanglement.png, this plot shows L1 coherence over time. It is not the same beta-sweep figure used in the manuscript.

The reproducibility break was not simply numerical. The manuscript figure, distributed raw_data.csv, and repository-side entanglement.png output did not describe the same reproducible path. The numbers in Table 1 did not come from raw_data.csv, and the repository-side plot followed a different L1-coherence-over-time path.

Whatever produced the manuscript figure was not bound to the distributed data artifact in a way an external reader could reproduce.

This was not a bug.

It was not a calibration issue.

It was the signature of a pipeline where paper figures and actual software outputs were generated through separate paths, then narrated as one result.

The technical reason is discernible. raw_data.csv stored Logarithmic Negativity, a bipartite entanglement measure. A single qubit evolving under local Kraus operators cannot be entangled with itself, so LogNeg = 0 was mathematically unsurprising. The beautiful decay curve in Figure 1 was generated using L1-norm coherence, a single-qubit superposition measure, not entanglement.

The measures were swapped. The figure was built separately from the data file. The paper was written around the figure.

Artifact trace for this claim:

Paper figure: Fig1_Relativistic_decay.png - the figure used in the PRA companion manuscript.
Paper table: Table 1 in the PRA companion manuscript - reports nonzero coherence and memory-backflow values across the beta sweep.
Distributed data: raw_data.csv - contains velocity, entanglement, and non_markovianity columns, with zero values across the sweep.
Repository output: entanglement.png - a separate L1-coherence-over-time plot, not the manuscript beta-sweep Figure 1.
Failure condition: the manuscript figure cannot be regenerated from the distributed data artifact without a separate, undocumented figure-generation path.

5.5 The Relativistic Boost: Level 3 Physics Fabrication

The core physical model of the project was:

We treat this equation here as the disputed object: a phenomenological amplitude-damping ansatz, not a result derived from relativistic quantum field theory.

The CPC paper claimed: "Equation (1) is derived for amplitude-damping channels."

This is the sentence that should stop a physicist.

The time-dilation argument goes: if the damping parameter p grows as p(t) = 1 - exp(-Gamma*t), and if the observer's proper time is dilated by t_prime = gamma*t, then:

p_prime(gamma*t) = 1 - exp(-Gamma*gamma*t) = 1 - (1-p)^gamma

This derivation is valid only for amplitude-damping channels with exponential Lindblad decay rates, under the assumption that the Lindblad equation keeps its form under the Lorentz boost. That assumption requires the noise coupling to the bath to behave as a proper-time scalar.

Relativistic quantum field theory does not generally grant that assumption. Moving accelerated detectors interact with the Unruh thermal bath. Inertial boosts in quantum field theory generate Bogoliubov transformations between particle modes [8]. The substitution t -> gamma*t in a Lindblad parameter is not a Lorentz-covariant treatment of open quantum systems.

The citations did not validate the formula. Peres and Terno discuss relativistic quantum information broadly [9]. Alsing et al. analyze entanglement degradation for Dirac fields in non-inertial frames [10]. Those works point toward the kind of QFT treatment that would have been required. They do not support our amplitude-damping substitution.

The formula produced smooth, intuitive results.

That was exactly why it was dangerous.

It was mathematically coherent, physically unjustified, and dressed in the language of derivation.

Part 6. The Architecture of Self-Deception

Five mechanisms produced the failure.

Mechanism 1: The Green Test as Epistemic Closure

When pytest exits with 28 passed, 0 failed, 82 percent coverage, a cognitive gate closes. The test suite becomes a proxy for correctness.

But a test suite built by the same LLM session that built the code can share the same blind spots. The code and the test can be wrong in the same direction.

The check_axiom2_conditionability test was green because it checked whether the function returned pass: True, not whether the function validated the right thing.

Green tests are necessary.

They are not enough.

Mechanism 2: Formal Register as Legitimacy Signal

The SCIENTIFIC_PUBLICATION_GUIDE.md file contained a section preparing answers to imagined peer-review questions:

"Peer Review Preparation - Common Reviewer Questions: (3) 'Is your code reproducible?' - Answer: Hash-chained trace, seed-based RNG, version pinning"

This document prepared for peer review before peer review had happened. It anticipated expert questions and pre-formulated answers. The effect on us was subtle: it simulated the feeling of having passed review.

The form of scholarly rigor was present.

The substance was not.

Mechanism 3: The Information Void Fills Silently

Research papers are not implementation manuals. They define mathematical objects, theorems, or experimental claims, while leaving many engineering choices implicit. That is normal. The problem was not that the QSOT paper failed us. The problem was that we treated those unstated implementation choices as if an LLM could safely fill them in.

The QSOT paper did not specify how to compute KD distributions numerically, how to implement a relativistic channel boost from first principles, how to implement a Transfer Tensor Method with appropriate spectral truncation, or what initial states and channel parameters make a physically meaningful test case.

LLMs fill information voids. They produce plausible, syntactically correct, type-consistent implementations of underspecified requirements. The implementation of boost_damping_channel shows this exactly. Given the task "implement relativistic correction to quantum channel damping," the LLM produced a formula that was dimensionally consistent, physically intuitive, and derivable by analogy to classical time dilation.

The analogy was not a theorem.

But it generated a smooth curve, passed tests, and matched the qualitative expectation.

That is the danger.

Mechanism 4: The Two-Figure Problem

The worst moment in the audit was not reading the code.

It was opening the data.

We had been looking at Figure 1 as if it were one of the strongest parts of the project. The curve was smooth. The velocity sweep was intuitive. The causal-horizon shading made the story feel complete. Quantum coherence decreased as observer velocity approached relativistic limits. Memory backflow increased. The picture looked like physics.

Then we opened raw_data.csv.

Every row was zero.

Not noisy.

Not weak.

Zero.

Twenty velocity points. Zero entanglement. Zero non-Markovianity. The distributed data did not reproduce the figure in the paper. It did not approximate it. It contradicted it.

That was the moment the project stopped being "possibly overstated" and became something else.

We had allowed two different pipelines to exist under one scientific story. One generated the paper figure. Another generated the reproducible artifact. The paper followed the figure. The repository shipped the data. They were not the same reality.

This is one of the most dangerous AI-native research failure modes: figure generation, data generation, and manuscript writing can happen as separate fluent tasks, each locally coherent, none forced to reconcile with the others.

The final paper then inherits the appearance of unity.

But unity of language is not unity of evidence.

Mechanism 5: Ambition-Scale Mismatch

The project framing was "Theory of Everything pipeline." The implementation was five Kraus channels applied sequentially to a 2 x 2 density matrix, with a fixed initial state |+><+|.

The ratio between the claim and the machinery was structurally incoherent. No implementation of five 2 x 2 matrix operations can contribute to a Theory of Everything framework, regardless of how sophisticated the surrounding language is.

The mismatch was invisible to us because we were reasoning at the level of ambition, not implementation.

Part 7. The Structural Problem Beyond QSOT

Our failure was specific in its artifacts, but not specific in its structure.

The DOI, the false KD artifact, the disconnected figure pipeline, the invented compatibility numbers, and the green tests that did not test the right thing were ours. But the underlying condition is larger.

AI-native research implementation now makes it possible to generate code, tests, figures, documentation, citations, reproducibility claims, and publication-style wrappers faster than the claims inside them can be verified.

That is why this post-mortem cannot stop at QSOT.

If the failure were only personal incompetence, the lesson would be simple: do not do what we did. The harder lesson is that the tooling environment now makes this failure pattern easy for competent people, teams, and agents to reproduce.

The same fluency that helps us implement papers also helps us hide the gaps between implementation and understanding.

DeepCode is one significant demonstration of the Paper2Code paradigm at scale [11]. It presents a framework for document-to-codebase synthesis using source compression, structured indexing, retrieval-augmented knowledge injection, and closed-loop error correction.

That work matters because it shows how far automated reproduction has come. It also clarifies the remaining risk: even strong Paper2Code systems operate inside the gap between what papers specify and what implementations must decide.

The DeepCode architecture includes serious engineering responses to the information-void problem. They reduce the gap.

They do not eliminate it.

The fundamental issue is epistemic. When a paper says "we implemented the Lorentz-boosted Kraus channel," the implementation requires decisions that the paper does not specify. An agent, however sophisticated, must make those decisions. If the agent has no domain-level falsification mechanism, decisions will be made by plausibility, not correctness.

This is the structural condition:

The speed at which formal artifacts can be produced has outpaced the speed at which correctness can be verified.

In traditional software development, the lag between "it compiles" and "it is correct" is bridged by code review, staged deployment, and domain expert oversight. In AI-native research implementation, that lag is compressed by the fluency of LLM output. The model produces not just code but all the surrounding artifacts that signal correctness.

The result is a coherent artifact with the appearance of verified work, before verification has happened.

Part 8. What Was Real, and What Was Fabricated

This is the part that matters most. Not everything was wrong.

What was genuinely correct

QSOT axioms are real science. The Linearity and Conditionability axioms, together with the uniqueness theorem connecting them to temporal quantum-state structure, are genuine mathematical results in Lie and Fullwood's QSOT work 3.
The quantum-channel machinery was mostly correct. Kraus operators, CPTP evolution, and 2 x 2 density-matrix simulations used standard quantum information methods 1.
Axiom 1 actually worked. The Linearity verification produced machine-precision agreement.
TTM is a real algorithm. The Transfer Tensor Method itself is legitimate 2, even if our chosen system made the output uninformative.
The coherence curve was internally consistent with the implemented model. Given boost_damping_channel, the L1-coherence decay was computed consistently with the implemented dynamics [12].
The audit trail was real. The hash-chained trace.jsonl logging system functioned as a tamper-evident record.

What was fabricated or unsupported

KD negativity was never computed. The core QSOT observable was replaced with a mock artifact.
The memory-kernel claim was unsupported. The implementation did not provide meaningful non-Markovian TTM analysis.
The relativistic boost was not derived physics. The formula followed from analogy, not Lorentz-covariant quantum channel theory.
The paper's Table 1 was not reproduced by the distributed dataset. raw_data.csv contained zeros throughout.

The correct and fabricated components were interwoven. The artifact was convincing because many parts were genuinely correct. The axiom verification worked. The coherence calculations worked. The audit logging worked. Those true components provided legitimacy cover for the parts that did not.

This is the defining property of High-Formality Slop: it is not uniformly false. It is selectively hollow, in exactly the places where hollowness is hardest to detect.

Part 9. Toward Legitimate AI-Native Research Implementation

The failure is diagnosable. It has a treatment. But the treatment cannot be a generic checklist floating above the failure. Every rule below comes from one place where our artifact broke.

1. Claim Boundary as a First-Class Artifact

This rule comes from the relativistic boost formula.

A claim boundary document for boost_damping_channel would have forced us to write:

"The derivation p_prime = 1 - (1 - p)^gamma is valid only as a phenomenological amplitude-damping ansatz under an assumed proper-time scaling of Lindblad rates. It is not derived from Lorentz-covariant quantum field theory."

That sentence would have changed the paper. It would have forced us to decide whether we were making a physics claim or presenting a phenomenological model.

QSOT Compiler v1.2.3 did not have that boundary.

2. The Adversarial Test Mandate

This rule comes from check_axiom2_conditionability.

The function accepted rhos and ignored them. A test suite existed. The test was green. The artifact looked validated. But the only adversarial test that mattered was absent: does the function fail when the supplied trajectory violates the claimed condition?

If a conditionability check can pass while ignoring the evolved states, the test does not test conditionability. It tests our willingness to accept a label.

3. Figure-Data Pipeline Integrity

This rule comes from Figure 1.

No publication figure should survive unless it can be regenerated from the distributed data by a named script in the repository.

This is not cosmetic reproducibility. It is a self-deception control. The moment a figure pipeline and a data pipeline separate, language begins stitching them back together even when the evidence does not.

4. Measure-Claim Alignment

This rule comes from the entanglement/coherence swap.

Logarithmic Negativity, L1-norm coherence, and heuristic memory backflow are not interchangeable because they can all be placed under a phrase like "quantum resources." Each measure answers a different physical question. A single-qubit coherence curve cannot silently stand in for a bipartite entanglement result.

Every measure must name the physical question it answers. If the measure changes, the claim must change with it.

5. The Governance Layer

This rule comes from the whole artifact.

The Flamehaven Verification Ledger was not a response to abstract concern about AI quality. It was a response to this failure: the false KD artifact, the green conditionability test, the two-figure split, the boost ansatz, and the formal register that made all of it feel coherent.

The core insight is that in AI-native development, the primary human contribution has shifted from production to acceptance. We do not merely write the code. We inspect, refuse, correct, and bear final responsibility.

The human's primary tool is not only an editor or a compiler. It is a governance framework that makes the gap between claimed and actual behavior visible.

Part 10. QSOT V2 and the Ongoing Repair

The current QSOT V2 work exists because of this failure.

It is not a defense of QSOT v1.2.3. It is an attempt to separate what was real from what was falsely claimed, and to rebuild only on the parts that can survive explicit verification.

The repair is not one change.

It is a change in posture.

We are discarding the claim that code execution proves a new physical principle, the implied claim that KD negativity has been computed when it has not, and the habit of treating polished figures or formal reports as substitutes for reproducible data. We are also discarding the practice of presenting phenomenological channel assumptions as derived physics.

What remains is narrower but stronger: the real QSOT axiom structure, standard quantum-channel machinery, artifact-first reporting, and the governance lesson that claims must be bounded before results are interpreted.

The V2 work adds a Phase 0 Temporal-State Axiom Contract layer, required axiom checks, machine-readable claim boundaries, stress tests across multiple physical assumptions, and a phase-separated runner architecture. The purpose is not to prove the Time-as-State idea true. The purpose is to make clear what the system computes, what it merely simulates, and what it has no right to claim.

None of this makes the Time-as-State idea true.

What it may do, if completed successfully, is make our handling of the idea less dangerous.

Sophistication is not the opposite of slop. QSOT v1.2.3 was already sophisticated. The opposite of slop is traceable humility: knowing exactly what a system computes, what it merely simulates, and what it has no right to claim.

Conclusion

On December 23, 2025, a twelve-month ambition compressed into six weeks of AI-assisted implementation, deposited on Zenodo with a DOI, and called done.

QSOT Compiler v1.2.3 had correct mathematics at its foundation, correct numerical implementation of some components, and fundamental hollowness at its core: KD quasiprobability was never computed; the relativistic boost formula was an analogy, not a derivation; the published figures could not be reproduced from the distributed data; and the axiom verification did not test the axioms it claimed to test.

The artifact passed every surface-level quality signal: CI/CD green, 82 percent test coverage, Zenodo DOI, LaTeX papers, Docker deployment, and a hash-chained audit trail.

It failed the one test that matters: can an independent expert reproduce the core claimed results from the distributed code and data?

The answer is no.

We call this High-Formality Slop because the formality is not accidental. It is what LLMs are especially good at producing.

Formal register, structured documentation, complete test scaffolding, and referenced citations are not separate from the hollow implementation. They are often produced by the same fluent process that generated the implementation in the first place.

The danger is not that the slop is unrecognizable. The danger is that it is too recognizable. It looks exactly like legitimate scientific software. That is why it is convincing, and why detecting it requires deliberate adversarial examination rather than casual reading.

The DOI 10.5281/zenodo.18035432 remains archived. We do not retract it. We document it here, as the record against which subsequent work is measured.

The Flamehaven Verification Ledger began the day we recognized what we had built. Approximately fifty experiments later, the tooling to detect, bound, and reduce this class of failure is now in active use. The lesson cost us six months. We are writing it down so it does not cost others the same.

References

1 M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge University Press (2000; 10th Anniversary Edition, 2010).

2 F. A. Pollock, C. Rodriguez-Rosario, T. Frauenheim, M. Paternostro, K. Modi, Non-Markovian quantum processes: Complete framework and efficient characterization, Phys. Rev. A 97, 012127 (2018).

3 S. H. Lie and J. Fullwood, Unique multipartite extension of quantum states over time, arXiv:2410.22630 (2024).

4 J. Cotler, C.-M. Jian, X.-L. Qi, F. Wilczek, Superdensity Operators for Spacetime Quantum Mechanics, JHEP 09 (2018) 093.

[5] O. Oreshkov, F. Costa, C. Brukner, Quantum correlations with no causal order, Nature Communications 3, 1092 (2012).

[6] M. Lostaglio, A. Belenchia, A. Levy, S. Hernandez-Gomez, N. Fabbri, S. Gherardini, Kirkwood-Dirac quasiprobability approach to the statistics of incompatible observables, arXiv:2206.11783 (2022).

[7] N. Yunger Halpern, B. Swingle, J. Dressel, The quasiprobability behind the out-of-time-ordered correlator, Phys. Rev. A 97, 042105 (2018).

[8] S. Takagi, Vacuum Noise and Stress Induced by Uniform Acceleration, Progress of Theoretical Physics Supplement 88, 1-142 (1986).

[9] A. Peres and D. R. Terno, Quantum information and relativity theory, Reviews of Modern Physics 76, 93 (2004).

[10] P. M. Alsing, I. Fuentes-Schuller, R. B. Mann, T. E. Tessier, Entanglement of Dirac fields in non-inertial frames, Phys. Rev. A 74, 032326 (2006).

[11] Z. Li, Z. Li, Z. Guo, X. Ren, C. Huang, DeepCode: Open Agentic Coding, arXiv:2512.07921 (2025).

[12] T. Baumgratz, M. Cramer, M. B. Plenio, Quantifying Coherence, Phys. Rev. Lett. 113, 140401 (2014).

We Built AI Verification Infrastructure. Then It Found Our Blind Spots.

Tue, 02 Jun 2026 17:44:46 +0000

Prologue: The Mirage & The Pivot

In the summer of 2024, our team began with an ambition that felt almost impossible: to use frontier AI for drug discovery, theoretical physics, and problems that seemed unreachable. We imagined systems that could reason about spacetime geometries and propose novel small-molecule bindings.

Within the first month, the boundary became clear. The models were powerful, but their claims were not something we could responsibly stand behind unless the path from input to result could be inspected.

So we made a choice. We could keep producing evocative claims we could not verify, or we could build infrastructure that would force our own work through the same scrutiny we would apply to anyone else's.

We stopped. We built.

For the last two years, we pursued this work without revenue. The original ambition was larger than the system we are publishing now, but the work became more precise. We began turning model-generated reasoning into executable Python harnesses, custody records, and governance gates before we had stable language for what to call them.

The practical question became narrower: can scientific claims expose the path by which they were generated, and can biomolecular AI systems be evaluated for transparency before their outputs are treated as evidence?

We shared progress publicly on dev.to, LinkedIn, Substack, Medium, and CoderLegion, usually one to three times a week. Most posts did not travel far. They were part research log, part signal flare, and part attempt to decide what should remain protected and what should be released.

Over time, the answer became clearer: protect the sensitive core, but publish verification surfaces, audit artifacts, and reproducible paths wherever possible.

We are not claiming to have solved drug discovery or the Theory of Everything. We are publishing the verification materials because even a small piece of reliable infrastructure may help others near those frontiers.

What the Flamehaven Verification Ledger Is

That choice — to verify before we believe — became the whole of what we built. Everything in the Flamehaven Verification Ledger answers one question:

Can this claim show the path by which it came to exist?

A mathematical or scientific claim earns operational trust only once the path behind it can be inspected, reproduced, or challenged.

A biomolecular AI pipeline is trusted only when its disagreements are surfaced, not smoothed over.

A bioscience repository is cleared only once its safety is checked line by line, with no model in the scoring loop.

Verification is not a feature here. It is the spine.

The ledger is public and inspectable by design. Each run is published as a bounded artifact: inputs, a score, a report, a traceable custody path, and, when it happens, a failure note.

Today it has three main verification lanes:

EQA — Equation-to-Artifact. Mathematical and physical claims are turned into runnable, deterministic Python checks, including string-theory beta-function tests at 200-bit precision and the OpenAI Erdos reproduction.
BAV — Biomolecular AI Validation. Biomolecular pipelines are evaluated as governed systems, not just prediction engines. Model disagreement is treated as a safety signal rather than noise.
BSC — Bioscience Compliance. The open-source STEM-BIO-AI scanner runs local, zero-execution safety and compliance scans of bioscience repositories and maps findings to external risk and traceability frameworks, with no LLM in the scoring path.
Open resources are rolling out as they are cleared for public release.

Public ledger

What Two Years Produced — and What It Cost Our Beliefs

This is the part that matters. Two years of runs left a body of results: some held up under scrutiny, and some broke beliefs we were attached to.

A verification engine earns credibility from both: results that survive and failures that tell us when we are wrong.

Success 1: Geometry Overruled the Narrative

The EQA lane showed something important in TOE-TEST-0004: it judged a background by its geometry, not by the words attached to it.

The test was built around Green-Schwarz anomaly cancellation, where the gravitational and gauge Pontryagin densities must match:

p1_R = p1_F

Two adversarial cases made the point.

A Schwarzschild black hole carrying a single-plane gauge field sounds like an anomaly waiting to happen. The engine returned PASS (Omega = 0.9997) because Schwarzschild is Ricci-flat (p1_R = 0) and a single-plane field is topologically trivial (p1_F = 0).

Flat 10-D Minkowski space sounds harmless. But with a two-plane gauge field, it returned FAIL:

p1_F = 2.0 != p1_R = 0

Without curvature, there is nothing to cancel the gauge anomaly. The ledger did not reward the background for sounding safe, nor punish it for sounding dangerous. It ran the case.

Success 2: Calibration Is Not Understanding

EXP-028 was built to catch a different confusion: a model can be beautifully calibrated and still not understand what it is looking at.

On the surface, the model looked excellent. Brier score improved from 0.204 to 0.0056 after calibration. AUC was 1.0.

By standard calibration metrics, this is the kind of model you might trust. But the cross-domain honesty gate refused it.

SR9 was about 0.26 against a gate of >= 0.80. DI2 was about 0.61 against a gate of <= 0.20.

So the pipeline did the rare thing: it returned "I cannot resolve this" and abstained, rather than let a well-calibrated number masquerade as comprehension.

Calibration measures whether confidence is honest about frequency. It does not prove understanding.

*SR9 and DI2 are advisory heuristics, not externally validated physical quantities. That boundary matters.

Success 3: A Repository Scored Against a Traceability Rubric, Deterministically

The third success is the least glamorous and the most reusable: a real bioscience repository scored against external traceability concerns by code anyone can re-run.

Our open-source scanner STEM-BIO-AI (pip install stem-ai) reads observable signals and grades three rubrics:

S1 = README / intent evidence
S2 = repo-local consistency
S3 = code / bio responsibility

They combine into a deterministic weighted score, with penalties and hard caps where needed:

raw = round(0.4*S1 + 0.2*S2 + 0.4*S3 - risk_penalty)

For yorkeccak/bio, the scan produced a final score of 48, mapped to T1 Quarantine.

The point is how the number is reached: README intent, repo-local consistency, dependency safety, exception handling, data provenance, clinical disclaimers, and AST-level code analysis using ast.parse.

Nothing in the target repo is executed. No network, no GPU, no model in the loop. The audit runs locally and reproduces from repository state.

The Failures We Are Publishing

Failure 0: Synthetic Data in Our Own Ledger

The most important failure was not in physics or biology. It was in our own record.

An early EQA dashboard shipped a "51-run calibration registry" that read as authoritative until an external reviewer showed it was procedurally generated.

It contained fabricated primes, fabricated field degrees, synthetic hash labels, and at least one failed check beside a PASS verdict. It was hallucinated scaffolding, not computation.

We deleted the synthetic registry, replaced it with real TOE-TEST foundational runs, sanitized local workspace paths, and added a deterministic synthetic_marker detector so a [synthetic] tag cannot re-enter the public ledger.

This is the failure we are most willing to show because it is what the system was built to catch — including when the fabricator was us.

Failure 1: One-Loop Weyl Curvature Blindness

The EQA engine evaluates spacetime geometries under one-loop worldsheet beta-functions of the non-linear sigma-model.

We stress-tested it with a "Planck-scale spacetime foam Schwarzschild metric with mass M = 0.1," expecting rapid failure due to extreme curvature near the horizon.

Instead, the solver returned PASS:

Omega = 0.9985

The reason is precise. Schwarzschild is Ricci-flat outside the singularity:

R_mn = 0

Our one-loop beta-function gate couples directly to Ricci curvature, not Weyl curvature. So the metric residuals vanished even though tidal curvature was physically severe.

The Kretschmann scalar made the blind spot visible:

K = R_abcd * R^abcd = 48 * M^2 / r^6 = 4800

This is not a broken engine. It is a specific boundary. The solver is correct where one-loop Ricci geometry is sufficient. T09 marks where that sufficiency ends.

We have not fixed this. The correction requires higher-order alpha-prime terms for Ricci-flat backgrounds with high tidal forces.

Failure 2: An Honest Rejection We Cannot Yet Confirm

In EXP-005, our internal candidate-generation engine screened three lipid carriers for topical Upadacitinib: SLN, NLC, and liposomal gel.

Each was scored by SR9, an advisory cross-domain consistency heuristic. Our bar was >= 0.80.

The scores were far below that bar:

SLN ~= 0.28
NLC ~= 0.23
liposomal gel ~= 0.26

Every candidate was rejected in under two hours, against an estimated months of bench work.

But because SR9 is advisory, we do not know whether these values reflect real formulation incompatibility or an over-conservative false reject. A fast negative is useful only if it is correct.

Failure 2b: When the Models Themselves Disagree

EXP-031 tested a 52-amino-acid target across AlphaFold2, AlphaFold3, Chai-1, and Boltz-2.

They did not converge. pTM stayed low:

AF2 pTM ~= 0.31
AF3 pTM ~= 0.40

The inter-model consensus drifted by 0.09 to 0.37.

The honest outcome was not a number. It was a refusal: the pipeline returned Unverified / Drift Detected and an observer-only decision for every arm.

The ledger's job was to refuse to launder that disagreement into a result.

Failure 3: The Multiplicative Reliability Fallacy

The BAV pipeline originally modeled end-to-end reliability as a multiplicative chain:

p_e2e = p_capture * p_transfer * p_model * p_clinical

These factors are not independent. Clinical reliability depends on model reliability, which depends on transfer and capture quality.

The correct direction is conditional:

p_e2e =
  p(clinical | model)
  * p(model | transfer)
  * p(transfer | capture)
  * p(capture)

We have not implemented this yet. Every reliability figure in the BAV lane carries this constraint.

From Experiments to Public Custody Paths

Before this infrastructure existed, experiments ended as local folders, private notes, or claims that required too much explanation. Now a weekly experiment can become a public record: a bounded run, score, report, artifact path, and failure note.

We do not see this as a replacement for peer review. We see it as the layer before peer review: the place where claims become inspectable, failures remain visible, and weekly experiments accumulate.

We also avoid claiming false determinism. Fold outputs vary with seed, GPU architecture, MSA depth, and compiler version. In EXP-031, we anchor the input FASTA, pin the seed and model versions, and record confidence metrics as a labelled reference run. The goal is regime-level comparison, not bit-exact reproduction.

The detailed repository tree is omitted here for space; the public ledger exposes the custody paths directly.

MICA and the CI Sanitizer Gate

Two layers keep the ledger honest before publication.

MICA, Memory Invocation & Context Archive for AI, is the governance memory. It encodes project invariants: internal metrics carry no external authority, fabricated data is prohibited, and public claims stay understated.

The sanitizer is the enforcement layer. It runs as a CI gate and fails the build when it finds workspace leaks, local-specific PII patterns, promotional language, or fabrication tags.

The synthetic_marker detector was added after Failure 0. The promotional_language detector flags public-facing hype such as "revolutionary," "state-of-the-art," or "authoritative."

The result is a repository that censors its own hype, leaks, and fabrications before publication — and keeps an auditable record of doing so.

Dashboard Heuristics: Labelled As Heuristics

The dashboard plots EQA metrics against an exponential curve.

Status: ADVISORY-HEURISTIC.

This curve is not a physical law. It is an empirical visual fitting heuristic for charting. Treating it as fundamental law would violate the ledger's credibility rules.

Omega(norm_R_F) = Omega_0 * exp(-lambda * norm_R_F) + epsilon_numerical

What We Need Others to Test

The ledger is not finished. It is a public audit brief.

We need formulation scientists to test whether EXP-005 reflects real carrier incompatibility or an over-conservative false reject.

We need structural biologists to test whether EXP-031 reflects genuine out-of-distribution behavior or a harness artifact.

We need string theorists to test whether TOE-TEST-0001 is physically correct in the Ricci-flat regimes where we claim results.

We need compliance experts to test whether the zero-execution Article 12 mapping in BSC is legally defensible or technically approximate.

All code, schemas, and run payloads: Zenodo DOI 10.5281/zenodo.20483364

Public Ledger: https://flamehaven01.github.io/Flamehaven-Verification-Ledger/
Citation ORCID: 0009-0009-2641-4280

Epilogue: The Invitation

We began by trying to reach the unreachable: spacetime geometries, novel small-molecule bindings, a Theory of Everything.

We did not get there. This ledger is not that.

What we found instead was smaller and more useful: a way to make a claim show the path by which it came to exist.

As AI fills every field with fluent, confident, publication-shaped output, the quiet casualty is the experiment itself: the runnable artifact, the failed run nobody deleted, the number traced back to code.

We are not announcing a solution. We are publishing a small, inspectable surface — failures included — and handing it to the people best equipped to break it.

If it proves useful against the confident noise filling everyone's feeds, that will be your verdict, not our claim.

We built this to learn from you.

Using Math in Programming Beautifully: Drawing with Turtle

Sat, 21 Mar 2026 12:46:08 +0000

Using Math in Programming Beautifully: Drawing with Turtle

"Programming is not just a skill; it is an art. And math is its brush." — Programming as an Art

Programming becomes truly beautiful when math guides your code. With Python’s turtle module, you can turn numbers and formulas into living art, making math visible, interactive, and mesmerizing.

1. Patterns with Loops

"Recognize the rhythm in numbers, and you will find rhythm in code." — Programming as an Art

Loops and math are perfect partners. For example, let’s draw a simple spiral:

import turtle

t = turtle.Turtle()
t.speed(0)

for i in range(100):
    t.forward(i * 2)
    t.right(45)

turtle.done()

The combination of a growing forward step and rotation creates a mathematical spiral—a visual echo of Fibonacci-like growth.

2. Geometry Comes Alive

"Lines, curves, and shapes are as much math as they are poetry." — Programming as an Art

Math lets us transform equations into visual art. For instance, using angles and loops:

import turtle
import math

t = turtle.Turtle()
t.speed(0)

for i in range(360):
    x = 100 * math.sin(math.radians(i))
    y = 100 * math.cos(math.radians(i))
    t.goto(x, y)

turtle.done()

Suddenly, a simple sine and cosine function becomes a perfect circle—a pure mathematical shape made visible.

3. Fractals: Math in Nature

"Every algorithm is a story, written with variables, loops, and functions." — Programming as an Art

Fractals are a stunning example of recursion and math:

import turtle

t = turtle.Turtle()
t.speed(0)
t.left(90)

def tree(branch_len):
    if branch_len > 5:
        t.forward(branch_len)
        t.left(30)
        tree(branch_len - 15)
        t.right(60)
        tree(branch_len - 15)
        t.left(30)
        t.backward(branch_len)

tree(100)
turtle.done()

Each branch splits recursively, creating a mathematical tree, full of natural elegance.

4. Controlled Randomness

"When uncertainty enters, math becomes your compass." — Programming as an Art

Math also controls randomness to create generative art:

import turtle
import random

t = turtle.Turtle()
t.speed(0)

colors = ["red", "blue", "green", "purple", "orange"]

for _ in range(50):
    t.color(random.choice(colors))
    t.penup()
    t.goto(random.randint(-200, 200), random.randint(-200, 200))
    t.pendown()
    t.circle(random.randint(10, 50))

turtle.done()

Here, probability and random numbers generate unique, colorful patterns every time.

5. Elegance is Key

"Elegance in code is the silent poetry of mathematics." — Programming as an Art

With turtle, the beauty isn’t just in the result—it’s in the logic and math behind it. Each loop, angle, and function is like a brushstroke, producing art from numbers.

Final Thought

Programming isn’t just solving problems—it’s painting with math. Turtle makes the invisible visible. Every function, formula, and loop is an opportunity to create beauty.

"To program without math is to paint without colors." — Programming as an Art

Key Math Concepts in Data Structures and Algorithms

Sat, 07 Mar 2026 23:13:34 +0000

At the core of data structures and algorithms are fundamental mathematical principles that guide their design and implementation. This article explores key math concepts essential for mastering data structures and algorithms, helping developers write optimized and effective code.

Number Systems

A number system is a structured way of representing numbers using specific symbols and rules. Different number systems are based on different base values (also called the radix).

We are most familiar with the decimal number system, which is used in everyday life. It uses ten digits (0 through 9) to represent all numbers. Because it uses ten distinct digits, its base is 10.

Computers, however, operate using only two digits: 0 and 1. Therefore, they use the binary number system, which has a base of 2. All data inside a computer is ultimately stored and processed in binary form.

The octal number system uses eight digits (0 to 7) so its base is 8. It is used as a more compact way to represent binary numbers, since each octal digit corresponds to a group of three binary digits (bits). Octal is still used in certain areas of computing, such as file permissions in Unix and Linux systems.

The hexadecimal number system uses sixteen symbols: the digits 0–9 and the letters A–F. Because it has sixteen symbols, its base is 16. Each hexadecimal digit corresponds to a group of four binary digits, making it a convenient way to represent memory addresses, machine code, and other low-level data in computers.

The octal number system has a base of 8, which means it uses eight symbols: 0 to 7.
Since 8 = 2³, each octal digit can represent exactly three binary digits (bits). Same goes for hexadecimal number system.

You can learn how to convert between these number systems from this detailed article.

Now, let’s talk about the binary number system. We are familiar with representing positive numbers using unsigned binary. But how do we represent negative numbers? For this, we use the two’s complement method, which is the standard way to represent signed binary numbers.

Two's complement method

Negative of any decimal number is mathematically found by subtracting that number from 0. The same applies to the binary number system. In the signed binary number system, the most significant bit (the leftmost bit) indicates the sign of the number.

See the image below if you are not familiar with the most and least significant bits in the binary representation of a number:

Let’s take the example of decimal 7 whose binary representation is 0111. Its binary number system is stored in 4 bits. Zero can be represented as 0000 (in 4 bits). In 4-bit representation, if a fifth bit is present in the leftmost position, it is discarded due to lack of space. So, we can represent zero as 10000 (the bit 1 will be discarded). Now, 10000 can be written as the sum of 1111 and 1. We can then proceed as follows:

Essentially, the negative of a binary number will be its complement added to 1. This method is called as the two’s complement method.

Bitwise Operators

In the decimal number system, we use arithmetic operators like +, -, *, and / to perform mathematical operations. In computing, numbers are represented in binary, and in addition to arithmetic operators, we also use bitwise operators like &, |, ^, ~, >>, and << to manipulate individual bits. Let’s discuss each of these operators.

AND Operator (&)

This operator performs bit-by-bit AND operation on two values. This means that for each bit in the operands, the resulting bit is set to 1 only if both corresponding bits in the operands are 1; otherwise, it is set to 0.

Observation: The AND operation of any number with 1 gives the least significant bit (LSB) of that number.
Example: 5 & 1 results in 1 because 101 & 001 gives its LSB, which is 1.

OR Operator (|)

This operator performs a bitwise OR operation between two operands, setting each resulting bit to 1 if at least one of the corresponding bits in the operands is 1; otherwise, it is set to 0.

XOR Operator (^)

XOR stands for exclusive OR. This operator performs a bitwise comparison between two operands, setting each resulting bit to 1 if the corresponding bits in the operands are different; otherwise, it is set to 0.

Observations:

The XOR operation on any number with 1, results in a number where the least significant bit (LSB) is flipped.
Example: 3 ^ 1 results in 2 because 11 ^ 01 changes the LSB to 0 in this case.
The XOR operation on any number with 0 results in the same number.
Example: 4 ^ 0 results in 4 because 100 ^ 000 gives 100, which is 4.
The XOR operation on any number with itself results in 0 because there are no differing bits.
Example: 5 ^ 5 results in 0 because 101 ^ 101 gives 0 as there are no bits that differ.

Complement Operator (~)

The complement operator gives the bitwise inverse of a number, flipping each bit from 0 to 1 and from 1 to 0.

Left Shift Operator (<<)

The left shift operator is a bitwise operator that shifts the bits of a number to the left by a specified number of positions. Example: 5 << 1 gives 10 because shifting the bits of 5 (which is 101 in binary) one position to the left results in 1010, which is 10 in decimal. If we shift twice (5 << 2), we will get 20 because shifting the bits of 5 (which is 101 in binary) two positions to the left results in 10100, which is 20 in decimal. Therefore, each left shift operation effectively multiplies the number by 2 for each position shifted. In general, if the number is shifted k times, it is multiplied by 2^k.

Right Shift Operator (>>)

The right shift operator is a bitwise operator that shifts the bits of a number to the right by a specified number of positions. Example: 8 >> 1 gives 4 because shifting the bits of 8 (which is 1000 in binary) one position to the right results in 0100, which is 4 in decimal. If we shift twice (8 >> 2), we would get 2 because shifting the bits of 8 (which is 1000 in binary) two positions to the right results in 0010, which is 2 in decimal. Therefore, each right shift operation effectively divides the number by 2 for each position shifted. In general, if the number is shifted k times, it is divided by 2^k.

Core Number Theory

Prime Number

A prime number is a natural number greater than 1 with exactly two positive divisors: 1 and itself. To determine if a number is prime, we typically check divisibility from 2 up to n−1 since it is always divisible by 1 and itself.

int n = 24;
boolean isPrime = true;
for (int i = 2; i < n; i++) {
    if (n % i == 0) {
        isPrime = false;
        break;
    }
}
if (isPrime) {
    System.out.println("Prime");
} else {
    System.out.println("Not prime");
}

The above algorithm makes a maximum of n - 2 (2 to n-1) checks. So, its time complexity is O(N). Let's see if we can optimise it. If we closely examine the factors of any number, they appear in pairs (one smaller and one larger). Mathematically, we can say that for every divisor d > √n (like d = 6), there exists a corresponding divisor \(\frac {n} {d} < \sqrt n\) (like 4). Therefore, checking beyond √n is redundant.

If we look into the above possible pairs of factors of 24, we can see that we're checking divisibility of each pair twice. It is sufficient to check divisibility only up to √n since √24 ≈ 4.89, checking up to 4 is enough. Hence, we can reduce the number of computations from the order of N to √N.

int n = 24;
boolean isPrime = true;
for (int i = 2; i * i <= n; i++) {
    if (n % i == 0) {
        isPrime = false;
        break;
    }
}
if (isPrime) {
    System.out.println("Prime");
} else {
    System.out.println("Not prime");
}

The time complexity of this algorithm is now O(√N).

Prime Numbers in Range

Let's say we need to find prime numbers between two certain numbers. The simplest idea would be to check if each of the numbers between them is prime or not.

public class Main {
    public static void main(String[] args) {
        int n = 50;
        displayAllPrimes(n);
    }
    
    public static void displayAllPrimes(int n){
        for (int i = 2; i <= n; i++) {
            if (isPrime(i)) {
                System.out.print(i + " ");
            }
        }
    }
    
    public static boolean isPrime(int n){
        if (n <= 1) return false;
        for (int i = 2; i * i <= n; i++) {
            if (n % i == 0) return false;
        }
        return true;
    }
}

In the above program, the isPrime() method takes O(√N) to check whether a certain number is prime. The displayAllPrimes() method calls the isPrime() method approximately N times. So, the time complexity of the entire algorithm is O(N√N). If we carefully observe the numbers we're checking for prime, we'll notice that there are a number of redundant checks. For example, if we already know that 2 is a prime number then its multiples won't be prime.

Similarly for the rest of the numbers, the below image shows the multiples of smaller prime numbers we've checked earlier.

The unmarked numbers turn out to be the prime numbers we're finding in that range of 2 to 50. This is known as the Sieve of Eratosthenes.

public class Sieve {
    public static void main(String[] args) {
        int n = 50;
        boolean[] isComposites = new boolean[n+1]; // size is n+1 to include n 
        sieve(n, isComposites);
    }

    public static void sieve(int n, boolean[] isComposite){
        for (int i = 2; i * i <= n; i++) {
            if (!isComposite[i]) {
                for (int j = 2 * i; j <= n; j += i) {
                    isComposite[j] = true;
                }
            }
        }

        for (int i = 2; i <= n; i++) {
            if (!isComposite[i]) {
                System.out.print(i + " ");
            }
        }
    }
}

In the program above, we check for multiples up to √N because large numbers can be factored using smaller numbers' multiples. The number of times the inner loop (the loop responsible for marking the composite numbers) runs for each prime number P is \(\frac {N} {P}\). Therefore, total number of times the inner loop runs will be:

Hence, the time complexity of this algorithm is O(N loglog(N)).

Newton Raphson Method

The Newton Raphson method is an iterative method of finding better approximations to the roots of a real-valued function. The best linear approximation to any differential function near the point \(x = x_i\) will be its tangent with the equation:

where x_i is the ith iterative approximation of root

The root of the tangent equation will be the next iterative approximation x_{i+1} when it intercepts the x-axis (y = 0).

Now, if we want to find the square root of any number (say n), we can take f(x) = x² - n. The root of that equation will give us the closest approximation of √n.

You can view this animated image from Wikipedia, which visually demonstrates all the iteration of approximated roots of f(x).

Let's try to write a program for finding the approximate square root of any number.

public class NewtonRaphsonSqrt {
    public static void main(String[] args) {
        System.out.println(sqrt(43));
    }

    public static double sqrt(double n) {
        double x = n;
        double root;
        while (true) {
            root = 0.5 * (x + (n/x));

            if (Math.abs(root - x) < 0.5) { // Error threshold, can be adjusted for more precision
                break;
            }
            x = root;
        }
        return root;
    }
}

Reducing the error threshold, currently set at 0.5, will result in a more accurate approximation of the square root.

Modular Arithmetic

It is a system of arithmetic operations for integers where the numbers "wrap around" when exceeding a certain number called as the modulus. For a given integer m ≥ 1, it is called a modulus if the difference of two integers a and b (a - b) is an integer multiple of m i.e. a - b = k*m (where k is an integer). The integers a and b are said to be congruent modulo m and is mathematically denoted by a ≡ b (mod m).

The parentheses mean that (mod m) applies to the entire equation, not just to the right-hand side (here, b). In particular, (a mod m) = (b mod m) is equivalent to a ≡ b (mod m), and this explains why "=" is often used instead of "≡" in this context.

The congruence relation is written as *a = b + k\m**.

Source: Wikipedia

In programming, we don’t write congruences directly. Instead, we use the % operator, which computes the remainder. The % operator is the practical realization of the congruence relation. For example: ≡ (mod m) ⇔ (a % m) == (b % m) in code.

The range of (a % m) is always in the range (\[0, m-1\]) when (m > 0). This ensures the remainder is non-negative, which is crucial for consistent modular arithmetic in programming.

Properties:

Addition Property: When adding two numbers, their remainders modulo (m) add up, but if the sum exceeds (m), subtracting (m) (i.e., taking modulo again) brings it back within the valid range.
Subtraction Property: Subtraction can lead to negative results, so adding (m) before taking modulo ensures the result stays within (\[0, m-1\]).
Multiplication Property: Multiplying the remainders and then taking modulo keeps the product within the modular system, preserving equivalence.
Division Property: Division is not straightforward in modular arithmetic. For division by (b) modulo (m) to be valid, (b) must have a modular inverse modulo (m), which exists only if gcd(b, m) = 1.

Modular multiplicative inverse of an integer b is the solution (which exists only when b and m are coprime) of the equation: ax ≡ 1 (mod m).

Bézout's Identity

It is a theorem which related two arbitrary integers with their greatest common divisor (GCD). Mathematically,

where, x and y are integers.

If gcd(a, b) = d, then every integer linear combination ax + by is a multiple of d.

For example, if a = 3 and b = 16, then 3*(-5) + 16*1 = 1 = gcd(3, 16).
Let's take another example: a = 8 and b = 18. Then, 8*(-2) + 18*1 = 2 = gcd(8, 18).

The coefficients x and y can be found using the Extended Euclidean algorithm and the gcd can be calculated using the Euclidean algorithm.

Euclidean Algorithm

The Euclidean Algorithm is one of the oldest and most efficient methods to compute the greatest common divisor (GCD) of two integers. Since Bézout’s identity relies on the GCD, understanding this algorithm is essential. It's based on the principle that if

Effectively, a is reduced by multiples of b until only the remainder is left. Therefore, gcd(a, b) = gcd(b, a % b).

Let’s compute gcd(514, 106) using the Euclidean Algorithm.

We start by dividing the larger number by the smaller one and continue replacing the pair with (b, a % b) until the remainder becomes zero:

514 = 106*4 + 90 → gcd(514, 106) = gcd(106, 90)
106 = 90*1 + 16 → gcd(106, 90) = gcd(90, 16)
90 = 16*5 + 10 → gcd(90, 16) = gcd(16, 10)
16 = 10*1 + 6 → gcd(16,10) = gcd(10, 6)
10 = 6*1 + 4 → gcd(10, 6) = gcd(6, 4)
6 = 4*1 + 2 → gcd(6, 4) = gcd(4, 2)
4 = 2*2 + 0 → gcd(4, 2) = 2

Therefore, gcd(514, 106) = 2.

public static int gcd(int a, int b) {
    if (a == 0) {
        return b;
    }
    return gcd(b % a, a);
}

We can also calculate the LCM of two numbers using their GCD with the formula:
\[\text{LCM}(a, b) = \frac{|a \times b|}{\text{GCD}(a, b)}\]

public static int lcm(int a, int b) {
    return a * b / gcd(a, b);
}

Extended Euclidean Algorithm

This algorithm is an extension of the Euclidean Algorithm which computes the coefficients of the Bézout's identity. We start with the gcd of the two numbers and eventually represent that as a linear combination of those two numbers.

For example, we have found gcd(514, 106) = 2. We start with the last non-zero remainder step: 6 = 4 + 2 ⇒ 2 = 6 - 4.
Then, from previous steps, we can write 6 = 16 - 10 and 4 = 10 - 6. We'll back-substitute them in the above equation.
∴ 2 = (16-10) - (10 - 6) = 16 - 2*10 + 6 = 16 - 2*10 + (16 - 10) = 2*16 - 3*10
∴ 2 = 2*16 - 3*(90 - 16*5) = 17*16 - 3*90
∴ 2 = 17*(106 - 90) - 3*90 = 17*106 - 20*90
∴ 2 = 17*106 - 20*(514 - 4*106) = (-20)*514 + 97*106

Hence, (-20)*514 + 97*106 = 2 = gcd(514, 106). So, the coefficients (Bézout's coefficients) of Bézout's identity would be -20 and 97.

We can utilise the recursion stack to keep track of the steps performed in Euclidean Algorithm. Below is the code for this algorithm:

static int gcdExtended(int a, int b, int[] xy) {
    if (a == 0) {
        xy[0] = 0; // x
        xy[1] = 1; // y
        return b;
    }

    int[] temp = new int[2];
    int gcd = gcdExtended(b % a, a, temp);

    // Update x and y using results of recursion
    xy[0] = temp[1] - (b / a) * temp[0];
    xy[1] = temp[0];

    return gcd;
}

Conclusion

In this article, we explored fundamental mathematical concepts that are crucial for understanding and implementing data structures and algorithms. We delved into number systems, including binary, octal, and hexadecimal, and discussed how computers represent numbers using the two's complement method for signed integers. We also covered bitwise operators and their applications in manipulating individual bits of data.

⭐ Check out the DSA GitHub repo for more code examples.

Kids Maths app - Datalaria

Wed, 04 Feb 2026 06:45:04 +0000

Looking for a fun and interactive way to reinforce basic arithmetic for children?

I built Kids Math Learning Game, a dynamic web application designed to turn math practice into an engaging game for children aged 5 to 7. Created with the help of ChatGPT, this tool proves that you can build functional educational software using just natural language. Key features include:

4 Fundamental Operations: Comprehensive practice for Addition, Subtraction, Multiplication, and Division.

Adaptive Difficulty: Features 4 specific learning levels (Infant to Primary) and adjustable digit complexity (1 or 2 digits) to grow with the child.

Gamified Experience: Keeps users engaged with a 10-second countdown per question, real-time scoring, and motivating instant feedback with emojis.

AI-Powered Development: The entire logic, structure, and design were generated through iterative ChatGPT prompt engineering.

Responsive Design: A clean, attractive interface styled with Tailwind CSS that works on any device.

Built with HTML5, Tailwind CSS, and Vanilla JavaScript with ChatGPT assistance. Lightweight, responsive, and fully self-contained in a single file.

If you want to know the details about how I built this educational tool using natural language prompts, please read the following article: Co-piloting with AI: Learning with Founderz and ChatGPT.

Descriptive Analysis - Basics

Wed, 07 Jan 2026 00:19:35 +0000

As a starting point in data analysis, it is necessary to know some mathematical and statistical foundations in order to work with the information to be analyzed.

Thus, we will start learning some basic concepts:

Population, Samples, Individuals and Variables:

Population: The set of all the elements under study
Sample: A subset of elements of the population (it should be representative of the population)
Individuals: Every individual element in the popularion
Variables: Characteristics of the individuals

Example: Titanic Dataset

Population: 2.224 individuals
Samples: files:
- train.csv: Set of 891 representative individuals of the population used for the training of models of Machine Learning.
- test.csv: Set of 418 representative individuals of the population used for the testing of models of Machine Learning.
Individuals: Each passenger and crew of the population (every row of the sample or population data collected).
Variables: The characteristics collected for each individual (every column of the sample or population data, ej.: survival (survivals), sex (sex) o Age (age)).

Types of Variables

The variables can be classified depending on the type of data they contain and, more specifically:

Numerical: the data is represented by numbers that are metric or measures of said variable. According to their values, these could be classified in:
- Continuous: they could take a countless or indeterminate number of values. For example, in the data set of the Titanic example, the variable fare could be considered a continuous variable by taking different values with up to 4 decimals of precision.
- Discretes: they could take a countable number within a list of values (the data of continuous variables can be truncated to discrete variables for easier analysis). For example, the variable Sibsp (# of siblings) would be a discrete variable.
Categorical: the data is represented by texts or numbers whose meaning is not a metric, but associated with a specific datum or category. Depending on the relationship between their values, the data would be classified in:
- Nominals: data that have no relation or intrinsic order in themselves. An example of this type would be the embarked variable that represents the port of embarkation of passengers.
- Discretes: data with a natural order or classification among themselves. As for example it could be the variable Pclass that represents the class in the Titanic according to the social status.

Note of interest: The typology of the variables is not always clear and they could be considered of different types depending on the criteria of each person and the objectives sought. An example could be the representation of age, which could be correctly considered both numerical (continuous or discrete depending on the precision) and ordinal, depending on whether you want to work with formulas to treat age as a continuous value or simply work with specific age ranges according to the relationship of the rest of the variables.

Histograms and Bar charts

Although I will make a specific post for the data visualization, it is important to briefly mention the histograms and bar graphs. These are two of the most useful representations in the data analysis and help to understand how the data are distributed with respect to a variable or characteristic of the dataset (continuous in the case of histograms and discrete in the case of bar graphs).

Example: Bar chart

Example: Histogram

Thus, significant information about the data distribution can be extracted, classifying it in symmetric or asymmetric, as we will see below.

Measures of descriptive statistics

A key step when analyzing a dataset is what is called "descriptive statistics", that is, in describing the data through a series of measures that allow to characterize its distribution and its dispersion.

Measures of central tendency

The measures of central tendency serve to give us an idea of the distribution of our data.

Arithmetic Mean (Average)

This measure seeks to obtain the average value of a dataset. That is, all the values in the dataset are summed and divided by the total number of values in that data.

Population mean:
\[\mu = \frac{1}{N}\displaystyle\sum_{i=1}^Nx_i\]
Sample mean:
\[\bar x = \frac{1}{n}\displaystyle\sum_{i=1}^Nx_i\]

Being N the total of values in the population and n the total of values in the sample.

Example:
If we have the following set of 5 data: [1, 2, 2, 5, 10], the average or mean of these values is their sum (20) divided by the total of data (5), that is, the average or mean is 4.
Example of the Titanic dataset:
Following up on our example of the Titanic dataset, and taking the Age variable in the sample train.csv, we visualize its distribution and its average or mean:

Median

This measure represents the middle value in the dataset. That is, for its calculation, we take the data point in the middle of all the dataset x_m:

\[\displaystyle x_{m} = x (\frac{n+1}{2})\]

The median, due to its dependence on the position of the values and not on their values, is less dependent on extreme values (outliers) than the arithmetic mean. That is why in many common statistics (for example, the average salary of the population) it would be more convenient to use the median instead of the average to avoid the presence of very high values that could misalign or mislead the analysis.

Example:
If we have the following set of 5 data: [1, 2, 2, 5, 10], the median of these values is the data n the middle of the dataset. That is, the value in the third position, the number 2.
Example of the Titanic dataset:
Following up on our example of the Titanic dataset, and taking the Age variable in the sample train.csv, we visualize its distribution and its median:

Mode

The mode in statistics represents the most frequently accruing value in the dataset.

Example:
Continuing with our set of 5 data: [1, 2, 2, 5, 10], the mode is the most repeated value, that is 2.
Example of the Titanic dataset:
Following with our example of the Titanic dataset, and taking the Age variable in the sample train.csv, we visualize its distribution and its mode:

Analysis of central tendency

As the examples show, first conclusions can be drawn regarding the distribution of the analyzed variable and its most representative values. Therefore, and comparing these values, we can characterize the distribution of a variable to be symmetric (those cases where the mean, median and mode are similar), or to be positive asymmetric (more values are below the mean and in that case the mean is greater than the median and this greater than the mode) or negative asymmetric (more values are above the mean and the mean is lower than the median and this lower than the mode).

As we have seen in our example of the Titanic and the Age of its passengers we have a certain positive asymmetry (Mean = 29.69 > Median = 28 > Mode = 24) which indicates that there were more people below 29.69 years that above.

Meaures of variability

The measures of variability serve to give us an idea about how spread apart the data values are.

Range

The range is simply a way to express full distribution of the data and it's calculated substracting the minimum value in a set from the maximum one.

\[\displaystyle Range = Max (i) - Min (i)\]

Example:
Analyzing our dataset of 5 values: [1, 2, 2, 5, 10], the range is 10 - 1 = 9.
Example of the Titanic dataset:
Following with our example of the Titanic dataset, the range in the Age variable is 80-0,42 = 79,58 years.

Variance

The variance is a measure of the dispersion of the values with respect to its average or mean value. It is calculated by obtaining the average value of the square of its residuals (differences of each value with respect to the average). That is:

Variance of the population:
\[\sigma^{2} = \frac{\displaystyle\sum_{i=1}^N (x_i-\mu)^2}{N}\]
Unbiased variance of the sample:
\[s^{2} = \frac{\displaystyle\sum_{i=1}^N (x_i-\bar x)^2}{n - 1} \]

The calculation is made on the square of the differences because if we add all their differences the result would be 0.

Regarding the difference in the denominator of both calculations, this is because we are using what is known as unbiased variance of the sample. This modification is based on the fact that we are calculating a statistical value of a sample that could be leaving the population mean outside or far from our values, if we calculate the variance of the sample dividing by n, we could be underestimating the real value of the population variance (more information can be found at Kahn Academy: Explanation about why we divide by n-1 for the unbiased variance of the sample

Example:
Continuing with our set of 5 data: [1, 2, 2, 5, 10], the variance of the population is 10,8.
Example of the Titanic dataset:
Following with our example of the Titanic dataset and the Age variable in the sample train.csv our unbiased variance is 211,01.

Standard deviation

The standard deviation is a measure of the dispersion of the values of the analyzed variable represented in their same units. Its calculation is made by the square root of the variance and its representation is simply sigma or s depending on whether it represents the population or the sample.

Example:
Continuing with our set of 5 data: [1, 2, 2, 5, 10], the standard deviation is 3,67.
Example of the Titanic dataset:
Following with our example of the Titanic dataset and the Age variable in the sample train.csv our, the standard deviation of the sample is 14,52 years.

Standard error

The standard error serves to measure how well a sample represents the population. The calculation is made by dividing the standard deviation by the square root of the total values of the sample.

\[\displaystyle SE_{\bar x} = \frac{s} {\sqrt n}\]

Example of the Titanic dataset:
Following with our example of the Titanic dataset and the Age variable in the sample train.csv our, the standard error of the sample is 0,54.

Analysis of the variability

As seen from the examples seen, these measures help to draw first conclusions regarding the dispersion or variability of our distribution values. This is useful when comparing datasets on variables with those we have worked with previously.

In addition, and in combination with the measures of central tendency, these measures are of special interest in order to extract information about the distribution of the data of our variable, and in this way, characterize said variable in one way or another.

Basic distributions

As final subject in this first article of the Fundamentals series, it is interesting to visualize the distributions of the data because they can give more clear information than the one extracted from their calculated values. According to the data treated, the most typical distributions that we can find could be:

Based on the asymmetry coefficient (according to the measures of the central tendency), and as mentioned above, we could distinguish:

Symmetric distribution:

The values of the mean, median and mode are similar

Asymmetric distribution:

Positive asymmetry:

More values are concentrated below the average or mean. Being this greater than the median and this greater than fashion.

Negative asymmetry:

More values are concentrated above the average or mean. Being this smaller than the median and this smaller than fashion.

According to the shape of the distribution, the most typical cases of representation would be:

Uniform distribution:

Almost all values are identical throughout the range of the variable. This type of distributions could be due to the fact that the representation intervals are too small or too large, and because the variables represented consist of a combination of several variables. Therefore, it is advisable to vary the way of representing the data to detect changes that induce more information:

Random distribution:

Those distributions without a clear pattern of representation where different modes or peaks can be distinguished. These could be due to the same cases as those indicated for the normal distribution, so it is recommended to try to make changes in their representation:

Bimodal distribution:

Distributions with 2 modes or peaks, which generally indicates that they are affected by 2 sources of variation that should try to be studied independently. It could be the case that the measures of central tendency and variability are similar, so their visualization is convenient for a better understanding.

Normal distribution:

The normal or Gaussian distribution consists of the desired distribution for all our data. Although it will have a specific post for analyzing its characteristics, briefly explained, it is a symmetrical bell-shaped function that allows us to model numerous phenomena of nature. The great advantage of having variables whose data are assimilated to this type of distribution is that their distribution can be characterized according to their standard deviation in the following way:

Sources of interest:

Kaggle – Titanic Machine Learning from Disaster