This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

The Haunted Frog is a tiny language model that lives in your terminal. You teach it Halloween, and then it whispers back.

🐸 THE GHOST IS LAUGHING
🐸 THE NIGHT IS HUNGRY
🐸 THE OWL SAYS WHO
🐸 THE GHOST
🐸 THE ZOMBIE IS FLICKERING
🐸 THE FOG IS HUNGRY

Those are six consecutive samples (seeds 1003-1008), not a script and not hand-picked. The frog learned 60 short Halloween phrases ("THE FOG IS ROLLING IN", "THE WITCH IS LAUGHING", "I AM HUNGRY"...) and recombined them. Some outputs reproduce a training line (THE OWL SAYS WHO is one), but "the zombie is flickering" and "the fog is hungry" appear nowhere in its training data.

The "ghost in the machine" here is literal and small. The model is CSE-Frog, an open-source next-symbol model I made earlier this year as a dissection specimen for learning how language models work. It is not a Transformer. It has no attention and no gradient descent, it looks back only about two symbols, and you can read the whole engine in one sitting. For this challenge I built a brand-new app around it that trains it, haunts a terminal with it, and measures honestly what it learned.

Demo

A real, unedited run. Every 🐸 line is model output, and the seed is printed next to it. Run python -m haunted_frog haunt --count 6 --seed 1003 and you get exactly these lines. The dim text ("Something small is breathing inside the machine...") is hand-written narration and is drawn without the frog, so you can always tell which is which.

Code

[

git clone https://github.com/kagioneko/the-haunted-frog && cd the-haunted-frog
python -m venv .venv && source .venv/bin/activate && pip install -e .
python -m haunted_frog train      # ~1 second on a CPU
python -m haunted_frog haunt      # Ctrl+C to leave
python -m haunted_frog evaluate   # the experiment below

No GPU, no API key, no network after install.

How I Built It

1. A corpus small enough to read. data/halloween.txt has 60 lines that I wrote for this project, deliberately sharing words (THE ... IS ..., I AM ..., ... IN THE ...) so the frog has joints to recombine at. You can edit it and retrain in a second.

2. Training. CSE-Frog learns by strengthening links between symbols that follow each other (Hebbian-style), plus a memory of the last pair of symbols. Training is one call: Frog(top_k_edges=16).learn(sequences, epochs=5).

The first bug was a good one. I passed the lines as a flat list of strings, and CSE-Frog read the whole list as one sequence of 60 giant symbols. The frog recited the corpus as one run-on sentence:

HAPPY HALLOWEENTRICK OR TREATGHOST IN THE MACHINETHE GHOST IS HEREI AM A FROG

The fix is a list of lists, one sequence per line, so the frog learns where a phrase ends. There is now a test for it.

3. Generation. frog.generate(prefix, n=max_len, seed=s) samples one symbol at a time until the frog picks <END>. My wrapper never edits, filters or re-ranks the output. An empty answer is printed as (silence).

4. The Haunted Terminal. Green frog, orange banner, typewriter effect, a line every few seconds, and a --no-halloween switch for plain output. Each line uses seed base+i, so a whole spooky session can be replayed.

5. An experiment instead of a vibe. python -m haunted_frog evaluate trains 10 frogs and scores each one on the same 100 seeds.

What worked

• Training is what makes it speak at all, and the corpus decides what it says. The control frog never said a Halloween word. The Halloween frog uses one in every 5-7 words.
• It recombines. With the settings I ship, 83-94% of word-level outputs are not exact lines from the corpus. Here are the first ten seeds of the 1-epoch frog, unpicked: COLD WIND AT MIDNIGHT · COLD WIND AT MIDNIGHT · A SPIDER IN THE WINDOW · I AM HUNGRY · THE NIGHT IS QUIET · I WANT A WITCH ON A GHOST IN THE WIRES · THE MACHINE · SOMETHING IS HUNGRY · A FROG · THE WITCH IS AWAKE

What failed

• It is obsessed with "THE". THE starts 38% of the corpus lines, but 99% of the outputs after 5 or 20 epochs (49% after 1 epoch). Its Halloween-word rate (15-19%) stays below the corpus's own 22.6%, because it over-produces THE and IS.
• More training made it duller. Unique outputs fell from 81% (1 epoch) to 38% (20 epochs). Asking the frog for its probabilities showed why: the chance that a line starts with THE goes from 0.435 after 1 epoch to 0.998 after 20. The strongest links keep getting stronger, the temperature stays fixed, and the softmax saturates.
• The default settings were too narrow. At CSE-Frog's default top_k_edges=3, only 20% of outputs were unique. I raised it to 16 after looking at seeds 0-39, and then scored everything on seeds 1000-1099 so the choice wasn't graded on the samples that picked it.
• The character-level frog almost never formed a Halloween word. In 200 outputs at 5 and 20 epochs it formed none, and at 1 epoch it formed exactly one. It keeps circling the same little loop:

  THE INGHE INGHE IN IN IN WIS INDYS ING     (5 epochs)
  THE IN IN IN ING                           (5 epochs)
  HE S L THE AUNIGIC                         (1 epoch)

My hypothesis is that with only about two characters of context, THE, IN, IS and -ING become attractors it can't escape. I haven't isolated that experimentally yet (that needs a variable context length), but it is the most vivid picture I've seen of why real language models need long context.

Full method, all ten conditions, unedited samples and threats to validity: docs/EXPERIMENT.md.

Why Does Open Innovation Matter?

With a closed model, you can only watch the ghost. With an open one, you can open the machine and find out why it says what it says.

Because the engine is open, small and inspectable, I could look straight at the numbers behind these behaviours. CSE-Frog can print the score breakdown behind any single prediction (frog.show(...)). When the 20-epoch frog got stuck on THE GHOST, I didn't have to guess: scripts/diagnose.py shows THE at 0.998 at the start of a line, and almost all of GHOST's score coming from one direct link. You can run the same script and see the same numbers.

Open also means anyone can repeat this. The corpus is a text file, the seeds are printed on screen, and the model file is plain JSON. python -m haunted_frog evaluate rebuilds the results table in about a minute on a single CPU core, and python scripts/diagnose.py prints the probabilities and score breakdown quoted above. If you think I cherry-picked, run them and check.

And open lets small things exist. A 60-line corpus and a frog-sized model aren't a product. They're a specimen you can dissect. I think more people would understand language models if they could start with one that fits in their head.

The Spirit of It

A ghost in the machine is something that seems alive but turns out to be mechanism. The frog is that idea at a scale you can hold: it says THE FOG IS HUNGRY, and you can look inside and see exactly which links made it say so.

Honest notes

• CSE-Frog itself predates this challenge (I published it on PyPI on 2026-10-04, MIT). It is used unmodified as a PyPI dependency. Everything in the-haunted-frog, including the corpus, app, terminal, experiment and tests, was built during Week 4.
• I wrote the code with an AI coding assistant (Claude Code). Every number above comes from running the repo.
• This is a toy. It does not understand English or Halloween. It recombines short phrases, and the experiment above shows where that breaks.