Coder Legion - Recent posts tagged emulation

Porting Test Drive II from SNES to PC, Part 5: Making scanline validation repo-owned

Sat, 21 Mar 2026 13:48:21 +0000

Porting Test Drive II from SNES to PC, Part 5: Making scanline validation repo-owned

The cleanup thread in this repo is still product work.

That point is easy to miss when a change is small. The commit I closed today, 224057d on 2026-03-19, does not add a new renderer, a new extractor, or a new visible gameplay result. What it does is remove another place where the archaeology loop could silently lean on mutable shared emulator output.

That matters more than it sounds.

The last cleanup checkpoint was only half the job

The previous cleanup commit, d06ac64, moved the boot-probe path onto repo-owned outputs by default.

That was a good checkpoint, but it did not close the whole class of problem.

There was still a committed scanline-validation path that defaulted to Mesen's shared LuaScriptData area:

validation/mesen_scanline_step_test.lua
tools/capture_visible_mode7_range.py

That meant a helper could still read from a mutable emulator-owned location unless the caller was careful.

In a repo like this, that is a real risk:

the same script can be run many times across different frame windows
multiple archaeology lanes can be active in parallel
old probe JSON can look plausible enough to reuse by mistake
automation becomes harder to reason about when "latest file wins"

The whole point of the cleanup track is to make that kind of accidental ambiguity harder.

What changed in this checkpoint

This iteration was intentionally narrow.

The generic Mesen launcher, validation/run_mesen_capture.sh, now prepares the parent directory for TD2_SCANLINE_TEST_OUTPUT_PREFIX before launching the emulator.

That means the scanline probe can be given a repo-owned destination without relying on the caller to pre-create the directory tree.

Then the committed visible-range helper, tools/capture_visible_mode7_range.py, was updated so that it no longer defaults to shared LuaScriptData.

Instead, when you run something like:

python3 tools/capture_visible_mode7_range.py \
  1094 1101 \
  --output tools/out/visible_mode7_1094_1101.json

the helper now derives a repo-owned raw probe prefix beside the aggregate output:

tools/out/visible_mode7_1094_1101_scanline_probe/td2_scanline_step_test.json

It also records that raw probe path in the aggregate JSON payload, so the run is easier to trace later.

That is the important pattern:

one command
one explicit aggregate output
one repo-owned raw probe surface attached to that run

No silent dependence on whatever happened to be left in .mesen-config/Mesen2/LuaScriptData.

Why this matters for archaeology

This project is already juggling several different kinds of evidence:

emulator captures
bridge extracts
direct ROM-derived builders
runtime manifests
scanline probes
contract checks

That is healthy, but only if the repo keeps a strong boundary between:

authoritative artifacts
scratch intermediates
stale leftovers from old experiments

Shared emulator output blurs that boundary.

Repo-owned per-run output sharpens it again.

That pays off immediately in a few ways:

validation commands become easier to rerun without cleanup guesswork
aggregate JSON can point back to the exact raw probe surface it used
it becomes clearer which artifacts belong to the repo's evidence chain
future automation can assume less hidden global state

None of that is glamorous, but it is exactly the kind of work that keeps a reverse-engineering-heavy port from becoming fragile.

What this checkpoint does not solve

It is important not to oversell this.

This commit does not finish the validation-output cleanup track.

In particular, it does not yet absorb the newer gameplay-oriented scanline wrapper that is currently sitting in the dirty worktree. That helper is part of active lane work and should be closed as its own checkpoint instead of being folded into a cleanup commit by accident.

So the state after 224057d is:

boot-probe output defaults are repo-owned
the committed visible-range scanline helper now follows the same policy
the broader scanline/gameplay cleanup surface is still open

That is the right level of scope for one iteration.

Where the project stands now

The repo's overall posture is still the same:

Lane 1 bank30 unresolved queue closure remains the official top gate
Lane 2 tilemap provenance has solid contiguous coverage through 1086..1117
Lane 3 gameplay archaeology is active and currently lives deep in visible-phase scanline work
the cleanup/refocus track is still running in parallel so those archaeology lanes stay trustworthy

This checkpoint belongs to that last category.

It is small, but it makes the validation surface more deterministic, and that gives the later archaeology results a better floor to stand on.

Next step

The next cleanup-side step is straightforward:

push the same repo-owned/per-run policy through the remaining scanline and gameplay wrappers
keep trimming lingering doc examples that still assume shared emulator output

After that, the project can keep pressing the more interesting blockers:

the 958..977 bootstrap gap
the 986+ final-screen composition gap
the callback-family replacement work beyond the current intro coverage

That is the shape of progress here.

Not every useful checkpoint produces a new frame.

Some of them make sure the next frame result is actually believable.

Porting Test Drive II from SNES to PC, Part 3: Provenance instead of screenshots

Sat, 21 Mar 2026 13:29:08 +0000

Porting Test Drive II from SNES to PC, Part 3: Provenance instead of screenshots

Once the SDL runtime could play a real intro path, the next bottleneck was not rendering. It was evidence.

If a frame looked close, that was no longer enough. The project needed to answer harder questions:

Which ROM chunk produced the visible tilemap?
Which callback path was active when that chunk was used?
Is this frame a stable source window, or just a screenshot that happens to match today?

That is the phase that starts on March 5 and dominates the repo through March 17.

The repo grew a dedicated archaeology workspace

Commit 6a12991 on 2026-03-05 added the rom_analysis/ workspace and a lot of the tooling that now defines the back half of the project:

build_bank30_chunk_registry.py
build_mesen_design_pack.py
build_mesen_design_pack_range.py
extract_compression_header_manifest.py
validate_td2_chunks.py
summarize_l001210_trace.py
new roadmap and checkpoint docs under rom_analysis/docs/

This was a useful structural move for two reasons.

First, it separated "runtime code that has to ship frames" from "analysis code that has to explain frames." Those concerns overlap, but they are not the same job.

Second, it made the archaeology lane explicit. The repo stopped treating bank 10, bank 11, and bank 30 as vague future work. It started giving them artifacts, checkpoints, and pass or fail surfaces.

`L001210` became the main provenance hook

The core trace target in this phase is L001210, the dispatcher at 00:9210 that is tied to chunk usage. The project added probe-side tracing and summary tooling so it could move from static bank scanning to runtime-backed provenance.

That flow looks like this:

make -C tools l001210-probe L001210_PROBE_TOTAL_FRAMES=3600 MESEN_TIMEOUT_SECONDS=90
make -C tools l001210-trace-summary
make -C tools bank30-registry

By itself, a bank scan tells you where candidate compressed blocks exist. It does not tell you which ones matter in a live frame window. The L001210 trace changed that.

The bank-30 registry could now classify candidates by both decode status and runtime visibility. As of the current snapshot, the unresolved queue is still concrete:

1E:E91F
1E:EE7F
1E:DA96
1E:9681

That is better than a hand-wavy "some bank 30 work remains." It is a small list of named unresolved producers with evidence attached.

Design packs made the frame data usable outside the emulator

The project also stopped treating raw Mesen dumps as the final output format.

mesen_ppu_extract and the design-pack builders turned frame windows into bundles that can be inspected like assets instead of debugger state:

per-layer visible renders
decoded tilemaps
tile sheets
sprite analysis
raw VRAM/CGRAM/OAM
stable manifests for a frame or a frame range

That gave the repo a middle layer between emulator internals and runtime playback. The port could still consume raw scene files, but the engineer could now inspect the same frame as:

raw SNES memory
decoded tilemap entries
sprite metadata
a runtime sequence window

That is why later provenance work stays readable. The project has actual frame packages, not just screenshots and prose.

Provenance windows got promoted one range at a time

The best example is the tilemap-to-ROM provenance chain around the late attract path.

Commit bd239a0 on 2026-03-13 promoted provenance windows through 1101. The next commit, 7614c8f, pushed that contiguous reading through 1117. Those windows are not just captured frames. They carry a source argument.

The current reading in the roadmap is very specific:

bg1 is the visible main-screen layer across the promoted window
chrBaseWords = 0x2000
runtime provenance steps through three chunk anchors:
- 1094..1095 -> 0D:C4DC
- 1096..1100 -> 07:BF49
- 1101..1117 -> 07:C112

That is the kind of statement that changes implementation strategy. Once a port effort knows which visible block comes from which chunk family, asset conversion and runtime replacement can be targeted. Without that, every new frame range looks like a one-off.

The repo is careful here in a way I like: it does not overclaim continuity. It explicitly treats 1117 as the current confidence edge and avoids pretending that later frames are solved by carry-over alone.

The same tooling started reaching gameplay

This archaeology phase is not limited to front-end cleanup.

Commit 36767ce captured the first deterministic track-1 seed window for frames 86..93. Commit 207d8f6 followed with a bounded sweep. The current roadmap now says:

.mesen-config/Mesen2/SaveStates/game_11.mss is a usable deterministic seed
b_hold is the first nontrivial dynamic lane
the queue cursor equalization path is directly observed through frames 90..92

That is exactly the right style of progress for gameplay-era reverse engineering. It is not "bank 10 is understood." It is "this input schedule on this savestate produces this specific observable state transition window."

The later timed-input extraction work in 802f804 extends the same method to a different provenance problem. The repo recovered a later direct-hit cluster around:

7051 -> 0D:C4DC
7059 -> 07:BF49
7064 -> 07:C112

Again, the useful part is not just that those frames were found. It is that they were found under a deterministic scripted-input lane and packaged with provenance outputs.

What changed in practice

Before this phase, the repo could say:

this frame range matches the emulator closely
this runtime path seems to work

After this phase, the repo could say:

this frame range belongs to callback family 01:9FE5
this visible tile block maps to specific chunk producers
this gameplay seed window is reproducible from a known savestate and input schedule
these bank-30 candidates remain unresolved even after matrix sweeps

That is a different level of engineering maturity.

It also changes what "port progress" means. A project like this does not move linearly from renderer to gameplay. It moves by reducing uncertainty in the places where rendering, content, and control flow meet.

By mid-March, asmdump had become good at that:

deterministic capture
design-ready frame bundles
trace-backed chunk provenance
promoted source windows with explicit confidence edges

That set up the next phase cleanly. Once the repo could explain its frame windows, the next pressure point was obvious: the validation surface had to become stricter, and the repo itself had to get cleaner so generated output did not drown the actual evidence.

Porting Test Drive II from SNES to PC, Part 2: The first native intro slice

Sat, 21 Mar 2026 13:28:35 +0000

Porting Test Drive II from SNES to PC, Part 2: The first native intro slice

The first two days in asmdump were about measurability. The next phase, from February 28 through March 2, was about replacing a narrow front-end slice with native code.

That is the point where the project stopped proving that it could extract data and started proving that it could replay behavior.

The runtime target was small on purpose

The port/ runtime was never meant to be a full game engine in week one. It was a controlled execution environment for known assets and known frame windows.

By this stage, port/README.md already described a focused runtime:

SDL2 window and renderer
fixed 60 Hz loop
256x224 framebuffer
direct SNES BG rendering from extracted VRAM + CGRAM + PPU state
optional OBJ composition from extracted OAM
sequence playback for extracted scene windows
indexed palette-animation playback for compact native clips

That last distinction mattered immediately. Some intro segments are better represented as sampled frame sequences. Others are better represented as a native state machine driven by extracted data.

The project did not try to answer that in the abstract. It answered it on one scene.

Why the Ballistic splash was the right first target

The Ballistic intro window is anchored cleanly at frame 654. It is also a good engineering target:

it is deterministic under no input
it is presentation-heavy but structurally simple
it is mostly a palette-driven BG scene, not a sprite-heavy gameplay state

The repo ended up carrying three versions of the same clip:

a measured reference clip from deterministic screenshots
a ROM-derived clip generated from helper-scene CGRAM plus the A39C palette ramp
a compact callback asset for direct runtime playback via ballistic_a39c

That progression matters. The measured clip proves what the frame window should look like. The ROM-derived clip proves that the asset path can reconstruct it from source data. The callback asset proves that the runtime can execute a compact native representation instead of replaying a bag of screenshots.

By the time commit aa67d27 landed on 2026-02-28, the repo had already marked the milestone directly: "reproduce intro animation natively."

Hybrid playback was a feature, not a compromise

One of the best early decisions in this project was to allow hybrid manifests instead of forcing every frame range into the same representation.

That produced several useful runtime artifacts:

tools/out/ballistic_native_sequence.txt
tools/out/ballistic_rom_sequence.txt
tools/out/ballistic_callback_sequence.txt
tools/out/intro_loop_hybrid_sequence.txt
tools/out/intro_loop_hybrid_bridge_visible_sequence.txt

The project used those manifests to keep progress moving while specific windows remained unresolved.

The no-input intro loop is the clearest example:

direct runtime ballistic_a39c playback covers the Ballistic segment
sampled bootstrap playback bridges the unstable 958..977 window
queue-driven snes_bg replacements cover frames 978 and 982
later attract states can be swapped between sampled and native bridge-visible windows as coverage improves

This is a better engineering shape than an all-or-nothing rewrite. Every new native window can replace a sampled window when it is good enough, without destabilizing the rest of the loop.

The bootstrap and bridge work forced more precise tooling

The easy story would have been "render the Ballistic clip and move on." The real story is that getting beyond it forced the repo to model the transition frames precisely.

The project added:

repeatable L00A00C scene building
decoded WRAM queue summaries for the bootstrap window
queue-driven scene builders for post-Ballistic frames
OAM-aware snes_bg playback
bridge-visible Mode 7 scene models for the late attract path

That is how the repo got to statements that are actually useful:

frame 978 and frame 982 can be reconstructed from the queued VRAM/OAM path
frame 986 is close on the BG side but still wrong in OBJ composition
late bridge-visible frames through 1093 can run natively in the SDL runtime, even when screenshot parity is still open for some of them

The runtime command surface reflects that evolution:

make -C port

./port/build/td2_port --sequence ./tools/out/ballistic_callback_sequence.txt
./port/build/td2_port --sequence ./tools/out/intro_loop_hybrid_sequence.txt
./port/build/td2_port --sequence ./tools/out/intro_loop_hybrid_bridge_visible_sequence.txt
./port/build/td2_port --headless --sequence ./tools/out/intro_loop_hybrid_bridge_visible_sequence.txt --sequence-no-loop --frames 1418 --dump-prefix ./port/build/intro_loop_hybrid_bridge_visible

At this point the port is no longer just "an SDL viewer for extracted images." It is executing multiple representations of front-end state:

pure extracted scene windows
compact callback assets
palette-driven indexed animations
queue-derived bridge scenes

That is the real milestone.

The code had to be reorganized early

Commit 8858823 on 2026-03-01 is one of the most practical commits in the whole history. It split a 3083-line main.c into:

td2_types.h
td2_io.c/.h
td2_ppu.c/.h
a much smaller main.c

That same commit also added:

docs/bootstrap_9D69_analysis.md
docs/bank10_11_30_notes.md
port/test_regression.sh

This is not incidental cleanup. It is the point where the runtime stopped being a prototype and started becoming an environment that other port work could live inside.

There are two reasons that matters:

The PPU and scene-loading path were already complicated enough to deserve their own modules.
The reverse-engineering notes and the runtime code had started to depend on each other in both directions.

Once the project reached hybrid intro playback, runtime work and archaeology work were no longer separate lanes. The runtime needed more exact hardware facts. The archaeology needed a runtime that could replay extracted states and make errors visible.

First native launch means more than "the window opened"

Commit 87a3f67 on 2026-03-02 is named "first native launch running intro anim." The important part of that message is not "launch." It is "running intro anim."

At that point, the project had already moved from:

raw ROM and disassembly
to deterministic captures and extraction
to a PC runtime capable of replaying a real front-end path with native logic in the loop

That is the point where the questions get more expensive and more interesting.

You are no longer asking:

Can I decode this chunk?
Can I render this tilemap?

You are asking:

Which frames still need sampled playback?
Which callbacks can be promoted to native code?
Where does the remaining mismatch come from: BG state, OAM state, queue timing, Mode 7 math, or presentation order?

That is why the next phase in the repo turns hard toward provenance, callback tracing, and window-by-window archaeology. Once the renderer can play the intro, the bottleneck is no longer drawing. The bottleneck is explaining exactly where each visible window comes from.

Porting Test Drive II from SNES to PC, Part 1: Day 0 to a measurable ROM

Sat, 21 Mar 2026 13:28:13 +0000

Porting Test Drive II from SNES to PC, Part 1: Day 0 to a measurable ROM

Between February 26 and March 19, 2026, the asmdump repo moved through 45 commits. It started as a raw reverse-engineering dump and ended this stretch with a C/SDL runtime, deterministic emulator probes, tilemap provenance tooling, and regression gates.

This first part covers the first two days: what was in the repo on day 0, why the project did not start with gameplay code, and how the initial extraction and validation toolchain changed the shape of the problem.

Day 0: the repo was a reverse-engineering base

Commit a683c61 on 2026-02-26 is the real starting point. It imported:

bank0.asm through bank31.asm
game.smc and game.sym
ROM build files like Makefile, linkfile, main.s, snes.asm, and spc700.asm

That is enough to rebuild and inspect the ROM. It is not enough to port it.

There was no extraction pipeline. There was no deterministic capture harness. There was no stable asset format that a PC runtime could load. The repo knew a lot about addresses, but very little about contracts.

The absence of a direct decompilation path matters here. bank0.asm already exposed recognizable reset, NMI, IRQ, and callback scheduling logic, but the gameplay-facing banks were not in a state where a straight assembly-to-C pass would be a good use of time:

bank10.asm and bank11.asm still mixed code and data heavily
bank30.asm looked more like structured tables and dispatcher data than a clean list of callable functions
game.sym exposed labels and coverage, not behavior

So the early plan was not "translate bank 0, then bank 1, then bank 10". The early plan was:

Make the original ROM measurable.
Extract typed data from it.
Rebuild systems on the PC side against those measured outputs.

That decision is still the right one. A reliable port needs a reference harness before it needs more C files.

The first burst of tooling

February 27 has nine commits. That entire day is the actual day-1 port effort.

The important commits are:

c867efd - SNES asset extraction toolchain scaffolding
fcc30b5 - scene, sprite, and road visualization tooling
f8785b9 - fully integrated Qt asset viewer UI
2bc9dc2 - initial scripts, sprite translation, capture data, and the first runtime skeleton

The repo stopped treating the ROM as a monolith and started turning it into inspectable outputs:

palette dumps as JSON
VRAM reconstruction and boot-screen manifests
decompressed chunk outputs
tile sheets rendered to neutral image formats
Mesen-based screenshot and state capture
scene visualizers that could be driven outside the emulator

Representative commands from that stage still describe the approach well:

./validation/run_mesen_capture.sh

python3 tools/extract_bank3_palettes.py game.smc tools/out/bank3_palettes.json
python3 tools/extract_boot_screen_manifest.py game.smc tools/out/bank1_boot_screen.json
python3 tools/build_boot_vram.py game.smc tools/out/bank1_boot_screen.json tools/out/bank1_boot_vram_variant0.bin --json-out tools/out/bank1_boot_vram_variant0.json
python3 tools/decompress_td2_chunk.py game.smc tools/out/bank7_42fb_8000.bin --bank 7 --addr 0x8000 --json-out tools/out/bank7_42fb_8000.json

None of that is glamorous. All of it is necessary.

The early validation/ scripts mattered because they made the emulator part of the build surface instead of a manual debugging tool. mesen_capture.lua gave the project deterministic frame dumps and input logs. mesen_probe_boot.lua started recording boot and title-state selectors frame by frame. Once those existed, "what does the ROM do here?" stopped being a memory question and became a file question.

Typed outputs changed the project

The most important change in those first two days was not the number of tools. It was the move toward typed outputs.

There is a real difference between:

"I have a PNG of the title screen"
and "I have VRAM, CGRAM, OAM, PPU state, decompressed source chunks, and the command sequence that built this screen"

The first is useful for discussion. The second is useful for engineering.

By the end of the initial toolchain burst, the repo already had examples of that second category:

tools/out/bank3_palettes.json
tools/out/bank1_boot_screen.json
tools/out/bank1_boot_vram_variant0.json
tools/out/bank7_42fb_8000.json
deterministic Mesen capture logs under .mesen-config/Mesen2/LuaScriptData/

That gave the project a clean boundary between "ROM-side fact" and "PC-side implementation". It also made later runtime work much less fragile. The SDL renderer could target stable files instead of scraping live emulator output or hard-coding one-off frame knowledge.

Why the visualizers mattered

fcc30b5 and f8785b9 are easy to dismiss as UI work, but they are part of the core engineering story.

The early scene, sprite, road, and Qt viewer tools turned opaque ROM state into something an engineer could inspect without rebuilding the entire mental model every session. That is not polish. That is throughput.

On a project like this, the fast path is rarely "write more code". The fast path is usually:

expose the data
make it inspectable
reduce the number of manual emulator steps between a hypothesis and a rendered result

That principle shows up all over the repo after day 1. The later design packs, provenance windows, regression gates, and scanline samplers are all extensions of the same idea.

What the repo looked like after the first real day

By the end of 2bc9dc2, the repo had crossed a line:

it still was not a gameplay port
it still was not a decompilation
but it was no longer just a disassembly dump

It had become a measurable workbench:

deterministic captures
extraction scripts
decompression tooling
typed manifests
visual inspection tools
the first C/SDL runtime skeleton under port/

That last item is where the next part starts. Once the reference side was strong enough, the obvious next step was not more extraction. It was to take one narrow front-end slice and get it running natively.

That slice ended up being the Ballistic intro path, and it forced the project to answer a harder question than "can I decode this asset?":

"Which parts of the intro are best represented as raw extracted state, and which parts need a native callback-driven implementation?"

Coder Legion - Recent posts tagged emulation

Porting Test Drive II from SNES to PC, Part 5: Making scanline validation repo-owned

Porting Test Drive II from SNES to PC, Part 5: Making scanline validation repo-owned

The last cleanup checkpoint was only half the job

What changed in this checkpoint

Why this matters for archaeology

What this checkpoint does not solve

Where the project stands now

Next step

Porting Test Drive II from SNES to PC, Part 3: Provenance instead of screenshots

Porting Test Drive II from SNES to PC, Part 3: Provenance instead of screenshots

The repo grew a dedicated archaeology workspace

L001210 became the main provenance hook

Design packs made the frame data usable outside the emulator

Provenance windows got promoted one range at a time

The same tooling started reaching gameplay

What changed in practice

Porting Test Drive II from SNES to PC, Part 2: The first native intro slice

Porting Test Drive II from SNES to PC, Part 2: The first native intro slice

The runtime target was small on purpose

Why the Ballistic splash was the right first target

Hybrid playback was a feature, not a compromise

The bootstrap and bridge work forced more precise tooling

The code had to be reorganized early

First native launch means more than "the window opened"

Porting Test Drive II from SNES to PC, Part 1: Day 0 to a measurable ROM

Porting Test Drive II from SNES to PC, Part 1: Day 0 to a measurable ROM

Day 0: the repo was a reverse-engineering base

The first burst of tooling

Typed outputs changed the project

Why the visualizers mattered

What the repo looked like after the first real day

`L001210` became the main provenance hook