[Chaoqi31]

Why

Codex 0.153+ writes a token_usage_record for every model response. A remote compaction writes that record and then a token_count whose last_token_usage is all zeros. The importer billed only token_count, so those calls became 0 rows. One local thread had 26,705 input and 491 output imported as nothing.

Scope

• src-tauri/src/services/session_usage_codex.rs
• A token_usage_record is held until the next token_count. If that last_token_usage matches the record, the token_count row stays the only row. If last_token_usage is present and all zeros, the record is imported as codex_thread:{root_thread_id}:usage:{response_id}.
• Existing token_count rows keep their event_index. The new suffix does not renumber later requests.
• Existing cursors stay. Compaction lines already scanned before the upgrade are left as they are on main. The new path runs only for lines read after the upgrade.

Tradeoffs

A token_usage_record that does not match a later non-zero last_token_usage stays timing-only. Billing every mismatch double-counted a latency fixture whose record was 1 input and 999 output while the following token_count last was 200 and 700.

The record is not billed at end of file. A live session often writes the record before token_count. Billing at end of file would insert a usage: row, and the next sync would also insert the matching token_count row.

The signal is a zero last_token_usage, not a zero total. Real files keep a non-zero total and a zero last after compaction.

A compaction record copied into a child fork is not covered here. That replay prefix is issue 7917 item 1.

Pre-upgrade compaction usage is not recovered. A one-time reread of already-scanned files would insert detail rows for days that already live in usage_daily_rollups.

Blast Radius

This touches Codex session import. New compaction rows use a separate request id, so existing detail rows are not rewritten. Files whose cursor is already current are not reread.

Verification

Before the change, remote_compaction_usage_record_is_imported_when_the_following_token_count_is_zero imported 0 rows and the test expected 1. The sibling case, a record followed by a matching token_count, imported exactly 1 row under the existing event_index.

After dropping the upgrade backfill, cargo test --manifest-path src-tauri/Cargo.toml --lib session_usage_codex::tests passed 87 tests, failed 0, and ignored 1. cargo clippy --manifest-path src-tauri/Cargo.toml -- -D warnings passed. rustfmt --check was applied to session_usage_codex.rs before the test run.

Section 4 item 3 of #7917. Issue 7917 stays open.

[farion1231]

Thanks for the fix — the pairing between token_usage_record and the following token_count looks right. Replaying a copy of my local ~/.codex (3,038 files) against main, this PR only adds the 102 missing compaction rows and leaves every other row unchanged.

One change before merging: please drop the upgrade backfill (the codex_token_usage_record_backfill marker and the forced reread of already-scanned files) and let the new logic apply only to lines read after the upgrade. Existing cursors stay where they are, so compaction usage from before the upgrade stays as it is on main.

The reasons:

• The first sync after upgrading rereads every already-scanned file. On those 3,038 files that took about 70 s in a debug build.
• The marker is written only when the whole pass succeeds. If one file keeps failing to parse, the marker is never written and the full reread repeats on every periodic sync.
• For users who also run the local route, rows older than 30 days have already been rolled into usage_daily_rollups, so the backfilled detail rows count that usage a second time.

Removing that layer also removes its tests, so the change should get noticeably smaller.

[Chaoqi31]

Dropped the upgrade backfill in 3c97d68. codex_token_usage_record_backfill and the forced reread are gone. Existing cursors stay, so compaction usage from before the upgrade stays as it is on main. New lines still import an unmatched token_usage_record.

[farion1231]

LGTM, thanks for your contribution!

[farion1231]

Thanks for tracking this down! Compaction calls showing up as 0 tokens was an easy one to miss, and pairing each token_usage_record against the following token_count (instead of billing both) keeps the regular path free of double counting. The regression tests cover exactly the cases that matter, and thanks for dropping the backfill layer along the way.

Two small edge cases I noticed, neither blocking, in case you'd like to follow up:

1. If a token_usage_record is the last line of the file when a sync runs, the cursor has already moved past it. On the next sync, the all-zero token_count that follows still uses the record's line number for its event, so it gets skipped and that call is missed. Using the token_count line number there should fix it. The window is tiny (tens of milliseconds) and it isn't a regression.
2. When a forked thread copies the parent's compaction record plus the zero snapshot, those usage events aren't filtered by replay_prefix, so the parent and child could both count them under different roots. I didn't find a real sample locally.

Merged, thanks again!