The Price of Everything ... checked by `jq empty`

Fri, 10 Jul 2026 09:50:57 +0000

Every AI usage cost monitoring tool runs the same multiplication: tokens from a log file times a price from a table.

The tokens are easy. Claude Code and its cousins write them to disk with model names attached.
The price is the interesting part, because almost the whole ecosystem gets it from one place: LiteLLM's community price map, about 2,783 models in a single JSON file, in a repo with roughly 51k stars. ccusage, at 16.4k stars itself, re-pins it hourly. tokencost vendors a copy. ccost embeds it. When that file is wrong, everyone is wrong together, in the same direction, with the same confidence.

The only automated quality check on that file is `jq empty`.

Your linter has stronger opinions about import order than the ecosystem has about whether a million Grok tokens cost 💲2.00 or 💲1.25. Money-adjacent data, maintained with less rigor than a lint rule. I don't know about you, but it bothers me.

That is not a simplification. jq empty confirms the JSON parses. No value validation, no cross-check against the provider's published page, no guard against swapping input and output prices. Nothing dates a record or says who verified it. The number your cost dashboard multiplies by a month of token counts clears CI the moment it is syntactically valid.

The 💲543 branch

One Saturday morning I grouped a month of my spend by project and branch, mostly out of curiosity. The month came to about 💲2,770. One feature branch, a big Rails plus SvelteKit UI refactor, had eaten 💲543 on its own, more than everything else on that client combined. The tiny PR branch next to it: 💲32.

The mechanism was the retry loop. A complex multi-file refactor fails halfway, and the retry re-sends the entire 80K-plus token context. You revert, you try again, same cost, nothing shipped. You never feel it per message. You feel it at month end, as a lump.

Tools existed to show me this after the fact. ccusage and its relatives parse the same logs and render perfectly good totals. What I wanted did not exist: something that enforces a limit instead of a dashboard you check once the damage is done.

So I built the open-source BudgetClaw, a single static Go binary that tails the session logs with fsnotify, computes cost in real time, and on a breach sends SIGTERM to the Claude process and mobile notifications via ntfy. ccusage tells you what you spent; BudgetClaw stops you from spending it.

Which is exactly where I hit the wall.

Enforcement is only as good as the multiplication

Cache pricing is what makes this unforgiving: a cache read bills at 0.1x the base input rate, a 5-minute cache write at 1.25x, a 1-hour cache write at 2x. Get one multiplier or one base rate slightly wrong and every per-branch figure is quietly off. For a tool whose entire value is "trust this number", quietly off is not acceptable in my book. So before shipping, I read how the price table everyone depends on is actually maintained.

The LiteLLM autopsy

Start with a concrete case. Four xai/grok-4.20-* rows in LiteLLM's map, each citing its source:

in the map:        $2.00 / $6.00 per 1M tokens (input / output)
cited docs.x.ai:   $1.25 / $2.50

The entry contradicts its own citation, and nothing in the pipeline can notice, because the pipeline checks that the file parses and stops there. It gets worse the deeper you read. Input and output prices have been transposed in the past with no guard to catch it. A source field is present on only about 31 percent of records. There are no last_validated, confidence, or date fields at all, and no history: when a price changes the old value is overwritten and gone, so nobody can answer what a model cost in March. Governance compounds it.

The default branch is litellm_internal_staging, the public main can run about nine days stale and serve a bug already fixed on staging, and a bulk staging sync can overwrite a community fix after it merges. A correction can land, merge, and silently regress.

The ccusage community already knows , the request is closed unresolved, and ccusage keeps applying today's prices to all historical data, so costs do not reflect what was actually charged at the time. The upstream data model cannot express it, because the upstream data model deletes the past.

A 28-line control experiment

I did not want to write about a bug I had not tried to fix, so I filed the fix. BerriAI/litellm#30849, Twenty-eight lines, touching both the root map and the bundled backup so they could not drift on this model, plus the cost-calculator test that did not exist, so the corrected number could not silently regress.

Weeks later it sits open and unmerged, reviewed by exactly three entities: a CLA signature bot, greptile, and codecov. No human has looked at it. The lazy reading is "maintainers bad", and the lazy reading is wrong. The repo fields roughly 1,158 pricing PRs. No volunteer team hand-verifies that volume against provider pages, which is precisely why the only gate is jq empty: it is the one check that scales without humans. The failure is structural. Money-adjacent data needs the rigor of money, and a JSON parse check is not that, however good the people running it are.

Correctness is a provenance problem

My first instinct was the obvious wrong one: fork it, cover more models, do it better. Then I realised: nobody's cost tool broke because model 2,784 was missing; mine would have broken because one number among the 94 I use was wrong, uncited, and undated. One bad price kills a cost tool; 2,700 correct ones do not save it.

So the thing I built is not a bigger table. It is an automated price sentinel, published as ai-price-index. A bot diffs each provider's own official pricing page; a human - mostly me, verifies every change before it lands. Every record carries a first-party source URL, an effective date, a last_validated date, and a confidence label. The dataset is bitemporal: a correction supersedes the old row, it never deletes it, so the audit trail survives its own fixes. Point-in-time lookup is a first-class operation:

priceOn("gpt-4", "2023-06-01") // the rate in effect that day, not today's

As of this writing the index holds 94 models across 11 providers, 202 current price rows, and history back to GPT-4's launch in March 2023. It ships as an open dataset (CC-BY-4.0 data, MIT tooling) with raw JSON, an npm library, and a zero-key MCP server, on the theory that a reference table is only useful if it is easier to reuse than to re-scrape. Found a wrong price? Send a PR with the source.

One table under everything

The payoff is boring, which is how I know it is real. That single audited table is now the shared backbone for BudgetClaw and a handful of small, free, fully client-side cost tools on roninforge.org: a Claude Code spend analyzer, an Anthropic cost analyzer with per-API-key attribution, the price explorer itself, a couple of Copilot bill calculators. They parse your local exports in the browser and upload nothing, and because every one reads the same sourced, dated rows, their numbers cannot drift apart. None of them has an interesting UI. What they have is a price table with receipts.

Parsing the logs is the easy 90 percent; the tokens sit on your own disk with model names attached. The dollar at the bottom of the report is a different kind of claim. It is only worth trusting if someone kept the receipt for the price, and today, for most of the AI cost stack, that receipt is a JSON file whose sole certification is that it parses. jq empty can tell you a price table is valid. It cannot tell you it is true.

Coder Legion - Recent posts tagged token-tracking