Quick Answer
LLM token cost is the price a provider charges per token a model reads or writes, quoted in dollars per million tokens. Input and output bill at separate rates, with output priced at roughly 5x input. As of September 2026, published rates range from under $0.10 to more than $180 per million tokens on top-end reasoning tiers.
In late 2025, Hardik Sonetta of Thomson Reuters Labs published a warning about the most common prompt caching mistake in production. The pattern: fire parallel calls at the same large document, caching enabled, and assume the cache is working. In the team’s real-world testing, that pattern produced a cache hit rate of 4.2%. Essentially zero, the post’s own words.
The culprit is a race condition. Cache creation takes two to four seconds, and parallel calls launch before any cache exists, so each call builds its own. In the post’s worked example, three questions against one 30,000-token research paper create three caches and reuse none, with wasted cost running up to 60% higher per session in their testing. One warming call before the parallel batch cuts the example workload’s cost 59%.
As Sonetta warns, optimizing for speed through parallelization can make an application “slower” and “dramatically more expensive.”
The story is not about a wrong model or an overused one. The meter counted the same work three times, and no rate card in the world would have shown it. That is why understanding tokens, the unit every AI invoice is denominated in, is worth 20 minutes of a budget owner’s life.
What is a token, and why do LLMs charge by it?
A token is the chunk of text a language model actually reads and writes: whole words, word fragments, punctuation, spaces. In English, one token averages about four characters, so 1,000 words of text runs roughly 1,300 tokens. Models process token by token, which makes the token the natural billing unit.
Providers charge by it because compute scales with it. Every token in your prompt gets processed, and every token in the response gets generated, one at a time, on expensive accelerators. Metering the token is metering the work, which is why no major provider prices text generation per request: a request can be 10 tokens or 200,000.
If you want LLM tokens explained in one sentence for a finance audience: the token is to AI spend what the kilowatt hour is to a utility bill, the unit the meter counts, whether or not anyone in the building is watching the meter.
Report
Finance needs to prove AIâs return: CloudZero report
260 senior finance leaders (more than half CFOs) told us why the speed of seeing AI spend, not the size of it, separates who pulls ahead on AI from who gets burned.
How does LLM token cost work?
Token pricing is quoted in dollars per million tokens, with separate rates for input (what you send) and output (what the model writes back). The bill for any call is simple arithmetic: input tokens times the input rate, plus output tokens times the output rate.
Two conventions trip people up. First, older documentation quotes cost per 1,000 tokens instead of per million; divide the per-million rate by 1,000 to convert, so $2 per million is $0.002 per 1K. Second, rates span an enormous range: small workhorse models run pennies per million while frontier models list at $5 to $25 or more per million output tokens. Our LLM API pricing comparison keeps the live provider-by-provider table.
The asymmetry is worth internalizing. Input tokens vs. output tokens is not a fair fight: reading happens in bulk, but every generated token requires its own full pass through the model, and rate cards price that difference at roughly 5x on today’s flagships. As of September 2026, Claude Sonnet 5 reads for $2 per million and writes for $10. A model that pads its answers is spending your money on the expensive side of the meter.
Prompt caching stores a reusable copy of a repeated prompt prefix so later calls bill it at a discounted cache-read rate instead of the full input rate. Anthropic discounts cache reads by 90%, and writing the cache costs 1.25x standard input for a five-minute lifetime or 2x for an hour. The savings only land when the cache actually hits.
Vendor-specific token math lives in our OpenAI pricing guide, alongside the Claude pricing and Azure OpenAI pricing breakdowns.
Why the token is not a standard unit
Different models tokenize the same text into different token counts. Each provider trains its own tokenizer, so a paragraph that costs 100 tokens on one model might cost 130 on another. The token is not a liter or a kilowatt hour. It is a house unit, defined by the house.
That has a sharp implication for comparing cost per token across providers: you are comparing prices quoted in different currencies without an exchange rate. A model with a lower per-token rate but a denser tokenizer can cost more per sentence than its “expensive” rival. A per-token comparison only means something when both sides count tokens the same way, which they rarely do.
Language makes it worse. Non-English text tokenizes less efficiently on most tokenizers, sometimes at double or triple the token count for equivalent meaning, so the same application serving the same feature can carry structurally higher token cost in some markets than others. Few budget forecasts account for it. The invoices do.
Illustrative math makes it concrete. Say Model A charges $2 per million input tokens and renders your average customer email as 900 tokens, while Model B charges $2.60 but renders the same email as 600 tokens. Model A is 23% cheaper per token and 15% more expensive per email. Every pricing table on the internet says A wins. Your invoice says B does.
The token is a fine unit for metering and a poor unit for deciding. Stable comparisons need a stable denominator, and the stable denominators are tasks and outcomes, not tokens.
What counts against your token meter?
Token metering covers more than the words you see. A production call bills for several categories, and the invisible ones explain most bill surprises:
- Input you sent on purpose. The user’s message, plus everything your application attaches: system prompts, tool definitions, retrieved documents, conversation history. In most production apps, the visible user message is a rounding error inside total token usage; the scaffolding around it is the real volume.
- Input you resent without noticing. Chat history and context get retransmitted on every turn, so a 20-turn conversation reprocesses its own past 20 times. This is where cache pricing matters: providers bill repeated cached content at heavy discounts, 90% off reads on Anthropic, but only when prompt structure lets the cache actually hit. The Thomson Reuters case above is what metering looks like when it does not.
- Output, at the premium rate. Everything the model writes back, priced at the 5x multiple.
- Reasoning you never see. Thinking models generate internal reasoning tokens before answering, billed as output. The answer might be 50 tokens; the thinking might be 5,000. If token consumption jumped without a usage jump, a reasoning default is the first suspect.
- Cache writes. Creating a cache costs a premium (1.25x to 2x input on Anthropic) before reads earn it back. Metered, like everything else.
The pattern across all five: the meter is honest, but it counts work, not value, and most of the work is invisible from the chat window. Instrumenting per-feature AI cost monitoring is how the invisible categories become line items someone can own.
Ownership is worth deciding explicitly, because token metering sits in the gap between two teams. Engineering can see the meter but does not carry the budget; finance carries the budget but cannot read the meter. Every unexplained AI bill in your company lives in that gap, and the fix is a shared view both teams trust, not a quarterly translation exercise.
How much do AI agents multiply token cost?
An agent’s token bill is a chatbot’s bill with a multiplier on every term. An agent completing a task reasons, calls tools, reads results, and self-corrects across 10 to 20 model calls per user action, and each call re-carries the accumulated context of everything before it. Volume compounds twice: more calls, and bigger calls as state piles up.
The compounding is why agent workloads dominate 2026 AI budgets even at collapsing per-token rates, and why the scariest line in an agent’s cost profile is context accumulation, not model choice.
Illustrative math again: a 12-step agent starts with 2,000 tokens of context and accumulates 1,000 per step as tool results pile up. Step one reads 2,000 tokens; step 12 reads 13,000. Total input across the task: 90,000 tokens, nearly 4x what a naive “12 calls at 2,000 tokens” estimate predicts. The last three steps alone out-consume the first six combined, and nothing about the workload was wrong. Accumulation is just what agents do.
Two mitigations matter most: caching the stable prefix so accumulated context bills at read rates instead of full price, and pruning state the later steps genuinely do not need. Both are metering problems before they are model problems.
Our agentic AI cost analysis covers budgeting for the multiplier, and the oversight question, whether every autonomous step should run unwatched, has its own economics in our human-in-the-loop cost breakdown.
From tokens to outcomes: the unit that actually matters
AI token cost answers what you paid. It cannot answer what you bought. Getting from one to the other is a three-rung ladder, and here it is with illustrative math for a support bot handling 50,000 tickets a month:
| Rung | Unit | Illustrative math | Number |
|---|---|---|---|
| 1. Rate | $ per million tokens | Workhorse model, $1 in / $4 out | The list price |
| 2. Task | $ per ticket handled | 5,500 input tokens + 400 output per ticket | ~$0.007 |
| 3. Outcome | $ per ticket resolved | Bot fully resolves 60% of tickets | ~$0.012 |
Rung one is what vendors publish. Rung two is what your architecture produces. Rung three is what your business buys, and it is the rung a CFO can actually compare against alternatives: Gartner puts the median cost per contact at $13.50 on assisted channels, more than a thousand times rung three. That comparison, cost per resolved ticket against cost per resolved ticket, is a real business case. A rate card is not.
Each rung fails for a different reason. Rung one fails as a decision tool because tokens are not standard units. Rung two fails at most companies because nobody allocates: tokens get metered by API key, and an API key is not a feature. Rung three fails because spend data and outcome data live in different systems that have never been introduced to each other.
Closing all three gaps per feature is the actual discipline: meter raw tokens, allocate them to the tasks they performed, then divide by the outcomes those tasks produced. A falling per-token rate does not help a team that cannot say what a token bought.
The teams that build this ladder stop fearing token growth. When cost per resolved ticket is visible and falling, a rising token bill is good news, more resolution volume at better unit economics. Without the ladder, the same bill is just a bigger number and a harder meeting.
This is precisely what CloudZero builds for AI spend: ingesting token-level usage from OpenAI, Anthropic, and cloud AI services alongside AWS, Azure, and GCP, then allocating it to products, features, teams, and customers.
Companies like Duolingo, Coinbase, and Skyscanner run spend-heavy engineering organizations on that unit-economics view, where the question “what did tokens cost us?” gets answered in the units the business actually ships. The broader framework lives in our guide to understanding AI costs.
How do you calculate LLM token cost?
The formula fits on an index card: (input tokens x input rate + output tokens x output rate) per call, summed across calls, with cached input billed at the cache-read rate. Estimate tokens from text with the 1.3-tokens-per-word English rule, then multiply by your traffic.
Worked example. A support bot on Claude Sonnet 5 handles one ticket with 5,500 input tokens and 400 output tokens. Input: 5,500 at $2 per million is $0.011. Output: 400 at $10 per million is $0.004. Total: $0.015 per ticket, or about $750 a month at 50,000 tickets. Cache the 4,000-token system prompt at Anthropic’s 90% read discount and input drops to $0.0038, cutting the monthly total to about $390. Rates as of September 2026.
The estimating mistakes are predictable. Teams count the user message and forget the scaffolding, which understates input five to ten times. They assume output matches input, when workloads skew hard one way (classification is input-heavy, generation is output-heavy). And they model the average request when the tail drives the bill.
Model the arithmetic before any launch: plug in tokens per request, requests per user, and users, then stress-test with an agent multiplier if workflows chain calls. Our LLM cost calculator guide walks the full arithmetic with worked examples.
Most calculators also convert between per-1K and per-million conventions, so legacy documentation stops causing 1,000x spreadsheet errors. And for turning estimates into enforced budgets, our AI cost optimization guide picks up where the math leaves off.
Estimates decay fast in production, though. Real workloads drift, prompts grow, and a calculator snapshot from launch week says little about month three. The durable version of this exercise is live metering allocated to features, which is the difference between predicting a bill and controlling one. Our LLM inference cost guide covers the system-level drivers and the levers that bring the number down.
How did token pricing change in 2026?
The rates moved, but the more important change is that the meter itself got more granular. Four shifts matter for anyone budgeting by token this year:
- One model, multiple prices. Context-tier pricing arrived: Google bills Gemini 3.1 Pro at $2 per million input tokens for prompts up to 200K, and $4 above that threshold (per Google’s pricing), with output jumping from $12 to $18 on the same boundary. The same call to the same model now costs different rates depending on how much you send, which makes prompt size a pricing tier, not just a volume knob.
- Reasoning became a billed line. Thinking models moved hidden reasoning tokens from research footnote to mainstream invoice line, billed at output rates. The meter now charges for deliberation, and defaults decide how much deliberating happens.
- Cached tokens became their own price class. With Anthropic at 90% read discounts and OpenAI’s newest flagships matching, cached input is now effectively a third rate alongside input and output. Prompt structure, which used to be a craft preference, converts directly into which of three prices your tokens pay.
- Per-million quoting standardized. The industry consolidated on per-million-token pricing, retiring the per-1K convention that still haunts older documentation and, occasionally, older spreadsheets by three orders of magnitude.
Net effect: a single token can now bill at one of several rates depending on direction, cache state, reasoning mode, and context size. The meter has grown more sophisticated than most of the budgets it reports to.
Meter tokens. Manage outcomes.
Every AI invoice you will ever receive is denominated in tokens, and no business decision you will ever make is. CloudZero closes that gap: token-level spend from every provider, allocated to the features, teams, and customers that consumed it, so the unit on the invoice converts to the units your business runs on.
Request a demo to see your AI spend as cost per feature and per customer.
Frequently asked questions about LLM token cost
