LLM Pricing Explained: Input Tokens, Output Tokens, Context, and Caching

Every LLM invoice is built from four moving parts: how many tokens you send, how many the model sends back, how much of that gets served from a cache, and whether you paid for synchronous or batch processing. Get the mechanics of any one of these wrong and your forecast can be off by an order of magnitude. Here's what's actually happening under each line item.

Tokens Are Not Words, Words Are Not Characters

A token is a chunk of text the model's tokenizer produces — sometimes a whole word, sometimes a fragment ("token" might become "tok" + "en"), sometimes a single punctuation mark. The commonly cited "1 token ≈ 4 characters" or "~0.75 words" is an average for everyday English prose, and it moves around more than most estimates account for:

• Code tokenizes less efficiently than prose — punctuation, whitespace, and identifiers fragment into more tokens per character.
• Non-English text, especially languages that don't use whitespace to separate words (Chinese, Japanese, Thai), can run 1.5-3x more tokens per character than English.
• Structured output (JSON, XML, Markdown tables) carries syntax overhead — brackets, quotes, and keys all cost tokens beyond the actual data.
• Numbers and rare tokens (UUIDs, hashes, unusual proper nouns) tokenize inefficiently since they don't match common subword patterns.

If your workload is heavy on any of these, don't trust a word-count estimate — run a sample through your provider's tokenizer, or measure actual usage from API response metadata, before you commit to a forecast.

Why Output Costs More Than Input

Across essentially every provider, output tokens are priced at 3-5x the input rate. This isn't arbitrary — it reflects how inference actually runs. Input tokens are processed in parallel in a single forward pass (the "prefill" stage); the GPU chews through the whole prompt at once. Output tokens are generated one at a time, autoregressively — each new token requires a full forward pass that attends over everything generated so far ("decode"). Decode is inherently sequential and far less GPU-efficient per token than prefill, and providers price accordingly.

The practical implication: a system prompt bloated with instructions is expensive, but a model that rambles in its replies is usually more expensive per token generated. Capping response length and asking for concision is often the single cheapest optimization available — see our prompt optimization playbook for specifics.

The Reasoning Token Gotcha

Reasoning models — OpenAI's o-series and similar "thinking" models from other providers — generate an internal chain of reasoning before producing the visible answer. That reasoning is billed as output tokens, even when the API doesn't show it to you by default. This is the most common cost surprise teams run into when they switch to a reasoning model expecting output pricing to track the visible response length.

Example: a request to o3 (Reasoning) that returns a 120-token visible answer but spends 2,200 tokens reasoning internally is billed for 2,320 output tokens, not 120. At o3 (Reasoning)'s rates, that's the difference between $0.0014 and $0.0190 for a single call — roughly a 19x difference from what the visible answer alone would suggest. Reasoning-token volume is genuinely hard to predict in advance because it depends on problem difficulty, not prompt length, which is exactly why reasoning models need wider error bars in any cost forecast than standard chat models.

Context Window: Capacity, Not a Bill Multiplier

A model's context window (128K, 200K, 1M tokens, etc.) is the maximum it can hold in a single call — it is not what you pay for by default. You're billed for the tokens you actually send and receive, not the window size. The connection to cost is indirect but real: a larger context window makes it easier to justify sending more — longer document context, deeper conversation history, more retrieved chunks — and all of that is billable input. The window being large doesn't cost you anything until you fill it.

One caveat worth knowing: a few providers apply a higher per-token rate once a request crosses a long-context threshold (commonly reported around the 128K-200K token mark on models that support very long context). Check your specific model's pricing page if you're routinely sending requests near the top of its context window — the rate for that tier can differ from the headline price.

Prompt Caching: Paying Less to Repeat Yourself

If part of your input is identical across calls — a system prompt, a long set of tool definitions, a document you query repeatedly — prompt caching lets you pay a reduced rate to reread that portion instead of the full input price every time. The mechanics vary by provider, but the shape is consistent:

• The cached segment must be a stable prefix of the prompt — content after the cached portion can change per call, but the cached portion itself has to match exactly.
• There's usually a minimum token length before content is eligible for caching (small system prompts often don't qualify).
• Cache entries expire after a period of inactivity, typically minutes, so caching helps most for bursty, repeated traffic rather than one call a day.
• Some implementations charge slightly more for the initial cache write and substantially less — commonly reported in the 75-90% discount range, though this varies and changes over time — for subsequent cache reads.

Caching is most valuable for the fixed parts of a request (see our chatbot cost estimation guide for how this interacts with growing conversation history) and for RAG systems that repeatedly query the same document set.

Batch APIs: Trading Latency for Discount

For workloads that don't need a synchronous response — nightly summarization, bulk classification, offline evaluation — most providers offer a batch endpoint priced at roughly half the standard rate, typically in exchange for a completion window measured in hours rather than seconds. If your workload can tolerate asynchronous processing, this is often the single largest cost lever available, larger than any prompt optimization.

How the Price Tiers Actually Compare

Providers publish a range of models at different price points, roughly clustered into "nano/lite," "mid-tier," and "flagship" tiers. The spread between cheapest and most expensive is large — often 50-100x on input price alone:

For a fixed workload of, say, 2M input tokens and 500K output tokens a month, the gap between the cheapest and most expensive model here is $22.50 vs. $1.85 — a difference of roughly 12x for the exact same volume. That spread is the entire reason model selection matters more than almost any other cost decision — see how to pick the cheapest model that's still good enough.

Computing Your Own Bill

The formula, stripped to its core:

cost = (input_tokens / 1,000,000 × input_price) + (output_tokens / 1,000,000 × output_price)

As a sanity-check example: a request with 2,000 input tokens and 500 output tokens on Claude Sonnet 5 costs $0.0090. Run the same request through GPT-4.1 and it's $0.0080, or through Gemini 3.1 Flash-Lite and it's $0.0013. Multiply by your expected monthly request volume and you have a real forecast — that's exactly what our Token Cost Calculator automates across every model in this catalog at once.

Common Misreadings of a Pricing Page

1. 1. Treating the headline price as the only price. Long-context surcharges, cached-token rates, and batch rates are often listed separately, lower on the page (or on a different page entirely).
2. 2. Forgetting reasoning tokens exist. If you're pricing a reasoning model from a visible-answer word count, you will underestimate the bill.
3. 3. Assuming price scales with context window size. A 1M-token context window doesn't mean you pay for 1M tokens on every call — only for what you send.
4. 4. Comparing sticker prices without weighting by your input:output ratio. A model with cheap input but expensive output can lose to a model with the opposite profile, depending on whether your workload is input-heavy (long documents, short answers) or output-heavy (short prompts, long generations).

FAQ

Why is output pricing always higher than input pricing?

Because of how inference actually executes: input tokens are processed together in one parallel pass, while output tokens are generated sequentially, one forward pass per token, which is far less efficient per token on the hardware. Every major provider prices output tokens at a multiple of the input rate as a result.

Do I pay for tokens in the context window I don't use?

No. You're billed for tokens actually sent and received on each call, not the maximum context window size the model supports.

Is prompt caching worth setting up for a low-traffic app?

Usually not by itself — caching helps most when the same prefix is reread frequently within the cache's active window (often minutes). Low, sporadic traffic may not hit the cache often enough to see meaningful savings, so it's worth modeling your actual call frequency before investing engineering time in it.

How accurate is the "1 token ≈ 4 characters" rule?

It's a reasonable starting point for English prose and a poor one for code, non-English text, or structured data formats. For anything beyond a rough order-of-magnitude estimate, tokenize a representative sample of your real content instead of relying on the rule of thumb.

Once you understand how the bill is built, the next question is which model to build on — read how to pick the cheapest model that's still good enough.