How does NumStack calculate AI token costs?

NumStack uses real published pricing from OpenAI, Anthropic, Google, and other AI providers. Enter your typical usage (input tokens, output tokens, requests per day) to see projected monthly costs across all models.

Why do different AI models have different token prices?

Different models have different computational costs. Smaller models like GPT-4o mini or Haiku are cheaper but less capable. Larger models like GPT-4o or Claude Sonnet are more powerful but cost more per token.

How can I reduce my AI API costs?

Use smaller models for simple tasks, cache repeated prompts, trim output length (output tokens cost 3-5x more than input), and batch requests instead of real-time when latency allows.

Cutting Token Spend Without Hurting Quality: A Prompt Optimization Playbook

Before you switch models or rebuild an architecture, spend an afternoon on the prompt itself. Most production prompts accumulate bloat over time — instructions that stopped mattering, few-shot examples nobody re-evaluated, no cap on how long the model is allowed to ramble — and trimming that bloat is usually the fastest, lowest-risk cost reduction available. Here's the playbook, with the techniques ranked roughly by effort-to-savings ratio.

Where the Waste Actually Lives

System prompts that grew by accretion — new instructions added, old ones never removed
Few-shot examples kept "just in case," long after the model stopped needing them
No output length cap, letting the model restate the question, hedge, or over-explain
Full conversation history resent verbatim instead of summarized
RAG context stuffed with more retrieved chunks than the answer actually needs
Free-form prose output where a structured, constrained format would be shorter and easier to parse

1. Trim the System Prompt First

This is almost always the highest-leverage, lowest-risk edit, because a bloated system prompt costs you on every single call, not just some of them. Read it fresh and ask, for each instruction: has this changed model behavior in a way I've actually verified, or did it just seem safer to leave in? Consolidate redundant rules into one clear statement. Cut few-shot examples down to the minimum that still holds accuracy on your eval set — often 2-3 well-chosen examples outperform 10 redundant ones, and cost a fraction as much per call.

2. Cap Output Length Explicitly

Output tokens cost 3-5x the input rate on most providers (see LLM pricing explained for why), which makes uncapped output the single most expensive kind of waste per token. Set an explicit token limit sized to what a good answer actually needs, use stop sequences to end generation at a natural boundary, and instruct the model directly: "respond in 2-3 sentences" or "answer with only the JSON object, no explanation" measurably reduces rambling on most models. Test the cap against your eval set — too aggressive a limit truncates legitimate answers and causes retries, which costs more than the tokens it saved.

3. Compress History Instead of Resending It Verbatim

In any multi-turn context, full conversation history resent on every call is a compounding cost — see the turn-by-turn math in our chatbot cost estimation guide for how fast this grows. A running summary of the conversation so far, updated every few turns and substituted for the verbatim transcript, typically holds most of the useful context at a small fraction of the token count. This costs an extra summarization call, so it pays off once conversations run long enough that the summarization overhead is smaller than the history it replaces — usually somewhere past 4-6 turns, though the exact point depends on your average message length.

4. Use Structured Output

Asking for a JSON object against a defined schema, instead of free-form prose, tends to produce shorter, more consistent output — the model isn't spending tokens on transitions, hedging, or restating the question, and you get a smaller, more reliably parseable response as a side effect. This also reduces a hidden cost: prose output that has to be re-prompted or re-parsed on a formatting miss is a retry, and retries are pure waste.

5. Right-Size RAG Retrieval

More retrieved chunks is not automatically a better answer — past a certain point it's just more tokens for the model to read, and can actively hurt answer quality by burying the relevant passage in noise. Rerank retrieved chunks and keep only the top few that genuinely inform the answer, rather than stuffing in everything above a similarity threshold. See our vector database pricing guide for how chunk size and count also affect retrieval infrastructure cost, independent of the LLM token savings here.

6. Cache the Stable Prefix

If your system prompt and tool definitions are identical across calls, prompt caching lets you pay a reduced rate to reread them instead of the full input price every time. This is a different lever from trimming — it doesn't reduce what you send, it reduces what you pay for the part that repeats. The two combine: a trimmed prompt that's also cached is cheaper than either change alone.

7. Batch What Doesn't Need to Be Synchronous

Anything that doesn't need a response in real time — nightly summarization, bulk classification, offline scoring — is a candidate for a batch API, typically priced at roughly half the standard rate. This is orthogonal to every technique above and stacks with all of them.

Measure Output Tokens, Not Just Input

Teams auditing token spend almost always start with the prompt — it's the part they wrote, so it's the part they think about. But since output is priced at a multiple of input, a verbose model response is frequently the larger cost driver, and it's the one that gets the least scrutiny because nobody wrote it by hand. Pull actual output token counts from your API response metadata across a sample of real traffic before you start optimizing — you may find the system prompt isn't where most of the money is going at all. It's common to discover that a handful of request types account for a disproportionate share of output tokens, usually the ones where the prompt never specified a length or format constraint, which makes them the highest-leverage place to start applying the output-capping technique above.

Worked Example: Stacking Two Changes

Trimming the system prompt from 900 to 350 tokens and capping average output from 400 to 150 tokens, at 200,000 requests/month:

Claude Haiku 4.5: $592.00/month before, $232.00/month after — 61% reduction
GPT-4o mini: $76.80/month before, $30.30/month after — 61% reduction

Notice the percentage reduction lands in roughly the same range on both models even though the absolute dollar figures are very different — trimming and capping are ratio-based savings, so they pay off proportionally no matter which model tier you're on. That also means this is worth doing even after you've already switched to the cheapest viable model; the two levers are independent and stack. Try your own before/after prompt in the Prompt Optimizer to see the exact savings on your text.

Common Mistakes

1. Cutting without re-running your eval. Every trim is a hypothesis, not a guaranteed win — verify accuracy didn't drop before shipping it, especially on few-shot example cuts.
2. Setting max_tokens too aggressively. A cap that truncates legitimate answers causes retries or unusable output, which costs more than the tokens the cap saved in the first place.
3. Over-compressing conversation history. A summary that drops a detail the user referenced three turns ago produces a visibly worse experience — test summarization quality with the same rigor as any other prompt change.
4. Optimizing low-volume endpoints first. The engineering time to trim, test, and validate a prompt change is roughly constant regardless of call volume — spend it on your highest-volume endpoints first, where the same effort returns far more savings.
5. Treating this as a one-time pass. Prompts drift back toward bloat as new edge cases get patched in with new instructions. A periodic re-trim, not just a launch- day one, keeps the savings from eroding.

FAQ

Which technique should I try first?

Trimming the system prompt and capping output length, in that order — both are low-risk, apply to every call, and don't require new infrastructure. Caching and history compression are worth adding once the basics are done and you're looking for the next increment.

How do I know if a trim hurt quality?

Run the same eval set you'd use for a model comparison (see how to pick the cheapest model that's still good enough for the methodology) before and after the change. If you don't have one yet, build a small one before making aggressive prompt edits — it's the only reliable way to distinguish a safe trim from a regression.

Is prompt optimization still worth it if I already use a cheap model?

Yes — the savings are proportional, as shown in the worked example above, so they apply regardless of which model tier you're on. A cheap model with a bloated prompt is still paying for waste, just at a lower absolute price.

Can I automate prompt trimming?

Partially — tools can flag redundant instructions or suggest shorter phrasing, but validating that a trim didn't change model behavior still requires running your eval set. Treat automated suggestions as a starting draft, not a change you ship without testing.

How often should I revisit an already-optimized prompt?

Whenever the underlying task changes, and as a light habit alongside any provider model upgrade — a new model version can respond differently to the same instructions, sometimes needing fewer examples to hit the same accuracy, sometimes more. A quarterly review of your highest-volume prompts is a reasonable default cadence even without a specific trigger, since small amounts of bloat tend to creep back in as edge cases get patched with new instructions nobody later removes.

Once your prompts are lean, the next lever is model choice — see how to pick the cheapest model that's still good enough.