Acme Support's LLM Cost Audit (sample)

Executive summary

Acme Support runs its support chatbot and ticket-classification pipeline entirely on GPT-4o, at 2,500 requests/day (75,000/month). Modeled from that volume and your typical request shape (2,200 input /undefinedoutput tokens), current spend is $638/month — 165.0M input tokens and 22.5M output tokens.

Nothing about this setup has changed since launchundefinedmonths ago, and it shows: no prompt caching, no batch processing, and no cheaper model for the lower-stakes half of the workload (ticket classification doesn't need the same model as a nuanced support reply). Three findings stand out:

1. Model choice is the single biggest lever — if your evals hold up. GPT-4.1 — the cheapest model in GPT-4o's own price band — runs the same workload for $106/month instead of $638. Price bands are not quality rankings; validate any swap with your own evals before switching.
2. The system prompt is doing more work than it needs to. The sample prompt you'd have submitted for this workload trims fromundefinedtoundefinedtokens (17%) with redundancy removal alone, before touching wording.
3. You're above the volume where fine-tuning is worth a real look. The break-even math favors fine-tuning — see "Architecture recommendations."

The projected-savings table below is a sequential plan, not independently-stacked totals — see "Assumptions and limits" for exactly how each row was built.

Your current cost model

Token volume here is modeled from requests/day × typical prompt/response size (Acme didn't have an exact token count on hand, which is common — the intake form accepts either). With a single model in production, the "blended price" above is just GPT-4o's own list price; teams running more than one model would see this section split cost across each one, weighted by an even-split assumption stated plainly in "Assumptions and limits."

At $638/month, Acme is a straightforward case: one model, two use cases, no optimization applied yet. That's exactly the profile where the biggest win is usually a single model swap, not a redesign.

Model options by use case

Both of Acme's use cases run on the same model today, but they don't need to. Below is the same price-band candidate set — priced by cost per token, not by quality — at Acme's actual volume for each use case. Price bands are not quality rankings; validate any swap with your own evals before switching.

Chatbot / customer support

Customer-facing replies are where quality risk is highest — a wrong or off-tone answer costs more in support escalations than the model swap saves. GPT-4.1 (Acme's own price band) is the recommended starting point precisely because it doesn't ask you to bet quality on a cheaper band; validate it against a sample of real transcripts before moving production traffic. The lower-priced band's 97% savings is real, but it's an "if your evals allow" option, not the default — run it head-to-head against production quality before moving support traffic there.

Classification / extraction

Ticket classification and routing is a narrow, structured task — exactly the kind of workload where the lower-priced band usually holds up fine, but "usually" isn't "verified." Run LlamaundefinedScout (or an equivalent model in that price band) head-to-head against GPT-4o on a labeled set of past tickets; if accuracy holds within a point or two, this is the lower-risk half of the savings to capture first.

Prompt and caching savings

Prompt trimming. Applied to your current monthly input volume:

The sample prompt Acme submitted is a good example of where this comes from — mostly filler and redundant instructions, not information the model needs:

"You are a customer support assistant for Acme Support, a SaaS platform for scheduling. Please always be extremely polite and friendly to the customer, and make sure to fully and completely answer their question in as much detail as possible so they feel well taken care of. It is important to note that you should never make up information about billing. Please try to resolve the issue in a single response if at all possible. The customer's account plan is included below along with their recent ticket history. Please review the ticket history carefully before responding, and please make sure to reference specific details from their account to personalize the response."

Running it through redundancy removal alone cuts it fromundefinedtoundefinedtokens (17%) — phrases like "please make sure to," "it is important to note that," and restating the same instruction twice add tokens without changing what the model does. Projected across your full monthly volume, that alone is worth about $68/month.

Prompt caching. Assumes 60% of input tokens are repeated context (system prompt, few-shot examples, retrieved documents) eligible for prompt caching, discounted ~90% on cache reads — Anthropic's published prompt-caching rate, used here as the reference regardless of provider. Both of Acme's use cases qualify — the system prompt and account/ticket-history preamble repeat on nearly every call. At your volume, that's $223/month.

These two levers compound with each other (a shorter prompt is also a smaller cache write), but not with a model swap in the simple additive way the projected-savings table presents them — see "Assumptions and limits."

Architecture recommendations

• Split the model by use case, not just for cost — a validated, same-band model for support replies and a lower-priced-band model for classification is also a smaller blast radius if a new model regresses on one task but not the other.
• Turn on prompt caching before anything else. It requires no model change, no quality risk, and captures $223/month on its own.
• Batching doesn't apply here — Acme's workload is tagged interactive/real-time, and support chat genuinely needs a synchronous response.
• Fine-tuning is worth a real evaluation. At 75,000 requests/month, fine-tuning is worth checking. Modeled with an illustrative 2,000-example training set, since the intake form doesn't collect your real dataset size — swap that in for a precise number. Note this uses its own illustrative training-set assumption (see "Assumptions and limits") and a different internal cost model than the "current cost model" above — treat it as a signal to scope a real fine-tuning pilot with your actual dataset, not a number to bank on directly.
• Instrument before you optimize further. Acme doesn't currently track token counts directly — add basic request/response token logging so the next audit can use reported volume instead of an estimate.

30-day action plan

Projected savings

This is a sequential plan on the recommended path, not a menu of independently-stackable savings: each row is the incremental saving on top of the row above it, starting from the current bill and ending at the projected cost — so the rows can never sum to more than Acme is actually spending today.

Current monthly cost: $638 Total projected savings: $336 Projected monthly cost: $301

Capturing prompt caching (week 1) plus a validated model switch (weeks 2–3) is the achievable near-term combination, and either lever alone already clears the $149 cost of this audit many times over in its first month.

Assumptions and limits

• Token volume is modeled, not measured. Acme reported requests/day and typical prompt/response size rather than an exact token count, so input/output token totals above are 2,200 × requests andundefined× requests respectively — real traffic varies request to request.
• The projected-savings table is a sequential plan, not independent totals. Each row is measured against the cost remaining after the row above it (model swap, then trimming, then caching, then batching where applicable) — never against the original bill a second time — so the rows never overstate what's achievable.
• Price bands are not quality rankings. Price bands are not quality rankings; validate any swap with your own evals before switching.
• Prompt-caching savings estimates in this report assume 60% of input tokens are repeated context eligible for a cache hit.
• Caching assumption: Assumes 60% of input tokens are repeated context (system prompt, few-shot examples, retrieved documents) eligible for prompt caching, discounted ~90% on cache reads — Anthropic's published prompt-caching rate, used here as the reference regardless of provider.
• Batching assumption: Assumes batch/async API pricing at 50% off synchronous rates (Anthropic's and OpenAI's batch APIs both price at 50% of standard), applied to the full workload since it's already tagged as batch-tolerant.
• Fine-tune check assumption: At 75,000 requests/month, fine-tuning is worth checking. Modeled with an illustrative 2,000-example training set, since the intake form doesn't collect your real dataset size — swap that in for a precise number. Its modeled monthly costs (fine-tune: $1,238, prompting: $1,069) come from a separate internal cost model built for that comparison specifically — they will not exactly match the "current cost model" section above, which is modeled directly from your reported usage.
• Traffic-split assumption: with more than one model in production, cost is split evenly across the models you selected — a real traffic split will shift each model's share.
• Prices are list prices as published by each provider, not any volume discount, committed-use pricing, or enterprise agreement Acme may already have.

This report is an engineering cost analysis based on the information you provide and public list prices. It is not financial, tax, or legal advice.