Every token you send to a large language model costs money and adds latency. That sounds harmless until you do the math on a production workload. A coding agent running 200 calls per session on a frontier model can burn through twenty dollars in a single conversation. Multiply that by a team, a month, a year, and the bill becomes significant. A couple of months ago, tokenmaxxing was the hype of the day. Now Token minimizing is the discipline of cutting that waste without touching the quality of what your model produces.
Why token minimizing matters more than model selection
LLM pricing follows a simple formula: input tokens times input price, plus output tokens times output price. The catch is that agents and chat applications re-send the full conversation history on every turn. A 2,000-token system prompt sent across 200 calls in one session equals 400,000 input tokens just for the repeated instructions. Wasted tokens compound on every subsequent call.
There is also a latency side to this story. Inference happens in two phases. Prefill processes input tokens in parallel and is relatively fast. Decoding generates output tokens one at a time and is memory-bandwidth-bound. That means output length usually dominates how slow your app feels. Cutting twenty output tokens often improves perceived speed more than cutting two hundred input tokens.
The financial impact scales fast. A customer support workload handling a million conversations a month on a flagship model can cost roughly $3,250. The same token volume on a budget-tier model lands closer to $195. Same input, same output count, sixteen times the difference. Token minimizing sits on top of that choice and pushes both numbers lower.
Where tokens actually leak
Most production systems waste tokens in a small set of predictable places. Once you can name them, you can fix them.
- Verbose system prompts. Polite framing, redundant examples and personality instructions that the model already follows by default add overhead on every single call.
- Unbounded conversation history. A 20-turn chat can carry 5,000 to 10,000 tokens of context when only the last 500 to 1,000 are actually relevant.
- Oversized RAG context. Retrieval pipelines often pull more documents than needed, filling the window with low-signal text.
- Unlimited output. Without a max_tokens cap or an explicit length instruction, models tend to write longer than necessary. Output tokens typically cost four to six times more than input tokens.
- Bloated function descriptions and few-shot examples. More demonstrations rarely translate to better results past a certain point.
The good news is that almost all of this is addressable without rebuilding your architecture.
Foundation techniques for token minimizing
Tighten the prompt
Lead with keywords. Ask for extraction instead of full prose. Request structured output. “Summarize the main points:” usually performs as well as a multi-sentence preamble. A distilled system prompt that reads “Act: ResearchBot. Task: Find X. Output: JSON. No fluff.” replaces a 42-token verbose version with roughly 12 tokens. Across a 100-step loop that single change saves about 3,000 tokens.
Constrain the output
Set max_tokens on every API call and tell the model what length you want inside the prompt itself. “Answer in 50 words” combined with max_tokens=100 gives the model both a soft target and a hard ceiling. Since output tokens drive both cost and latency disproportionately, this is one of the highest-leverage changes you can make.
Semantic chunking for document-heavy work
Splitting documents on meaning rather than fixed character counts keeps concepts intact. You end up retrieving fewer chunks for the same answer quality because no idea is sliced across a boundary.
Context compaction beats summarization
When agent conversations grow past 100K tokens, something has to give. The traditional answer is summarization: ask a model to rewrite the history in fewer words. The problem is that summaries lose specifics. File paths become “a configuration file.” Error codes become “an error.” Function signatures vanish into “a function in the auth module.” The agent then spends tokens re-asking for what the summary discarded.
Compaction takes a different route. Instead of rewriting, it deletes low-signal tokens verbatim. Redundant formatting, repeated boilerplate and verbose metadata get removed, while every surviving sentence remains character-for-character identical to the original. File paths, numbers, code snippets and error messages stay intact. A 200K-token conversation can compact down to 80K with zero hallucination risk, because nothing was paraphrased.
Run compaction before every call, not only when you hit the context window. By the time auto-compact fires at the capacity cliff, you have already paid full price for a hundred turns of bloated context.
Caching repeated content
Prompt caching stores the processed representation of stable content so the model does not recompute it on every call. The savings are immediate and require no quality trade-offs.
Anthropic charges 90% less on cache reads. A 2,000-token system prompt sent 200 times costs roughly $2 without caching on a frontier model. With caching, the same volume drops to about $0.21. OpenAI applies a 50% discount on cached prefixes automatically once the prefix exceeds 1,024 tokens, with no code changes required.
On top of provider-level caching, semantic caching at the application layer eliminates the LLM call entirely for queries that mean the same thing. “What is the weather today?” and “How is the weather right now?” can hit the same cached answer based on a similarity threshold. Workloads with natural repetition, like customer support and FAQ-style interactions, see the largest gains because each cache hit replaces a full inference round-trip with a sub-millisecond vector lookup. In high-repetition scenarios, semantic caching has driven cost reductions of around 73%.
Smart model routing
Not every request needs your most capable model. Adding a comment, formatting JSON, renaming a variable or generating boilerplate runs just as well on a small model as on a frontier one. A router classifies each prompt by difficulty and sends easy work to a cheap tier, medium work to a mid-tier, and hard reasoning to the flagship.
Because 60 to 80% of agent requests are routine, the weighted average cost per request drops by 40 to 70% without any visible change in output quality. Classification adds about a millisecond of overhead and a fraction of a cent per call. For multi-agent setups, the same principle applies at the role level: give the planner a strong model and let the executor run on something cheaper.
Batching for non-urgent work
Both Anthropic and OpenAI offer batch APIs at a flat 50% discount with a 24-hour SLA. Anything that does not need a real-time response qualifies: evaluation pipelines, data labeling, nightly reports, content backfills, bulk migrations. For most teams, 20 to 40% of total LLM spend falls into this category and can be moved to batch with minimal effort. Batch also stacks with caching, pushing combined savings on repeated content to around 95%.
Stacking the levers
Each technique works on its own, but the real wins come from combining them. Picture a coding agent running 200 calls per session, averaging 20K input tokens per call and 500 output tokens, all on a frontier model. Baseline cost lands near $22.50 per session.
- Add model routing and the weighted price drops the session to roughly $7 to $9.
- Layer in context compaction and the input shrinks by 60%, pulling the session toward $3.50 to $5.
- Cache the system prompt and stable reference docs, saving another dollar or so.
- Trim the prompt and request structured output for an additional 10 to 20%.
Conservative estimates land at 70 to 85% total reduction. Aggressive setups with high cache hit rates and tight routing reach 90%+. The same $22.50 session can finish under $2.
The discipline behind the numbers
What makes token minimizing work is not any single technique but the habit of treating tokens as a budget rather than an afterthought. Measure your system prompt length. Track cache hit rates by query type and user segment. Watch which conversations grow fastest and ask whether the model actually needs that history. Most teams discover their biggest leaks only after they start instrumenting.
Aggressive compression can backfire if it strips context the model needs to stay accurate. Token savings that force the agent to re-ask for information cost more than they save. The goal is never the smallest prompt, it is the smallest prompt that still gets the job done on the first try.
