Model budget controls in customer-facing AI
Every LLM feature has an unbounded cost tail. The controls that prevent a support agent from spending your quarter's budget in an afternoon.
Any LLM feature exposed to customers or agents has an unbounded cost tail. A misbehaving loop, a prompt injection, a single high-volume user — any of these can spend a month's model budget in an hour. The controls that prevent that are boring, and they're the difference between a feature that ships and one that gets pulled.
Written June 2026 from a review of production LLM integrations.
The layered controls
- Per-request token cap. Every request has a maximum output token count. This is the last line of defense against runaway generation.
- Per-user daily cap. A single user cannot generate more than N tokens per day, regardless of how many requests they make.
- Per-feature monthly cap. A whole feature has a monthly budget. When it's exhausted, the feature disables and returns a clear message.
- Global monthly cap. All LLM usage across the platform has a ceiling. Approaching it triggers alerts to on-call.
- Per-request cost estimation. Before dispatching a large context, estimate the cost. Reject requests over a threshold.
The alerts that matter
- Sudden per-user cost spike. Might be legitimate, might be an attack. Investigate.
- Approaching monthly budget. At 70%, warn. At 90%, throttle. At 100%, disable.
- Requests that hit the token cap. These are usually a signal of a misbehaving prompt or a prompt injection.
The one control everyone forgets
Model version pinning per feature. Model providers deprecate versions on their own schedule. If your integration silently falls back to a "compatible" model, the cost model can shift under you. Pin explicitly and monitor for the deprecation notices.
Cost control is not a feature. It's a hygiene requirement without which the feature doesn't ship.