LLM FinOps: Per‑Feature Cost Attribution and Token Budgets
Add a feature_id tag to every LLM call, log provider response tokens, and route to Haiku first with fallback to Sonnet.
Wrap all LLM calls with a middleware that injects feature_id, tenant_id, and model_used metadata, logs usage, and implements Haiku‑first routing.
Summary
A B2B SaaS product that shipped 12 AI features by 2026 faces a $48,000 monthly Anthropic bill, but without per‑feature attribution it cannot determine which feature drives the cost. The article outlines four levers: tagging every LLM call with a feature_id, counting tokens from provider responses instead of estimates, aggregating per feature, and enforcing per‑feature budgets. Adding a feature_id tag to every call is a single line of code at the call site, and both Anthropic and OpenAI accept metadata that flows to their consoles and logs. Counting tokens from the provider response (response.usage.input_tokens, response.usage.output_tokens) eliminates the 5–15% drift of estimates. Model routing is the biggest cost lever: routing to Haiku first and falling back to Sonnet only on low‑confidence responses can reduce a 10:1 success ratio from $7,500 to $2,375 per 1M requests. The article provides a detailed usage log schema and demonstrates how to detect accidental model upgrades and track prompt‑cache hit rates per feature.
Key changes
- Add a feature_id tag to every LLM call to enable per‑feature monthly roll‑up
- Log provider response tokens (input_tokens, output_tokens) instead of estimates to avoid 5–15% drift
- Implement Haiku‑first routing with Sonnet fallback to reduce cost by ~70% for 10:1 success ratio
- Track prompt‑cache hit rate per feature via cached_input_tokens
- Detect accidental model upgrades by logging model_used metadata
- Enable query attribution to track per‑tenant cost via tenant_id
- Wrap all LLM calls with middleware that injects metadata and logs usage
- Use the detailed usage log schema to support future dashboards and alerts