Short answer: cost per task equals input tokens times the input rate, plus output tokens times the output rate, plus retries and cache reads. Output tokens usually cost several times more than input tokens, so response length drives the bill.
The basic formula
cost = (input_tokens / 1M × input_rate) + (output_tokens / 1M × output_rate)
Rates are quoted per million tokens and differ by model. Use the current price list from your provider; the arithmetic matters more than the exact numbers here.
A worked example
Suppose a request sends 4,000 input tokens and receives 800 output tokens. With example rates of $0.30 per million input and $1.20 per million output:
- Input: 4,000 / 1,000,000 × $0.30 = $0.0012
- Output: 800 / 1,000,000 × $1.20 = $0.00096
- Total: about $0.0022 per request
At 10,000 requests per day, that is roughly $22 per day. Small per-request costs become real at volume, which is why caching and output limits matter.
Where the money actually goes
| Driver | Effect | Control |
|---|---|---|
| Long system prompts | Repeated input cost on every call | Prompt caching, shorter instructions |
| Verbose answers | Output tokens at the higher rate | Max tokens, format instructions |
| Retries | Full request paid again | Backoff, validation, fallback model |
| Context growth | Each turn resends history | Summarise, trim, window history |
Cached tokens
Prompt caching stores a stable prefix, such as a long system prompt or reference document, so repeat calls read it at a lower rate. Cache reads are cheaper than fresh input, and cache writes may carry a small premium. The saving depends on reuse: a cache only pays off when the same prefix is sent many times within its lifetime. Our prompt caching guide covers the implementation details.
Compare cost per success
Add retries and human review to the per-request price. A model that costs half as much but fails one in five tasks may cost more per finished task. Track successful completions, not API calls, and review the ratio weekly.
Practical budget rules
- Set a max output token limit for every endpoint.
- Log input, output and cached tokens per request.
- Alert on cost per successful task, not total spend alone.
- Test a smaller model on classification and extraction first.
Model choice affects both quality and unit cost; see DeepSeek vs Qwen vs Kimi. Top-ups are listed in the TOKEN category.
Frequently asked questions
Are input and output tokens priced the same?
No. Output tokens usually cost more, so answer length has a larger effect on the bill than a long prompt.
Do cached tokens reduce cost?
Cache reads are billed at a lower rate. The saving appears when the same prefix is reused often enough to offset any cache write cost.
How do I estimate monthly spend?
Multiply cost per request by daily volume and days, then add retries and peak usage. Measure real token counts instead of estimating from character counts.
No comments yet. Be the first to share your thoughts!