Last updated: 2026-06-22

Why multi-turn cost grows

Language model API calls are stateless: the model retains no memory between calls, so the client must re-send the prior conversation as context on every request for the model to stay coherent. That re-sent history is billed as input tokens each time. As a conversation lengthens, the input on each turn carries the accumulated history of all earlier turns, so the per-turn input grows and the cumulative input grows faster than the turn count.

The formula

Let $N$ be the number of turns, $n_{sys}$ the system prompt re-sent each turn, $n_{turn}$ the new history added per turn (the user message plus the prior reply), and $n_{out}$ the output per turn. On turn $k$ the input is $n_{sys} + (k-1)\,n_{turn}$. Summing over all turns:

The triangular term $\tfrac{N(N-1)}{2}$ is quadratic in $N$, which is why long conversations cost far more than a single turn would suggest. With per-million prices $p_{in}$ and $p_{out}$:

Worked example

Take N = 10 turns, an 800-token system prompt, 600 new tokens per turn, 400 output tokens per turn, at $3 per million input and $15 per million output:

The conversation costs about 16.5 cents. The history term (27,000 tokens) dwarfs the 8000 tokens of repeated system prompt — most of the input bill comes from re-sending the growing transcript, not from the fixed instructions.

The quadratic effect

Because $T_{in}$ scales with $N^2$, doubling the number of turns roughly quadruples the input cost rather than doubling it. A 20-turn version of the example above would re-send $600 \cdot \tfrac{20 \cdot 19}{2} = 114{,}000$ history tokens — over four times the 27,000 at ten turns. This compounding is a property of re-sending full history, independent of the per-token price.

Mitigation

The growth is reduced by paying for the accumulated history fewer times or at a lower rate: summarizing or truncating old turns caps how much history is re-sent, while prompt caching lets the provider bill repeated context at a reduced rate. The savings from caching the static portion of the context can be estimated with Prompt Caching Savings Calculator. To confirm that a growing transcript still fits the model before cost even becomes the concern, see Context Window Fit Calculator.