Last updated: 2026-06-22

Batch API savings explained

Batch API savings is the cost reduction from submitting a language-model workload through a provider's asynchronous batch endpoint instead of the real-time endpoint. A batch API accepts a collection of requests and processes them within a stated completion window rather than immediately, and because the provider can schedule the work flexibly across its capacity, it bills the throughput at a flat discount. The savings is the difference between the real-time cost and the discounted batch cost.

How it works

The real-time cost combines input and output tokens at their respective per-million rates. With $n_{\text{in}}$ and $n_{\text{out}}$ tokens priced at $p_{\text{in}}$ and $p_{\text{out}}$ per million:

The batch discount $d$ is a flat proportion removed from that whole cost, covering both input and output:

Because the discount is flat, the savings scales directly with the size of the workload and with the discount rate.

Worked example

Take a workload of 1,000,000 input tokens and 200,000 output tokens, priced at $3 per million input and $15 per million output, with a 50% batch discount. The real-time cost is:

The batch cost is half of that:

so the savings is \6 - $3 = $3, a 50% reduction. The input and output contributions happen to be equal here — \3 each — because the larger output price offsets the smaller output volume.

Variations

The discount applies uniformly, so the dollar savings grows in proportion to the bill: a workload ten times larger saves ten times as much at the same discount. Because output is billed higher than input, completion-heavy batches carry a larger absolute saving for the same token counts. The key constraint is latency. Batch processing is asynchronous — results arrive within the provider's completion window, not synchronously — so it suits work that does not need an immediate response: bulk classification, summarization, evaluation runs, or offline content generation. Interactive, user-facing requests are not a fit.

Application

Batch savings is an offline-throughput lever, applied after a workload's token volume is known. The real-time and batch costs both feed the broader estimates in Token Cost Calculator and LLM Monthly Cost Calculator. Batch and caching are independent discounts that can stack on the same job; for the input discount from repeated prefixes see Prompt Caching Savings Calculator, and to read a per-million rate as tokens per unit of spend see Tokens per Dollar Calculator.