Last updated: 2026-06-22

Function calling token overhead

Function calling token overhead is the input-side cost of making tools available to a large language model. To let a model decide whether and how to call a tool, the provider serializes each tool’s definition — its name, a natural-language description, and a JSON schema for its parameters — into the prompt the model reads. Those definitions are made of tokens and are sent on every request, whether or not a tool ends up being used. This calculator isolates that fixed per-call overhead and the recurring cost it produces across a call volume.

Why tools are billed on every call

A model is stateless between requests: it remembers nothing from one call to the next, so the tools it may use have to be described afresh each time. There is no way to register a tool once and refer to it cheaply later — the full definition travels in the input of every call. A request where the model answers directly, invoking nothing, still pays for the definitions, because they were supplied so the option existed. The cost therefore scales with the number of tools exposed and the verbosity of their schemas, multiplied by how often you call the model.

The formula

With $m$ tools at $t_{tool}$ tokens each and $t_w$ wrapper tokens framing the tool block, the per-call overhead is

Across $n$ calls the total overhead and its cost at an input price $p$ per million tokens are

The division by $10^6$ converts the per-million quote into a per-token rate. Tool definitions are read by the model, so they are billed at the input rate, typically the lower of the two published prices.

Worked example

Consider an agent exposing 14 tools whose definitions average 120 tokens each, with 18 wrapper tokens framing the block. The per-call overhead is

Run across 25,000 calls at an input price of $3 per million tokens, the overhead cost is

That $127.35 is spent before any user message or model reply is counted — it pays only to describe the tools on each request. Routing the agent to expose only the handful of tools relevant to each task, rather than all 14, would cut this figure several-fold.

Notes and variations

The per-tool figure depends heavily on how richly each tool is described: detailed parameter descriptions and many optional fields inflate the schema, while a terse definition stays compact. The overhead measured here is purely the tool block; the conversation tokens and any tool-call arguments the model emits are separate. Prompt caching can reduce the repeated cost when the tool block is stable across calls, since a cached read is billed at a reduced rate.

Application

Function calling overhead lands on the input side of the bill. Its output-side counterpart — the JSON scaffolding the model writes back when constrained to a schema — is handled by the Structured Output Overhead Calculator. To price the underlying per-call token usage from which this overhead is drawn, see the Token Cost Calculator.