GPU Utilization Cost Calculator
Inputs
| Hourly cost | 2.5 $ |
|---|---|
| Utilization | 60 % |
| Hours per month | 730 |
GPU Utilization Cost Calculator
See the real cost of a reserved accelerator once idle time is counted. Enter the hourly price and utilization to get the effective cost per useful hour and the money spent on idle capacity.
Inputs
Accelerator
Results
Enter a value to see results.
Details
GPU Utilization Cost
A reserved or owned accelerator costs the same whether it is saturated with work or sitting idle: you pay for every hour you hold it. That makes utilization — the share of paid time actually spent serving work — the hidden multiplier on your real compute cost. This calculator turns an hourly price and a utilization figure into the effective cost of a useful hour, the full monthly bill, and the money lost to idle capacity.
Why idle time is expensive
On-demand, reserved, and owned hardware are all billed by the hour held, not by the work performed. When a node is idle, those hours still accrue charges, and the only way to account for them honestly is to spread them over the hours that did produce work. A node that is busy only half the time costs twice as much per productive hour as its sticker rate suggests — the idle half is paid for by the busy half.
The formulas
With an hourly cost , a utilization fraction , and hours held per month:
ceffCW=uc=c⋅H=C⋅(1−u)where is the effective cost per useful hour, is the total monthly cost, and is the idle cost. The effective cost is the headline number: it climbs steeply as utilization drops, doubling when utilization halves.
Worked example
Take an accelerator at $2.50 per hour, held around the clock (about 730 hours a month) but busy only 60% of the time:
ceffCW=0.62.50≈$4.17 per useful hour=2.50×730=$1,825=1,825×(1−0.6)=$730Although the rate is $2.50, each hour of real work effectively costs $4.17, and $730 of the $1,825 monthly bill — 40% — pays for idle time. Pushing utilization from 60% to 90% would cut the effective hourly cost to about $2.78 and shrink the idle waste to roughly $180.
Recovering the waste
Utilization rises with batching, workload consolidation, autoscaling during quiet hours, and spot capacity for interruptible jobs. For bursty or low-volume traffic, per-token or serverless pricing can be cheaper than reserved hardware even at a higher headline rate, because it shifts the idle risk to the provider. Compare the effective cost per useful hour here against those alternatives — the Self-Host vs API Break-even Calculator works through that trade-off, and the Effective Tokens per Second Calculator turns utilization into a sustained throughput figure.
Frequently Asked Questions (FAQ)
What does utilization mean here?
Utilization is the fraction of the time you pay for that the accelerator is actually doing useful work. It can be measured as the share of wall-clock time the device is busy, or more strictly as the share of its compute that serves real requests.
Either way, the gap between what you reserve and what you use is the source of waste. A node held continuously but busy only 60% of the time is at 60% utilization, and the other 40% of the bill buys nothing.
Why does idle time make useful work more expensive?
Reserved and owned accelerators are billed for every hour they are held, not for the work they perform. When a node sits idle, that time still accrues charges, and those charges have to be absorbed by the hours that do produce work.
Dividing the hourly price by utilization spreads the idle cost over the productive hours, which is why the effective cost of a useful hour climbs as utilization falls — halving utilization doubles the real cost of every working hour.
How can utilization be improved?
Common levers include batching requests so the accelerator stays busy, consolidating workloads onto fewer nodes, autoscaling down during quiet periods, and using spot or preemptible capacity for interruptible jobs.
Serverless or per-token pricing shifts the idle risk to the provider entirely, which can be cheaper for bursty or low-volume workloads even at a higher headline rate. The right choice depends on how steady the traffic is.
Disclaimer
This is a straightforward accounting of held cost against utilization and does not include data transfer, storage, networking, or staffing. Compare the effective cost per useful hour against per-token or serverless pricing before committing to reserved capacity.