# Inference Economics

> Source: https://aiwiki.ai/wiki/inference_economics
> Updated: 2026-11-03
> Categories: AI Compute, AI Infrastructure
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Inference Economics." aiwiki.ai, 3 Nov 2026. https://aiwiki.ai/wiki/inference_economics
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**Inference economics** is the study of the resources, costs and revenue associated with running a trained [AI model](https://aiwiki.ai/wiki/model). It connects the price of [compute](https://aiwiki.ai/wiki/ai_compute) with the work a serving system delivers. A rental rate, an API token price and a cost per completed task describe different parts of that system. Comparing deployment options requires a defined workload and service requirements.[1][2]

## Units and scope

An hourly GPU price describes access to hardware over time. It does not state how many requests the system can complete. Throughput depends on the model, serving software, precision, input and output lengths, concurrency and hardware configuration. A benchmark must also state its latency requirement: a configuration that produces more total tokens may make each user wait longer.[1]

| Quantity | What it measures | What must accompany it |
| --- | --- | --- |
| Dollars per GPU-hour | A hardware access rate | GPU count, instance resources, region and billing terms |
| Output tokens per second | Generation throughput | Aggregate system or per-request basis, tokenizer and workload |
| Dollars per million tokens | A usage price or computed serving cost | Input/output distinction, cache rules and included costs |
| Dollars per task | Spend divided by task attempts | Task definition, tools and retry policy |
| Dollars per accepted task | Spend divided by outputs meeting a criterion | Acceptance rule and all attempted runs |

Published [benchmark](https://aiwiki.ai/wiki/benchmark) results have their own scope. MLPerf Inference, for example, defines workloads, scenarios, model requirements and reproducibility rules. A result under one benchmark's rules cannot establish performance on every application.[2]

## From billed hours to delivered tokens

Let `H` be the total hosting cost per billed hour, `q` the aggregate output-token throughput during active serving, and `u` the productive serving fraction of billed time. For a fixed workload, a simplified hosting cost per million output tokens is:

`H * 1,000,000 / (3,600 * q * u)`

This is an accounting identity for the stated inputs, not a performance prediction. Input processing must already be included in the throughput measurement. If measured throughput uses the entire paid wall-clock interval, including idle time and restarts, a separate utilization penalty would count those losses twice. The calculation also needs a separate treatment for any costs excluded from `H`.[1][3]

For a hypothetical system costing $6 per billed hour and producing 1,000 output tokens per second during active serving, the formula gives about $1.67 per million output tokens at full productive utilization. At 50% productive utilization it gives about $3.33. These are arithmetic examples with assumed inputs, not rental quotes or measured model results.

## API prices and self-hosting

An API bill can include different rates for uncached input, cached input and output. The amount of each category matters as much as its unit price. [Reasoning models](https://aiwiki.ai/wiki/reasoning_models) can consume additional output or reasoning tokens, so two models charging the same price per token can have different costs on the same task. Artificial Analysis calculates its cost per task using token consumption across its evaluation workload; this is a benchmark-specific measure.[4]

A self-hosting comparison needs the cost of the whole deployment. GPU rent alone may omit CPU, memory, storage, network traffic, software and operations. Owned hardware requires a cost-allocation method rather than treating its purchase price as the cost of one run. [GPU depreciation](https://aiwiki.ai/wiki/gpu_depreciation) and cash expenditure are separate measures.[1]

The options should be evaluated against the same acceptance and latency requirements. A cheaper output that needs more retries or human correction can have a higher [AI task cost](https://aiwiki.ai/wiki/ai_task_cost). Conversely, a larger or more expensive model may be unnecessary if a smaller system meets the task's requirements.

## Utilization, interruptions and commitments

Productive utilization is not the same as a GPU's hardware-utilization counter. A reserved server can be billed while no requests arrive. An interruptible server can lose work or incur restart time. Azure's Spot VM documentation explicitly distinguishes its interruptible capacity from a service with availability guarantees.[5]

Commitment terms also affect comparisons. In its November 2, 2026 registration-statement amendment, [Anthropic](https://aiwiki.ai/wiki/anthropic) reported that short-term compute cost materially more than comparable compute procured under multi-year commitments.[6] A lower committed hourly rate should therefore be assessed alongside minimum duration, prepaid credits, cancellation terms and unused capacity. Prepaid credit applied to a future invoice is not an additional charge to add to that same invoice.

## Revenue, contribution and profit

A serving provider's revenue depends on the work it sells and the prices customers actually pay. A contribution calculation subtracts the stated serving costs from that revenue. It does not establish net profit if research, administration, financing or other expenses remain outside the calculation.

The break-even productive utilization for a fixed-capacity scenario is the utilization at which included revenue equals included costs. A scenario that requires utilization above 100% cannot break even under those assumptions. Missing throughput or selling-price evidence should remain unknown rather than being replaced with a favorable estimate.

## Reading price indices

A GPU price index and a token-price index can describe different customer populations and contract types. Ornn's OCPI methodology measures on-demand transactions and excludes bespoke long-term contracts. A comparison with another series must retain those scope differences.[7]

Lower unit prices also do not imply lower total spending: expenditure depends on both price and purchased volume. Platform token counts are another distinct measurement. OpenRouter's rankings count tokens processed through its own API, exclude private requests and do not constitute a census of worldwide AI spending.[8]

## References

1. Vinh Nguyen and Sergio Perez. ["LLM Inference Benchmarking: How Much Does Your LLM Inference Cost?"](https://developer.nvidia.com/blog/llm-inference-benchmarking-how-much-does-your-llm-inference-cost/) NVIDIA Technical Blog, June 18, 2025.
2. MLCommons. ["MLPerf Inference Rules."](https://github.com/mlcommons/inference_policies/blob/master/inference_rules.adoc) Benchmark policies, accessed November 3, 2026.
3. vLLM Project. ["Benchmark Suites."](https://docs.vllm.ai/en/latest/benchmarking/) Documentation, accessed November 3, 2026.
4. Artificial Analysis. ["Artificial Analysis Benchmarking Methodology."](https://artificialanalysis.ai/methodology) Definitions of price, tokens and cost per task, accessed November 3, 2026.
5. Microsoft. ["Azure Spot Virtual Machines."](https://learn.microsoft.com/en-us/azure/virtual-machines/spot-vms) Microsoft Learn, accessed November 3, 2026.
6. Anthropic, PBC. [Form S-1/A.](https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm) Filed November 2, 2026, management discussion of compute commitments and quarterly results.
7. Ornn Data. ["Ornn Compute Price Indices (OCPI): Methodology."](https://data.ornn.com/methodology) Universe creation, accessed November 3, 2026.
8. OpenRouter. ["AI Model Rankings."](https://openrouter.ai/rankings) Measurement methodology, accessed November 3, 2026.