New AI cost tools target inference bills as GPU pricing data multiplies

Developer Flavio Copes published an inference cost calculator on 23 September. It compares 27 models from OpenAI, Anthropic, Google, xAI, Mistral, OpenRouter and Workers AI. The tool ranks every model on the same assumptions so teams can see cheap versus capable trade-offs rather than just list prices. It turns product assumptions into a monthly bill: daily active users, LLM calls per user, input and output tokens per call, and an optional prompt-cache hit rate. Everything runs in the browser, and a share link only encodes the inputs in the URL.

The framing matters more than the individual numbers.

The calculator's documentation says the figures come from published API rates. It ignores batch pricing, enterprise discounts, image or tool surcharges, and any caching layer the customer runs. The output is a planning estimate, not a vendor quote. That is an unusually blunt caveat for a pricing tool, and it points at the real state of this market.

Two ways to price the same GPU

The same week brought a second category of tool: trackers that convert hourly GPU rates into cost per million tokens. FitMyLLM reads rates live from each provider's API, refreshing every three minutes and caching for ten minutes at the edge. Its table currently lists 47 cards and 74 quotes, of which 19 are measured rather than estimated. It sorts cheapest to most expensive by cost per million tokens.

The finding is counterintuitive. The cheapest row on FitMyLLM is a Vast.ai RTX 3090 at $0.13 per hour, producing $0.291 per million tokens, marked as measured. The fastest row is an RTX 5090 at 227 tokens per second, costing $0.576 per million tokens, also measured. An RTX 3090 on RunPod at a published $0.22 per hour comes out at $0.494 per million tokens.

Datacenter cards rank badly here, and that is the finding, not a fault. An H100 or H200 earns its price by serving many streams at once; at one stream its enormous bandwidth sits mostly idle.

FitMyLLM is explicit that this ranking inverts for anyone serving a crowd rather than a single user. It notes that the H100 and H200 rows are its least certain. The tool also warns that marketplace prices are one host's listing, not a published rate, and that hosts may run a card below its stock power limit.

GPUAdvisor takes the opposite approach, tracking 13 cloud providers and 132 instances per node or per GPU, grouped by region and sorted cheapest first. Its snapshot, last tracked on 28 September, shows an RTX A4000 16GB at $0.10 per hour on GPU.ai as the cheapest entry, and an RTX 3090 24GB at $0.17 per hour. GPU-specialist clouds offer two to four times lower per-GPU pricing than hyperscalers. GCP Spot offers up to 70 percent off A100 and H100 instances, and one-year commitments save 35 to 40 percent. The tool closes with the same caveat everyone else uses: verify with provider pricing pages before procurement.

Utilization, not sticker price

Krapton Engineering last updated its guide on 25 September. It argues the decisive metric is neither hourly rate nor token price but utilization. An H100 idle 60 percent of the day still bills, and its effective cost per inference rises accordingly. The firm describes a client engagement with a conversational AI chatbot whose on-demand GPU usage sat at 30 to 40 percent during off-peak hours even with aggressive autoscaling. Moving predictable base load to reserved instances and adding a Kubernetes scheduler to co-locate inference requests cut the overall compute bill by 30 percent.

Krapton's worked example is illustrative, and it says so. It models a mid-range GPU at 1,000 tokens per second for a 7B parameter model. The illustrative reserved cost is $0.75 per hour after a 50 percent discount, plus $0.25 per hour of operational overhead that it does not fold into API cost. Real prices from AWS, GCP or Azure vary and should be verified. The number is a shape, not a quote.

aicost.ai frames the same problem as a three-way choice rather than a binary: pay per token to a serverless host, rent a dedicated GPU by the hour, or run on your own cloud account. Its calculator defaults to 2,500 output tokens per second and 45 percent utilization, with presets of 20 percent for real-world low and 70 percent for well-batched. The guidance is conditional. Serverless wins on low or bursty volume and cheap per-token models. Dedicated rental wins on steady high volume with expensive-per-token models. Self-hosting wins at very high steady volume with reserved or spot pricing, assuming you already run infrastructure. The crossover is blended per-token price times volume against GPU dollars per hour times fleet size.

What the trackers admit they miss

Every one of these tools publishes its own limitations, which is worth more than the tables. aicost.ai states that all three GPU legs assume full utilization at the stated throughput and exclude setup, ops, networking and reliability overhead. Throughput varies by model, hardware, batch size and prompt shape, so it is an editable input rather than a constant. Reserved and spot pricing assumes you can actually hold or reclaim the capacity, and spot can be interrupted.

FitMyLLM notes that batched serving is cheaper per token, and that an idle card still bills. It discloses that provider links may carry a referral code, while stating the code does not change what is listed or how the table sorts. GPUAdvisor says its prices are approximate, vary by region and availability, and reflect September 2026 estimates.

The practical takeaway is narrow. On 28 September, the cheapest published on-demand entry in GPUAdvisor's tracker was $0.10 per hour, and the cheapest measured cost per million tokens in FitMyLLM's table was $0.291. Both numbers are planning inputs, not commitments, and both change with the workload assumptions a team feeds in.

The tools also disagree in emphasis. FitMyLLM ranks consumer cards ahead of datacenter cards for single-stream serving and calls that the finding. Krapton treats reserved capacity as the middle ground for predictable sustained workloads. aicost.ai treats own-cloud as cheapest only with reserved or spot commitment and high utilization. They are not averaging to a single answer, because there is not one.

What is new is that the assumptions are now exposed. A year ago, the hourly rate was the only public number, and the conversion into cost per token happened in a spreadsheet nobody shared. As of this week, three separate tools publish their throughput figures, their utilization defaults and their error bars, and a fourth publishes its estimate date. That does not make the numbers right. It makes them arguable, which is the prerequisite for budgeting against them.