Article URL: https://runinfra.ai/news/inference-cost-truth Comments URL: https://news.ycombinator.com/item?id=49133617 Points: 4 # Comments: 3

Those are both the price of one million output tokens. Same unit, same dataset, read off vendor pages in the same week. The first is gpt-oss-120b on a single MI355X. The second is Qwen3.5 on eight H100s. Neither is a typo and neither is a trick. And almost none of the gap between them is the model. So I went looking for what it actually is. Here is what I found, measured across 24 providers and 378 GPU rental rates, ranked smallest to largest: Five levers. The two everyone talks about are the two smallest. This post walks up the list. We build RunInfra, which makes models run faster on your own GPUs. We make money when people self-host. That is a direct conflict of interest with the question this post answers, so read the numbers adversarially. Every one of them carries a source URL and a retrieval date in the repo at the end. Is self-hostingRunning model inference on compute you rent or own instead of buying each request from a hosted model API. cheaper than the API? Here is the like-for-like comparison. Same weights, same model id, cheapest published option on each side. At 90% utilizationThe fraction of paid accelerator time during which the hardware is doing useful inference work. self-hosting takes two of three. At 30%, which is generous for most real deployments, the API takes two of three.