Inference Cost & Performance

Cost and speed of AI inference: serverless GPU providers, pricing and benchmarks.

• TTFT, TPOT and Tokens per Second: How to Read LLM Inference BenchmarksLLM speed claims rest on different metrics that can move in opposite directions; MLPerf's interactive target for Llama 2 70B, for example, is a 450 ms time to first token and 40 ms per output token. This guide explains each metric and how to compare vendors fairly.Nov 1, 2026