Spot, Reserved or On-Demand GPUs? How to Cut Cloud AI Compute Costs Safely

Renting GPUs by the hour is the most expensive way to run AI work in the cloud, and the three biggest providers all sell cheaper alternatives. Amazon Web Services says its Spot Instances run at up to 90% off On-Demand prices, Google Cloud quotes discounts of up to 91% on Spot VMs, including GPUs and TPUs, and long-term commitments on AWS can save up to 72%.

The catch is that every discount trades away something: the right to keep your machine, the freedom to walk away, or the ability to start whenever you like. Picking the wrong model can mean a training run killed halfway through or a year of payments for GPUs you no longer need. This guide explains the four main ways to pay for cloud GPUs, what each provider's own documentation says about the rules, and how to match each one to a real workload.

Key facts at a glance

• Spot capacity is the cheapest and least reliable. AWS gives a two-minute interruption notice, Azure gives up to 30 seconds, and Google's notice can be 120 seconds or zero.
• Commitments cut steady costs but can't be cancelled. AWS Savings Plans run for one or three years, and Google says its committed use contracts can't be cancelled after purchase.
• Short reservations exist for GPU-heavy projects. AWS Capacity Blocks for ML can be booked up to eight weeks ahead, and Google's calendar-mode reservations cover 1 to 90 days.
• Discounts rarely stack. AWS says Savings Plans add nothing on top of Spot, and Google says its flexible commitments can't be used for Spot VMs.

Who this guide is for

This is for developers, small AI teams, startup founders and IT leads in the US, UK, Canada and Australia who pay their own cloud bill and want to bring down the cost of model training, fine-tuning, batch inference or experiments. It assumes you already know which GPU type you need. It does not cover renting from smaller GPU clouds, though the same trade-offs usually apply there.

The core question is simple: how much interruption and commitment can your workload tolerate? A chatbot answering customers needs capacity that never vanishes. A nightly job that scores a million documents can usually stop and restart without anyone noticing.

Option 1: On-demand, the flexible baseline

On-demand is the default. You start a GPU machine, pay a fixed hourly rate, and stop whenever you like. AWS describes the On-Demand hourly price as static, and you decide when the instance stops or terminates.

Flexibility is what you pay for. There is no contract and no early warning to handle, which makes on-demand the right choice for short tests, unpredictable demand and anything customer-facing while you work out your real usage.

There is one weakness that matters more for GPUs than for ordinary servers: availability. AWS notes that if capacity isn't there when you ask, you simply get an insufficient capacity error. Popular GPU types can be hard to get in busy regions, so on-demand is flexible on price but does not promise you a machine.

Option 2: Spot capacity, the deepest discount

Spot machines are spare capacity that the provider sells cheaply and can take back. The three big clouds handle that reclaim step differently, and the difference is the most important thing to understand before you use them.

AWS sets the Spot price for each instance type and zone, adjusting it gradually with long-term supply and demand. Besides the two-minute notice, it sends an earlier "rebalance recommendation" when an instance is at elevated risk of interruption, giving you time to move work before the final warning.

Google Cloud lets you choose a preemption notice of 120 seconds or the default of zero, followed by a best-effort shutdown period of up to 30 seconds. Google does not bill Spot VMs that are preempted within their first minute, and it excludes Spot VMs from the Compute Engine service level agreement. It also says GPU Spot VMs are preempted by default during maintenance events.

Azure evicts Spot VMs when it needs the capacity back or when the price rises above a maximum you set. Setting the maximum price to -1 means the VM won't be evicted for price reasons, and you will never pay more than the standard rate. Azure's portal shows eviction rates per hour; Microsoft explains that a 10% rate means a 10% chance of eviction within the next hour, based on the last seven days.

Spot fits training with frequent checkpoints, batch inference, data processing and experiments. It is a poor fit for anything that must answer users in real time, and for long jobs that cannot save their progress.

Option 3: One- and three-year commitments

If you know you will use GPUs steadily for a long time, a commitment buys a lower rate in return for a promise to pay whether you use the capacity or not.

AWS Savings Plans commit you to a set amount of compute spend per hour for one year (365 days) or three years (1,095 days). AWS says they can save up to 72% compared with On-Demand prices, and you can pay all upfront, partly upfront or nothing upfront. Compute Savings Plans apply across instance families, sizes and regions, so you are not locked into one GPU type.

Google Cloud committed use discounts come in two forms. Resource-based commitments cover specific hardware, including GPUs, in one region, with discounts of up to 55% for most machine series and up to 70% for memory-optimised ones. Compute flexible commitments are spend-based and work across regions; for GPU machines they apply only to the G2 series with Nvidia L4 chips (21% for one year, 43% for three) and the G4 series with Nvidia RTX PRO 6000 chips (16% and 42%).

The rule that matters most: you can't back out. Google states that a commitment can't be cancelled after purchase and is billed monthly until the term ends, regardless of use. For a fast-moving field where a better GPU generation can arrive within a year, that is a real risk. Commit only to the baseline you are confident you will still need.

Option 4: Short-term GPU reservations

Between on-demand and long commitments sit reservations built for projects that need many GPUs for days or weeks, such as a fine-tuning run.

AWS Capacity Blocks for ML let you reserve GPU instances for a future date. Each block can hold up to 64 instances, with up to 256 across blocks, and you can book a start time up to eight weeks ahead. AWS places the instances close together for low-latency networking. Two rules catch people out: Capacity Blocks can't be cancelled, and they end at 11:30 a.m. UTC, with instance termination starting at 11:00 a.m. UTC on the final day.

Google Cloud calendar-mode reservations use its Dynamic Workload Scheduler to book GPU or TPU capacity for 1 to 90 days. You can check GPU availability up to 60 days ahead, and a GPU request must start at least 87 hours after you submit it. Google says it reviews requests within about a minute and charges a discounted rate only once the capacity is provisioned. You can cancel only if the start is more than 30 days away, or within 24 hours of approval when the start is at least four days away.

These options suit planned training and fine-tuning runs where the timeline is known and an interruption would waste days of work.

How to choose and apply the right mix

Most teams end up combining models rather than picking one. A practical way to decide:

1. List your workloads by how they fail. Mark each one as "must not stop" (live inference), "can pause and resume" (training with checkpoints, batch jobs) or "temporary and planned" (a fine-tuning sprint).
2. Measure before you commit. Run on on-demand for a few weeks and record your steady minimum GPU use. AWS Cost Explorer offers Savings Plans recommendations based on your history.
3. Commit only to the floor. Cover the usage you are sure of with a one-year plan first; the three-year rate is lower but ties you to today's hardware.
4. Move interruptible work to Spot. Make sure jobs save checkpoints and can restart automatically. On Google Cloud, a managed instance group recreates preempted VMs when capacity returns.
5. Book reservations for big planned runs. Check availability early, because Capacity Blocks and calendar-mode requests depend on capacity being free on your dates.
6. Check eviction and price history. Azure shows 90 days of Spot pricing and 28 days of eviction rates through Azure Resource Graph, and AWS shows three months of Spot price history in the console.

What to do if something goes wrong

If a Spot machine is reclaimed mid-job, your first defence is the notice. Have your code watch for it (AWS's interruption notice, Google's preempted metadata value, or Azure's Scheduled Events) and write a checkpoint immediately. Then restart from the last checkpoint on a different instance type or zone, since capacity varies by pool.

If you can't get capacity at all, try nearby regions or another GPU type. Microsoft's own advice for evicted Azure users who still need capacity right away is to switch to standard VMs.

If you over-committed, you will keep paying. On Google Cloud, compute flexible commitments can at least shift to other eligible services such as GKE or Cloud Run, and between G2 and G4 GPU machines, so look for other workloads that can use the spend.

Conclusion

The cheapest GPU hour is the one that matches how your workload behaves. Useful takeaways:

• Use Spot for work that can stop, and build checkpointing in before you rely on the up-to-90% discount.
• Commit only to proven baseline use, remembering that AWS and Google commitments can't be cancelled.
• Reserve short blocks for planned training runs rather than paying on-demand rates or locking in for a year.
• Don't expect discounts to stack: Savings Plans don't add to Spot savings on AWS, and Google's flexible commitments don't cover Spot VMs.
• Review your mix every quarter, because GPU generations and prices change faster than most contracts.