GPU Cloud Cost Calculator
Estimate GPU compute costs for AI training and inference. Compare on-demand, spot, and reserved pricing — and find the break-even point for reserved commitments.
94+
Providers
3
Pricing modes
Workload presets
Total GPUs in your training cluster
Wall-clock hours for the full run
Effective compute utilization (MFU)
$454.18
182 GPU-hours
$198.82
Save $255.36 (56%)
$344.74
Save $109.44 (24%)
$2.490
at stated utilization
Cost comparison
Cost optimization tips
- Use spot instances with checkpointing to save 56% on this run
- Mixed-precision (BF16/FP8) can reduce training time 30–50% on H100 80GB
- Gradient checkpointing trades ~30% speed for lower VRAM, enabling larger batch sizes
How these estimates work
Training cost
GPU-hours × rate × utilization factor. GPU-hours = GPU count × wall-clock hours. Utilization accounts for idle time during data loading, checkpointing, and communication overhead.
Inference cost
GPU-hours are fixed (GPUs run 24/7 for serving). Cost per request = daily GPU cost ÷ daily requests. Concurrency = requests/sec × latency in seconds.
Break-even
Reserved pricing is billed for all hours in the commitment period regardless of usage. Break-even is the daily utilization at which reserved total cost equals on-demand total cost.
On-demand vs reserved GPU pricing
On-demand GPU instances are billed by the hour with no upfront commitment and can be terminated at any time. Reserved instances require a 1-month, 3-month, or 6-month commitment in exchange for a 20–40% discount on the hourly rate. The break-even point — the daily utilization at which reserved total cost equals on-demand total cost — is typically 60–70% for a 3-month commitment. If your GPU workload runs more than 16–17 hours per day on average, a reserved instance almost always saves money. For sporadic or unpredictable workloads (research experiments, occasional fine-tuning runs), on-demand is more cost-effective. The calculator above models this break-even precisely for any GPU, provider, and commitment length.
Spot instances and fault-tolerant training
Spot (preemptible) GPU instances use spare cloud capacity at 40–70% discounts versus on-demand rates. H100 spot prices can fall below $2/hr on providers like Vast.ai and RunPod — compared to $2.49–$3.50/hr on-demand. The trade-off is interruption risk: spot instances can be reclaimed with 30 seconds to 2 minutes notice. For training jobs, this means implementing checkpointing — saving model state every N steps so a job can resume after interruption without losing all progress. With checkpointing every 15–30 minutes, spot instances are viable for most training workloads and can reduce total training cost by 40–60% versus on-demand. For inference serving and real-time applications, on-demand or reserved instances are required — interruptions are not acceptable in production.
Hidden costs in GPU cloud billing
The hourly GPU rate is only part of the true cost of cloud compute. Egress fees — charged when data leaves the cloud provider's network — range from $0.08–$0.09/GB on AWS and Google Cloud to free on Lambda Labs and CoreWeave. For large model checkpoints (a 70B model checkpoint is ~140GB at FP16), egress costs can add $11–$13 per checkpoint transfer on hyperscalers. Storage costs ($0.02–$0.10/GB/month) matter for persistent volumes holding model weights and datasets. Minimum billing increments vary: some providers bill per second, others per hour — for short experiments, per-hour billing can significantly inflate effective cost. Networking costs for multi-GPU training (InfiniBand fabric, NVLink) are sometimes included in the GPU rate, sometimes charged separately. Always check the provider's full pricing page before committing to a long training run.
GPU cost for common AI workloads
Training costs vary enormously by model size and hardware. Fine-tuning a 7B model (Llama 3.1 7B, Mistral 7B) on a single RTX 4090 ($0.44/hr) for 3 hours costs approximately $1.32 — accessible for individual researchers. Fine-tuning a 70B model requires at least 2× A100 80GB or 1× H100 80GB; at $2.49/hr for 8 hours, that's $19.92. Pre-training a 7B model from scratch on 1T tokens requires approximately 140,000 H100-hours — at $2.49/hr, that's $348,600 in compute alone, explaining why only well-funded labs train frontier models. For inference, a single H100 can serve approximately 100–200 concurrent users for a 7B model at reasonable latency, making the per-user compute cost $0.012–$0.025/hr at $2.49/hr GPU rental. Use the calculator above to model your specific workload duration and utilization.
Choosing between GPU providers for cost
For pure compute cost, specialist providers (Lambda Labs, CoreWeave, RunPod, Fluidstack, Vast.ai) consistently undercut hyperscalers (AWS, Google Cloud, Azure) by 40–60% for equivalent hardware. AWS p5.48xlarge (8× H100 SXM) costs $98.32/hr on-demand — Lambda Labs offers the same 8× H100 SXM configuration for approximately $19.92/hr. The hyperscaler premium buys ecosystem integration (IAM, VPC, managed storage, compliance certifications), not better hardware. For teams already invested in AWS or GCP tooling, the integration value may justify the premium. For teams running self-contained training jobs or inference servers with minimal cloud ecosystem dependencies, specialist providers offer the same hardware at a fraction of the cost. The GPU cost calculator lets you compare total cost across providers for any workload duration and commitment type.