RTX 3090
Previous-gen consumer GPU. Cheapest option for 24GB VRAM workloads. Limited BF16 support.
RTX 3090 Overview
The RTX 3090 is an Ampere consumer GPU that remains relevant because it offers 24GB of GDDR6X at a comparatively low price. It provides 71 TFLOPS of FP16 performance and 936 GB/s of bandwidth, giving it considerably more memory headroom than many newer midrange consumer cards.
The 24GB pool can support smaller full-precision models and larger models after quantization, while 936 GB/s is respectable for its generation. Its shortcomings are equally important: it lacks ECC, does not provide NVLink memory pooling, and does not have native BF16 acceleration or Hopper-era FP8 support.
The card is a useful budget option for experimentation, small-model fine-tuning, and inference where 24GB matters more than newest-generation features. It is not intended for reliability-sensitive data-center deployments or for fast modern transformer training that relies on BF16 or FP8 paths.
Memory
Compute Performance
Hardware Specifications
Chip
Processors
Clocks
Memory
Power
Relative Performance
Relative to highest-spec GPU in database
Limitations
Live Cloud PricingOn-demand hourly rates
Compare RTX 3090 vs…
Use Case Guidance
LLM Model Size Guidance
Estimates only. Actual capacity depends on context length, KV cache, and framework overhead.
Related Guides
LLM APIs Running on This GPU Class
Providers that serve frontier LLM inference on Ampere-class hardware.
RTX 3090 vs Alternatives — Spec Comparison
| Spec | RTX 3090 this | RTX 5060 Ti | RTX 3080 Ti | RTX A5500 24GB |
|---|---|---|---|---|
| VRAM | 24GB GDDR6X▲ | 16GB GDDR7 | 12GB GDDR6X | 24GB GDDR6▲ |
| Memory Bandwidth | 936 GB/s▲ | 672 GB/s | 912 GB/s | 768 GB/s |
| FP16 TFLOPS | 71▲ | 70.4 | 68.2 | 68.2 |
| BF16 TFLOPS | 71▲ | 70.4 | 68.2 | 68.2 |
| FP8 TFLOPS | — | — | — | — |
| INT8 TOPS | 142 | 140.8 | 136 | 272.8▲ |
| TDP | 350W▲ | 180W | 350W▲ | 230W |
| Process Node | Samsung 8nm▲ | TSMC 4NP | Samsung 8nm▲ | Samsung 8nm▲ |
| Architecture | GA102 | GB206▲ | GA102 | GA102 |
| Release Year | 2020 | 2025▲ | 2021 | 2021 |
| Max model (FP16) | ~12B params▲ | ~8B params | ~6B params | ~12B params▲ |
| Max model (INT4) | ~48B params▲ | ~32B params | ~24B params | ~48B params▲ |
Related GPUs
Frequently Asked Questions
How much VRAM does the RTX 3090 have?
The RTX 3090 has 24GB of GDDR6X memory with 936 GB/s bandwidth. This enables running models up to approximately 48B parameters at INT4 precision, 24B at INT8, or 12B at FP16.
What is the FP16 performance of the RTX 3090?
The RTX 3090 delivers 71 TFLOPS of FP16 performance and 71 TFLOPS BF16. INT8 throughput is 142 TOPS. For transformer inference, memory bandwidth (936 GB/s) is often the binding constraint rather than raw TFLOPS.
What is the RTX 3090 best used for?
The RTX 3090 is best suited for: Budget inference, Small model fine-tuning, Experimentation. Previous-gen consumer GPU. Cheapest option for 24GB VRAM workloads. Limited BF16 support.
What interconnect does the RTX 3090 use?
The RTX 3090 uses PCIe 4.0. Without NVLink, VRAM cannot be pooled across multiple cards — the single-card capacity is the hard ceiling for model size.
What LLM model sizes can the RTX 3090 run?
With 24GB of GDDR6X, the RTX 3090 can run models up to approximately 12B parameters at FP16 (2 bytes/param), 24B at INT8 (1 byte/param), or 48B at INT4/GGUF (0.5 bytes/param). These are estimates — actual capacity depends on context length, KV cache size, and framework overhead. Longer context windows require more KV cache memory, reducing the effective model size that fits.
How does the RTX 3090 compare to the A100 for LLM inference?
The RTX 3090 has 71 TFLOPS FP16 vs the A100 80GB's 312 TFLOPS, and 936 GB/s memory bandwidth vs the A100's 2,039 GB/s. For memory-bound autoregressive LLM inference, bandwidth is the primary determinant of tokens-per-second. The A100's higher bandwidth gives it a throughput advantage for large model inference, despite the RTX 3090's lower cost.
What is the power consumption of the RTX 3090?
The RTX 3090 has a TDP (Thermal Design Power) of 350W. This is the maximum sustained power draw under full load. For data center deployments, total rack power consumption is typically 1.2–1.5× the GPU TDP when accounting for CPU, memory, networking, and cooling overhead. At 350W, the RTX 3090 is in the mid-range tier — compatible with standard data center power infrastructure.
Ready to rent?
Compare RTX 3090 prices across 102+ providers
Live on-demand & spot rates · monthly cost estimates · availability status