RunsOnMy ModelsGPUs & MacsBy VRAMBest GPUCan I run…?OllamaStart hereAPI pricesVRAM calculatorNew

GPUs and Macs for running AI models locally

Pick your hardware to see every open model that fits, the best quant, and an estimated speed ceiling. Or browse by VRAM size, the Apple Silicon guide, or head-to-head GPU comparisons (RTX 3060 vs 4060, RTX 4090 vs 5090, RX 7900 XTX vs RTX 4090 and more).

Consumer GPUs

HardwareMemoryUsableBandwidthModels that fit (8K)
RTX 3060 12GB12 GB11.4 GB360 GB/s173
RTX 3060 Ti8 GB7.6 GB448 GB/s161
RTX 30708 GB7.6 GB448 GB/s161
RTX 3080 10GB10 GB9.5 GB760 GB/s165
RTX 3080 12GB12 GB11.4 GB912 GB/s173
RTX 309024 GB22.8 GB936 GB/s209
RTX 3090 Ti24 GB22.8 GB1008 GB/s209
RTX 40608 GB7.6 GB272 GB/s161
RTX 4060 Ti 8GB8 GB7.6 GB288 GB/s161
RTX 4060 Ti 16GB16 GB15.2 GB288 GB/s200
RTX 407012 GB11.4 GB504 GB/s173
RTX 4070 Super12 GB11.4 GB504 GB/s173
RTX 4070 Ti12 GB11.4 GB504 GB/s173
RTX 4070 Ti Super16 GB15.2 GB672 GB/s200
RTX 408016 GB15.2 GB717 GB/s200
RTX 4080 Super16 GB15.2 GB736 GB/s200
RTX 409024 GB22.8 GB1008 GB/s209
RTX 50608 GB7.6 GB448 GB/s161
RTX 5060 Ti 16GB16 GB15.2 GB448 GB/s200
RTX 507012 GB11.4 GB672 GB/s173
RTX 5070 Ti16 GB15.2 GB896 GB/s200
RTX 508016 GB15.2 GB960 GB/s200
RTX 509032 GB30.4 GB1792 GB/s218
RX 76008 GB7.6 GB288 GB/s161
RX 7700 XT12 GB11.4 GB432 GB/s173
RX 7800 XT16 GB15.2 GB624 GB/s200
RX 7900 GRE16 GB15.2 GB576 GB/s200
RX 7900 XT20 GB19.0 GB800 GB/s209
RX 7900 XTX24 GB22.8 GB960 GB/s209
RX 907016 GB15.2 GB640 GB/s200
RX 9070 XT16 GB15.2 GB640 GB/s200
Arc B58012 GB11.4 GB456 GB/s173
Arc A770 16GB16 GB15.2 GB560 GB/s200

Multi-GPU rigs

HardwareMemoryUsableBandwidthModels that fit (8K)
2x RTX 30902×24 GB45.6 GB936 GB/s225
2x RTX 40902×24 GB45.6 GB1008 GB/s225
2x RTX 50902×32 GB60.8 GB1792 GB/s228

Apple Silicon Macs

HardwareMemoryUsableBandwidthModels that fit (8K)
Mac M1 16GB16 GB10.7 GB68 GB/s168
Mac M2 24GB24 GB16.1 GB100 GB/s206
Mac M3 24GB24 GB16.1 GB100 GB/s206
Mac M4 16GB16 GB10.7 GB120 GB/s168
Mac M4 32GB32 GB21.4 GB120 GB/s209
Mac M1 Pro 32GB32 GB21.4 GB200 GB/s209
Mac M3 Pro 36GB36 GB24.1 GB150 GB/s209
Mac M4 Pro 24GB24 GB16.1 GB273 GB/s206
Mac M4 Pro 48GB48 GB32.2 GB273 GB/s218
Mac M4 Pro 64GB64 GB48.0 GB273 GB/s226
Mac M1 Max 64GB64 GB48.0 GB400 GB/s226
Mac M2 Max 96GB96 GB72.0 GB400 GB/s229
Mac M3 Max 128GB128 GB96.0 GB400 GB/s238
Mac M4 Max 64GB64 GB48.0 GB546 GB/s226
Mac M4 Max 128GB128 GB96.0 GB546 GB/s238
Mac M2 Ultra 192GB192 GB144 GB800 GB/s257
Mac M3 Ultra 256GB256 GB192 GB819 GB/s261
Mac M3 Ultra 512GB512 GB384 GB819 GB/s292

Unified-memory PCs

HardwareMemoryUsableBandwidthModels that fit (8K)
DGX Spark128 GB109 GB273 GB/s245
Ryzen AI Max+ 395 128GB128 GB109 GB256 GB/s245

Workstation GPUs

HardwareMemoryUsableBandwidthModels that fit (8K)
RTX A600048 GB45.6 GB768 GB/s225
RTX 6000 Ada48 GB45.6 GB960 GB/s225
RTX PRO 6000 Blackwell96 GB91.2 GB1792 GB/s238

Data-center GPUs

HardwareMemoryUsableBandwidthModels that fit (8K)
L40S48 GB45.6 GB864 GB/s225
A100 40GB40 GB38.0 GB1555 GB/s218
A100 80GB80 GB76.0 GB2039 GB/s230
H100 PCIe80 GB76.0 GB2000 GB/s230
H100 SXM80 GB76.0 GB3350 GB/s230
H200141 GB134 GB4800 GB/s254