RunsOnMy ModelsGPUs & MacsBy VRAMAPI pricesVRAM calculatorNew

Can Inkling run on the Mac M3 24GB?

No. Even at 2-bit with 8K context, Inkling needs about 335 GB, more than the 16.1 GB usable on the Mac M3 24GB.

Memory by quant and context

Quant4K ctx8K ctx32K ctx128K ctxMax tok/s (est., 8K)
2-bit (Q2_K)334 GB No335 GB No342 GB No366 GB No—
3-bit (Q3_K_M)435 GB No436 GB No443 GB No467 GB No—
4-bit (Q4_K_M)544 GB No545 GB No551 GB No576 GB No—
5-bit (Q5_K_M)634 GB No635 GB No641 GB No666 GB No—
6-bit (Q6_K)729 GB No730 GB No736 GB No761 GB No—
8-bit (Q8_0) •800 GB No801 GB No807 GB No832 GB No—
16-bit (F16/BF16) •1766 GB No1767 GB No1773 GB No1798 GB No—

• real GGUF file size from unsloth/inkling-GGUF. Speed is a bandwidth ceiling (100 GB/s), not a benchmark. Methodology.

Hardware that does fit it

Mac M3 Ultra 512GB

More

Inkling on every GPU and Mac · Every model that runs on the Mac M3 24GB · Apple Silicon guide · Try other settings in the calculator

Data: Hugging Face Hub (thinkingmachines/Inkling), Apple specs. Data as of 2026-11-17.