Methodology
Every number on RunsOnMy is computed by one small, deterministic JavaScript module that runs the same way at build time, in the Worker and in your browser. Nothing is written by hand per model.
Total memory
total = weights + KV cache + overhead
Weights
When a GGUF build exists on Hugging Face (from the model maker, Unsloth, bartowski, LM Studio Community or ggml-org) we use the real file size of each quant, summing split files. Otherwise we estimate parameters × bits-per-weight ÷ 8 with llama.cpp's average bits per weight:
| Quant | Bits per weight | Quality |
|---|---|---|
| Q2_K | 3 | rough |
| Q3_K_M | 3.91 | usable |
| Q4_K_M | 4.89 | good, most popular |
| Q5_K_M | 5.7 | very good |
| Q6_K | 6.56 | near lossless |
| Q8_0 | 8.5 | effectively lossless |
| F16 | 16 | original weights |
Parameter counts come from the safetensors metadata on Hugging Face. For mixture-of-experts models, all experts must be in memory, so memory uses total parameters. Active parameters (used for speed) are computed from the config: total − MoE layers × (experts − experts per token) × 3 × hidden × expert FFN size.
KV cache
- Standard / GQA / MQA:
2 × layers × KV heads × head dim × context × 2 bytes(FP16). - MLA (DeepSeek, Kimi, GLM-5 families):
layers × (KV latent rank + RoPE dim) × context × 2 bytes, the compressed cache llama.cpp stores. Sparse-attention indexer keys are added when present. - Sliding-window layers (Gemma, gpt-oss and others): those layers are capped at the window size.
- Hybrid linear-attention / state-space layers (Qwen3.5+, Nemotron-H, LFM): only full-attention layers grow with context; the small fixed state is covered by overhead.
- 8-bit or 4-bit KV cache (calculator option) scales the KV term by 0.5 or 0.25.
Pages for models with newer compressed-attention designs we can't model exactly are labeled as upper bounds.
Overhead
0.4 GiB + vocabulary size × 512 × 4 bytes, covering the runtime context and the logits/compute buffer at the default 512-token batch.
Usable memory
- Discrete GPUs: 95% of VRAM (driver and display reserve). Multi-GPU rigs add up their cards.
- Apple Silicon: macOS lets the GPU wire about 75% of unified memory on 64 GB+ Macs and about 67% below that, by default.
- Unified-memory PCs (DGX Spark, Ryzen AI Max): 85% of system memory.
Fits = total ≤ 90% of usable memory. Tight = 90–100%. No = over.
Speed ceiling
Generating one token reads all active weights once, so tokens/sec ≤ memory bandwidth ÷ (weights × active/total parameters). This is an upper bound; real llama.cpp speeds are usually 50–80% of it. We label it an estimate everywhere and will replace it with measured, opt-in benchmark results.
Runs with offload (slower)
When a model does not fit in a discrete GPU's VRAM, llama.cpp can keep part of it in system RAM. Fit pages then show a Runs with offload result with the exact command and an estimated speed range. Every offload figure is an estimate, not a benchmark.
- Which quant and how much RAM. We try 4-bit, then 3-bit, then 2-bit (Q4_K_M, Q3_K_M, Q2_K) at 8K context, with 32, 64, 128 GB of system RAM, keeping 6 GB for the OS and apps. The page shows the first one that fits, so it names the smallest PC that runs it.
- Dense models:
-ngl Nputs the first N layers on the GPU.N = floor((usable VRAM − overhead − 0.5 GiB) ÷ ((weights + KV cache) ÷ layers)). - Mixture-of-experts models:
-ngl 999 --n-cpu-moe Nkeeps attention, shared weights and the KV cache on the GPU and moves the routed experts of the first N layers to RAM. Expert bytes per layer =weights × (MoE layers × experts × 3 × hidden × expert FFN size ÷ total parameters) ÷ MoE layers. Leading dense layers count toward N because llama.cpp numbers layers from 0. - Speed:
tokens/sec = 1 ÷ (GPU bytes per token ÷ (GPU bandwidth × 0.7) + CPU bytes per token ÷ RAM bandwidth). For MoE, only the top-k ÷ experts share of the experts is read per token, and the result is capped at 0.7× the in-VRAM speed ceiling for the same quant. The range runs from dual-channel DDR4-3200 (51.2 GB/s × 0.45 = 23.0 GB/s effective) to dual-channel DDR5-6000 (96 GB/s × 0.5 = 48 GB/s effective). - Not offered on Macs and unified-memory PCs (the CPU and GPU share one memory pool, so offload adds no room), on multi-GPU rows, or when the card can't hold a single layer.
How the estimate compares with published llama.cpp results
The 0.45–0.5 RAM efficiency was calibrated on these public runs. Our N is the value our formula picks; theirs is what the tester used.
| Model, hardware | Their N | Measured | Our N | Our estimate | Source |
|---|---|---|---|---|---|
| gpt-oss-120b, RTX 3090 24 GB, DDR5-5200 | --n-cpu-moe 25 | 26.9 tok/s | 24 | 17–34 | llama.cpp gpt-oss guide, comment Jan 20 2026 |
| gpt-oss-120b, RTX 5090 32 GB | --n-cpu-moe 21 | ~30 tok/s | 19 | 22–44 | llama.cpp gpt-oss guide (ggml-org) |
| gpt-oss-120b, RTX 4060 Laptop 8 GB, DDR5-5200 | --n-cpu-moe 34 | 18.9 tok/s | 33 | 12–23 (desktop 4060) | llama.cpp gpt-oss guide, comment Nov 10 2025 |
| gpt-oss-20b, RTX 4060 Laptop 8 GB, DDR5-5200 | --n-cpu-moe 14 | 42.3 tok/s | 12 | 27–44 (desktop 4060) | same comment |
| gpt-oss-120b, RTX 4070 12 GB, DDR5-6000 | -ngl 37 + tensor overrides | 23.9 tok/s | 31 | 13–26 | carteakey.dev |
| Llama 7B Q4_0 (dense), CUDA GPU, hardware not stated | -ngl 10 / 20 of 33 | 13.5 / 21.4 tok/s | — | 8.5–17.5 / 14.5–28.6 | llama-bench README |
Measured numbers are single-user decode speed (llama-bench tg128 or llama-server). Our estimates use the model's Q4_K_M GGUF size at 8K context; the testers mostly used MXFP4 files of nearly the same size. We found no primary benchmark for 70B-class dense models with partial offload, so those estimates (about 1–2 tok/s on a 24 GB card) rest on the formula alone. Ranges are wide on purpose: RAM speed changes offload speed about 3× (DDR5 at 2000 vs 6000 MT/s).
What fits instead
Fit pages list the cheapest listed GPU that runs the model fully in VRAM at 8K context (by US launch MSRP from the vendor's announcement; street prices differ, and Macs and data-center cards have no list price here) and the largest popular model that fits the card at 4-bit or better (popular = the 60 most-downloaded models we track).
GPU specs. Memory, bandwidth and launch MSRP come from NVIDIA, AMD, Intel and Apple product pages and launch announcements, last re-checked Nov 17, 2026. NVIDIA no longer lists bandwidth for RTX 30 and 40 series cards on its comparison page, only bus width and memory speed; for those cards the bandwidth is bus width × data rate, matching NVIDIA's launch materials where they state it.
API prices
Prices come from OpenRouter's public models API every 6 hours. Each snapshot is compared to the previous one and every change is logged (change log). Where LiteLLM's MIT-licensed price map has the model maker's own entry, we show it side by side and flag differences.
Automated accuracy checks
Before every build, a check script runs golden tests against reference values computed independently from the models' published configs and sanity gates on every page. A model page is held back (not published) if its numbers fail, for example missing architecture or memory below the file size. GGUF sizes that disagree with the parameter count are discarded in favour of estimates. A model is also held when the size in its name (for example "27B", "8x7B" or "17B-16E") is more than 1.8× away from both its total and active parameter counts, which usually means a draft head or broken metadata. Speculative-decoding draft models (DSpine, DFlash, EAGLE, Medusa, MTP heads) are left out of the catalog because their parameter count is the small draft, not the model in the name. The build is blocked if the golden tests fail or more than 20% of pages are held.
Latest run: Nov 18, 2026 · 298 models checked, 0 held · golden tests: 3/3 passed · 147 prices cross-checked with LiteLLM, 30 differ.
What we don't model
- Offload is estimated, not measured. The "Runs with offload" speed comes from the formula above and a handful of public benchmarks. PCIe transfer, CPU core count and prompt processing speed are not modeled.
- Runtime differences. vLLM, MLX, Ollama and llama.cpp allocate memory differently. Our figures follow llama.cpp's defaults.
- Measured speed. Tokens per second is a memory-bandwidth ceiling, not a benchmark. Real speeds are usually lower.
- Very new attention designs. Where we can't model the KV cache exactly, the page says its long-context figures are an upper bound.
Refresh schedule
- Hugging Face models: hourly (new releases get a page within the hour).
- API prices: every 6 hours.
- GPU and Mac specs: from vendor spec pages, re-checked monthly.
Spotted something wrong? Tell us. Confirmed fixes are listed on the corrections log.