Can I run Yi 34B Chat locally?
Yi 34B Chat by 01-ai has 34B parameters. At 4-bit with an 8K context it needs about 21.6 GB of GPU memory. It fits on 36 of 65 GPUs and Macs we track. See all models that fit in 24 GB of VRAM.
Memory needed by quant and context length
GB of GPU memory = weights + KV cache (FP16) + runtime overhead. Weights use real file sizes from TheBloke/Yi-34B-Chat-GGUF where available (marked •), otherwise parameter count × bits per weight. How we calculate.
| Quant | Quality | 4K ctx |
|---|---|---|
| 2-bit (Q2_K) • | rough | 15.0 GB |
| 3-bit (Q3_K_M) • | usable | 17.0 GB |
| 4-bit (Q4_K_M) • | good, most popular | 20.7 GB |
| 5-bit (Q5_K_M) • | very good | 24.1 GB |
| 6-bit (Q6_K) • | near lossless | 27.7 GB |
| 8-bit (Q8_0) • | effectively lossless | 35.5 GB |
| 16-bit (F16/BF16) | original weights | 65.5 GB |
Which GPUs and Macs run Yi 34B Chat?
Best quality quant that fits at 8K context. "Tight" = within 10% of usable memory. Speed is a ceiling from memory bandwidth (real speed is typically 50–80% of it).
Questions
How much VRAM does Yi 34B Chat need?
About 21.6 GB at 4-bit (Q4_K_M) with an 8K context: 19.2 GB of weights (real GGUF file size), 1.9 GB of KV cache and 0.5 GB of runtime overhead. At 8-bit it needs about 36.4 GB.
What is the smallest GPU that runs Yi 34B Chat?
At Q4_K_M and 8K context, the smallest single consumer GPU in our table that fits it is the RTX 5090 (32 GB).
Can I run Yi 34B Chat on a Mac?
Yes. At Q4_K_M with 8K context it fits on a Mac M3 Pro 36GB or larger, using the default macOS GPU memory limit.
Similar models
Same family or a similar parameter count, with the VRAM each needs at Q4_K_M and 8K context.
Yi 34B
34B · 21.6 GB at 4-bitYi 1.5 34B Chat
34B · 21.6 GB at 4-bitYi 34B 200K
34B · 21.6 GB at 4-bitNous Hermes 2 Yi 34B
34B · 22.1 GB at 4-bitQwen AgentWorld 35B A3B
35B · 21.6 GB at 4-bitQwen3.6 35B A3B
36B · 21.8 GB at 4-bitEmbed this
Free badge and "Will it run on my GPU?" widget for model cards, READMEs and blogs. No cookies, no tracking. All options.
Copy a snippet
Markdown badge
[](https://runsonmy.com/model/yi-34b-chat/) Widget (script)
<div class="runsonmy-widget" data-model="yi-34b-chat"><a href="https://runsonmy.com/model/yi-34b-chat/">Will Yi 34B Chat run on my GPU? Check on RunsOnMy</a></div>
<script async src="https://runsonmy.com/embed.js"></script> Widget (iframe)
<iframe src="https://runsonmy.com/embed/m/yi-34b-chat/" width="100%" height="220" style="max-width:440px;border:0" loading="lazy" title="Will Yi 34B Chat run on my GPU? (RunsOnMy)"></iframe> Model facts
| Hugging Face repo | 01-ai/Yi-34B-Chat |
| Parameters | 34,388,917,248 (34B) |
| Layers / hidden size | 60 / 7168 |
| Attention | GQA; 56 heads, 8 KV heads, head dim 128 |
| KV cache per token (FP16) | 240.0 KiB |
| Max context | 4,096 tokens |
| License | apache-2.0 |
| Released on HF | Nov 22, 2023 |
| Downloads (30d) / likes | 30,546 / 358 |
Sources: Hugging Face model API · config.json (01-ai/Yi-34B-Chat) · GGUF files · GPU specs from vendor pages (list). Data fetched Nov 17, 2026. Spot an error? Report it.