What LLM can I run on an RTX 3060 12GB?
The 12 GB RTX 3060 is one of the most popular budget cards for local AI because it has more VRAM than many faster cards. With ~90% usable, about 10.8 GB is available for the model.
That is enough for 7B–14B models at good quantizations and for small mixture-of-experts models. The tables show the best GGUF quantization that fits fully, and for bigger models how many layers would stay on the GPU with offload.
Estimates with default assumptions: 10% runtime overhead, ~90% usable VRAM (~72% on Macs), FP16 KV cache, one sequence. Model data synced from Hugging Face on 2026-10-18. Methodology.
Models on NVIDIA GeForce RTX 3060 12GB (12 GB)
22 of 49 models fit fully at 8K context. Memory shown is the total estimate for the best fitting quantization. “Offload” assumes 64 GB system RAM.
| Model | Params | 8K context | 32K context |
|---|---|---|---|
| Devstral Small 2 24B | 23.6B | Q2_K (10.4 GB, big quality loss) | Offload Q2_K (~30 layers on GPU) |
| Mistral Small 3.2 24B | 23.6B | Q2_K (10.4 GB, big quality loss) | Offload Q2_K (~30 layers on GPU) |
| Qwen2.5 Coder 14B Instruct | 14.8B | Q4_K_M (10.7 GB) | Offload Q2_K (~43 layers on GPU) |
| DeepSeek R1 Distill Qwen 14B | 14.8B | Q4_K_M (10.7 GB) | Offload Q2_K (~43 layers on GPU) |
| Qwen3 14B | 14.8B | Q4_K_M (10.5 GB) | Offload Q2_K (~39 layers on GPU) |
| Phi-4 14B | 14.7B | Q4_K_M (10.7 GB) | Q3_K_M (10.5 GB) |
| Mistral Nemo 12B Instruct | 12.3B | Q5_K_M (10.2 GB) | Q2_K (9.91 GB, big quality loss) |
| Gemma 4 12B | 11.9B | Q6_K (10.5 GB) | Q5_K_M (9.43 GB) |
| Gemma 3 12B | 11.8B | Q6_K (10.7 GB) | Q4_K_M (9.79 GB) |
| Qwen3.5 9B | 9.0B | Q8_0 (10.1 GB) | Q6_K (8.69 GB) |
| Qwen3 8B | 8.2B | Q8_0 (10.0 GB) | Q5_K_M (10.5 GB) |
| Llama 3.1 8B Instruct | 8.0B | Q6_K (7.76 GB) | Q6_K (10.8 GB) |
| Qwen2.5 7B Instruct | 7.6B | Q8_0 (8.73 GB) | Q8_0 (10.0 GB) |
| Qwen2.5 Coder 7B Instruct | 7.6B | Q8_0 (8.73 GB) | Q8_0 (10.0 GB) |
| Mistral 7B Instruct v0.3 | 7.2B | Q8_0 (8.89 GB) | Q6_K (10.1 GB) |
| Qwen3.5 4B | 4.2B | F16 (8.93 GB) | F16 (9.68 GB) |
| Qwen3 4B Instruct 2507 | 4.0B | F16 (9.37 GB) | Q8_0 (8.89 GB) |
| Gemma 3 4B | 3.9B | F16 (8.23 GB) | F16 (8.70 GB) |
| Phi-4 mini 3.8B | 3.8B | F16 (8.87 GB) | Q8_0 (8.18 GB) |
| Llama 3.2 3B Instruct | 3.2B | F16 (7.47 GB) | F16 (10.1 GB) |
| Qwen2.5 3B Instruct | 3.1B | F16 (6.61 GB) | F16 (7.45 GB) |
| Llama 3.2 1B Instruct | 1.2B | F16 (2.79 GB) | F16 (3.54 GB) |
| DeepSeek V3 | 685B | Doesn’t fit | Doesn’t fit |
| DeepSeek R1 0528 671B (MoE) | 671B | Doesn’t fit | Doesn’t fit |
| DeepSeek V3 0324 | 671B | Doesn’t fit | Doesn’t fit |
| Qwen3 235B-A22B Instruct 2507 (MoE) | 235B | Doesn’t fit | Doesn’t fit |
| Qwen3.8 Flash Next | 177B | Doesn’t fit | Doesn’t fit |
| gpt-oss 120B (MoE) | 117B | Doesn’t fit | Doesn’t fit |
| GLM-4.5 Air 106B (MoE) | 110B | Offload Q2_K (~10 layers on GPU) | Offload Q2_K (~9 layers on GPU) |
| Llama 4 Scout 17B-16E (MoE) | 108B | Offload Q2_K (~12 layers on GPU) | Offload Q2_K (~12 layers on GPU) |
| Qwen2.5 72B Instruct | 72.7B | Offload Q2_K (~26 layers on GPU) | Offload Q2_K (~21 layers on GPU) |
| Llama 3.3 70B Instruct | 70.6B | Offload Q2_K (~29 layers on GPU) | Offload Q2_K (~23 layers on GPU) |
| Mixtral 8x7B Instruct (MoE) | 46.7B | Offload Q2_K (~20 layers on GPU) | Offload Q2_K (~17 layers on GPU) |
| Qwen3.6 35B-A3B (MoE) | 35.5B | Offload Q2_K (~30 layers on GPU) | Offload Q2_K (~29 layers on GPU) |
| Qwen3.5 35B A3B | 34.7B | Offload Q3_K_M (~25 layers on GPU) | Offload Q3_K_M (~24 layers on GPU) |
| Qwen2.5 32B Instruct | 32.8B | Offload Q2_K (~47 layers on GPU) | Offload Q2_K (~33 layers on GPU) |
| DeepSeek R1 Distill Qwen 32B | 32.8B | Offload Q2_K (~47 layers on GPU) | Offload Q2_K (~33 layers on GPU) |
| Qwen3 32B | 32.8B | Offload Q2_K (~47 layers on GPU) | Offload Q2_K (~33 layers on GPU) |
| Qwen2.5 Coder 32B Instruct | 32.8B | Offload Q2_K (~47 layers on GPU) | Offload Q2_K (~33 layers on GPU) |
| Gemma 4 31B | 30.7B | Offload Q3_K_M (~39 layers on GPU) | Offload Q3_K_M (~35 layers on GPU) |
| Qwen3 Coder 30B-A3B (MoE) | 30.5B | Offload Q2_K (~42 layers on GPU) | Offload Q2_K (~35 layers on GPU) |
| Qwen3 30B-A3B Instruct 2507 (MoE) | 30.5B | Offload Q2_K (~42 layers on GPU) | Offload Q2_K (~35 layers on GPU) |
| GLM-4.7 Flash (MoE) | 30.0B | Offload Q2_K (~42 layers on GPU) | Offload Q2_K (~38 layers on GPU) |
| Qwen3.8 27B | 27.3B | Offload IQ4_XS (~45 layers on GPU) | Offload IQ4_XS (~41 layers on GPU) |
| Gemma 3 27B | 27.0B | Offload Q2_K (~56 layers on GPU) | Offload Q2_K (~48 layers on GPU) |
| Qwen3.6 27B | 26.9B | Offload Q3_K_M (~47 layers on GPU) | Offload Q3_K_M (~43 layers on GPU) |
| Qwen3.5 27B | 26.9B | Offload Q3_K_M (~47 layers on GPU) | Offload Q3_K_M (~43 layers on GPU) |
| Gemma 4 26B-A4B (MoE) | 25.3B | Offload Q2_K (~28 layers on GPU) | Offload Q2_K (~26 layers on GPU) |
| gpt-oss 20B (MoE) | 21.0B | Offload MXFP4 (~20 layers on GPU) | Offload MXFP4 (~19 layers on GPU) |
Try your own setup
Loading model list…
Questions
Can an RTX 3060 12GB run 14B models?
Yes, at around 4-bit with a moderate context: a 14B Q4_K_M file is about 9 GB, leaving room for a few thousand tokens of KV cache. Check the 8K and 32K tables for each model.
Is 12 GB better than an 8 GB faster card for LLMs?
For local LLMs, memory capacity usually matters more than compute: a model that fits fully on a slower 12 GB card is typically much faster than the same model offloaded from an 8 GB card.
Can I run a 30B mixture-of-experts model on 12 GB?
Not fully on the GPU, but MoE models are good offload candidates: llama.cpp can keep expert weights in system RAM with --n-cpu-moe while attention stays on the GPU. You need enough system RAM for the rest of the model.