KoboldCpp requirements: what your computer needs
KoboldCpp runs on 64-bit Windows 10 and 11, Linux and Apple Silicon Macs, with or without a graphics card. The model decides how much memory you need: as a rule, the model file plus 0.5 to 2 GB more, and more for a long context. A CPU with AVX2 (most from 2013 onwards) gives full speed; older CPUs work with a slower build.
Checked against KoboldCpp 1.124.2 · updated 13 Nov 2026
- lab-lnx: Linux test VM · 4 vCPU · 15 GB · no GPU
- gha-w25: Windows Server 2025 test machine · 4 vCPU · 16 GB · no GPU
- gha-w22: Windows Server 2022 test machine · 4 vCPU · 16 GB · no GPU
- gha-u24: Ubuntu 24.04 test machine · 4 vCPU · 16 GB · no GPU
- gha-mac15: macOS 15 test VM (Apple Silicon) · 3 vCPU · 7 GB · paravirtual Metal
On this page
KoboldCpp requirements at a glance
| Windows | 64-bit Windows 10 or 11 | Tested on Windows Server 2022 and 2025. Windows 7 and 8 are not confirmed: users report PrefetchVirtualMemory not found. No 32-bit build, no ARM build. |
| Linux | 64-bit x86 with glibc 2.29 or newer | From the symbols the build needs (F71); we tested Ubuntu 24.04 only. ARM Linux needs a build from source. |
| macOS | Apple Silicon (M1 or newer) | There is no build for Intel Macs. |
| CPU | Any 64-bit CPU; AVX2 for full speed | AVX2 arrived about 2013. Without it use koboldcpp-oldpc, --noavx2 (about 33% slower) or --failsafe (about 8 times slower). |
| RAM | The model file plus 1 to 2 GB, and more for long context | Never a single number: it depends on the model. See below. |
| GPU (optional) | NVIDIA (CUDA), any Vulkan GPU, or Apple Metal | NVIDIA owners use the CUDA build (CUDA 12.1; an older 11.4 build exists). AMD and Intel use Vulkan. |
| Disk | The program, 65 to 620 MB, plus the model files | The CUDA builds are about 600 MB; the Mac build is 65 MB and the nocuda builds 86 to 102 MB. |
The file to download for your hardware is on the download page: for Windows, Linux or Mac.
How much memory a model needs
Three things sit in memory: the model’s weights, the KV cache (the memory of the conversation) and some working space. The weights are about the file size: on a tiny model, peak RAM was the file plus about 200 MB F14. The cache grows with the context you ask for: we measured 51 MiB at 4K and 1539 MiB at 128K on a 0.5B model F10, and peak RAM rose from 616 MB at 2K to 2,123 MB at 128K F11. A real 8B model at 8K needs about 5.2 GB in total on a GPU, from measured allocations:
- Model weights 3.9 GB
- KV cache 1.0 GB
- Compute buffer 124 MB
- Free 201 MB
This is what each model needs to run completely, at 8K of context. The first column of each pair is the download size. These are calculated from the real file headers with the same code as the calculator.
| Model | Q4_K_M file | Needs (Q4_K_M) | Q8_0 file | Needs (Q8_0) |
|---|---|---|---|---|
| Qwen2.5 0.5B Instruct | 469 MB | 628 MB | 644 MB | 804 MB |
| Llama 3.2 3B Instruct | 1.88 GB | 3.25 GB | 3.19 GB | 4.65 GB |
| Qwen2.5 7B Instruct | 4.36 GB | 5.17 GB | 7.54 GB | 8.35 GB |
| Llama 3.1 8B Instruct | 4.58 GB | 5.99 GB | 7.95 GB | 9.36 GB |
| Qwen3 8B | 4.68 GB | 6.22 GB | 8.11 GB | 9.65 GB |
| Gemma 2 9B It | 5.37 GB | 8.57 GB | 9.15 GB | 12.6 GB |
| Mistral Nemo Instruct 12B | 6.96 GB | 8.67 GB | 12.1 GB | 13.8 GB |
| Qwen2.5 14B Instruct | 8.37 GB | 10.4 GB | 14.6 GB | 16.6 GB |
| Qwen3 30B-A3B (MoE) | 17.3 GB | 18.7 GB | 30.3 GB | 31.6 GB |
| Qwen2.5 32B Instruct | 18.5 GB | 21.2 GB | 32.4 GB | 35.1 GB |
| Llama 3.3 70B Instruct | 39.6 GB | 43.2 GB | – | – |
For the same model with a longer context, add the KV cache: see context size. A smaller quant saves memory: quantisation.
What can I run on my computer?
Choose your card (or “none”) and your memory. The list shows every model that fits completely, largest first.
Llama 3.1 8B Instruct Q4_K_M · 16K context · KV f16
Fits completely
All 32 layers fit on the GPU, with the context and buffers.
Launch values
- --gpulayers 33
- --contextsize 16384
KoboldCpp counts one extra layer for the output layer, so --gpulayers 33 puts all 32 repeating layers on the GPU (F65). Leaving --gpulayers on auto (-1) lets autofit choose; it keeps a 1 GB padding and may offload fewer layers than this calculation. GPU layers guide
koboldcpp --model Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf --gpulayers 33 --contextsize 16384
The tiers by memory, with the models in each, are on the models page.
CPU only: is it usable?
Yes, with small models. On four cloud vCPUs, a 0.5B model generated 43 to 61 tokens per second with four threads, and a 3B to 8B model is much slower F07 F13. We measured an 8B model on four CPU cores of our Linux test machine: 5.3 tokens per second, about reading speed F37. Treat that as the speed above 8B on a modest CPU. Real desktop numbers are not in yet. See speed for the thread count.
Old computers
KoboldCpp’s normal builds need a CPU with AVX2, which most processors from 2013 onwards have. To check on Linux, run grep -m1 -o avx2 /proc/cpuinfo: it prints avx2 when the CPU has it (this printed avx2 on our Linux test machine). On Windows, look up your CPU’s model name in its manufacturer’s specification page, which lists the instruction sets. If it has none, the options are koboldcpp-oldpc, --noavx2 and --failsafe, each with its speed cost, in Will not start F03 F15. For Windows 7 and 8 see Windows 7. There are no 32-bit builds.
Phones, Chromebooks and Raspberry Pi
Android phones can run it through Termux, which is outside the official downloads; iPhones cannot. A Raspberry Pi or another ARM Linux machine needs a build from source. These guides are still to come: Android and build from source.
No suitable computer?
Use a cloud service instead: Colab for a free GPU for a few hours, a rented cloud GPU, or Kobold Lite with the AI Horde, where other people’s computers answer.
Frequently asked questions
Does KoboldCpp need a GPU?
No. It runs on the CPU alone, using system RAM. A GPU makes it much faster when the model fits in its memory. In our test an 8B model on four CPU cores generated 5.3 tokens per second, which is about reading speed, and a 0.5B model on four cloud vCPUs generated 18 to 61.
Is 8 GB of VRAM enough?
For 7B to 9B models at Q4_K_M with a normal context, yes: an 8B model needs about 5.2 GB at 8K of context in our calculation. Larger models run with some layers on the CPU, which is slower. See the calculator on this page.
Will KoboldCpp run on Windows 7?
We cannot confirm it. Users report the error PrefetchVirtualMemory not found in KERNEL32.dll on Windows 7 and 8, and we have no Windows 7 machine to test. Windows 10 and 11 are the safe choice.
How much RAM do I need for a 7B or 8B model?
About the size of the file plus 1 to 2 GB: a Q4_K_M 8B file is 4.6 GB, so 8 GB of RAM is the least that works and 16 GB is comfortable. Memory also grows with the context size. The table on this page lists the figure for each size.
Does KoboldCpp run on 32-bit Windows?
No. There are no 32-bit builds. KoboldCpp needs a 64-bit system.