Which model should you run in KoboldCpp? What fits your VRAM

Last reviewed

Last reviewed

2026-11-14

Next review

2026-12-07

Checked against KoboldCpp

1.124.2

This list is reviewed once a month. What changed.

What can I run?

Pick your card and context. The list shows every model of ours that fits completely, largest first.

koboldcpp --model Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf --gpulayers 33 --contextsize 16384

How we choose and test these models

• Shortlist. A model is listed only if its licence allows personal use, it comes from its original publisher on Hugging Face or a named, widely used quantizer, and the current KoboldCpp supports its architecture.
• Sizes are calculated, not guessed. Layers, KV heads and every tensor size are read from the real file headers on Hugging Face with the same code as the calculator. The context is 8K, the KV cache is f16, and the margin is KoboldCpp’s autofit padding of 1024 MB.
• What “tested by us” means. We loaded the model in our lab and recorded what happened. Two models qualify: Qwen2.5 0.5B, which every one of our tests used, and Llama 3.1 8B, whose memory we measured F36 and whose CPU speed we measured F37.
• What we have not done. We have not scored any model on answer quality, so there is no ranking and no “best” below. We have no graphics card in the lab, so there are no GPU speeds. The tables show what fits; the quality scoring and the real-card runs are still to come, and each pick will be added with its evidence when they exist.
• Review. We recheck the list monthly and after any KoboldCpp release that adds an architecture. See how we test.

KoboldCpp best model by memory tier

Every table below is the largest quantisation of each model (IQ4_XS or better) that fits completely in that memory at 8K context, calculated. A model that would need a 3-bit or lower quant is not listed for that tier. “Tight” means less than 5% of the usable memory is left. Each model name goes to its Hugging Face page.

No GPU (CPU only)

The model sits in system RAM. RAM use is about the file size plus a fixed overhead F14, and the calculation keeps 3 GB for the system. Speed is the catch: Llama 3.1 8B generated 5.3 tokens per second on four CPU cores F37, and smaller models are faster.

With 32 GB of RAM:

8 GB of VRAM

12 GB of VRAM

16 GB of VRAM

On an AMD card these are Vulkan numbers; see AMD when it is published.

24 GB of VRAM

Mac (unified memory)

A Mac shares one pool between the CPU and the GPU, and the GPU may use only a part of it. The calculation uses about two thirds of RAM up to 36 GB and about three quarters above, which is an estimate we have not measured on a real Mac. A 24 GB Mac:

More on Apple Silicon: Mac.

Next steps for any tier: download the file when that page is published, load it, and set the GPU layers.

Models by use case

We have not scored models for any use case yet, so we do not name a best one. What we can say:

Chat and general assistant

Pick an instruct or chat version of a model that fits your tier above. KoboldCpp reads the chat template from the file, so most work without setup; if replies come out in the wrong format, see chat templates when it is published.

Roleplay and story writing

Many community fine-tunes target roleplay, and some are uncensored. We list only models we have tested, and we have not tested any for roleplay, so none is named here. Whatever you choose, read the model card and licence first. The chat template and the sampler settings change the result as much as the model for many people (samplers when it is published). What we will score when we do: staying in character, following the scene, and not repeating itself.

Coding

Choose a coder-tuned model of the largest size that fits your tier. We have not tested any.

Vision (images as input)

A vision model needs its matching projector file, the mmproj, from the same repository: a projector from another model family does not work. Our only test used a tiny model, which recognised shapes and missed text F28, so it says nothing about larger ones. See vision when it is published.

Big models that run fast: MoE

A mixture-of-experts model holds many expert networks but uses only a few for each word. Its whole file still has to fit in memory, but the experts can sit in system RAM while the attention layers stay on the GPU, which is what --moecpu does. Calculated for Qwen3-30B-A3B at Q4_K_M with 64 GB of RAM and 8K context (we have not run it):

8 GB card:

12 GB card:

We have not measured the speed of a mixture-of-experts model. More on the flag: GPU layers, MoE.

How to read a model file name

Meta-Llama

The family and who made it

3.1

The version

8B

About 8 billion parameters

Instruct

Tuned to follow instructions (some use "it" or "chat")

Q4_K_M

The quantisation: about 4 bits per weight, k-quant, medium

-00001-of-00002

A split file: download every part

.gguf

The file format KoboldCpp loads

The full file name in this example is Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf. See quantisation and, for split files, the Hugging Face page when it is published.

Older favourites

Models such as Tiefighter and Airoboros still load, because GGUF versions exist, but they date from 2023 and 2024. We did not test them, so we do not compare them with current models.

LoRA adapters

A text LoRA is a small add-on file that changes a model’s behaviour. It must be converted to GGUF and loaded with --lora, with --loramult setting its strength. Image LoRAs go in the image settings. Most readers do not need one. This is from KoboldCpp’s own help; we have not tested a LoRA.

Changelog

• 2026-11-14. First version of the page. The tier tables are calculated from real file headers (F63, F70). The only models we have run are Qwen2.5 0.5B and Llama 3.1 8B (F36, F37). Nothing is ranked yet.