koboldcpp.dev

Let your model see images: vision in KoboldCpp

Vision needs two files: a vision-capable GGUF model and the mmproj (projector) file made for that exact model. Put the mmproj in the launcher's Vision mmproj field (or --mmproj), launch, then attach an image in KoboldAI Lite or send it as an image_url in a chat request. We tested this on a CPU with KoboldCpp 1.124.2; a projector from a different model family stops the start with an error, and with no projector the model still answers, without seeing the picture.

Checked against KoboldCpp 1.124.2 · updated 13 Nov 2026
  • lab-lnx: Linux test VM · 4 vCPU · 15 GB · no GPU
  • gha-w25: Windows Server 2025 test machine · 4 vCPU · 16 GB · no GPU
  • gha-w22: Windows Server 2022 test machine · 4 vCPU · 16 GB · no GPU
  • gha-u24: Ubuntu 24.04 test machine · 4 vCPU · 16 GB · no GPU
How we test
On this page
  1. What an mmproj is: KoboldCpp vision in one paragraph
  2. Pick a vision model
  3. Load it
  4. What --visionmaxres costs
  5. Ask about an image in Lite
  6. Send images through the API
  7. What our tests show
  8. Audio input
  9. When it does not work

What an mmproj is: KoboldCpp vision in one paragraph

A vision model is a text model plus a small projector. The projector (the mmproj file) turns a picture into tokens the language model reads like words. Each projector is made for one model, so it must come from the same repository as the model file. The projector also takes memory on top of the model.

Pick a vision model

Vision models
SmolVLM-256M-InstructModel Q8_0 (175 MB) and its mmproj (104 MB)Works on CPU; a plumbing test, not a quality pick <EvidenceBadge id="F89" />
Gemma 3 (vision sizes)Model plus mmproj from the same repositoryUntested
Qwen VL familyModel plus mmprojUntested

We name no best vision model: our test model is far too small to judge quality (see what the tests show). The koboldcpp organisation on Hugging Face hosts some projector files (Hugging Face).

Load it

  1. Download the model and mmproj from one repository

    A mismatched pair stops the start (see when it does not work).

  2. Loaded Files tab, then Vision mmproj

    Or start with .

  3. Optional: limits

    Vision MaxRes (, default 1024) sets how large a picture may be before it is shrunk; puts the projector on the CPU.

  4. Launch and check

    /api/extra/version must say "vision": true F27.

What --visionmaxres costs

We sent one 1600 by 1000 picture at three settings F89:

One 1600 by 1000 picture, SmolVLM-256M on CPU
--visionmaxres 5121554.2
1024 (default)2217.2
204848616.0

A larger limit keeps more detail and costs more tokens and time. Users report that small text is lost at low limits; we did not measure that.

Ask about an image in Lite

In Lite, Add File opens “Add Audio or Image”; “Upload a File (Image / Audio)” takes the picture. We did this with a vision model loaded: Lite sent a POST /v1/chat/completions with an image_url part holding the picture as a data URI, and showed the picture in your turn F105. The tiny SmolVLM-256M answered “Triangles.” to a picture of a red circle and a blue rectangle, so the plumbing worked and the model did not.

Lite after uploading a test picture through Add File.gha-u24-lite-upload-image-01 · gha-u24 · KoboldCpp 1.124.2 · 14 Nov 2026
Lite chat showing an uploaded picture of a red circle and a blue rectangle in the user turn
Lite with a vision model: the picture and the answer Triangles.gha-u24-lite-vision-reply-01 · gha-u24 · KoboldCpp 1.124.2 · 14 Nov 2026
Lite chat with the uploaded red circle and blue rectangle and the tiny vision model answering Triangles

Send images through the API

In /v1/chat/completions a message’s content is a list with a text part and an image_url part holding a data URI. This worked on every machine we tried (F27) and took 7.2 s with our tiny model on a CPU F89. The native /api/v1/generate takes base64 pictures in images with an <image> tag in the prompt, which worked in 6.4 s F89. Other routes (/sdapi/v1/interrogate, Ollama images) are untested. See the chat API and native API.

What our tests show

Our test picture holds a red circle, a blue rectangle and the text “KOBOLD 42”. SmolVLM-256M called them “circles and squares” and did not read the text F28. The same model, prompt and settings gave different answers on different machines F33, and in our latest run changing only --visionmaxres changed the answer to a formula: the answers of a 256M model are noise. The lessons: small models miss text and detail, judge a model on your own pictures, and do not trust one answer.

Audio input

Some models carry audio in the projector too (upstream, since 1.96; the audio flag of /api/extra/version). We did not test it. See voice.

When it does not work

Vision problems
<code>mismatch between text model (n_embd = 896) and mmproj (n_embd = 576)</code>, then <code>failed to load mmproj model!</code>, exit 3The projector belongs to another model <EvidenceBadge id="F89" />Download the projector from the same repository as the model.
The model answers as if it saw nothing, or invents a descriptionNo projector loaded: <code>vision</code> is false and the image is dropped, with no error <EvidenceBadge id="F89" />Check /api/extra/version; add --mmproj.
<code>key clip.vision… not found</code>Users report a projector that does not match or is outdatedGet the matching, current projector. Not reproduced by us.
Out of memory with the projectorThe projector takes GPU memory tooTry --mmprojcpu or a smaller --visionmaxres.

Old advice says vision does not work on Vulkan: that is from 2024, and we could not check it without a GPU.

Frequently asked questions

What is an mmproj file?

The projector that turns an image into tokens the language model can read. It is made for one model: SmolVLM's projector would not load with a Qwen model, and KoboldCpp said the embedding sizes (576 and 896) did not match.

Which vision models work with KoboldCpp?

Models that have a GGUF with a matching mmproj file. We ran SmolVLM-256M; larger families such as Gemma 3 and Qwen VL are named upstream but we did not run them.

Why does my model ignore the image?

Usually no projector is loaded. With no --mmproj, our image request returned 200 and a confident description, using only 27 prompt tokens: the picture was dropped without an error. Check that /api/extra/version says vision true.

How do I send an image through the API?

In /v1/chat/completions, add a content part of type image_url with a data URI. It took 7.2 s on our CPU with the tiny model. The native /api/v1/generate takes base64 images in an images field.

Does vision work on CPU?

Yes. Everything above ran on a CPU, and --mmprojcpu forces the projector onto the CPU on a machine with a GPU.

Does vision work with Vulkan or AMD?

Old advice says it does not. We have no GPU to check, so we cannot confirm or deny it for the current version.