koboldcpp.dev

Run KoboldCpp in a Docker container

Use the official koboldai/koboldcpp image from Docker Hub: publish port 5001, mount a folder for models, pass the model and flags in KCPP_ARGS, and give the container your GPU (NVIDIA Container Toolkit for NVIDIA). The project describes the image as for experienced users, mainly on rented cloud GPUs. It downloads the newest KoboldCpp every time it starts and opens a public tunnel unless you set KCPP_DONT_TUNNEL=true. We ran it on a CPU-only machine; we could not test a GPU in a container.

Checked against KoboldCpp 1.124.2 · updated 13 Nov 2026
  • lab-lnx: Linux test VM · 4 vCPU · 15 GB · no GPU
How we test
On this page
  1. The official KoboldCpp Docker image in one minute
  2. Prerequisites
  3. Run it
  4. Docker Compose
  5. Check the GPU is used
  6. Security
  7. Problems

The official KoboldCpp Docker image in one minute

Image
koboldai/koboldcpp (tag latest)
Size
1.26 GB on disk after pulling (our measurement)
What it does at start
Downloads the newest KoboldCpp binary into /workspace, then runs it
Time to ready
12 s with the model on a mounted folder, CPU only

The entrypoint reads these variables (from the reference script in the project’s repository): KCPP_ARGS, KCPP_MODEL, KCPP_IMGMODEL, KCPP_MMPROJ, KCPP_WHISPERMODEL, KCPP_TTSMODEL, KCPP_EMBEDMODEL, KCPP_DONT_TUNNEL, KCPP_DONT_UPDATE, KCPP_REMOVE_FILES, KCPP_BIN_OVERRIDE, KCPP_GIT and KCPP_CONFIG_GIT. Community images exist (by korewaChino and noneabove1182) but may be out of date.

Prerequisites

Before you start 0 of 4 done

Run it

This is the command shape we ran (CPU only, local model, no tunnel):

Docker
docker run -d --name kcpp --network host \
-v /path/to/models:/models:ro \
-v /path/to/workspace:/workspace \
-e KCPP_DONT_TUNNEL=true \
-e KCPP_ARGS="--model /models/qwen2.5-0.5b-instruct-q4_k_m.gguf --port 5001 --contextsize 2048" \
koboldai/koboldcpp:latest

Use -p 127.0.0.1:5001:5001 instead of --network host on a normal setup. We used host networking because our sandbox has no Docker bridge.

The command field by field
-p 127.0.0.1:5001:5001Publishes the port to this PC only.
-v …:/models:roYour models folder, read only.
-v …:/workspaceWhere the container keeps the KoboldCpp file; use a named volume so it survives image updates.
KCPP_DONT_TUNNEL=trueStops the default public tunnel. Leave it out only if you want one.
KCPP_ARGSAll KoboldCpp flags: the model, context, GPU layers. Build them in the <a href="/tools/config-builder/">command builder</a>.

The container first downloaded the nocuda build (103 MB, version 1.124.2) into /workspace, then started it: ready after 12 s F95. On a second start with the same folder it downloaded the file again; set KCPP_DONT_UPDATE=true to reuse it (from the script). Models persist in the mounted folder, or with pointed at the volume (not tested).

Docker Compose

The project’s reference compose file (in its examples/docker-reference folder) passed docker compose config. It sets the image, a volume at /workspace, KCPP_DONT_TUNNEL=true, a KCPP_ARGS line, port 5001 and restart: unless-stopped, and reserves an NVIDIA GPU. A copy of it without the GPU reservation, with host networking and our local model, started and served the model; the restart policy read back as unless-stopped F95. Compose warns that its version: line is obsolete; remove it. Run it next to Open WebUI as described in other apps.

Check the GPU is used

Look for CUDA0 or Vulkan0 buffer lines in docker logs; a CPU-only run shows Initializing dynamic library: koboldcpp_default.so and no GPU buffers, which is what we saw. KoboldCpp does not fail when a GPU is missing F17. See not using the GPU. We did not test an NVIDIA or AMD GPU in a container.

Security

Problems

  • The container stalls for three minutes and exits: the script does this on an error so you can read it. Read the log above it; ours was the TLS error.
  • CUDA is not visible in the container: check the Container Toolkit and nvidia-smi inside the container (users report this; issue 1015).
  • Health checks: users report trouble with them (issue 1114); we did not test.
  • Image updates: the binary is fetched at start, so pulling the image is not how KoboldCpp updates.

Frequently asked questions

Is there an official KoboldCpp Docker image?

Yes: koboldai/koboldcpp on Docker Hub. The project's README says it is for experienced users, mainly for cloud GPU rentals, on x86-64 with an NVIDIA or AMD GPU. It was 1.26 GB on disk after we pulled it.

How do I pass my GPU to the KoboldCpp container?

For NVIDIA, install the NVIDIA Container Toolkit and run with --gpus all, then add --usecuda and --gpulayers to KCPP_ARGS. For AMD or Intel the project's compose example passes /dev/dri. We could not test a GPU.

How do I set launch flags in Docker?

Put them in the KCPP_ARGS environment variable, for example --model /models/file.gguf --contextsize 4096. We ran exactly that with a mounted model.

Does the image work with AMD GPUs?

The entrypoint switches to the ROCm build when it finds /dev/dri and no NVIDIA tools. We did not test AMD.

Does the container update itself?

It downloads the newest KoboldCpp binary into /workspace at every start. A second start downloaded again in our test; set KCPP_DONT_UPDATE=true to reuse the file.

Why is a tunnel opened without my asking?

The image was built for rented GPUs, so it starts a public Cloudflare tunnel unless KCPP_DONT_TUNNEL is true. The project's compose example sets it to true.