Compile KoboldCpp from source

Do you need to build?

KoboldCpp is open source under the AGPL-3.0 licence (what it is).

What you need

Git, Python 3, and a C and C++ compiler. The project’s requirements file lists the Python packages: numpy, sentencepiece, transformers (with torch), gguf, customtkinter, protobuf, psutil, darkdetect and jinja2; the GUI needs customtkinter and Tk, and chat templates need jinja2. Running from source worked for us with a system Python 3.13 and no GPU libraries. On macOS you need the Xcode command line tools; on Windows, w64devkit (see below).

KoboldCpp from source on Linux: make with a backend

We cloned the repository (commit 6d2fedf, version 1.124.2) and ran make -j4:

git clone --depth 1 https://github.com/LostRuins/koboldcpp.git
cd koboldcpp
make -j4
python3 koboldcpp.py --model /path/to/model.gguf --skiplauncher

On our 4-core Linux machine with gcc 13.3 the CPU build took 451 seconds and produced koboldcpp_default.so and koboldcpp_common.so in the folder; starting koboldcpp.py then reported 1.124.2, picked the default backend and served a model F97.

For a build you copy to another machine add LLAMA_PORTABLE=1, which stops the compiler from tuning only for the build machine (from the README); we did not test it.

Linux: the koboldcpp.sh way

The koboldcpp.sh script uses a local micromamba environment to fetch dependencies and build: ./koboldcpp.sh builds as needed and opens the launcher (it needs X11), ./koboldcpp.sh rebuild refreshes after an update, and KCPP_PORTABLE=1 ./koboldcpp.sh dist packages a binary. We did not run it. untested

Intel Mac

There is no binary for Intel Macs (Mac), so the README sends them to a source build. We have no Intel Mac. untested Users report a CPU-only result.

Raspberry Pi and ARM Linux

The README says source builds also cover boards such as the Raspberry Pi. untested Users report models up to about 3B; we have no board. Expect the build to take much longer than our 451 seconds on a fast four-core machine.

Windows with w64devkit

Install the x64 version of w64devkit, open its terminal in the repository, run make (or make LLAMA_VULKAN=1) and start with python koboldcpp.py --model "C:\Models\model.gguf" (from the README). CUDA builds need Visual Studio, CMake and the CUDA Toolkit. We did not test Windows builds. untested

Run your build

Start koboldcpp.py with Python and the usual flags (flags). The launcher window needs customtkinter and Tk; --skiplauncher runs without it, as we did. --unpack <folder> extracts the files bundled in a release binary for inspection.

Convert and quantize models (the tools)

make -j4 quantize_gguf gguf-split built two 10 MB programs in 4 seconds after the main build F97. We tried both:

• Splitting: gguf-split --split-max-tensors 100 cut Qwen2.5-0.5B into three files in 0.7 s. KoboldCpp loaded it when pointed at part 1; part 2 alone failed with model must be loaded with the first split; with part 3 missing it failed to open that file F97. See Hugging Face downloads.
• Quantizing: quantize_gguf --allow-requantize made a Q4_0 file (241 MB) from a Q8_0 one (292 MB) in 0.6 s, and KoboldCpp generated text with it. Requantizing a quantized file lowers quality: start from a full-size file when you can.

Converting a safetensors model uses convert_hf_to_gguf.py from the repository, which needs a large Python environment; we did not run it. The project also publishes the tools as koboldcpp_tools.zip (tools).

Build errors