KoboldCpp and llama.cpp: same engine, different jobs

How they are related

KoboldCpp’s history begins with llama.cpp’s first commits in March 2023; its first release was named llamacpp-for-kobold and was renamed in 1.0.8beta (upstream). It merges llama.cpp changes regularly. The licences differ: llama.cpp is MIT, KoboldCpp is AGPL-3.0 with MIT-licensed parts. KoboldCpp puts the engine in one file with stable-diffusion.cpp, whisper.cpp and text-to-speech backends around it (what it is).

KoboldCpp vs llama.cpp: measured on one PC

We ran the same two model files on one CPU-only Ubuntu machine with 4 vCPUs F107, and timed every tool the same way: through its OpenAI-style streaming endpoint, with the time to the first token giving prompt speed and the rest of the stream giving generation speed. The prompts changed every run so none could reuse a cache. On this machine KoboldCpp, llama-server and TextGen all started with 2 threads by default, so we ran each again with 4 set explicitly; the 4-thread columns are the matched comparison. The llama.cpp column is its own llama-server, build b12377.

With threads set to 4 for both, generation after the long prompt was 4% (0.5B) and 3% (1.5B) lower in KoboldCpp; after a short prompt it was 9% faster on the 0.5B and 4% slower on the 1.5B. Prompt processing was the clear gap: KoboldCpp read the long prompt 39% (0.5B) and 54% (1.5B) slower. llama-server was ready in 0.5 to 1 s against 3.5 to 5 s, and used more memory on the 1.5B model (probably its default of several slots). At default settings both used 2 threads and the generation speeds were close. This is a CPU test with a 0.5B and a 1.5B model on one machine type. GPU speed, bigger models and other CPUs can differ; absolute numbers moved by up to 2 times between GitHub machines in earlier runs F44, so compare only rows from the same table. A second, cruder check through llama-cpp-python is in the findings F92.

What KoboldCpp adds

• One file per operating system, no installer (download).
• A launcher with 11 tabs (launcher) and KoboldAI Lite (Lite).
• KoboldAI, OpenAI, Ollama and A1111-style routes on one port F23.
• Image, voice and vision, which we ran on a CPU: images, voice, vision.
• ContextShift and FastForward, .kcpps config files (config files), a Horde worker and a tunnel, autofit, and old GGML formats (upstream).

What llama.cpp does first or better

New model architectures land in llama.cpp first and reach KoboldCpp in a later release; an unknown architecture stops KoboldCpp with a clear message (model won’t load). We could not measure the lag: the llama.cpp repository was not reachable from our lab, so we publish no number. llama.cpp also offers backends KoboldCpp does not ship, such as SYCL for Intel Arc (users report; we have no Intel card), a lighter set of files, and the MIT licence.

Switching flags between them

KoboldCpp accepts llama.cpp-style short flags since 1.99: -m, -c, -ngl, -ts, -b, -fa, -ot and --n-cpu-moe (upstream). Differences that bite: KoboldCpp adds 256 cells to the context it allocates (a 4096 context showed 4352 in the log F96, F06), flash attention is on by default F09, and GPU layers default to automatic. Check a command in the command builder.

When to choose llama.cpp

You need a model released this week, the lightest possible server, SYCL for Intel Arc, or you embed an engine in your own program under the MIT licence. See llama.cpp’s own documentation for installing it.

When to choose KoboldCpp

You want one file with a launcher and a chat and story page, images, voice and vision in one program, several API styles on one port, and .kcpps files.

Which should you use?

• Intel Arc user: llama.cpp’s SYCL builds are reported; KoboldCpp reaches Intel through Vulkan.
• Developer embedding a server: llama.cpp.
• Beginner on Windows: KoboldCpp.
• Roleplay or story user: KoboldCpp, with Lite.
• You need a model released this week: llama.cpp first, KoboldCpp’s rolling build second.