How koboldcpp.dev tests KoboldCpp
Every "we tested" on this site maps to a finding below and to raw results you can open. Automated runs on Windows, Linux and macOS machines without a GPU check behaviour, error messages and relative effects; speed, VRAM and model claims need real PCs, Macs and phones, and wherever we do not have one the page says so. Each result carries the KoboldCpp version and the date.
Checked against KoboldCpp 1.124.2 · updated 13 Nov 2026
- lab-lnx: Linux test VM · 4 vCPU · 15 GB · no GPU
- gha-w25: Windows Server 2025 test machine · 4 vCPU · 16 GB · no GPU
- gha-w22: Windows Server 2022 test machine · 4 vCPU · 16 GB · no GPU
- gha-u24: Ubuntu 24.04 test machine · 4 vCPU · 16 GB · no GPU
- gha-mac15: macOS 15 test VM (Apple Silicon) · 3 vCPU · 7 GB · paravirtual Metal
- sw-vulkan: Software Vulkan (Mesa lavapipe) · reports exact VRAM allocations, speeds meaningless
On this page
How we test KoboldCpp: our rules
- One fact, one source. A page may say “we tested” only for something listed in the findings below.
- Same inputs. A fixed model set and a fixed prompt set, so runs can be compared.
- Official files only, checked against the project’s SHA-256 values F01.
- Label the limits. Lab numbers are lab numbers; real hardware gets its own result with its own label.
- Three runs, median, wherever speed is quoted.
Every claim on the site carries one of three labels: tested by us, upstream says it (the project’s own documentation or release notes), or reported by users, with links.
Machines
The machines we have used so far:
lab-lnx
Linux test VM · 4 vCPU · 15 GB · no GPU
gha-w25
Windows Server 2025 test machine · 4 vCPU · 16 GB · no GPU
gha-w22
Windows Server 2022 test machine · 4 vCPU · 16 GB · no GPU
gha-u24
Ubuntu 24.04 test machine · 4 vCPU · 16 GB · no GPU
gha-mac15
macOS 15 test VM (Apple Silicon) · 3 vCPU · 7 GB · paravirtual Metal
sw-vulkan
Software Vulkan (Mesa lavapipe) · reports exact VRAM allocations, speeds meaningless
termux-docker
Termux container (Android userland, x86_64)
Planned, and not yet run: a Windows PC with an NVIDIA GPU, a high-memory NVIDIA PC, a PC with an AMD GPU, an old PC without AVX2, a laptop with two GPUs, a Mac, an Android phone, a Windows 11 virtual machine, Colab and a rented cloud GPU. Pages that depend on them say “users report” until their first result exists.
Test runs
run1 (test-data/run1-2026-11-13-local-linux/) | Our cloud Linux box (4 vCPU, 15 GB RAM, no GPU) | lab-lnx | Full harness: 34 cases (base, bench, threads, ctx, api, conf, feat, fail) + Lite and Swagger screenshots |
ci-1 (test-data/ci-1-2026-11-13-gha/) | GitHub Actions, dencyclopedia/test-KoboldCpp workflow lab.yml run #1 | lab-w25 (Windows Server 2025, 4 vCPU AMD EPYC, 16 GB), lab-w22 (Windows Server 2022, same class), lab-u24 (Ubuntu 24.04, 4 vCPU, 16 GB), lab-mac15 (macOS 15 VM on Apple Silicon, 3 vCPU, 7 GB, paravirtual Metal) | Full harness with the -nocuda build on each OS + koboldcpp.exe/-linux-x64 (CUDA builds) and -oldpc builds on the same machines; asset manifest; Windows launcher screenshots |
run2 (test-data/run2-2026-11-13-local-swvulkan/) | Agent container + Mesa lavapipe | sw-vulkan | Vulkan backend on a software GPU; Llama 3.1 8B allocations and CPU speed (F35–F37) |
lab2-1 (test-data/lab2-1-2026-11-13-gha/) | GitHub Actions lab2.yml run 1 (partial: cancelled after the fast jobs) | gha-w25, gha-u24 | Launcher tabs, Defender scans, CPU image/TTS, Termux container start (F38–F40) |
lab2-2 (test-data/lab2-2-2026-11-13-gha/) | GitHub Actions lab2.yml run 2 | gha-u24, gha-w25, sw-vulkan | Vulkan allocation sweeps on Linux and Windows, SillyTavern connection (F41–F43) |
lab2-3 (test-data/lab2-3-2026-11-13-gha/) | GitHub Actions lab2.yml run 3 | gha-u24, termux-docker | Runner comparison at defaults, Termux installer (F44, F45) |
lab2-4 (test-data/lab2-4-2026-11-13-gha/) | GitHub Actions lab2.yml run 4 | gha-u24 | Runner comparison with matched threads; SillyTavern URL history (F46, F47) |
lab2-6 (test-data/lab2-6-2026-11-13-gha/) | GitHub Actions lab2.yml run 6 | gha-u24 | Complete SillyTavern connection matrix with a chat reply (F48) |
r1-lnx (test-data/r1-lnx-2026-11-13/) | Agent container (Linux, 4 vCPU, 15 GB, no GPU) | lab-lnx | Round 1 error reproductions with lab/scripts/err_repro.py (T-ERR-03, -05, -08, -10, -11): F49–F53 |
r2-lnx (test-data/r2-lnx-2026-11-14/) | Agent container (Linux, 4 vCPU, 15 GB, no GPU) | lab-lnx | Round 2 runs written while building the Tune and Connect pages: ContextShift, SmartCache and SWA chats (lab/scripts/ctx_shift.py), CPU batch size and draft model (perf_extra.py), OpenAI and Anthropic SDK probe, connection audit, Lite probe, Linux download checks: F59–F73 |
ci-2 (test-data/ci-2-2026-11-13-gha/) | Same workflow, run #2 | same | Repeat of ci-1 + multimodal feature group on every OS + macOS --usecpu comparison (see F27–F29, F33) |
The harness is a Python script (kcpp_lab.py) that starts the official binary with a fixed list of cases, writes one folder per case with a machine-readable result and the server log, and is run on GitHub Actions for the Windows and macOS machines. Case groups: base, bench, threads, ctx, api, conf, feat and fail. The test model is Qwen2.5-0.5B-Instruct (Apache-2.0) in four quantisations, with SmolVLM-256M, Whisper tiny.en and all-MiniLM-L6-v2 for the multimodal cases. The lab scripts are public in dencyclopedia/test-KoboldCpp for as long as it exists.
Findings
Every finding has a permanent number and anchor (for example #F36): it is never renumbered. If a later version changes one, a dated line is added under it.
F01 The 1.124.2 release has exactly seven official binaries, with these sizes and hashes
koboldcpp.exe 594.0 MB, koboldcpp-nocuda.exe 85.7 MB, koboldcpp-oldpc.exe 458.2 MB, koboldcpp-linux-x64 599.8 MB, koboldcpp-linux-x64-nocuda 102.4 MB, koboldcpp-linux-x64-oldpc 454.0 MB, koboldcpp-mac-arm64 65.2 MB. SHA-256 for each is in ci-1-2026-11-13-gha/assets/assets-v1.124.2.json. Names koboldcpp-mac-x64, koboldcpp-linux-arm64, koboldcpp_cu12.exe, koboldcpp_nocuda.exe return 404 for this version (probe on run1). Evidence: assets job of ci-1; run1 probe log in this file's history.
F02 The CUDA builds run on machines with no NVIDIA GPU; they silently fall back to CPU and start 2–4× slower
koboldcpp.exe on Windows Server 2022/2025 (no NVIDIA GPU): ready in 11.7 s / 12.3 s vs 6.5 s / 6.6 s for koboldcpp-nocuda.exe; generation 42.3 / 36.6 tok/s vs 47.4 / 43.2 tok/s. Linux koboldcpp-linux-x64: ready 9.1 s vs 3.5 s. No error is shown. Implication for P03: a wrong download still works but costs ~500 MB and start-up time. Results: R-ci1-w22-cuda-base-q4_k_m, R-ci1-w25-cuda-base-q4_k_m, R-ci1-u24-cuda-base-q4_k_m vs the -nocuda rows.
F03 The `oldpc` builds are much slower on a modern CPU, because they always load the no-AVX2 library
koboldcpp-oldpc.exe logs Initializing dynamic library: koboldcpp_noavx2.dll on AVX2-capable EPYC CPUs; generation 28.2 tok/s (w22), 23.2 (w25), 28.7 (u24) vs 47.4 / 43.2 / 46.3 with -nocuda; ready 10.2–25.4 s. Download it only for CPUs without AVX2 or old NVIDIA GPUs (CUDA 11.4 build, upstream facts §4). Results: R-ci1-*-oldpc-base-q4_k_m.
F04 First start includes unpacking the single file; small builds are ready in 3.5–7 s with a 0.5B model
Time from process start to a working API (/api/v1/model answers): nocuda builds 3.5 s (Linux) to 6.6 s (Windows), macOS 5.2–6.6 s; the 600 MB CUDA builds 9–12 s; oldpc up to 25 s. Results: every base-* row, column ready_s.
F05 Defaults in 1.124.2: context 16384, GPU layers auto, flash attention on, chat adapter AutoGuess, port 5001, default generation length 2048
The server log's Namespace(...) line and /api/v1/config/max_length (2048) on every machine; base-default-args allocates n_ctx = 16640. Results: R-run1-lnx-nocuda-base-default-args, R-run1-lnx-nocuda-api-all (api.json).
F06 KoboldCpp adds 256 tokens to the context you ask for
Requested 4096 → n_ctx = 4352; 6144 → 6400; 16384 → 16640; 32768 → 33024; 131072 → 131328 (log line llama_context: n_ctx). Readers see slightly larger KV-cache numbers than a naive calculation. Results: ctx-*, conf-load, base-default-args on all machines.
F07 Auto threads uses half the vCPUs on these machines, leaving speed unused
With no --threads, the log says Threadpool set to 2 threads and 2 blasthreads on every 4-vCPU x86 runner (1 thread on the 3-vCPU mac VM). Generation with 4 threads vs auto (2): lab-lnx 24.6 vs 18.9 tok/s (+30%), w22 60.8 vs 48.4 (+26%), w25 49.9 vs 44.2 (+13%), u24 48.7 vs 46.0 (+6%). 1 thread: 11.0 / 33.8 / 26.5 / 26.1. Real desktops with SMT may behave differently: re-test as T-PERF-03 before advising a rule. Results: threads-*, base-default-args.
F08 `.kcpps` export/import round-trips exactly
--exportconfig lab.kcpps (exit 0) writes a JSON file with every argument (model path, contextsize: 6144, threads: 2, gpulayers: -1, …); --config lab.kcpps starts a server with the same settings (n_ctx = 6400). Identical on Windows, Linux and macOS. Results: conf-export, conf-load.
F09 Flash attention is already on by default; `--flashattention` changes nothing
llama_context: flash_attn = enabled appears without the flag, and peak RAM at 32K context is identical with and without it (976 MB lab-lnx, 957 MB w22). Results: ctx-32768 vs ctx-32768-fa.
F10 KV-cache memory matches the standard formula, and KV quantisation saves a lot of RAM at long context
Logged KV sizes for Qwen2.5-0.5B (24 layers, 2 KV heads × 64 dims): 51 MiB at 4352 cells, 195 MiB at 16640, 1539 MiB at 131328 (all f16), 108.8 MiB at 33024 with --quantkv q4_0. That is exactly 2 (K and V) × layers × cells × KV heads × head size × bytes per element, so the T2 calculator formula is validated. Peak RAM at 32K context: f16 976 MB → q8_0 794 MB → q4_0 698 MB (lab-lnx; same deltas ±3 MB on every OS). Results: ctx-*.
F11 Context size drives RAM, not generation speed, for short prompts
Generation stays within ±10% from 2K to 128K context on CPU; peak RAM rises 616 → 688 → 976 → 2123 MB for 2K / 8K / 32K / 128K (lab-lnx). Results: ctx-*.
F12 mmap is off by default; turning it on lowers the reported working set on Windows only
Peak RSS with --usemmap 530 MB vs 620 MB (w22) and 533 vs 623 (w25); unchanged on Linux and macOS. Results: mmap-on vs fail-gpulayers-no-gpu (same context).
F13 On CPU, Q8_0 processes prompts faster than Q4_K_M, and generates at about the same speed
Long-prompt (1650 tokens) processing: u24 133 vs 111 tok/s, w22 173 vs 133, w25 153 vs 113, lab-lnx 136 vs 113. Generation: within ±5%. F16 generates 25–35% slower than Q4_K_M on x86 (30.3 vs 46.3 u24). Q2_K is not faster than Q4_K_M at generation (46.3 vs 46.3 u24). Use: P22 quantization page ("smaller is not automatically faster on CPU"). Results: base-q2_k, base-q4_k_m, base-q8_0, base-fp16 on all x86 machines.
F14 RAM use ≈ file size + a fixed overhead
Peak RSS at 4K context: Q2_K 613 MB (file 396 MiB), Q4_K_M 686 (463 MiB), Q8_0 861 (644 MiB), F16 1434 (1208 MiB) on u24. Results: base-*.
F15 `--failsafe` and `--noavx2` work, at a large speed cost
Failsafe: 5.2–5.9 tok/s vs 43–49 (x86, 8× slower); --noavx2: 27.7–31.7 vs 43–49 tok/s on the GitHub runners (−33 to −36%), 14.8 vs 18.5 on lab-lnx (−20%). On the macOS VM, failsafe (CPU) is *faster* than the default Metal path (F29). Results: fail-failsafe, fail-noavx2.
F16 Built-in `--benchmark` fills the whole context and writes one CSV row
With --contextsize 4096 it processes ~4000 tokens then generates 100; output columns Timestamp, Backend, Layers, Model, MaxCtx, GenAmount, ProcessingTime, ProcessingSpeed, GenerationTime, GenerationSpeed, TotalTime, Output, Flags. Example lab-lnx Q4_K_M: 97.8 tok/s processing, 16.3 tok/s generation (lower than chat-style tests because the context is full). Results: bench-*.
F17 GPU flags on a machine without a GPU do not fail, and the log can say "offloaded N/N layers to GPU" while everything runs on the CPU
With --usecuda (nocuda build), --usevulkan (no Vulkan device) or --gpulayers 99, the server starts and generates at CPU speed; the log prints load_tensors: offloaded 25/25 layers to GPU but the KV cache line says CPU KV buffer size. Readers must check for CUDA0/Vulkan0/MTL0 buffer lines, not the "offloaded" line, to know where the model really is (P48 not using GPU, P45 errors, T4). Results: fail-usecuda-on-nocuda, fail-usevulkan-no-gpu, fail-gpulayers-no-gpu on all machines.
F18 Without a detectable GPU, the log says so in three lines
No GPU or CPU backend was selected. Trying to assign one for you automatically..., Unable to detect VRAM., Auto Selected Default Backend (flag=0) (Linux: koboldcpp_default.so, Windows: koboldcpp_default.dll). Used by the log analyzer (T4). Results: base-default-args.
F19 Missing model file:
Cannot find text model file: <path>, exit code 2, within 5–10 s. Results: fail-missing-model.
F20 A file that is not GGUF:
load_model: error: invalid or unsupported model file '<path>', then Load Text Model OK: False and Could not load text model: <path>, exit code 3. Results: fail-not-gguf.
F21 A truncated (incomplete) download:
llama_model_load: error loading model: tensor 'output.weight' data is not within the file bounds, model is corrupted or incomplete, exit code 3. This is the message users see after an interrupted download (P49 model won't load, P45, T4). Results: fail-truncated.
F22 Out-of-range context size:
rejected before loading: error: argument --contextsize/--ctx-size/-c: must be within [256, 524288], exit code 2. Results: fail-ctx-too-big.
F23 One server answers four API families at once
On every machine, 23 of 25 probes returned 200: KoboldAI (/api/v1/generate, /api/v1/model, /api/v1/config/*, /api/extra/*), OpenAI (/v1/models, /v1/completions, /v1/chat/completions, /v1/embeddings), Ollama (/api/tags, /api/generate, /api/chat), A1111 (/sdapi/v1/sd-models returns [] without an image model); streaming works for /api/extra/generate/stream and OpenAI stream: true (SSE). The two 404s are expected: /swagger.json (the spec lives at /api?json=1) and an invented path, which returns Error: KoboldCpp HTTP Server is running, but this endpoint does not exist. Please check the URL and METHOD. Results: api-all (api.json).
F24 API version gotchas
/api/v1/info/version returns 1.2.5 (the KoboldAI API version, not KoboldCpp's); the KoboldCpp version is at /api/extra/version ("version": "1.124.2", plus capability flags llm, vision, transcribe, txt2img, tts, embeddings, jinja…). /v1/embeddings without an embeddings model returns 200 with an empty vector and "model": "inactive": apps must check the vector length. Results: api-all.
F25 `--password` protects generation, not discovery
Without the key or with a wrong key: /api/v1/generate and /v1/chat/completions return 401; with Authorization: Bearer <key>: 200. But /api/v1/model and the web UI page (/) stay reachable without the key (200). The help text adds that image endpoints are not secured. Use: P38 remote access ("a password is not a firewall"). Results: api-password.
F26 The default request queue serves one request at a time; `--parallelrequests 2` serves two at once but does not add throughput on CPU
Two simultaneous 64-token requests: default → 1.69 s and 3.15 s (second waits) on u24; --parallelrequests 2 → 3.28 s and 3.28 s. Total time is the same on CPU; the benefit is fairness for multi-user setups. GPU behaviour must be tested (T-PERF-08). Results: api-multiuser-1, api-parallel-2.
F27 Vision, Whisper and embeddings all work from one CPU-only server
One process with --mmproj, --whispermodel and --embeddingsmodel reports vision: true, transcribe: true, embeddings: true at /api/extra/version. lab-lnx: image description via /v1/chat/completions with an image_url data URI in 4.6 s; Whisper tiny.en transcribed the 11 s JFK sample word for word in 1.2 s via /v1/audio/transcriptions (multipart file); embeddings via /v1/embeddings returned 384-dimension vectors with sensible similarity (cat/kitten 0.659 vs cat/stock-market 0.077). ci-2 repeated it on every OS: all capabilities loaded everywhere; Whisper transcribed the sample correctly on all five machines (0.9 s w22, 1.7 s u24, 1.9 s w25, 15–16 s on the mac VM); embedding similarity was identical to three decimals (0.657–0.659); image description took 3.8–6.6 s. Results: feat-multimodal in run1 and ci-2 (all machines, plus macos-15-arm64-usecpu).
F28 A tiny vision model sees shapes but misses details
SmolVLM-256M described our test image (red circle, blue rectangle, text "KOBOLD 42") as "circles and squares" and did not read the text. Model choice, not KoboldCpp, limits vision quality: P41 must recommend tested larger vision models (T-MM-02 with Gemma 3 4B on CPU).
F29 On GitHub's macOS VM the Metal path is slower than the CPU, so VM numbers say nothing about real Macs
The mac build uses Metal by default (MTL0 buffers). In the VM, Q4_K_M generation was 4.3–5.7 tok/s with Metal vs 7.5–10.1 tok/s with --usecpu (ci-2) and 44.7 tok/s in failsafe mode (ci-1); Q8_0 4.6–4.9 vs 9.1–10.3. VM CPU scheduling is also noisy (three runs vary by up to 30%). Real Apple Silicon must be measured on a real Mac (T-MAC-02). Results: R-ci1-mac15-*, R-ci2-mac15-*, R-ci2-mac15-usecpu-*.
F30 The Windows launcher opens with 11 tabs; Quick Launch defaults are Launch Browser, ContextShift and FlashAttention on, context 16384
Tabs: Quick Launch, Hardware, Context, Loaded Files, Network, Horde Worker, Image Gen, Audio, Agent, Admin, Extra. Quick Launch also shows Backend ("Use CPU" on a GPU-less machine), Use MMAP, Remote Tunnel, Quiet Mode, Force AutoFit, Use Jinja, a GGUF Text Model field with Browse and HF Search buttons. Shots lab-w25-launcher-quick-01, lab-w22-launcher-quick-01 (bottom buttons cut off by the runner's 1024×768 screen: retake on a real PC, T-GUI-01).
F31 KoboldAI Lite (v353, embedded) greets first-time visitors with a UI-style picker that must be closed first
"Welcome To KoboldAI Lite" offers Classic, Aesthetic and Corpo UI, plus "Launch llama.cpp UI | Launch StableUI | Launch MusicUI"; behind it, Lite says "Instruct Mode Selected", names the model and the endpoint http://localhost:5001. A prompt typed before choosing is not sent. Shots lab-lnx-lite-welcome-01, lab-lnx-lite-firstprompt-01.
F32 The `/api` page is an interactive Swagger UI titled "KoboldCpp API 2026.11.01" grouping OpenAI-compatible and Anthropic endpoints as "v1, recommended for beginners"
Shot lab-lnx-api-swagger-01.
F33 The same model, prompt, seed and temperature 0 can give different answers on different machines
In ci-2, SmolVLM-256M answered our image prompt with "The shapes are circles and squares." on Linux (twice, run1 and ci-2) but with a LaTeX-like fragment (\red\quad\blue\quad…) on Windows Server 2022, Windows Server 2025 and the macOS VM. Floating-point differences between CPU code paths change which token wins when probabilities are close, and a tiny model is especially sensitive. Use: P25 samplers ("seed is not portable"), P61 methodology (quality tests must run on the named machine). Results: R-run1-lnx-nocuda-feat-multimodal, R-ci2-*-feat-multimodal.
F34 The quant name does not tell you the real size: the load log prints the true bits per weight
For Qwen2.5-0.5B the log line print_info: file size = … (… BPW) read Q2_K 390.28 MiB (5.20 BPW), Q4_K_M 462.96 MiB (6.16 BPW), Q8_0 638.74 MiB (8.50 BPW), F16 1.17 GiB (16.00 BPW), identical on every machine. Small models keep large embedding and output tensors at higher precision, so "2-bit" and "4-bit" files are much bigger than the name suggests; for 7B+ models the gap shrinks (measure in T-PERF-05). Use: P22 quantization ("read the BPW line"), T2 (use real file sizes, not quant-name maths), T4 (extract BPW). Results: base-q2_k, base-q4_k_m, base-q8_0, base-fp16 on all machines (stdout.txt).
F35 KoboldCpp ignores a software Vulkan device unless you select it explicitly; selected, the real Vulkan backend runs and offloads layers
With Mesa 25.2.8 lavapipe (llvmpipe, Vulkan 1.4.318) installed, --usevulkan logs ggml_vulkan: No devices found. and runs on CPU (KV and compute on CPU, offloaded 25/25 layers to GPU still printed: compare F17). With GGML_VK_VISIBLE_DEVICES=0 the log shows ggml_vulkan: Found 1 Vulkan devices, using device Vulkan0 … 15464 MiB free, Vulkan0 model buffer size, and partial offload works (offloaded 12/25 → 106 MiB on Vulkan0; all layers → 236 MiB; output.weight stays on CPU). Use: the agent's GPU-path test environment (testing guide §3.1 sw-vulkan), P29/P48 (why "Unable to detect VRAM" and CPU-type devices are skipped; the same filter applies to real systems that only expose software Vulkan). Logs: vgl0.log, vgl12.log, vgl99.log, qwen05b-vulkan-novisibledevice-gl0.log. Software-Vulkan speeds (1.5–20 tok/s) are meaningless and never published.
F36 Exact GPU memory needed by Llama 3.1 8B Instruct Q4_K_M (all 33 layers offloaded)
Measured from Vulkan allocation lines: weights 3,992.5 MiB on the device (+ 692.8 MiB token-embedding table kept in system RAM, which llama.cpp-based runners always leave on the CPU), KV cache 1,056 MiB at 8K context (8,448 cells) and 4,128 MiB at 32K (33,024 cells), device compute buffer 124.3 MiB at 8K and 148.3 MiB at 32K, plus a 266.5 MiB host-side compute buffer in system RAM. Totals on the GPU: ≈ 5,173 MiB (5.05 GiB) at 8K and ≈ 8,269 MiB (8.08 GiB) at 32K. The KV figures equal 2 × 32 layers × cells × 8 KV heads × 128 × 2 bytes exactly (same formula as F10). The file is 4.58 GiB, 4.89 BPW (F34). Use: P21, P26, P27, P47, T2 calibration ("an 8B Q4_K_M with 8K context needs about 5.1 GiB of VRAM; with 32K about 8.1 GiB: fits a 12 GB card, not an 8 GB card"), labelled "calculated from measured allocations". Logs: l8b-ctx8192.log, l8b-ctx32768.log.
F37 An 8B model on 4 CPU cores: usable but slow, and it can be confidently wrong
Llama 3.1 8B Q4_K_M, CPU only, 4 threads, lab-lnx: ready in 55–65 s, prompt processing 16.9 tok/s, generation 5.3 tok/s (about reading speed). Asked what a GGUF file is, it answered that GGUF is a Google Groups Usenet file format: wrong. Use: P05 ("CPU only: is it usable?"), P21 (why our model list scores answers instead of trusting names), P37/P34 (verify outputs in apps). Log: l8b-cpu.log, session.txt.
F38 Windows Defender finds no threats in the official Windows binaries
On Windows Server 2025 (Defender engine 4.18.26100.4, signatures 1.463.283.0, 13 Nov 2026), a custom MpCmdRun -Scan -ScanType 3 of koboldcpp.exe (SHA-256 E306B58F…) and koboldcpp-nocuda.exe (C66550F4…) reported "found no threats"; Get-MpThreatDetection was empty. This is a point-in-time result for these exact hashes, not a guarantee for future signatures or other antivirus products. Use: P06, P08 (the measured counterpart to users' antivirus reports). Results: w25-defender/.
F39 The launcher can be driven and photographed automatically; at 100% scaling on 1920×1080 its window is 606×629 px
win_launcher_tabs.py opened koboldcpp-nocuda.exe, clicked each of the 11 tabs and captured the full window including the bottom bar (Update, Save Config, Load Config, Get Help, Launch). Defaults seen without a GPU: Hardware tab Backend "Use CPU", Threads 2, Batch Size 512, Launch Browser and Use FlashAttention ticked, Use MMAP off; Image Gen tab lists Image Model, Compress Weights, ImgThreads, Clamp Resolution, Image LoRAs, Image LLM, Clip-1/Clip-2, PhotoMaker, Upscaler, Image VAE, Audio VAE, Automatic VAE, Conv2D Direct, VAE tiling threshold 512, SD Flash Attention, Model Offload, VAE/CLIP device, VRAM Limiter. Use: P17 LauncherMap (our own shots, all tabs), P26/P28 (defaults), P40 (image settings). Shots: gha-w25-launcher-*-01.
F40 Image generation and text-to-speech work on a CPU-only machine with small models
One KoboldCpp process with --sdmodel sdxs-512-tinySDdistilled_Q8_0.gguf --ttsmodel Kokoro_no_espeak_Q4.gguf (no text model) was ready in 5 s and reported txt2img: true, tts: true, llm: false. A 512×512 1-step image took 8.4 s via /sdapi/v1/txt2img and 7.7 s via OpenAI /v1/images/generations; Kokoro spoke a 92-character sentence in 6.1 s via /v1/audio/speech (300 KB WAV); /v1/audio/voices listed 7 voices (kobo, cheery, sleepy, shouty, chatty, random, instruct). Use: P40, P42, P34, P02 (feature table), P05 (no GPU needed for basic media). Results: u24-media/media.json, shots gha-u24-sd-*, audio gha-u24-tts-kokoro-01.wav.
F41 Layer offload moves weights to the GPU in equal steps, identically on Linux and Windows, and auto offloads everything that fits
Software Vulkan (Mesa 26.2.6 lavapipe on Windows Server 2025; Mesa on Ubuntu 24.04), Qwen2.5-0.5B Q4_K_M, 4K context: --gpulayers 0 → 0 MiB on Vulkan0 (325.0 MiB in a pinned Vulkan_Host buffer + 137.9 MiB CPU); 6 → 48.6 MiB; 12 → 106.4 MiB; 18 → 166.4 MiB; 99 or auto (-1) → 235.8 MiB (all 25 layers; the remaining 89.3 MiB Vulkan_Host + 137.9 MiB CPU are the token-embedding/output tensors kept in system RAM). Values are byte-identical on both OSes. Without GGML_VK_VISIBLE_DEVICES the same command logs No devices found on Windows too (F35). Use: P26 (what one more layer costs), P48, P29, T2 per-layer model. Results: u24-vk-gl*, w25-vk-gl*.
F42 What context, KV quantisation and `--lowvram` do to GPU memory (measured allocations)
Same model, all layers on the device: KV cache 51 MiB at 4K, 99 MiB at 8K, 387 MiB at 32K; with --quantkv q4_0 at 32K 108.8 MiB (−72%); compute buffer 39.8 MiB (4K) → 43.8 (8K) → 67.8 (32K); --lowvram keeps the 99 MiB KV cache in system RAM (CPU KV buffer size) and the device compute buffer grows slightly (47.9 MiB); Q8_0 weights take 362.9 MiB vs 235.8 MiB for Q4_K_M. Together with F36 (8B) these calibrate T2. Results: u24-vk-alloc/*.
F43 SillyTavern's KoboldCpp connection: what it really accepts and what confuses people
SillyTavern 1.19.0 (release branch, 13 Nov 2026) on first start shows a "Welcome to SillyTavern!" dialog that requires a persona name before anything else works. In API → Text Completion → API Type "KoboldCpp", the URL field shows the example http://127.0.0.1:5001; entering http://127.0.0.1:5001/v1 still connected (green status with the model name koboldcpp/qwen2.5-0.5b-instruct-q4_k_m), so the "/v1 breaks it" advice is wrong for this mode; a wrong port did not connect. The field labelled "koboldcpp API key (optional)" shows a red ✗ "Missing key" even when the server has no password and the connection works, a likely source of "koboldcpp api key" searches (F25: a key is only needed with --password). Use: P36, P33, P59/T5 presets, P45 (non-error). Shots: gha-st-firstrun-01, gha-st-connect-01/02, gha-st-textcomp-v1-suffix-01. Completed in F48.
F44 Out of the box on CPU, KoboldCpp generated about 30% slower than llama.cpp and Ollama, and took longer to start
Same machine (GitHub Ubuntu 24.04, 4 vCPU), same file (Qwen2.5-0.5B Q4_K_M), 4K context, greedy 128 tokens, three runs each, each tool's own timing fields: KoboldCpp 1.124.2 27.0 tok/s generation / 137 tok/s prompt (short) and 23.7 / 146 (1,650-token prompt), ready 3.5 s; llama.cpp llama-server b12362 39.4 / 215 and 31.2 / 227, ready 0.5 s; Ollama 0.40.16 39.0 / 219 and 32.1 / 229, model load 0.8 s. Install footprint: KoboldCpp single file 103 MB; llama.cpp release folder 57 MB. Caveat measured separately (F07): KoboldCpp's auto setting uses 2 of the 4 vCPUs while the others use all cores; the matched-threads comparison is lab2 run 4 (koboldcpp --threads 4 row). Use: P51, P53, P50 (honest "defaults vs tuned" table), P28 (set threads). Results: u24-compare/compare.json, versions.txt, install-sizes.txt.
F45 The official Android installer builds KoboldCpp in Termux; running it needs a real Android system
In the official termux/termux-docker:x86_64 image (uname -o = Android), android_install.sh 2 (install without a model) installed wget/git/python with pkg, cloned the repository, compiled with make -j 2 and ended with "Your KoboldCPP Installation is Complete!" (exit 0; koboldcpp_default.so 3.9 MB, koboldcpp_common.so 30.8 MB; whole job 11 min). Starting it then failed with dlopen failed: cannot locate symbol "__android_log_write" because the container lacks Android's system log library: Termux in Docker can verify install and build steps, the Android emulator (testing guide §3.1 emu-android) is needed for running it. Use: P11 (installer steps and timing), testing guide. Logs: termux-docker/install.log, run.log.
F46 With matched threads, KoboldCpp generates as fast as llama.cpp and Ollama on CPU, but reads prompts 25–40% slower; and GitHub runners differ so much that results may only be compared within one run
Same runner, same file as F44, three runs each: generation (short / long prompt) KoboldCpp defaults 49.0 / 37.9 tok/s, KoboldCpp --threads 4 54.9 / 39.7, llama.cpp b12362 55.9 / 40.7, Ollama 56.8 / 43.5; prompt processing KoboldCpp 148.8 / 145.9 vs llama.cpp 198.6 / 236.4 and Ollama 207.0 / 239.7. The same KoboldCpp defaults case ran at 27.0 tok/s on the run-3 runner and 49.0 on this one: absolute numbers depend on which CPU GitHub assigns, so every comparison table uses rows from a single run, and machine.json/lscpu is recorded with each run. Use: P51, P53, P28 ("set threads to your core count"), P50. Results: u24-compare/compare.json.
F47 SillyTavern remembers every URL you try and shows them as a dropdown that can cover the Connect button
After a few connection attempts the API URL field opens a history list (http://localhost:5001, http://127.0.0.1:5002/, http://127.0.0.1:5001/v1) directly over Connect; press Escape or click elsewhere first. The plug icon turns red with a "!" and the chat box says "Not connected to API!" while disconnected. Use: P36 troubleshooting. Shot: gha-st-url-history-01.
F48 The complete SillyTavern ↔ KoboldCpp connection matrix (SillyTavern 1.19.0, KoboldCpp 1.124.2)
Text Completion → API Type KoboldCpp: http://127.0.0.1:5001 ✔, http://localhost:5001 ✔, http://127.0.0.1:5001/v1 ✔ (tolerated), wrong port ✘ "No connection...", https://127.0.0.1:5001 against a normal http server ✘. A chat message sent through the connected backend returned a reply ("Good morning!"). Chat Completion → Custom (OpenAI-compatible): SillyTavern reports "Valid" and lists the loaded model both for http://127.0.0.1:5001/v1 and for the base URL without /v1 (its own hint says to add /v1 if it doesn't work; generation over this route not yet exercised: T-INT-01 extension). In every mode the optional API-key field shows "Missing key" when empty (F43). Use: P36 (exact settings table, every shot ours), P33, T5 SillyTavern preset (known-good values), P45 ("No connection..." causes). Results: lab2-6-2026-11-13-gha/u24-sillytavern/st-results.json.
F49 A second instance on a busy port prints a normal start-up banner, then dies with the PyInstaller wrapper error; the first instance keeps running
With one server already on the port, the second process logged Starting Kobold API on port N at http://localhost:N/api/, then a traceback ending OSError: Cannot start the server: no socket could bind. and [PYI-…:ERROR] Failed to execute script 'koboldcpp' due to unhandled exception!, and exited with code 1 within 25 s; the first server was unaffected. This is one real cause of the generic "Failed to execute script" string (the real reason is the line above it). Use: P45, P46, T4 (rule: no socket could bind), P36. Results: err-port-in-use.
F50 A projector that does not belong to the text model is rejected at load with the embedding sizes in the message, exit code 3
Qwen2.5-0.5B (n_embd 896) with the SmolVLM-256M mmproj (n_embd 576): mtmd_init_from_file: error: mismatch between text model (n_embd = 896) and mmproj (n_embd = 576), hint: you may be using wrong mmproj, gpttype_load_model: error: failed to load mmproj model!, exit code 3; no server starts. Use: P41, P45, P49, T4. Results: err-wrong-mmproj.
F51 Starting the launcher on a machine with no display prints a clear message and exits with code 2
With DISPLAY and WAYLAND_DISPLAY unset and no arguments: Warning, GUI failed to start: Reason: no display name and no $DISPLAY environment variable, File selection GUI unsupported., customtkinter python module required!, then You must use the command line instead, e.g. python ./koboldcpp.py --help, exit code 2. Fix: pass --model (and --skiplauncher). Use: P09 Linux, P46, P45, P14 Docker/headless. Results: err-gui-no-display.
F52 KoboldCpp tolerates most wrong URL shapes; only a wrong method or a wrong `/api` path returns the "endpoint does not exist" text
Probed on 1.124.2: GET /v1/v1/models, /models, /v1/models all return 200 with the model list; POST /chat/completions (no /v1) and POST /api/v1/completions work; trailing slashes are accepted (/api/v1/model/ → 200). These return 404 with Error: KoboldCpp HTTP Server is running, but this endpoint does not exist. Please check the URL and METHOD.: GET /v1/chat/completions (needs POST), GET /api/v1/v1/model, GET /api/models. So a 404 from KoboldCpp usually means the wrong method or a path under /api/ that does not exist, not a missing /v1. Use: P33, P34, P36 (matches F43/F48), T5, P45. Results: err-api-urls.
F53 A client that allows a larger context than the server silently loses the start of its prompt: no error, no warning
Server --contextsize 2048 (allocated n_ctx = 2304, F06), client request with max_context_length: 4096, max_length: 16 and a prompt of about 2,700 words: HTTP 200, prompt_tokens: 2032 (= 2048 − 16), finish_reason: length; the server log shows only CtxLimit:2048/2048, Processed:2032. Everything the model never saw is the beginning of the prompt. Use: P27, P36 (set the app's context to the server's), P45 ("model forgets my character card"), P25. Results: err-context-mismatch.
F54 The Vulkan "force a small maximum allocation" environment variables did not change anything on the software GPU, so we could not reproduce a Vulkan out-of-memory error
With GGML_VK_FORCE_MAX_ALLOCATION_SIZE (50 MB and 10 MB) and GGML_VK_FORCE_MAX_BUFFER_SIZE (50 MB) set on Mesa lavapipe, a 0.5B Q4_K_M model still allocated the same 235.76 MiB on Vulkan0 and started normally (same lines as the baseline). ErrorOutOfDeviceMemory strings therefore stay "users report" (links in errors.json) until a real GPU run. Results: vk-baseline-ok, vk-max-allocation-*, vk-max-buffer-50mb.
F55 When the KV cache does not fit in system RAM, the load fails with four ggml lines, then `Could not load text model`, exit code 3. A 524,288-token context needs 6,147 MiB of KV cache even for a 0.5B model
Reproduced by limiting the process address space (prlimit --as=4000000000, an emulation of a machine without enough free RAM): ggml_aligned_malloc: insufficient memory (attempted to allocate 6147.00 MB), ggml_backend_cpu_buffer_type_alloc_buffer: failed to allocate buffer of size 6445596672, llama_init_from_model: failed to initialize the context: failed to allocate buffer for kv cache, gpttype_load_model: error: failed to load model '<path>', Could not load text model: <path>. Without the limit the same command starts: n_ctx = 524544, CPU KV buffer size = 6147.00 MiB, CPU compute buffer 549.52 MiB (matches the F10 formula). The 1.5 GB limit case failed earlier with a Python thread error and is not a representative OOM: not published. Use: P47, P45, P27 (huge contexts), T4. Results: host-oom-huge-context, host-oom-huge-context-nolimit.
F56 The CUDA builds fail at start on a machine with no NVIDIA driver when `--usecuda` is set: exit code 1, with a missing-library message that depends on the OS (found while building the log analyzer; rows from ci-1 and ci-2)
Linux koboldcpp-linux-x64 on Ubuntu 24.04 without a driver: OSError: libcuda.so.1: cannot open shared object file: No such file or directory, then PyInstallerImportError: Failed to load dynlib/dll '…/koboldcpp_cublas.so', then [PYI-…:ERROR] Failed to execute script 'koboldcpp' due to unhandled exception!, exit 1 after 7 s. Windows koboldcpp.exe on Windows Server 2022 and 2025: FileNotFoundError: Could not find module '…\koboldcpp_cublas.dll' (or one of its dependencies) plus the same two wrapper lines, exit 1. The -nocuda builds do not fail in this case (F17). This refines F02: the CUDA build runs on a PC without an NVIDIA GPU only when no CUDA flag is given. Use: P46, P48, P45 (the "dll not found" rows are not always antivirus), P03/T1, T4. Results: R-ci1-*-cuda-fail-usecuda-on-nocuda, R-ci2-*-cuda-fail-usecuda-on-nocuda.
F57 On Windows Server runners (no graphics driver), `--usevulkan` fails at start with a missing-module error, exit code 1, on both the CUDA and `-nocuda` builds
FileNotFoundError: Could not find module '…\koboldcpp_common.dll' (or one of its dependencies), PyInstallerImportError: Failed to load dynlib/dll '…\koboldcpp_common.dll', Failed to execute script 'koboldcpp', exit 1. On Linux and macOS the same flag only logs "no devices" and runs on the CPU (F17, F35). The missing dependency is the Vulkan runtime that a graphics driver normally installs, so on a normal Windows PC with a GPU driver this error is not expected; it shows what a missing or broken GPU driver looks like. Use: P46, P48, P31, T4. Results: R-ci1-w22-nocuda-fail-usevulkan-no-gpu, R-ci1-w22-cuda-fail-usevulkan-no-gpu.
F58 KoboldCpp does not write a log file by default: the console window is the log
Started on Linux (koboldcpp-linux-x64-nocuda 1.124.2, --skiplauncher, 25 s) in an empty working directory with output not redirected by us: no file was created in the working directory, the home folder or the system temp folder (the single-file build unpacks into a temporary _MEI… folder that is removed on exit). The only record of a session is what the terminal shows, which is why a window that closes immediately leaves nothing to read (P46) and why users must copy the console text or start from a terminal. Not yet repeated on Windows and macOS (T-WIN-02 and T-MAC-01 record the leftovers there). Use: P58 ("Where does KoboldCpp save its log?"), P46, P44. Results: test-data/r1-lnx-2026-11-13/log-file-check.txt.
F59 On Linux, a freshly downloaded binary does not run until `chmod +x` ("Permission denied"); started with no model and no display it prints the GUI failure and exits with code 2
koboldcpp-linux-x64-nocuda 1.124.2 (107,362,352 bytes, SHA-256 e7bc9457…a05f4, identical to the hash in F01's assets list) downloaded with curl -L from releases/download/v1.124.2/…. Run before chmod +x: Permission denied. After chmod +x, with DISPLAY and WAYLAND_DISPLAY unset and no --model/--config: Warning, GUI failed to start: Reason: no display name and no $DISPLAY environment variable, File selection GUI unsupported., customtkinter python module required!, You must use the command line instead, e.g. python ./koboldcpp.py --help, exit code 2 (it does not fall back to a server). Use: P09 (headless step), P46 (#launcher-wont-open), P45 (rows gui-no-display), T4. Results: r2-lnx-2026-11-14/no-display/ (stdout.txt, result.json).
F60 All seven `releases/latest/download/<asset>` links used by the download picker resolve (HEAD: two 302 redirects, then 200) on 14 Nov 2026
Checked for koboldcpp.exe, koboldcpp-nocuda.exe, koboldcpp-oldpc.exe, koboldcpp-linux-x64, -nocuda, -oldpc and koboldcpp-mac-arm64. Use: P03 / T1 acceptance (links resolve); repeat before launch.
F61 What Lite 353 really sends in Story mode: Memory and World Info go in the separate `memory` field (Memory first, then the triggered World Info entry), the story is the `prompt`, and the Author's Note is appended to the end of the prompt. Lite also sends its own context size, not the server's
Playwright drove the embedded Lite (Classic UI, Story mode) on KoboldCpp 1.124.2 started with --contextsize 2048. Inputs: Memory MEMORY_TEXT…, World Info key dragon with text WI_TEXT… (insert location "After Memory", the default), Author's Note ANOTE_TEXT…, story The knight met a dragon. The request to /api/extra/generate/stream carried memory = "MEMORY_TEXT: …\nWI_TEXT: …\n" and prompt = "The knight met a dragon.[Author's note: ANOTE_TEXT: …]", stop_sequence = [], max_length = 2048 and max_context_length = 16384 while the server reported 2048 (/api/v1/config/max_context_length). So the order the model sees is Memory, World Info, story, Author's Note; and Lite's own context setting must be lowered to match a smaller server context (cf. F53: the server cuts the start of the prompt without an error). The Context dialog has tabs Memory, World Info and TextDB, an "AutoGenerate Memory" button, Author's Note strengths Weak/Medium/Strong/Immediate, and World Info options "After Memory / Before A/N", a search depth and a case-sensitive toggle. Use: P18, P27, P25. Results: r2-lnx-2026-11-14/lite-probe/ (result.json), script lab/scripts/lite_probe.mjs.
F62 Lite keeps its autosave in the browser (IndexedDB `e_kaihordewebui_`, with small markers in localStorage), and the server exposes six built-in pages that all answer 200
After a generation: localStorage keys e_kaihordewebui_settings and e_kaihordewebui_story (20 characters each) and an IndexedDB database e_kaihordewebui_ with one object store holding 2 records. Browser storage is separate for each origin, so a different address or port starts empty (browser rule; not separately tested). Built-in pages: / (Lite), /lcpp/, /sdui, /musicui, /noscript and /api, all 200 text/html. Mode selector: Instruct (the start mode, F31), Story, Adventure, Chat. Use: P18 (#where-data-is-stored, built-in pages table), P08, P12. Results: r2-lnx-2026-11-14/lite-probe/.
F63 In F36's 8B run the 692.8 MiB buffer kept in system RAM is the token-embedding table (281.8 MiB) plus `output.weight` (411.0 MiB): the output matrix stayed in RAM even with all layers offloaded; the same split holds for the 0.5B model (F42)
Computed from the real GGUF tensor tables (src/lib/gguf.ts on the Hugging Face headers, 14 Nov 2026): Llama 3.1 8B Q4_K_M token_embd.weight Q4_K 281.8 MiB and output.weight Q6_K 411.0 MiB add up to the logged 692.8 MiB, and the remaining blocks add up to the logged 3,992.5 MiB device buffer. For Qwen2.5-0.5B Q4_K_M output.weight is 137.9 MiB (Q8_0) and the token embedding 89.3 MiB (Q5_0), and the logged device weights are 235.8 MiB. Seen on a software Vulkan device; a real GPU run (T-MEM-01) may place the output matrix on the card, so T2 keeps "count the output matrix on the GPU" as an opt-in. Use: P56 assumptions, accuracy table.
F64 A browser can read a GGUF header straight from Hugging Face with HTTP range requests: CORS allows it, the request is redirected to a `*.hf.co` CDN host, and 4 to 12 MB covers the header and tensor table of every model we tried
Chromium (Playwright) on a page served locally, Range: bytes=0-4194303 against https://huggingface.co/<repo>/resolve/main/<file>: answered with 206 after a redirect to us.aws.cdn.hf.co; only those two hosts were contacted. Models with a large vocabulary (Gemma 2 9B, Qwen3 30B-A3B) needed a second read of 8 MB. A two-part split model (Llama 3.3 70B Q8_0) needs each part's own header (12 MB in total) and lists 80 layers. A repository link (https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct, no GGUF files) answers "No GGUF files". The page's content-security-policy therefore allows huggingface.co, *.huggingface.co and *.hf.co for connect-src on /tools/vram-calculator/ only. Gated repositories (HTTP 401) were checked only against a mocked response. Use: P56 network and privacy sections, T-TOOL-2 part 5.
F65 `--gpulayers N` puts N−1 repeating layers on the device: "all layers" needs layers + 1, and autofit (`-1`) prints its own lines
Layer sweep of Qwen2.5-0.5B Q4_K_M (24 repeating layers, offloaded N/25), 4096 context, software Vulkan device (u24-vk, KoboldCpp 1.124.2, R-lab2-3-u24-vk-gl{0,6,12,18,99,-1}-alloc-q4-ctx4096). Vulkan0 model buffer size 0 / 48.56 / 106.39 / 166.44 / 235.76 MiB and Vulkan0 KV buffer size 0 / 10.62 / 23.38 / 36.12 / 51.00 MiB for --gpulayers 0 / 6 / 12 / 18 / 99: the KV figure is 2.125 MiB per layer, so 6, 12 and 18 gave 5, 11 and 17 layers (the log says offloading 11 repeating layers to GPU for 12). --gpulayers 25 or more (and -1) moves all 24. output.weight (137.94 MiB) stayed in CPU model buffer size at every setting, and the embedding table sat in Vulkan_Host (89.26 MiB) at full offload. With -1 the log adds Autofit Reserve Space: 1024 MB and Autofit Success: 1, Autofit Result: -c 4224 -ngl -1, and on this machine Unable to detect VRAM, but autofit may still be used if applicable. Use: P26 (what a layer number means, the log lines), T2 launch values (the calculator now prints layers + 1). Software device: allocation sizes only, no speeds.
F66 In a scripted 32-turn chat at 2048 context, ContextShift keeps each turn to about 23 processed tokens once the context is full; with `--noshift` every turn reprocesses about 1,946; a changed Memory costs one full reprocess; and `--quantkv q8_0` still shifts on 1.124.2 (CPU)
lab/scripts/ctx_shift.py (Qwen2.5-0.5B Q4_K_M, lab-lnx, 4 threads, KoboldCpp 1.124.2, context 2048, max_length 80, greedy sampling, Memory in the memory field, story in prompt; results in test-data/r2-lnx-2026-11-14/ctx/). "Processed" is the number in KoboldCpp's own CtxLimit … Processed:N line. The context fills at turn 20. Default: 23 tokens a turn from then on (the log prints [Context Shifting: Erased 102 tokens at position 22]), 749 processed in total, 6.9 s of prompt time. --noshift: 1,946 tokens a turn from turn 20, 25,729 in total, 155.0 s. --nofastforward: the whole chat is reprocessed from turn 2 (101 more tokens each turn, then 1,967), 43,714 in total, 258.7 s. Memory edited at turn 21 (default settings): that one turn processed 1,946 tokens, then turns went back to 23 (2,672 in total). --quantkv q8_0: identical to default (749 in total, shifting lines present). Two alternating chats (8 turns each): without SmartCache every turn after the first pair reprocessed the whole other chat (125, 125, 226, 226 … 731, 731: 6,064 in total); with --smartcache 2 (log: SmartCache: Prepared 2 KV slots, SmartCache Match of 1.00 restored from slot 0) 22 tokens a turn from the third turn (380 in total against 6,064); peak RSS 726 MB against 693 MB (2 slots, chats up to about 800 tokens). CPU run only: it says nothing about the CUDA crash reports for quantised KV. Use: P27 #contextshift, #smartcache, #quantkv; P28.
F67 Sliding-window models: KoboldCpp turns SWA on automatically, disables ContextShift (every turn reprocesses about 1,967 tokens once the context is full), and keeps `window + batch + 128` cells in the window layers' cache; `--noswa` restores shifting at the cost of memory
Gemma 3 270M Q8_0 (18 layers, 1 KV head × 256, window 512, 5 window layers per full layer), KoboldCpp 1.124.2, CPU. Log: SWA Mode is ENABLED!, Context shifting is DISABLED!, Note that using SWA Mode cannot be used with Context Shifting!. KV cache at 2048 requested: 2304 cells for the 3 full layers (6.75 MiB) and 1152 cells for the 15 window layers (16.88 MiB); with --noswa 2304 cells for all 18 (6.75 + 33.75 MiB). Window cells by batch size at 4096 context: --batchsize 256 → 896, 512 → 1152, 1024 → 1664 (that is batch + 640 = batch + window + 128); at 8192 context the window cache stays 1152. Same 32-turn chat as F66: default 1,967 tokens a turn from turn 20, --noswa 24 a turn. Peak RSS 499 MB with SWA against 629 MB with --noswa at 2048 context. A model without SWA logs This model does not use SWA. Use: P27 #swa, T2 sliding-window cache formula. One small model only: the +128 and the 5:1 layer pattern come from this one model.
F68 Tokens per word depend on the text: 1.07 for a simple story sentence and 2.20 for technical text full of numbers and names (Qwen2.5 tokenizer)
/api/extra/tokencount on KoboldCpp 1.124.2 with the 0.5B model: a 900-word story sentence repeated gave 961 tokens; 1,500 words of our own findings text gave 3,306 tokens (test-data/r2-lnx-2026-11-14/ctx/token-ratio.json). Another model's tokenizer will differ. Use: P27 #what (no single rule of thumb is published).
F69 On a CPU, batch sizes of 32 to 128 process prompts about 20 to 25% slower than 256 and above, with no clear winner above 256; a draft model made a tiny model about three times slower even though about 84% of its tokens were accepted
lab/scripts/perf_extra.py (lab-lnx, 4 threads, KoboldCpp 1.124.2, context 4096, results in test-data/r2-lnx-2026-11-14/perf/). Qwen2.5-0.5B Q4_K_M, 1,650-token prompt, median of three runs: --batchsize 32 / 64 / 128 / 256 / 512 / 1024 / 2048 gave 135.1 / 146.8 / 149.5 / 180.8 / 170.3 / 180.0 / 174.3 tokens per second of prompt processing (generation, which the batch should not affect, varied from 25.7 to 35.7 between cases: the machine is noisy). Speculative decoding with Qwen2.5-0.5B F16 as the main model and Q2_K of the same model as the draft: generation 23.7 tok/s without a draft, 7.4 with --draftamount 4 and 8.1 with 8; /api/extra/perf reported 69, 75, 64 accepted and 13, 1, 18 rejected draft tokens for the three runs at amount 4. CPU and tiny-model worst case only; it does not show what a draft does for a large model on a GPU. Use: P28 #prompt, #draft.
F70 The load log prints file size and bits per weight, and the effective bits per weight of a quant depend on model size: small models carry more bits than the name suggests, and the `file type = (guessed) unknown, may not work` line appears on files that run normally
KoboldCpp 1.124.2 print_info lines for Qwen2.5-0.5B (630.17 M parameters), ubuntu-24.04-nocuda/base-*/stdout.txt: Q2_K file size = 390.28 MiB (5.20 BPW), Q4_K_M 462.96 MiB (6.16 BPW), Q8_0 638.74 MiB (8.50 BPW), F16 1.17 GiB (16.00 BPW); each of these runs printed file type = (guessed) unknown, may not work and worked normally. From the real Hugging Face files and tensor tables (F63 method): Llama 3.1 8B Q4_K_M 4.90 BPW (4.58 GiB), Q5_K_M 5.71, Q6_K 6.57, Q8_0 8.51, Q2_K 3.17; Mistral Nemo 12B Q4_K_M 4.88; Llama 3.2 3B Q4_K_M 5.03; Qwen2.5-0.5B Q4_K_M 6.24 (computed). A Q4_K_M file therefore costs about 4.9 bits per weight on 8B to 12B models and 6.2 on a 0.5B model. Use: P22 #which, #what, and the BPW sentence.
F71 The Linux nocuda build needs glibc 2.29 or newer: its engine libraries ask for glibc 2.27 and libstdc++ 3.4.30, and the newest glibc symbol across the 91 bundled libraries is 2.29 (libpng); libstdc++ and libgcc are bundled
Started koboldcpp-linux-x64-nocuda 1.124.2 on Ubuntu 24.04 (glibc 2.39), let it unpack to its temporary folder, and read the dynamic symbol versions of every shared library there with objdump -T: koboldcpp_default.so, koboldcpp_vulkan.so, koboldcpp_noavx2.so and koboldcpp_failsafe.so each need up to GLIBC_2.27 and GLIBCXX_3.4.30; across all 91 libraries the maximum is GLIBC_2.29. The unpack folder holds its own libstdc++.so.6, libgcc_s.so.1 and libvulkan.so.1. We did not run it on an older system, so 2.29 is the requirement implied by the symbols, not a tested lower bound. Use: P05 (Linux row), P09.
F72 The official OpenAI Python, OpenAI JavaScript and Anthropic Python SDKs all work against KoboldCpp 1.124.2 with a placeholder key; the `model` value is ignored; `tools` return a tool call on the default settings but come back as plain text with `--jinja`; JSON schema output parsed; an unloaded embeddings model returns an empty vector
lab/scripts/openai_probe.py and openai_js_probe.mjs (lab-lnx, Qwen2.5-0.5B Q4_K_M, context 4096; results test-data/r2-lnx-2026-11-14/openai/*.json; SDK versions as installed on 14 Nov 2026: openai Python 3.51.0, anthropic 2.8.0, openai JS 7.57.0). With base_url …/v1 and api_key="placeholder": models.list returned koboldcpp/qwen2.5-0.5b-instruct-q4_k_m; chat.completions returned model: "koboldcpp/qwen2.5-0.5b-instruct-q4_k_m" for the model names anything-at-all and gpt-4o alike; completions finished with length; streaming produced chunks ending in finish_reason: "stop"; the base URL without /v1 also worked; an empty api_key is refused by the SDK itself before any request ("Missing credentials"). tools on default settings: finish_reason: "tool_calls" and one parsed tool call; with --jinja the same request returned finish_reason: "stop", an empty tool_calls list and the call as text (<tool_call> … in the message; a 0.5B model, so this shows the response shape, not model quality). response_format json_schema (strict) returned {"name": "Ada", "age": 36} with an integer age; json_object parsed. embeddings.create without an embeddings model: dimension 0, model: "inactive"; with the 22 MB MiniLM model loaded: 384 dimensions, model: "all-MiniLM-L6-v2-Q8_0". responses.create returned status: "completed" and output_text. Anthropic SDK messages.create and messages.stream returned stop_reason: "end_turn" with base_url the bare address and also with /v1 added. Ollama routes (raw HTTP): /api/tags, /api/chat, /api/generate, /api/show 200; /api/version returned 0.7.0 (an emulated Ollama version); /slots answered 501. With --password: right key 200, wrong key AuthenticationError 401 on chat, and /v1/models still answered with the real model id for a wrong key (the masked name koboldcpp/protected-model appears on /api/v1/model, F25). Use: P34, P33.
F73 On Linux, starting KoboldCpp 1.124.2 with a model and generating once made no network connection except to the machine itself
strace -f -e trace=connect,sendto on koboldcpp-linux-x64-nocuda (Qwen2.5-0.5B, --skiplauncher, no flags that use the network) from process start, through a model load and one /api/v1/generate request: three connect calls in total, none to a public address: one connection to 127.0.0.1 (the server's own port) and two failed attempts to reach the local name-service cache socket (/var/run/nscd/socket, not present), which is a local lookup (test-data/r2-lnx-2026-11-14/net/connect-calls.txt). No connection to any other address, no update check and no download appeared. This is the Linux nocuda build with default settings; it does not cover the launcher window, the Windows or Mac builds, --remotetunnel, --websearch, a model URL or the Horde worker, all of which use the network by design. Partial T-NET-04. Use: P02, P06.
F74 The official Linux ROCm build (`rocm-rolling`) starts without any AMD GPU on its CPU path; asking for the GPU path with `--usecuda` aborts with exit code 134 after a ROCm error
koboldcpp-linux-x64-rocm from the rocm-rolling pre-release: 648,081,632 bytes (618 MiB), SHA-256 starting e3372cfb, reports KoboldCpp 1.124.2, downloaded 14 Nov 2026 and run on Ubuntu 24.04 with no GPU (test-data/r2-lnx-2026-11-14/rocm/). With no backend flag: Unable to detect VRAM., No GPU backend found, or could not automatically determine GPU layers., Auto Selected Default Backend (flag=0), library koboldcpp_default.so, offloaded 0/25 layers to GPU, CPU KV buffer size = 195.00 MiB and the server started. With --usecuda: library koboldcpp_hipblas.so, Backend 0: ROCm, then ggml_cuda_init: failed to initialize ROCm: no ROCm-capable device is detected and rocBLAS error: Could not initialize Tensile host: No devices found, and the process aborted with exit code 134 (SIGABRT), where the nocuda build would run on the CPU (F17). No extra packages had to be installed for the CPU path. Use: P31 (#rocm-linux), P45. Not covered: a real AMD GPU, and the Windows fork (koboldcpp-rocm, not reachable from our session).
F75 On our Linux test machine, `--parallelrequests N` with N simultaneous requests raised total generation speed on the CPU: 18.8 → 31.1 → 41.3 tokens per second in total for 1, 2 and 4 requests (automatic threads), and 32.5 → 52.0 → 68.4 with `--threads 4`; each request slowed down
lab/scripts/parallel_probe.py (lab-lnx, 4 vCPU, KoboldCpp 1.124.2, Qwen2.5-0.5B Q4_K_M, context 2048, 64 generated tokens per request, greedy, three runs each after a warm-up, results in test-data/r2-lnx-2026-11-14/parallel/). Median wall time of the whole batch, automatic threads: 3.40 s for one request, 4.11 s for two, 6.20 s for four (one request alone: 3.4 s, so four sequential requests would take about 13.6 s). With --threads 4: 1.97 s, 2.46 s, 3.74 s. Every request in a batch finished at nearly the same time (4.61 and 4.65 s for two). This differs from the reading of F26 (CI runner, one default-queue run against one --parallelrequests 2 run, where the two requests took the same total time either way): here the batch finishes far sooner than the same requests one after another. The machines, thread settings and prompts differ, so the size of the gain is not general. Use: P28 #parallel, P33 #queue.
F76 A `.kcpps` file that holds only the settings you changed loads with every other setting at its default, and a command built from flags gives the same exported settings as the same file: 20 configurations checked against the real binary
tests/tools/config-binary.test.ts (KoboldCpp 1.124.2 koboldcpp-linux-x64-nocuda; results test-data/r2-lnx-2026-11-14/config/export-check.json). For each of 20 configurations (8 presets of our command builder and 12 more: threads and batch size, KV quantisation, tensor split, MoE, lowvram, SmartCache, RoPE, draft model, Whisper and TTS, embeddings, admin mode, paths with spaces and a password with an apostrophe) the test ran the binary with --exportconfig and the generated flags, checked every setting we set against the exported file, then ran it again with --config on the partial .kcpps our builder writes (changed keys only) and compared all 160 exported keys with the first export: no key differed. Two facts from the same run: --exportconfig does not need the model file to exist, and numbers come back typed (tensor_split as [3.0, 1.0]). Use: P57 (what the .kcpps download contains, #assumptions), P20. Linux build only; the Windows and macOS builds share the Python launcher code but were not run.
F77 KoboldCpp answers cross-origin browser requests permissively: a page on another origin can call it, including with an `Authorization` header, because the preflight echoes the page's origin and allows that header
lab/scripts/cors_probe.mjs (Chromium from the Playwright build, a page on http://127.0.0.1:<port> calling KoboldCpp 1.124.2 on another port; test-data/r2-lnx-2026-11-14/api-tester/cors-chromium.json). Without a password: GET /api/extra/version, POST /api/v1/generate with a JSON content type, with a Bearer header, and GET /v1/models all returned 200 to the page. With --password: the same page got 401 for generation without the key and 200 with it, and could still read /api/extra/version. The preflight (OPTIONS with Access-Control-Request-Headers: content-type,authorization) returned 200 with Access-Control-Allow-Origin equal to the asking origin, Access-Control-Allow-Methods: * and an allow-headers list that includes Authorization, Content-Type, apikey and genkey. Meaning for readers: a browser tool on a web page can check your server, and so can any other web page you open while a KoboldCpp without a password is running on your PC. Use: P59 (#browser-limits), P06, P38. Chromium only, plain http pages; https pages and other browsers: see the second table in P59.
F78 In Chrome 153 an https page can call a KoboldCpp on `http://127.0.0.1` or `http://localhost`, but not one on the machine's own network address: that is blocked as mixed content
lab/scripts/mixed_content_probe.mjs (Chromium 153 from the Playwright build; page served over https from https://localhost with a self-signed certificate; KoboldCpp 1.124.2 on plain http; test-data/r2-lnx-2026-11-14/api-tester/mixed-content-chromium.json). fetch("http://127.0.0.1:5281/api/extra/version") and the localhost form returned HTTP 200 from the https page; the same request to http://192.0.2.2:5281 (an address on the machine's own network interface) failed with TypeError: Failed to fetch (blocked mixed content). So the live check of the API tester works from our https site for a server on the same PC, and for a server on another device (a phone, a LAN address, http://) it cannot: a tunnel (https://…trycloudflare.com) or the command fallback is needed. We did not see a "local network access" permission prompt in this headless run; Chrome's prompt behaviour on a desktop, Firefox, Edge and Safari were not tested. Use: P59 (#browser-limits), P38.
F79 KoboldCpp says which chat template it picked: the log line `Chat completion heuristic: <rule name>` appears when the first chat completion request arrives, not when the model loads
KoboldCpp 1.124.2 with --debugmode 1 and the default adapter (AutoGuess), one /v1/chat/completions request each: Qwen2.5-0.5B-Instruct → Chat completion heuristic: ChatML (Qwen 2.5 based); Gemma 3 270M it → Chat completion heuristic: Google Gemma 3; Qwen3-0.6B → Chat completion heuristic: ChatML (Qwen 2.5 based) (test-data/r2-lnx-2026-11-14/thinking/*.log). At start-up only Loading Chat Completions Adapter: …/kcpp_adapters/AutoGuess.json and Chat Completions Adapter Loaded are printed. The rule names are the name fields of the 33 rules in AutoGuess.json; the unpacked kcpp_adapters folder holds 30 JSON files (29 adapters and the detection list AutoGuess.json; AutoGuess-NoThink.json is a second detection list). Not printed when an adapter is chosen with --chatcompletionsadapter. Use: P24 #autoguess.
F80 On Qwen3-0.6B, the thinking switches work as documented: `--jinjathink false` and the `ChatML-NoThink` adapter turned thinking off (12 tokens, 0.7 s against 288 tokens, 10.2 s), `--reasoningeffort none` shrank it, and reasoning arrives in its own field, not in the reply text
lab/scripts/thinking_probe.py (lab-lnx, 4 threads, KoboldCpp 1.124.2, Qwen3-0.6B Q4_K_M, one chat request "What is 17 times 3? Answer briefly.", temperature 0, 500 tokens; test-data/r2-lnx-2026-11-14/thinking/thinking.json). Message fields returned: content, reasoning_content, role, tool_calls. Default and --jinja and --jinja --jinjathink true: 794 characters of reasoning, 23 of answer, 288 tokens, 10.2 to 10.8 s. --jinja --jinjathink false: no reasoning, answer "17 multiplied by 3 equals 51.", 12 tokens, 0.7 s. --reasoningeffort none: 50 characters of reasoning, 27 tokens, 0.9 s (not zero). --chatcompletionsadapter ChatML-NoThink: no reasoning, 12 tokens, 0.7 s. No <think> tag appeared in content in any run. One tiny model and one question: it shows the switches, not answer quality with thinking off. Use: P24 #thinking.
F81 A `.kcpps` file with dashed keys is silently ignored, a 1.95-era key set still loads, `--config` prints one override line per flag you also type, and `--exporttemplate` differs from `--exportconfig` in six keys
lab/scripts/kcpps_probe.py (lab-lnx, KoboldCpp 1.124.2, test-data/r2-lnx-2026-11-14/kcpps/kcpps-probe.json). (1) A file with only model_param, contextsize: 2048, threads: 2 loaded and served a context of 2048, every other setting at its default (confirms F76). (2) A file with "ctx-size": 3072 instead of contextsize loaded without any message and served 16384, the default: dashed spellings are ignored. (3) A file with noblas: true and blasbatchsize: 128 (old names) loaded without a warning and applied them (usecpu=True, batchsize=128, n_batch = 128). (4) The same file with --contextsize 3072 on the command line served 3072 and printed Overriding Config Value: contextsize. (5) Same flags with --exportconfig and --exporttemplate (both 160 keys) differ only in gpulayers (10 → -1), istemplate (false → true), password ("secret" → null), sdmaingpu (null → -1), threads (3 → -1), usecpu (false → null). Evidence labels: tested on Linux CPU only; the Windows double-click, the Windows legacy-file run and the launcher's Save Config dialog were not run (no Windows PC or display).
F82 A `.safetensors` file given as a text model fails with exit 3 and "does not seem to be a GGUF file"; `--model <url>` downloads into `--downloaddir` and the next start reuses the file
Lab-lnx, KoboldCpp 1.124.2, a small fake safetensors file (valid header) and a Gemma 3 270M Q8_0 GGUF served from a local web server (test-data/r2-lnx-2026-11-14/hf/). (1) Safetensors as --model: load_model: error: invalid or unsupported model file, WARNING: Selected Text Model does not seem to be a GGUF file!, Could not load text model, exit 3. (2) --model http://…/tiny-Q8_0.gguf --downloaddir dl: prints Downloading <url>, writes dl/tiny-Q8_0.gguf (291,546,144 bytes, same as the source) and serves it as koboldcpp/tiny-Q8_0. (3) The same command again prints <path> already exists, using existing file. and downloads nothing. (4) --analyze on a GGUF prints a metadata block (version, tensor count, key count). Not run: Hugging Face itself (no access from the lab), the launcher's HF Search button (no display), split GGUF files, and the image-model safetensors path.
F83 Sampler fields measurably change repetition: on a loop-inducing prompt a repetition penalty of 1.1 cut repeated 4-grams from 45% to 6%, DRY 0.8 to 6%, and temperature 2.0 with no truncation made gibberish; the API default behaves like a penalty of 1.0, seeds reproduce text, and `--gendefaults` fills omitted fields
lab/scripts/sampler_probe.py (lab-lnx, CPU, 4 threads, KoboldCpp 1.124.2, Qwen2.5-0.5B-Instruct Q4_K_M, /api/v1/generate, test-data/r2-lnx-2026-11-14/samplers/sampler-probe.json). Prompt: "The cat sat on the mat." repeated four times plus "The cat"; 100 tokens, seeds 1-4; metric: share of repeated word 4-grams (0 = none). Results: request defaults 0.454 (identical to rep_pen 1.0), rep_pen 1.1 range 256 0.063, rep_pen 1.3 0.000, DRY multiplier 0.8 base 1.75 allowed length 2 0.062, DRY plus temperature 1.0 and Min-P 0.05 0.130, temperature 1.0 and Min-P 0.05 alone 0.295, temperature 2.0 with no truncation 0.000 but unreadable text. The metric counts repetition only: lower is not better (temperature 2.0 scores best and is the worst text), and a 0.5B model on one prompt cannot say which preset is best, so no preset here is called tested-best. Seeds: the same sampler_seed gave the same text twice, another seed a different text, temperature 0 gave the same text for different seeds. max_length 20 returned 10 words; /api/v1/config/max_length reports 2048. --gendefaults '{"temperature":0}': two seeds gave the same text when the request left temperature out; with --gendefaultsoverwrite a request asking for 1.5 was still forced to 0 (same text for two seeds). Not run: Lite and SillyTavern sampler panels, Mirostat, Adaptive-P, XTC, other models.
F84 The Linux builds ship these backend libraries: the `nocuda` build has default, failsafe, no-AVX2, Vulkan and Vulkan no-AVX2 (no CUDA); the `rocm` build adds an 806 MB hipBLAS library and a Vulkan failsafe one
--unpack of koboldcpp-linux-x64-nocuda and koboldcpp-linux-x64-rocm (1.124.2), lists in test-data/r2-lnx-2026-11-14/backends/libs-*.txt. nocuda: koboldcpp_default.so 2.9 MB, _failsafe 2.7, _noavx2 2.9, _vulkan 4.7, _vulkan_noavx2 4.5 (plus shader libraries 18.2 and 37.4 MB, and koboldcpp_common.so 31.6 MB). rocm: the same plus koboldcpp_hipblas.so 806.1 MB and koboldcpp_vulkan_failsafe.so 4.2 MB. This is why the launcher's Backend menu differs between files: it lists only options whose library is in your file. Windows and macOS file contents were not unpacked (no machine).
F85 The native KoboldAI routes all answered as documented on 1.124.2: generate, streaming, check, abort, token count and decode, logprobs, grammar conversion; the spec lists 42 generation parameters, 19 marked "KoboldCpp ONLY"
lab/scripts/kobold_api_probe.py (lab-lnx, CPU, Qwen2.5-0.5B, test-data/r2-lnx-2026-11-14/kobold-api/, spec saved as spec.json, extracted from the /api?json=1 page, which is HTML with the spec inside). Results: POST /api/v1/generate returns results[0] with text, tool_calls, finish_reason, logprobs, prompt_tokens, completion_tokens. /api/extra/tokencount and its alias /api/extra/tokenize return value, ids, prompt ("Hello world" gave 3 ids, the first the end-of-text token 151643); /api/extra/detokenize returns result and success. /api/extra/generate/stream sent 12 lines (6 event: message plus data: {"token":…,"finish_reason":…} pairs) for 12 requested tokens, the last with an empty token and finish_reason "length"; the first event held several tokens. /api/extra/generate/check with a genkey returned the partial text of a running request (732 characters at 5 s); POST /api/extra/abort with that genkey returned {"success":"true","done":"true"} and the waiting generate call ended with 705 characters of partial text and status 200. logprobs: true returned logprobs.content per token with top_logprobs (at temperature 0 every logprob was 0.0); /api/extra/last_logprobs with the genkey returned the same. /api/extra/json_to_grammar turned a JSON schema into GBNF text. /api/v1/info/version gave 1.2.5, /api/extra/version gave KoboldCpp and 1.124.2 plus flags (llm true, savedata false, admin 0, jinja false, parallelrequests 1). /api/extra/perf returned last speeds (34.6 tokens/s eval for the 8-token request), queue, uptime. A request with max_context_length 8192 on a server started with 2048 was accepted without an error. Without their flags /api/admin/list_options answered [] and /api/extra/preloadstory answered {}. Not run: the flag-gated data, multiplayer and admin routes with their flags, images, grammar in a generation, XTC and DRY effects (F83 covers DRY), a shared-server test of what a call without genkey sees.
F86 LangChain (`langchain-openai` 1.8.0) and the Ollama Python client (0.6.3) both work against KoboldCpp 1.124.2; with a password set, a wrong key and a placeholder key both give 401
lab/scripts/integrations_probe.py (lab-lnx, CPU, Qwen2.5-0.5B, --password secretkey, test-data/r2-lnx-2026-11-14/integrations/integrations.json). ChatOpenAI(base_url="http://127.0.0.1:5361/v1", api_key="secretkey", model="koboldcpp"): invoke returned a reply with usage (14 in, 4 out), stream gave 10 chunks. Wrong key and the placeholder not-needed each raised OpenAIAuthenticationError: Error code: 401. The Ollama client with host http://127.0.0.1:5361 and header Authorization: Bearer secretkey answered chat and list (models koboldcpp/qwen2.5-0.5b-instruct-q4_k_m:latest and the same without the tag). Not run: a tunnel (the sandbox refused to open a public tunnel), Janitor AI, Chub, Open WebUI, Agnaistic, RisuAI, Continue, n8n, Home Assistant: no display or account, so those recipes are the generic values above plus the apps' documentation, and are marked untested.
F87 By default KoboldCpp 1.124.2 listens on every network interface, so another device on the same network can already reach it; `--host 127.0.0.1` closes that; `--ssl` serves https only; `--ratelimit` answers a second request with 503; `KCPP_PASSWORD` works like `--password`
lab/scripts/network_probe.py (lab-lnx, Linux, Qwen2.5-0.5B, test-data/r2-lnx-2026-11-14/network/network-probe.json; sockets read from /proc/net/tcp). Default (no --host): the port is open on 0.0.0.0, and a request to the machine's own private network address returned 200 as well as the loopback one. --host 127.0.0.1: open on 127.0.0.1 only, the network address failed to connect. --host 0.0.0.0: like the default. --ssl cert.pem key.pem (self-signed, 2 days): https://127.0.0.1 returned 200 when certificate checks were off and failed verification with checks on; plain http:// on the same port was reset. --ratelimit 8: the first generate call returned 200, the next call right after returned 503, /api/v1/model still returned 200, and a generate call after 9 s returned 200. KCPP_PASSWORD=envkey in the environment: generate without a key 401, with Authorization: Bearer envkey 200. Not run: Windows or macOS firewall prompts, a phone or second device, the tunnel (the sandbox refused to open a public tunnel), the launcher Network tab, a long-generation time-out through a tunnel. Caution: "the lab machine's address" is a private sandbox address, not a home Wi-Fi.
F88 Image generation on a CPU with the tiny sdxs-512 model: a 512×512 image takes about 6 s, 256×256 about 1.8 s and 768×768 about 22 s; the seed reproduces the picture exactly; this model ignores the step count; `--sdclamped 256` shrinks a 512 request to 256; a text model given as the image model stops with exit 3
lab/scripts/image_probe.py (lab-lnx, CPU, 4 threads, KoboldCpp 1.124.2, sdxs-512-tinySDdistilled_Q8_0.gguf 683 MB, /sdapi/v1/txt2img, test-data/r2-lnx-2026-11-14/image/). /api/extra/version reported txt2img: true, llm: false; /sdapi/v1/sd-models listed the model; /sdui answered 200. Timings: 512×512 1 step 6.0 s; the same request and seed again gave a byte-identical PNG (same SHA-256 prefix 926e930a156c); another seed gave a different picture; 256×256 1.8 s; 768×768 21.7 s; steps 4 gave the identical PNG in the same time as steps 1 (this distilled model is made for one step, so the step count had no visible effect here: other models differ). img2img with init_images returned an image in 7.9 s; /v1/images/generations at 256×256 1.5 s. With --sdclamped 256 a 512×512 request came back 256×256, identical to the native 256 picture. --sdquant 2 still gave a 512×512 image in 6.4 s (a different picture). --sdmodel pointed at a Gemma text GGUF: Could not load image model: <path> then Error: Image generation failed to setup!, exit 3. Not run: any larger family (SD 1.5, SDXL, Flux and others need far more memory than this sandbox runs quickly), the encoder and VAE files, black-image causes, Lite's image panel, StableUI interaction, ComfyUI emulation, video and music. All prompts are our own and the pictures are ours.
F89 Vision on a CPU: a mismatched mmproj stops the start with exit 3 and a clear embedding-size message; with no mmproj the model still answers an image question, as if it had seen the image; `--visionmaxres` sets how many tokens an image costs (155, 221 and 486 for a 1600×1000 picture at 512, 1024 and 2048)
lab/scripts/vision_probe.py (lab-lnx, CPU, 4 threads, KoboldCpp 1.124.2, SmolVLM-256M-Instruct Q8_0 with its mmproj, our 1600×1000 test picture of a red circle, a blue rectangle and the text "KOBOLD 42", test-data/r2-lnx-2026-11-14/vision/). With the matching mmproj /api/extra/version showed vision: true; the OpenAI chat image_url call (data URI) took 7.2 s at the default 1024 and used 221 prompt tokens; at --visionmaxres 512 4.2 s and 155 tokens; at 2048 16.0 s and 486 tokens; --mmprojcpu 7.7 s, 221 tokens. The native images field of /api/v1/generate (prompt with an <image> tag) took 6.4 s and answered "A red circle and a blue square." The answers were not stable between settings: 512 gave "The shapes in this image are circles and squares.", the other settings gave \frac{1}{2}: this 256M model is a plumbing test, not a quality judgement (see F28, F33). Without --mmproj (vision: false) the same image_url request returned 200 in 0.8 s with only 27 prompt tokens (the image was dropped) and a confident description of shapes: no error tells you the image was ignored. The Qwen2.5-0.5B text model given SmolVLM's mmproj: mtmd_init_from_file: error: mismatch between text model (n_embd = 896) and mmproj (n_embd = 576), gpttype_load_model: error: failed to load mmproj model!, Could not load text model, exit 3. Not run: a larger vision model, Vulkan or CUDA, audio input, /sdapi/v1/interrogate, Lite's attach button, projector memory on a GPU.
F90 Voice on a CPU with no text model loaded: Whisper tiny.en transcribed the 11 s JFK sample in 2.0 s, also as 8 kHz stereo; Kokoro speaks at about 1.2 seconds of computing per second of audio (slower than real time on 4 CPU threads); a file that is not WAV comes back as success with empty text; a text model given as the TTS model stops with exit 3
lab/scripts/voice_probe.py (lab-lnx, CPU, 4 threads, KoboldCpp 1.124.2, --nomodel --whispermodel ggml-tiny.en.bin --ttsmodel Kokoro_no_espeak_Q4.gguf, test-data/r2-lnx-2026-11-14/voice/). /api/extra/version showed transcribe: true, tts: true, llm: false. /v1/audio/transcriptions (multipart file) with jfk.wav (16 kHz mono) returned the full sentence in 2.0 s; the same audio resampled to 8 kHz stereo gave the same sentence in 1.6 s; /api/extra/transcribe with base64 audio_data and langcode en gave the same sentence in 1.0 s; a file named fake.mp3 holding random bytes returned status 200 and {"text": ""} in 0.1 s (no error status: an app sees an empty transcript; the console line for it was not captured). /v1/audio/voices listed kobo, cheery, sleepy, shouty, chatty, random, instruct. /v1/audio/speech (voice kobo) returned 24 kHz mono WAV: 12 characters in 1.48 s (real-time factor 1.28), 70 characters in 4.83 s (1.20), 182 characters in 10.93 s (1.19); response_format mp3 returned audio/mpeg (23,824 bytes, starts fffb) in 2.3 s; /api/extra/tts returned WAV in 2.1 s; the XTTS-style /tts_to_audio/ returned WAV in 2.3 s; /speakers_list and /speakers listed the voices. --ttsmodel pointed at a Gemma text GGUF: Could not load TTS model!, Warning: KCPP OuteTTS missing a file! Make sure both TTS and WavTokenizer models are loaded., exit 3. Not run: Lite's microphone and TTS settings, the microphone on https or over LAN, voice cloning (--ttsdir), OuteTTS and Qwen3-TTS, other Whisper sizes, languages other than English, music transcription, GPU TTS. The audio samples are ours (generated from our own sentences).
F91 A model whose architecture KoboldCpp does not know stops with `unknown model architecture: '<name>'`, exit 3, and prints `The reported GGUF Arch is: <name>`; an HTML page saved as `.gguf` gives the same "invalid or unsupported model file" as F20; a path with spaces and non-ASCII letters loads fine on Linux
lab/scripts/load_probe.py (lab-lnx, KoboldCpp 1.124.2, test-data/r2-lnx-2026-11-14/load/). (1) A copy of Qwen2.5-0.5B whose general.architecture value was overwritten with zzzzz (same length): llama_model_load: error loading model: unknown model architecture: 'zzzzz', llama_model_load_from_file_impl: failed to load model, gpttype_load_model: error: failed to load model '<path>', Could not load text model: <path>, then The reported GGUF Arch is: zzzzz; exit code 3. This is how a model newer than your KoboldCpp fails (the architecture is the only change here, so the real-world message is the same by construction, not observed with a real new model). (2) A small HTML page saved as page.gguf (what a Hugging Face "blob" link gives): load_model: error: invalid or unsupported model file, Could not load text model, exit 3. (3) --model "…/my models/модель größe.gguf" (space, Cyrillic, ö and ß) started and served on Linux; Windows paths were not tested. (4) A normal start does not print the reported GGUF Arch line (it appears on failure only); --analyze <file> prints str: general.architecture = qwen2 in its metadata block, exit 0. Not run: a real newer-architecture model, a missing-tensor or wrong-shape file, split files, mmproj mismatch beyond F89, Windows paths, the launcher's Analyze Model button.
F92 Same model file, same 4 CPU threads: KoboldCpp generated faster than llama.cpp's Python server on both small models and started slower; prompt speed was similar. Ollama and LM Studio could not be measured
lab/scripts/compare_engines.py (lab-lnx, CPU only, 4 threads, 2048 context, test-data/r2-lnx-2026-11-14/compare/compare.json; KoboldCpp 1.124.2 against llama-cpp-python 0.3.36 built from source on this machine and run with its llama_cpp.server; the official llama-server binary could not be downloaded, so this is the llama.cpp engine behind a Python layer and not llama-server; a different llama.cpp version from KoboldCpp's). Three runs each, median shown. Qwen2.5-0.5B Q4_K_M: generation KoboldCpp 26.7 tokens/s (28.9, 26.7, 26.6) against 16.6 (17.7, 16.6, 13.7); prompt processing (505 tokens) 150.4 against 158.4; ready 5.5 s against 1.5 s; memory of the process group after the runs 665 MB against 1,605 MB. Gemma 3 270M Q8_0: generation 41.3 against 23.0; prompt (506 tokens) 438.9 against 431.0; ready 5.0 s against 1.5 s; memory 465 MB against 2,281 MB. The Python server's slower generation and larger memory use are probably caused by the Python layer and its defaults, so they say nothing about llama-server. Ollama: the installer and release files come from GitHub, which this sandbox blocks for that project; LM Studio is a closed desktop application with no headless install. Neither was run. text-generation-webui was not run either (it needs a large Python environment).
F93 The official Android installer (`android_install.sh`, read from the project's repository on 14 Nov 2026) shows a six-choice menu only when run from a terminal; piped from `curl` it silently picks choice 1 (install and download Gemma 3 1B) after a 3-second pause
We saved the script (test-data/r2-lnx-2026-11-14/android/android_install.sh, 179 lines) and read it; with F45 (a real run of choice 2 in a Termux container) this covers the page. Menu text: [1] Proceed to install and launch with default model Gemma3-1B, [2] Proceed to install without a model, you can download one later., [3] Download GGUF model from web URL (Requires already installed), [4] Load existing GGUF model from disk (Requires already installed), [5] Rebuild existing KoboldCPP installation, [6] Exit script, prompt Enter your choice [1-6]:. A number given as the first argument skips the menu. When stdin is not a terminal and no argument is given, the script prints Defaulting to normal install and model download… Install will start in 3 seconds. and runs choice 1. Choice 1 downloads gemma-3-1b-it-Q4_K_M.gguf from the ggml-org/gemma-3-1b-it-GGUF repository by passing the link to --model, asks Enter desired context size [8192] (accepts 256 to 524288) and launches python koboldcpp.py --contextsize N --model <link>. The build is make -j 2 after pkg install -y wget git python. The script refuses to run when uname -o is not Android. Our run of choice 2 (F45) finished in 11 minutes in a container. Not run: a real phone or emulator, the F-Droid install, model speed and temperature on a phone, "Illegal instruction" on older ARM chips.
F94 The official Colab notebook (read on 14 Nov 2026) has no password option, always starts a public Cloudflare tunnel, and deletes downloaded models at every start; its 15 presets set the context size and, for text models, all layers on the GPU
We saved colab.ipynb from the concedo branch (test-data/r2-lnx-2026-11-14/colab/colab.ipynb; 4 cells, GPU type T4) and read it; we did not run it (no Google account). The start command is ./koboldcpp_linux $MdCommand --usecuda 0 --gpulayers $Layers --contextsize $ContextSize --websearch --quiet --remotetunnel … plus per-model arguments; it contains no --password, so anyone with the link can use the instance. It downloads koboldcpp-linux-x64 from the latest GitHub release at each start, installs aria2, and stops with "Colab did not give you a GPU due to usage limits" if there is no nvidia-smi. Presets (context size set by the notebook): Gemma4 E4B Uncensored 40960, Tiefighter 13B 4096, Estopia 13B 4096, Qwen3 VL 8B (vision, with its mmproj) 32768, Cydonia 24B 10240, Broken Tutu 24B 10240, Dans PersonalityEngine 24B 10240, Harbinger 24B 10240, Muse 12B 49152, Mistral Small 24B 10240, GPT OSS 20B 32768, Gemma3 12B 16384, GLM4 9B 49152; image presets Z-Image Turbo and Flux Klein 4B (model, text encoder and VAE links set). Options in the form: custom model link, layers (Auto or 999), context size (4096 to 40960 or typed), vision mmproj, image model with clip and VAE, Whisper, TTS with WavTokenizer, embeddings, music (AceStep or YuE2), delete existing models (on by default), multiplayer, save to Google Drive (adds --savedatafile), and a LocalTunnel fallback that needs a password. The notebook's own text says: keep the page open and check for captchas so the instance is not shut down. Not run: the notebook, any preset's load time, the GPU assigned on the free tier, disconnect times, costs.
F95 The official Docker image downloads the newest KoboldCpp binary every time it starts; with no GPU it picks the nocuda build and was ready in 12 s; the reference compose file validates and runs
koboldai/koboldcpp:latest (pulled from Docker Hub on 14 Nov 2026, 1.26 GB on disk) on the Linux lab machine, no GPU, a local model mounted at /models (test-data/r2-lnx-2026-11-14/docker/, with the reference docker-helper.sh and docker-compose.yml from the project's examples/docker-reference). The entrypoint printed NVIDIA/AMD have not been detected, generic KoboldCpp will be used., followed redirects from koboldai.org/cpplinuxnocu to the v1.124.2 release asset and saved the 103 MB koboldcpp-linux-x64-nocuda as /workspace/koboldcpp; the server was ready 12 s after docker run, answered /api/extra/version with 1.124.2 and generated text (log: Initializing dynamic library: koboldcpp_default.so). A second start with the same /workspace downloaded the binary again (the helper skips it only with KCPP_DONT_UPDATE=true and an existing file). In our sandbox the first run failed with SSL/TLS handshake failure because the sandbox proxy re-signs HTTPS: this is a sandbox effect, so we mounted its certificate bundle for the real runs. The reference compose file passed docker compose config (with a warning that version is obsolete); a copy without the NVIDIA reservation, with host networking and our model, started, showed restart: unless-stopped and served the model. Not run: an NVIDIA or AMD GPU in a container, the NVIDIA Container Toolkit, the --remotetunnel default (we set KCPP_DONT_TUNNEL=true), health checks, KCPP_MODEL downloads of a real Hugging Face file (we mounted a local file), AMD /dev/dri.
F96 Sharing: `--queuelimit 1` lets one request run and one wait and answers the rest with 503 "Server is busy"; `--parallelrequests 2` keeps one shared context pool (`--paralleltotalcontext` sets its size); `--multiplayer` routes keep a shared story with a turn counter. The AI Horde status API was reachable and showed 46 text workers
lab/scripts/share_probe.py (lab-lnx, CPU, Qwen2.5-0.5B, KoboldCpp 1.124.2, test-data/r2-lnx-2026-11-14/horde/). Four generate requests sent 0.3 s apart with --queuelimit 1: the first took 1.9 s, the second 3.4 s (it waited), the third and fourth returned 503 with Server is busy; please try again later. in 0.0 s. With --contextsize 4096 --parallelrequests 2 the log showed n_seq_max = 3, n_ctx = 4352, n_ctx_seq = 4352, kv_unified = true (one pool, 256 cells added to the context size, as in F67), and /api/extra/version reported parallelrequests: 2; adding --paralleltotalcontext 8192 changed n_ctx to 8448, while /api/v1/config/max_context_length stayed 4096. With --multiplayer: /api/extra/multiplayer/status returned {"turn_major": 1, "turn_minor": 1, "idle": true, "data_format": ""}; setstory with a story returned success and moved the turn to 2; getstory returned the stored story. The public AI Horde status API (read-only GETs with curl, no key, no prompt; 14 Nov 2026): /status/heartbeat OK (version 6.0.0), /status/performance showed 46 text workers, 44 queued text requests and about 32,000 tokens in the past minute, /status/modes showed no maintenance. Not run: a Horde worker (it needs an account and would serve strangers' prompts), kudos, jobs, --routermode, --admin model switching, --rpcmode, the Lite multiplayer screen, a GPU. A Python client without a browser-like user agent got 403 from the Horde API; curl worked.
F97 Building KoboldCpp 1.124.2 from source on Linux took 7.5 minutes (4 cores) with plain `make`; the tools build in seconds afterwards; splitting a GGUF works and KoboldCpp loads it from part 1 only; the program runs from `python3 koboldcpp.py`
git clone --depth 1 of the project's repository (commit 6d2fedf, 2 Nov 2026, KcppVersion = "1.124.2") on the Linux lab machine (4 cores, 15 GB, Ubuntu 24.04, gcc 13.3, cmake 3.28, Python 3.13), test-data/r2-lnx-2026-11-14/build/. make -j4 (CPU, no flags): 451 s, 31 compiler warnings, produced koboldcpp_default.so (3.2 MB) and koboldcpp_common.so (33.7 MB) in the repository folder. python3 koboldcpp.py --model … --skiplauncher then started from the source folder, reported 1.124.2, loaded the bundled chat adapters, auto-selected the default backend and served a model. make -j4 quantize_gguf gguf-split took 4 s on top (objects were built) and made two 10 MB programs. gguf-split --split-max-tensors 100 cut Qwen2.5-0.5B Q4_K_M into three files (330, 86 and 75 MB; 291 tensors) in 0.7 s. KoboldCpp started with part 1 (…-00001-of-00003.gguf) and loaded all three. Part 2 alone failed: illegal split file idx: 1 (file: …-00002-of-00003.gguf), model must be loaded with the first split. With part 3 missing: gguf_init_from_file: failed to open GGUF file '…-00003-of-00003.gguf' (No such file or directory) and failed to load GGUF split from …-00003-of-00003.gguf. quantize_gguf --allow-requantize turned the Gemma 3 270M Q8_0 file (291.5 MB) into Q4_0 (241.4 MB) in 0.6 s and KoboldCpp generated text with it. Not run: CUDA, Vulkan, ROCm and Metal builds, koboldcpp.sh, Windows w64devkit, Intel Mac, ARM boards, convert_hf_to_gguf.py (it needs a large Python environment), CMake builds, LLAMA_PORTABLE.
F98 A Vulkan build with Ubuntu 24.04's packaged shader compiler fails: `GL_EXT_integer_dot_product` and `tensorLayoutNV` are not supported by shaderc 2023.8
Same machine as F97, after apt-get install libvulkan-dev glslc (shaderc 2023.8, spirv-tools 2023.6). make -j4 LLAMA_VULKAN=1 stopped after 70 s in the shader step with error: '#extension' : extension not supported: GL_EXT_integer_dot_product (in mul_mmq.comp and mul_mat_vecq.comp), 'dotPacked4x8EXT' : no matching overloaded function found, and 'tensorLayoutNV' : undeclared identifier (in mul_mm_cm2.comp), then make: *** [Makefile:918: ggml/src/ggml-vulkan-shaders.cpp] Error 2. The project's README lists the Vulkan SDK as the prerequisite, which ships a newer compiler; the distribution package is too old. We downloaded LunarG's SDK (1.4.363.1, shaderc 2026.4) but our sandbox policy stopped us from running it for the build, so we did not complete a Vulkan build. Error counts from the log: 90 for mul_mat_vecq.comp, 84 for mul_mmq.comp (test-data/r2-lnx-2026-11-14/build/vulkan-apt-glslc-errors.txt). Not run: any CUDA, ROCm, Metal, Windows or ARM build.
F99 Embeddings with a small model: 384-dimension unit-length vectors, a batch of six in 0.04 s on a CPU; inputs longer than the limit are cut, not rejected; without a model the endpoint still answers 200 with an empty vector; web search returned an empty list both when off and in our lab when on
lab/scripts/embed_probe.py (lab-lnx, CPU, KoboldCpp 1.124.2, --nomodel --embeddingsmodel all-MiniLM-L6-v2-Q8_0.gguf --embeddingsmaxctx 512 --websearch, test-data/r2-lnx-2026-11-14/embed/). /api/extra/version: embeddings: true, websearch: true, mcp: false, llm: false. /v1/embeddings with six inputs: 200, six objects with embedding, index, object, 384 dimensions, vector length 1.0 (normalized), usage.prompt_tokens 35, 0.04 s; the same text twice gave identical vectors. Cosine similarities: cat/kitten 0.786, cat/stock market 0.197, "The cat sat on the mat."/"A kitten rests on the rug." 0.621, cat sentence/earnings sentence -0.002. A 3000-word input with --embeddingsmaxctx 512 returned 200 and a normal vector with prompt_tokens 511 (the text was cut to the limit, no error). /api/extra/embeddings returned data, model, object, usage; Ollama's /api/embed returned embeddings (2 vectors), model and timing fields. With no embeddings model: /v1/embeddings returned 200 with "embedding": [] and "model": "inactive", and the start-up log listed VectorEmbeddings under "Inactive Modules". POST /api/extra/websearch {"q": "koboldcpp"} returned 200 and [] with --websearch on and also with it off: an empty list does not tell you which. In our lab the same query sent straight to DuckDuckGo's HTML page returned status 202 (no results page), which may be why it was empty; we could not tell. GET /mcp returned 405; POST /mcp with tools/list and no MCP servers configured returned an empty tool list. The earlier run (F27) measured cat/kitten 0.659 with a different copy of the model; similarities depend on the file, so compare only within one file. Not run: TextDB, MCP servers (--mcpfile), the Agent, tool calling beyond F72, rerankers, a GPU, a second embedding model.
F100 RPC works: one KoboldCpp can serve its CPU as a device and another can use it; over loopback it was about 6 times slower than running locally; the host listens on every interface by default; connecting to a host that is not there crashes the client
Lab-lnx, KoboldCpp 1.124.2, Qwen2.5-0.5B Q4_K_M, test-data/r2-lnx-2026-11-14/rpc/. Host: --rpcmode host --rpchost 127.0.0.1 --rpcport 5551 --nomodel printed Starting RPC server v8.0.1, endpoint : 127.0.0.1:5551, Devices: CPU: Intel(R) Xeon(R) Processor @ 2.10GHz (16087 MiB, 16087 MiB free), transport : TCP. Client: --rpcmode connect --rpctargets 127.0.0.1:5551 --gpulayers 99 logged using device RPC0 (127.0.0.1:5551) (unknown id) - 16087 MiB free, offloaded 25/25 layers to GPU, RPC0[127.0.0.1:5551] model buffer size = 373.71 MiB, RPC0[…] KV buffer size = 27.00 MiB, RPC0[…] compute buffer size = 306.02 MiB, Number of Backends: 2, Backend 0: RPC; the host logged Accepted client connection. A 40-token request took 7.2 s through RPC and 1.27 s and 1.13 s running locally with the same flags (this is a loopback test where the "remote device" is the same CPU, so it measures the overhead of the protocol, not of a real network or GPU). Without --rpchost the host printed endpoint : 0.0.0.0:5552 and the port was open on 0.0.0.0 (all interfaces). A client pointed at a host that was not running stopped with ggml/src/ggml-rpc/ggml-rpc.cpp:656: Failed to connect to 127.0.0.1:5551 and a backtrace through ggml_backend_rpc_add_server. Not run: a real second computer, a GPU on either side, --rpccache, --rpcdevice, tensor split, main GPU, split modes, autofit across two GPUs (we have no GPU; a software Vulkan device is a single device).
F101 text-generation-webui is now called TextGen and ships as a desktop app; its README (commit c93f887, 17 Aug 2026) lists backends, APIs and licence; we did not install or run it
git clone --depth 1 https://github.com/oobabooga/text-generation-webui.git worked from the lab (the clone's own remote still shows that name); the README (saved as test-data/r2-lnx-2026-11-14/compare/textgen-README.md) is titled "TextGen", says "A desktop app for local LLMs. Open source, no telemetry. Text, vision, tool-calling, web search. UI + API.", and points downloads to github.com/oobabooga/textgen/releases: portable builds for Linux, Windows and macOS with CUDA, Vulkan, ROCm and CPU-only options, "compatible with GGUF (llama.cpp) models"; ExLlamaV3 and Transformers models, training, image generation and extensions need the full installation, not the portable build. Backends listed: llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3 and TensorRT-LLM; API: OpenAI/Anthropic-compatible with tool calling; web UI at http://127.0.0.1:7860; tool-calling includes web search and MCP servers; image generation through diffusers (Z-Image-Turbo named); extensions include TTS and voice input; licence AGPL-3.0 (LICENSE file). Not run: any install, speed, install time, first reply, SillyTavern connection. No measurement of TextGen against KoboldCpp exists on this site.
F102 The same .kcpps and model-load behaviour holds on macOS (Apple M1 virtual machine, macOS 15.8.2) and Windows Server 2025 as on Linux; on Windows a model path with non-ASCII letters crashed when the output was redirected to a file
lab3 run 1 (test-data/lab3-1-2026-11-14-gha/, GitHub-hosted runners, KoboldCpp 1.124.2: koboldcpp-mac-arm64 and koboldcpp-nocuda.exe). macOS (mac15-probe, 7.5 GB, Apple M1 (Virtual)): a partial .kcpps served 2048, a file with a dashed key (ctx-size) served the default 16384, a legacy file (noblas, blasbatchsize) loaded with 2048, a flag typed on top of a file printed Overriding Config Value: contextsize and served 3072; --exporttemplate differed from --exportconfig in the same six keys as on Linux (F81); an unknown architecture stopped with exit 3 and unknown model architecture: 'zzzzz', a web page saved as .gguf with exit 3 and invalid or unsupported model file (F91); a path with a space and non-ASCII letters started; --analyze printed general.architecture = qwen2. The Metal start-up log said default.metallib not found, loading kernel sources and tensor API disabled for pre-M5 and pre-A19 devices. The unpacked mac build holds koboldcpp_default.so and koboldcpp_failsafe.so (list in mac15-probe/unpack-libs.txt). The downloaded file had no quarantine attribute (xattr printed nothing: a file fetched with curl is not quarantined, so this does not show what a browser download does). Windows (w25-probe): a legacy .kcpps with blasbatchsize 128 and noblas loaded and applied the batch size (batchsize=128); a missing model stopped with exit 2 and Cannot find text model file: <path> (as F19); --analyze exit 0 with the metadata block; a model path with a space, Cyrillic, ö and ß failed with UnicodeEncodeError: 'charmap' codec can't encode characters (exit 1) while stdout was redirected to a file (the Windows default codepage), so this is a redirection effect that a console window may not show; we did not test a normal console. Not run: Windows double-click association, the launcher's file dialogs, a real Mac, a 1.95-era file from the real old version.
F103 All 267 external links of the site return 200 from a normal network
lab3 run 1, job links (GitHub-hosted Ubuntu runner, scripts/lab3_links.py, HEAD then GET, test-data/lab3-1-2026-11-14-gha/links/links.json), 14 Nov 2026: 267 links to github.com (issues, releases, repositories), huggingface.co, ollama.com, lmstudio.ai, aihorde.net and other sites, all status 200 after redirects. This closes the 241 github.com links that returned 403 from the agent sandbox's proxy.
F104 KoboldAI Lite 353 (embedded in 1.124.2) has seven settings tabs and its own sampler defaults; it found the loaded image model by itself; TextDB can use the KoboldCpp embeddings model
lab3 run 3 (test-data/lab3-3-2026-11-14-gha/u24-screens/, Chrome on the Ubuntu runner, scripts/lab3_screens.mjs; KoboldCpp with a text, image, TTS, Whisper and embeddings model and --websearch) plus the Lite source unpacked from the binary (klite.embd). Settings tabs: General, GUI, Samplers, Media, Tokens, Tools, Misc. Usage modes: Instruct Mode, Story Mode, Adventure Mode, Chat Mode. UI styles: Classic, Messenger, Aesthetic, Corpo. Samplers tab defaults with the preset [Default] ("Good default settings, same as Simple Balanced"): context size 16384 (slider 512 to 16384), max output 2048 (16 to 2048), temperature 0.75, repeat penalty 1.05, top-P 0.92, top-K 100; advanced: Top-A 0, Typical 1, TFS 1, Min-P 0, Pr. Pen 0, Seed -1, Range 360, Slope 0.7, Sm.Fac 0, Sm.Crv 1, sampler order 6,0,1,3,4,2,5, T.NSigma 0; further groups DRY (Mult., Base, A.Len, L.Len, Seq. Breaks), XTC (Thresh., Prob.), Adapt.P. (Target, Decay) and Mirostat. Presets offered: Simple Logical, Simple Balanced, Simple Creative, Basic Min-P, Creative Min-P, Basic Top-nsigma, Basic DynaTemp, Basic SmoothSample, Basic SillyTavern, Immortal, Neutral (Disabled), and three legacy ones. Media tab: Image Backend choices Disabled, AI Horde, KCPP / Forge / A1111, OpenAI Compatible Img, ComfyUI, Pollinations.ai; with our image model loaded Lite had already chosen "KCPP / Forge / A1111" and listed the model sdxs-512-tinySDdistilled_Q8_0; defaults step count 20, CFG scale 5, sampler Euler, scheduler default, resolution 512 x 512, clip skip -1, Img2Img strength 0.6, autogenerate images Off (Off, Basic, Smart); text to speech choices Off, Browser Inbuilt TTS, KoboldCpp TTS API, XTTS API Server, AllTalk API Server, OpenAI-Compat. API Server; speech to text choices Off, Detect Voice, Push-To-Talk, Toggle-To-Talk. Tokens tab: stop sequences, logit biases, phrase and word ban, regex replace, thinking and reasoning tags, placeholder tags, classifier-free guidance. Tools tab: MCP tool calling (enable, automatic execution, custom tools, CORS proxy, include the KoboldCpp MCP bridge, server URLs) and KoboldCpp classic web search (enable, multiple passes). Context dialog tabs: Memory, World Info, TextDB; the TextDB provider list is Disabled, Local TextDB (searches in the browser with strings), KoboldCpp Embeddings (needs a loaded embeddings model) and OpenAI API Embeddings; defaults search results 3, added history 300, snippet size 800. /sdui (StableUI: Generate, Images, About, Options, Return to Lite), /musicui, /lcpp and /noscript all rendered; screenshots in the manifest. Not run: sending a picture from Lite's attach button, generating inside Lite, recording speech, a TextDB search with documents, the Scenarios and Save / Load dialogs (the click timed out).
F105 Lite's actions work end to end against a real KoboldCpp: text to speech, automatic image, picture upload, TextDB with the KoboldCpp embeddings provider, and speech input
lab4 run 2 (test-data/lab4-2-2026-11-14-gha/lite-actions/, Chrome on the Ubuntu runner with a fake microphone playing the JFK sample, scripts/lab4_lite_actions.mjs, KoboldCpp 1.124.2 with Qwen2.5-0.5B plus image, TTS, Whisper and embeddings models; a second server with SmolVLM-256M and its projector for the picture test). (1) Settings, Media, Text to speech = "KoboldCpp TTS API", then Test: one POST /api/extra/tts with {"input":"Hello there, this is a test.","voice":"kobo"}. (2) The Add File button opens "Add Audio or Image": Generate Image (Automatic, From Prompt), Generate Speech (From Prompt), Capture From (Camera, Microphone), Upload a File (Image / Audio), Drag Drop / Paste from Clipboard, Customize Media Settings, Launch StableUI, Launch llama.cpp UI. With the image backend set to KCPP / Forge / A1111, Automatic sent POST /sdapi/v1/txt2img with the last AI reply as the prompt ({[OUTPUT]}}A red apple on a wooden table exudes warmth, color, and tradition, symbolizing joy and nostalgia.), 20 steps, sampler Euler, and the picture appeared under the reply. (3) Upload a File with a vision model loaded: Lite sent POST /v1/chat/completions with an image_url part holding a data:image URI (request 9,361 characters), not the native images field; SmolVLM-256M answered "Triangles." to a picture of a red circle and a blue rectangle (the tiny model got it wrong; the plumbing worked). With the text-only Qwen server Lite sent its instruct turns to /api/extra/generate/stream. (4) TextDB with the provider KoboldCpp Embeddings and two short documents: Lite sent three POST /api/extra/embeddings (the question and each document, with truncate: true) before the generate request, and Qwen2.5-0.5B answered "Orsolya Brandt is the pastry chef at the Falcon Hotel." (5) Speech to text = Detect Voice with the fake microphone: Lite posted the recorded clips to /api/extra/transcribe (4 in 25 s), put each transcript in as a user turn without any click ("so my fellow america", "What your country can do for you and", "What you all my fellow America?") and the model replied to each; clips are cut mid-sentence and Whisper tiny.en misheard words. Not run: Push-To-Talk and Toggle-To-Talk, Capture From Camera, voices other than kobo, a real microphone, Smart autogenerate, the Scenarios and Save / Load dialogs.
F106 The macOS file is ad-hoc signed and not notarized; Gatekeeper's check (`spctl --assess`) rejects it, with or without the quarantine attribute; run from a terminal it still started on our test Mac
lab5 runs 1 and 2 (test-data/lab5-2026-11-14-gha/mac15-quarantine/, GitHub macOS 15 runner, koboldcpp-mac-arm64 1.124.2 downloaded with curl). file: Mach-O 64-bit executable arm64. codesign -dv: Signature=adhoc, TeamIdentifier=not set, flags adhoc. spctl --status: assessments enabled. spctl --assess --type execute --verbose=4 printed rejected (exit 3) both before and after we set com.apple.quarantine by hand (0083;5f3b0000;Safari;, the attribute a browser adds). ./bin/kcpp --version printed 1.124.2 and exit 0 with the quarantine attribute set and after removing it. So on this machine, started from a shell, the quarantine did not stop it; double-clicking in Finder or opening it from a downloaded folder may show the "unidentified developer" prompt that the page describes, but we could not show the prompt (no screen) and a CI machine may differ from a normal Mac. Not run: Finder launch, the Privacy & Security "Open Anyway" button, macOS on an Intel Mac.
F107 Same machine, same file, six runners, measured the same way (streaming `/v1/completions`): with threads set explicitly KoboldCpp generates within a few percent of llama.cpp, Ollama and TextGen, reads long prompts about 30 to 55% slower, and starts in 3.5 to 6 s against 0.5 to 2 s
lab4 run 4 (test-data/lab4-4-2026-11-14-gha/compare2/, one GitHub Ubuntu 24.04 runner, 4 vCPUs, AMD EPYC 9V45, scripts/lab4_compare.py): KoboldCpp 1.124.2, llama.cpp llama-server b12377, Ollama 0.40.16, TextGen 4.9 portable CPU build (llama.cpp loader), LM Studio headless (llmster 0.0.27 through lms). Files: Qwen2.5-0.5B-Instruct Q4_K_M (469 MB) and Qwen2.5-1.5B-Instruct Q4_K_M (1,066 MB), 4096 context, 128 tokens at temperature 0, three runs per prompt (a changing prefix stops any prompt cache), median. Method for every runner: time to the first streamed token gives prompt speed (prompt tokens divided by that time; it includes request overhead, so only the 1,652-token prompt is used for prompt speed) and the gap from first to last token gives generation speed. Effective threads from the logs: KoboldCpp, llama-server and TextGen all used 2 threads by default on this 4-vCPU machine (n_threads = 2, Threadpool set to 2 threads); each was also run with 4 set explicitly (--threads 4, -t 4, Ollama PARAMETER num_thread 4). Ollama's default thread count is not in its log. 0.5B, 4 threads, generation after a short prompt / after the 1,652-token prompt (tokens/s) and prompt speed on the long prompt: KoboldCpp 78.5 / 51.8 / 206.8; llama-server 71.9 / 54.1 / 338.6; Ollama 68.5 / 60.3 / 357.4; TextGen 69.6 / 57.9 / 294.9. 1.5B, 4 threads: KoboldCpp 38.8 / 31.0 / 87.9; llama-server 40.4 / 31.8 / 192.8; Ollama 40.0 / 32.8 / 195.4; TextGen 37.2 / 31.3 / 148.4. LM Studio (1.5B only, loaded with context 4096, one slot, GPU off, its default threads): 26.4 / 21.7 / 46.2; its 0.5B run failed to find the imported file, and its 143 s "ready" figure includes importing the file and starting the daemon, so it is not a start-up time. Start-up (0.5B / 1.5B): KoboldCpp 3.5 / 5.0 s with 4 threads, llama-server 0.5 / 1.0, Ollama 0.9 / 1.3 (model load, service already up), TextGen 1.5 / 1.5. Memory (sum of resident memory of the runner's processes after the runs, 1.5B): KoboldCpp 1,407 MB, Ollama 1,467, llama-server 2,030, TextGen 2,204, LM Studio 2,860; llama-server's default of several slots and a larger cache probably explains part of its figure. Install sizes: KoboldCpp 103 MB (one file), llama.cpp 57 MB (folder), Ollama 43 MB plus 2.4 GB of libraries, TextGen portable 916 MB, LM Studio 3.4 GB of runtime. A CPU test with 0.5B and 1.5B models on one machine type: GPU, larger models and other CPUs can differ, and earlier runs on other GitHub machines (F44, F46) had different absolute speeds with the same ordering for prompt speed.
Evidence browser
Every recorded result row. Filter by machine, search by case, and download the table. The raw log and result file open from the last column (these files are not indexed by search engines; user names, addresses and keys are removed).
| Raw files | ||||||||
|---|---|---|---|---|---|---|---|---|
| ci-1 | macos-15-arm64 | api-all | 4.16 | 726.7 | log · json | |||
| ci-1 | macos-15-arm64 | api-multiuser-1 | 3.88 | 748.8 | log · json | |||
| ci-1 | macos-15-arm64 | api-parallel-2 | 4.89 | 759.3 | log · json | |||
| ci-1 | macos-15-arm64 | api-password | 4.21 | 747.5 | log · json | |||
| ci-1 | macos-15-arm64 | base-default-args | 5.24 | 7.39 | 82.07 | 875.1 | log · json | |
| ci-1 | macos-15-arm64 | base-fp16 | 13.68 | 5.53 | 144.6 | 1187.8 | log · json | |
| ci-1 | macos-15-arm64 | base-q2_k | 29.64 | 5.98 | 108.44 | 671.4 | log · json | |
| ci-1 | macos-15-arm64 | base-q4_k_m | 6.62 | 6.27 | 92.51 | 753.8 | log · json | |
| ci-1 | macos-15-arm64 | base-q8_0 | 7.34 | 5.19 | 111.19 | 940.2 | log · json | |
| ci-1 | macos-15-arm64 | bench-q4_k_m | 76.1 | 4.13 | 84.75 | 0 | log · json | |
| ci-1 | macos-15-arm64 | bench-q8_0 | 52 | 8.29 | 126.02 | 0 | log · json | |
| ci-1 | macos-15-arm64 | conf-export | 0 | log · json | ||||
| ci-1 | macos-15-arm64 | conf-load | 4.66 | 8.21 | 74.95 | 767.1 | log · json | |
| ci-1 | macos-15-arm64 | ctx-131072-q4 | 10.68 | 8.26 | 66.09 | 2230.7 | log · json | |
| ci-1 | macos-15-arm64 | ctx-2048 | 4.65 | 8.96 | 82.61 | 727.9 | log · json | |
| ci-1 | macos-15-arm64 | ctx-32768 | 4.85 | 8.86 | 107.34 | 1101.8 | log · json | |
| ci-1 | macos-15-arm64 | ctx-32768-fa | 5.27 | 5.89 | 73.79 | 1076.6 | log · json | |
| ci-1 | macos-15-arm64 | ctx-32768-fa-q4kv | 4.07 | 7.96 | 66.2 | 808.2 | log · json | |
| ci-1 | macos-15-arm64 | ctx-32768-fa-q8kv | 5.3 | 8.11 | 76.31 | 900.1 | log · json | |
| ci-1 | macos-15-arm64 | ctx-8192 | 4.13 | 8.51 | 64.74 | 783.3 | log · json | |
| ci-1 | macos-15-arm64 | fail-ctx-too-big | 1.12 | 1.4 | 2 | log · json | ||
| ci-1 | macos-15-arm64 | fail-failsafe | 3.39 | 44.66 | 95.72 | 649.5 | log · json | |
| ci-1 | macos-15-arm64 | fail-gpulayers-no-gpu | 3.61 | 8.2 | 70.24 | 737.9 | log · json | |
| ci-1 | macos-15-arm64 | fail-missing-model | 5.83 | 45.9 | 2 | log · json | ||
| ci-1 | macos-15-arm64 | fail-noavx2 | 3.66 | 7.64 | 49.35 | 755 | log · json | |
| ci-1 | macos-15-arm64 | fail-not-gguf | 5.49 | 47 | 3 | log · json | ||
| ci-1 | macos-15-arm64 | fail-truncated | 5.37 | 160.6 | 3 | log · json | ||
| ci-1 | macos-15-arm64 | fail-usecuda-on-nocuda | 4.26 | 7.72 | 76.61 | 743.7 | log · json | |
| ci-1 | macos-15-arm64 | fail-usevulkan-no-gpu | 3.98 | 8.02 | 74.36 | 752.9 | log · json | |
| ci-1 | macos-15-arm64 | mmap-on | 4.22 | 8.69 | 73.64 | 741.2 | log · json | |
| ci-1 | macos-15-arm64 | threads-1 | 4.43 | 9.06 | 130.28 | 784.9 | log · json | |
| ci-1 | macos-15-arm64 | threads-2 | 4.24 | 7.41 | 155.62 | 784.4 | log · json | |
| ci-1 | macos-15-arm64 | threads-3 | 4.26 | 7.26 | 176.91 | 788.9 | log · json | |
| ci-1 | ubuntu-24.04-cuda | base-q4_k_m | 9.05 | 41.53 | 92.44 | 685.6 | log · json | |
| ci-1 | ubuntu-24.04-cuda | fail-usecuda-on-nocuda | 7.01 | 26 | 1 | log · json | ||
| ci-1 | ubuntu-24.04-cuda | fail-usevulkan-no-gpu | 9.51 | 46.26 | 122.19 | 642.1 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | api-all | 4.01 | 657.2 | log · json | |||
| ci-1 | ubuntu-24.04-nocuda | api-multiuser-1 | 4.01 | 642.2 | log · json | |||
| ci-1 | ubuntu-24.04-nocuda | api-parallel-2 | 4.01 | 643.8 | log · json | |||
| ci-1 | ubuntu-24.04-nocuda | api-password | 4.01 | 639.1 | log · json | |||
| ci-1 | ubuntu-24.04-nocuda | base-default-args | 4.01 | 46.66 | 122.98 | 784.3 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | base-fp16 | 4.51 | 30.25 | 82.95 | 1434 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | base-q2_k | 3.54 | 46.33 | 111.31 | 613 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | base-q4_k_m | 3.51 | 46.34 | 111.22 | 685.7 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | base-q8_0 | 3.51 | 45.25 | 132.51 | 861 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | bench-q4_k_m | 50.1 | 22.86 | 95.44 | 0 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | bench-q8_0 | 45.4 | 20.76 | 109.12 | 0 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | conf-export | 0 | log · json | ||||
| ci-1 | ubuntu-24.04-nocuda | conf-load | 4.01 | 46.72 | 121.79 | 664.5 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | ctx-131072-q4 | 4.51 | 46.56 | 121.02 | 2129.7 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | ctx-2048 | 4.01 | 46.61 | 121.79 | 615.7 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | ctx-32768 | 4.01 | 46.34 | 122.98 | 975.8 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | ctx-32768-fa | 4.01 | 45.96 | 117.65 | 975.8 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | ctx-32768-fa-q4kv | 4.01 | 44.31 | 113.77 | 697.7 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | ctx-32768-fa-q8kv | 4.01 | 45.31 | 117.65 | 794.5 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | ctx-8192 | 4.01 | 46.63 | 118.38 | 688.4 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | fail-ctx-too-big | 1.5 | 23.9 | 2 | log · json | ||
| ci-1 | ubuntu-24.04-nocuda | fail-failsafe | 8.51 | 5.73 | 6.66 | 639.6 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | fail-gpulayers-no-gpu | 4.01 | 46.83 | 122.19 | 639.7 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | fail-missing-model | 5.51 | 47.1 | 2 | log · json | ||
| ci-1 | ubuntu-24.04-nocuda | fail-noavx2 | 4.01 | 29.58 | 66.32 | 639.8 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | fail-not-gguf | 5.51 | 47.2 | 3 | log · json | ||
| ci-1 | ubuntu-24.04-nocuda | fail-truncated | 6.01 | 60.5 | 3 | log · json | ||
| ci-1 | ubuntu-24.04-nocuda | fail-usecuda-on-nocuda | 4.01 | 46.51 | 113.1 | 640 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | fail-usevulkan-no-gpu | 4.01 | 46.63 | 118.75 | 642.2 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | mmap-on | 3.51 | 46.48 | 115.5 | 639.8 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | threads-1 | 4.01 | 26.07 | 56.59 | 685.7 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | threads-2 | 4.01 | 46.03 | 111.16 | 685.7 | log · json | |
| ci-1 | ubuntu-24.04-nocuda | threads-4 | 4.01 | 48.71 | 119.39 | 685.4 | log · json | |
| ci-1 | ubuntu-24.04-oldpc | base-q4_k_m | 8.03 | 28.65 | 66.72 | 685.8 | log · json | |
| ci-1 | windows-2022-cuda | base-q4_k_m | 11.73 | 42.31 | 96 | 656.4 | log · json | |
| ci-1 | windows-2022-cuda | fail-gpulayers-no-gpu | 10.57 | 47.42 | 145.04 | 619.2 | log · json | |
| ci-1 | windows-2022-cuda | fail-usecuda-on-nocuda | 10.05 | 33.8 | 1 | log · json | ||
| ci-1 | windows-2022-cuda | fail-usevulkan-no-gpu | 10.08 | 33.7 | 1 | log · json | ||
| ci-1 | windows-2022-nocuda | api-all | 5.55 | 619.2 | log · json | |||
| ci-1 | windows-2022-nocuda | api-multiuser-1 | 5.56 | 619.1 | log · json | |||
| ci-1 | windows-2022-nocuda | api-parallel-2 | 5.56 | 624.3 | log · json | |||
| ci-1 | windows-2022-nocuda | api-password | 5.56 | 616.9 | log · json | |||
| ci-1 | windows-2022-nocuda | base-default-args | 6.07 | 49.61 | 162.39 | 764.4 | log · json | |
| ci-1 | windows-2022-nocuda | base-fp16 | 6.04 | 34.9 | 200.68 | 1396.5 | log · json | |
| ci-1 | windows-2022-nocuda | base-q2_k | 7.74 | 48.08 | 133.51 | 582.9 | log · json | |
| ci-1 | windows-2022-nocuda | base-q4_k_m | 6.53 | 47.37 | 133.17 | 655.6 | log · json | |
| ci-1 | windows-2022-nocuda | base-q8_0 | 5.54 | 47.25 | 173.34 | 831.1 | log · json | |
| ci-1 | windows-2022-nocuda | bench-q4_k_m | 47.7 | 15.46 | 119.46 | 0 | log · json | |
| ci-1 | windows-2022-nocuda | bench-q8_0 | 37.6 | 16.09 | 147.49 | 0 | log · json | |
| ci-1 | windows-2022-nocuda | conf-export | 0 | log · json | ||||
| ci-1 | windows-2022-nocuda | conf-load | 5.54 | 49.46 | 144.49 | 644.3 | log · json | |
| ci-1 | windows-2022-nocuda | ctx-131072-q4 | 6.04 | 46.36 | 127.95 | 2113.6 | log · json | |
| ci-1 | windows-2022-nocuda | ctx-2048 | 7.14 | 45.63 | 141.79 | 595 | log · json | |
| ci-1 | windows-2022-nocuda | ctx-32768 | 6.05 | 48.97 | 157.68 | 957.3 | log · json | |
| ci-1 | windows-2022-nocuda | ctx-32768-fa | 6.05 | 48.05 | 129.25 | 957.1 | log · json | |
| ci-1 | windows-2022-nocuda | ctx-32768-fa-q4kv | 14.07 | 47.87 | 131.03 | 680 | log · json | |
| ci-1 | windows-2022-nocuda | ctx-32768-fa-q8kv | 6.03 | 45.81 | 134.28 | 776.6 | log · json | |
| ci-1 | windows-2022-nocuda | ctx-8192 | 6.05 | 47.94 | 136.69 | 668 | log · json | |
| ci-1 | windows-2022-nocuda | fail-ctx-too-big | 5.05 | 27.8 | 2 | log · json | ||
| ci-1 | windows-2022-nocuda | fail-failsafe | 11.57 | 5.75 | 6.34 | 619 | log · json | |
| ci-1 | windows-2022-nocuda | fail-gpulayers-no-gpu | 5.04 | 46.31 | 145.04 | 619.8 | log · json | |
| ci-1 | windows-2022-nocuda | fail-missing-model | 10.06 | 36.4 | 2 | log · json | ||
| ci-1 | windows-2022-nocuda | fail-noavx2 | 5.58 | 31.67 | 71.56 | 618.8 | log · json | |
| ci-1 | windows-2022-nocuda | fail-not-gguf | 7.55 | 36.9 | 3 | log · json | ||
| ci-1 | windows-2022-nocuda | fail-truncated | 7.56 | 38.1 | 3 | log · json | ||
| ci-1 | windows-2022-nocuda | fail-usecuda-on-nocuda | 6.03 | 49.73 | 163.79 | 619.7 | log · json | |
| ci-1 | windows-2022-nocuda | fail-usevulkan-no-gpu | 5.03 | 33.8 | 1 | log · json | ||
| ci-1 | windows-2022-nocuda | mmap-on | 5.54 | 47.25 | 134.75 | 529.9 | log · json | |
| ci-1 | windows-2022-nocuda | threads-1 | 6.04 | 33.82 | 78.94 | 655.5 | log · json | |
| ci-1 | windows-2022-nocuda | threads-2 | 6.05 | 48.41 | 136.77 | 655.5 | log · json | |
| ci-1 | windows-2022-nocuda | threads-4 | 6.04 | 60.78 | 161.37 | 655.2 | log · json | |
| ci-1 | windows-2022-oldpc | base-q4_k_m | 10.17 | 28.24 | 72.56 | 655.7 | log · json | |
| ci-1 | windows-2025-cuda | base-q4_k_m | 12.27 | 36.57 | 89.3 | 658.9 | log · json | |
| ci-1 | windows-2025-cuda | fail-gpulayers-no-gpu | 13.12 | 44.76 | 114.11 | 622.6 | log · json | |
| ci-1 | windows-2025-cuda | fail-usecuda-on-nocuda | 12.6 | 36.7 | 1 | log · json | ||
| ci-1 | windows-2025-cuda | fail-usevulkan-no-gpu | 14.1 | 44.77 | 103.26 | 625.8 | log · json | |
| ci-1 | windows-2025-nocuda | api-all | 6.02 | 622.2 | log · json | |||
| ci-1 | windows-2025-nocuda | api-multiuser-1 | 6.05 | 622.7 | log · json | |||
| ci-1 | windows-2025-nocuda | api-parallel-2 | 6.56 | 625.6 | log · json | |||
| ci-1 | windows-2025-nocuda | api-password | 6.54 | 620.2 | log · json | |||
| ci-1 | windows-2025-nocuda | base-default-args | 6.58 | 38.39 | 68.72 | 767.5 | log · json | |
| ci-1 | windows-2025-nocuda | base-fp16 | 7.06 | 31.21 | 169.4 | 1399.7 | log · json | |
| ci-1 | windows-2025-nocuda | base-q2_k | 6.75 | 45.63 | 109.22 | 586.2 | log · json | |
| ci-1 | windows-2025-nocuda | base-q4_k_m | 6.55 | 43.18 | 113.42 | 659.2 | log · json | |
| ci-1 | windows-2025-nocuda | base-q8_0 | 8.06 | 46.7 | 152.51 | 834.5 | log · json | |
| ci-1 | windows-2025-nocuda | bench-q4_k_m | 57.1 | 14.37 | 90.83 | 0 | log · json | |
| ci-1 | windows-2025-nocuda | bench-q8_0 | 44.4 | 14.87 | 125.47 | 0 | log · json | |
| ci-1 | windows-2025-nocuda | conf-export | 0 | log · json | ||||
| ci-1 | windows-2025-nocuda | conf-load | 6.06 | 43.93 | 84.82 | 647.3 | log · json | |
| ci-1 | windows-2025-nocuda | ctx-131072-q4 | 6.05 | 44.28 | 106.74 | 2116.9 | log · json | |
| ci-1 | windows-2025-nocuda | ctx-2048 | 6.04 | 45.33 | 106.15 | 598.4 | log · json | |
| ci-1 | windows-2025-nocuda | ctx-32768 | 6.55 | 44.85 | 92.91 | 960.1 | log · json | |
| ci-1 | windows-2025-nocuda | ctx-32768-fa | 6.08 | 44.44 | 109.51 | 960.7 | log · json | |
| ci-1 | windows-2025-nocuda | ctx-32768-fa-q4kv | 6.56 | 43.45 | 102.43 | 682.9 | log · json | |
| ci-1 | windows-2025-nocuda | ctx-32768-fa-q8kv | 6.05 | 42.27 | 101.06 | 780.2 | log · json | |
| ci-1 | windows-2025-nocuda | ctx-8192 | 6.08 | 44.38 | 121.79 | 671.3 | log · json | |
| ci-1 | windows-2025-nocuda | fail-ctx-too-big | 10.07 | 30.5 | 2 | log · json | ||
| ci-1 | windows-2025-nocuda | fail-failsafe | 11.11 | 5.18 | 5.92 | 622.1 | log · json | |
| ci-1 | windows-2025-nocuda | fail-gpulayers-no-gpu | 6.06 | 44.61 | 123.38 | 622.6 | log · json | |
| ci-1 | windows-2025-nocuda | fail-missing-model | 7.54 | 39.4 | 2 | log · json | ||
| ci-1 | windows-2025-nocuda | fail-noavx2 | 6.56 | 27.66 | 60.61 | 622.2 | log · json | |
| ci-1 | windows-2025-nocuda | fail-not-gguf | 7.56 | 39.8 | 3 | log · json | ||
| ci-1 | windows-2025-nocuda | fail-truncated | 20.13 | 44.6 | 3 | log · json | ||
| ci-1 | windows-2025-nocuda | fail-usecuda-on-nocuda | 11.63 | 37.42 | 79.83 | 622.9 | log · json | |
| ci-1 | windows-2025-nocuda | fail-usevulkan-no-gpu | 7.1 | 44.8 | 113.1 | 625.7 | log · json | |
| ci-1 | windows-2025-nocuda | mmap-on | 6.03 | 44.94 | 110.47 | 532.8 | log · json | |
| ci-1 | windows-2025-nocuda | threads-1 | 7.05 | 26.51 | 62.33 | 658.7 | log · json | |
| ci-1 | windows-2025-nocuda | threads-2 | 6.52 | 44.2 | 104.21 | 659.3 | log · json | |
| ci-1 | windows-2025-nocuda | threads-4 | 6.59 | 49.86 | 127.19 | 659.2 | log · json | |
| ci-1 | windows-2025-oldpc | base-q4_k_m | 25.36 | 23.18 | 57.7 | 658.5 | log · json | |
| ci-2 | macos-15-arm64 | api-all | 4.21 | 748.7 | log · json | |||
| ci-2 | macos-15-arm64 | api-multiuser-1 | 4.74 | 742.9 | log · json | |||
| ci-2 | macos-15-arm64 | api-parallel-2 | 4.92 | 753.9 | log · json | |||
| ci-2 | macos-15-arm64 | api-password | 4.52 | 750.9 | log · json | |||
| ci-2 | macos-15-arm64 | base-default-args | 6.95 | 4.88 | 57.66 | 894.2 | log · json | |
| ci-2 | macos-15-arm64 | base-fp16 | 12.35 | 4.33 | 114.07 | 1177.5 | log · json | |
| ci-2 | macos-15-arm64 | base-q2_k | 33.19 | 6.99 | 122.05 | 661.9 | log · json | |
| ci-2 | macos-15-arm64 | base-q4_k_m | 6.57 | 5.46 | 96.53 | 762.1 | log · json | |
| ci-2 | macos-15-arm64 | base-q8_0 | 7.49 | 4.79 | 127.78 | 909.5 | log · json | |
| ci-2 | macos-15-arm64 | bench-q4_k_m | 70.3 | 5.05 | 85.79 | 0 | log · json | |
| ci-2 | macos-15-arm64 | bench-q8_0 | 59.6 | 4.92 | 126.47 | 0 | log · json | |
| ci-2 | macos-15-arm64 | conf-export | 0 | log · json | ||||
| ci-2 | macos-15-arm64 | conf-load | 5.19 | 8.17 | 75.25 | 778.5 | log · json | |
| ci-2 | macos-15-arm64 | ctx-131072-q4 | 8.45 | 7.94 | 68.59 | 2206.4 | log · json | |
| ci-2 | macos-15-arm64 | ctx-2048 | 4.43 | 8.7 | 96.69 | 706.2 | log · json | |
| ci-2 | macos-15-arm64 | ctx-32768 | 4.13 | 9.1 | 89.2 | 1083.6 | log · json | |
| ci-2 | macos-15-arm64 | ctx-32768-fa | 4.2 | 9.61 | 86.17 | 1087.2 | log · json | |
| ci-2 | macos-15-arm64 | ctx-32768-fa-q4kv | 3.72 | 7.52 | 87.36 | 814.1 | log · json | |
| ci-2 | macos-15-arm64 | ctx-32768-fa-q8kv | 4.35 | 8.2 | 69.6 | 884.6 | log · json | |
| ci-2 | macos-15-arm64 | ctx-8192 | 4.07 | 8.83 | 85.01 | 804 | log · json | |
| ci-2 | macos-15-arm64 | fail-ctx-too-big | 1.18 | 20.5 | 2 | log · json | ||
| ci-2 | macos-15-arm64 | fail-failsafe | 4.07 | 39.6 | 97.94 | 659.3 | log · json | |
| ci-2 | macos-15-arm64 | fail-gpulayers-no-gpu | 4.7 | 7.67 | 54.13 | 742.3 | log · json | |
| ci-2 | macos-15-arm64 | fail-missing-model | 5.54 | 46.2 | 2 | log · json | ||
| ci-2 | macos-15-arm64 | fail-noavx2 | 4.7 | 8.71 | 73.22 | 739.9 | log · json | |
| ci-2 | macos-15-arm64 | fail-not-gguf | 5.94 | 45.9 | 3 | log · json | ||
| ci-2 | macos-15-arm64 | fail-truncated | 6.37 | 161.8 | 3 | log · json | ||
| ci-2 | macos-15-arm64 | fail-usecuda-on-nocuda | 5.36 | 8.24 | 78.84 | 758.8 | log · json | |
| ci-2 | macos-15-arm64 | fail-usevulkan-no-gpu | 5.22 | 8.58 | 80.85 | 744.2 | log · json | |
| ci-2 | macos-15-arm64 | feat-multimodal | 4.53 | 912.2 | log · json | |||
| ci-2 | macos-15-arm64 | mmap-on | 4.06 | 7.43 | 47.92 | 742.4 | log · json | |
| ci-2 | macos-15-arm64 | threads-1 | 5.64 | 7.07 | 119.37 | 787.8 | log · json | |
| ci-2 | macos-15-arm64 | threads-2 | 4.09 | 6.48 | 188.94 | 801.2 | log · json | |
| ci-2 | macos-15-arm64 | threads-3 | 4.72 | 6.91 | 157.4 | 768.8 | log · json | |
| ci-2 | macos-15-arm64-usecpu | base-q4_k_m | 4.28 | 9.7 | 114 | 780.4 | log · json | |
| ci-2 | macos-15-arm64-usecpu | base-q8_0 | 5.41 | 9.31 | 189.44 | 931.6 | log · json | |
| ci-2 | macos-15-arm64-usecpu | feat-multimodal | 4.06 | 921.9 | log · json | |||
| ci-2 | ubuntu-24.04-cuda | base-q4_k_m | 10.53 | 25.11 | 101.86 | 685.7 | log · json | |
| ci-2 | ubuntu-24.04-cuda | fail-usecuda-on-nocuda | 8.51 | 2.1 | 1 | log · json | ||
| ci-2 | ubuntu-24.04-cuda | fail-usevulkan-no-gpu | 10.51 | 31.91 | 116.56 | 642.1 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | api-all | 4.01 | 657.3 | log · json | |||
| ci-2 | ubuntu-24.04-nocuda | api-multiuser-1 | 4.01 | 642.4 | log · json | |||
| ci-2 | ubuntu-24.04-nocuda | api-parallel-2 | 4.01 | 641.7 | log · json | |||
| ci-2 | ubuntu-24.04-nocuda | api-password | 4.01 | 639.2 | log · json | |||
| ci-2 | ubuntu-24.04-nocuda | base-default-args | 4.01 | 31.58 | 110.47 | 784.5 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | base-fp16 | 4.51 | 19.13 | 92.79 | 1434.1 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | base-q2_k | 4.03 | 34.35 | 103.23 | 613.2 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | base-q4_k_m | 4.01 | 31.27 | 105.79 | 685.8 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | base-q8_0 | 4.01 | 27.34 | 133.16 | 860.9 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | bench-q4_k_m | 52.3 | 20.58 | 92.18 | 0 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | bench-q8_0 | 45.5 | 18.08 | 111.64 | 0 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | conf-export | 0 | log · json | ||||
| ci-2 | ubuntu-24.04-nocuda | conf-load | 4.01 | 31.47 | 114.8 | 664.5 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | ctx-131072-q4 | 5.01 | 31.72 | 112.43 | 2129.8 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | ctx-2048 | 4.01 | 31.59 | 115.5 | 615.8 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | ctx-32768 | 4.01 | 31.27 | 114.46 | 975.9 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | ctx-32768-fa | 4.01 | 31.39 | 111.76 | 975.8 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | ctx-32768-fa-q4kv | 4.01 | 31.09 | 112.76 | 697.7 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | ctx-32768-fa-q8kv | 4.01 | 31.3 | 110.14 | 794.3 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | ctx-8192 | 4.01 | 31.2 | 114.11 | 688.5 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | fail-ctx-too-big | 2 | 32.8 | 2 | log · json | ||
| ci-2 | ubuntu-24.04-nocuda | fail-failsafe | 8.01 | 6.46 | 7.7 | 639.6 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | fail-gpulayers-no-gpu | 4.01 | 31.66 | 115.15 | 639.9 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | fail-missing-model | 6.01 | 47.1 | 2 | log · json | ||
| ci-2 | ubuntu-24.04-nocuda | fail-noavx2 | 4.01 | 24.27 | 67.26 | 639.8 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | fail-not-gguf | 6.01 | 47.3 | 3 | log · json | ||
| ci-2 | ubuntu-24.04-nocuda | fail-truncated | 6.01 | 60.8 | 3 | log · json | ||
| ci-2 | ubuntu-24.04-nocuda | fail-usecuda-on-nocuda | 4.01 | 31.69 | 115.5 | 640.1 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | fail-usevulkan-no-gpu | 4.01 | 31.75 | 115.85 | 642.1 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | feat-multimodal | 3.51 | 772.6 | log · json | |||
| ci-2 | ubuntu-24.04-nocuda | mmap-on | 4.01 | 31.55 | 108.88 | 639.6 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | threads-1 | 4.01 | 17.23 | 54.94 | 685.8 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | threads-2 | 4.01 | 31.5 | 106.15 | 685.9 | log · json | |
| ci-2 | ubuntu-24.04-nocuda | threads-4 | 4.01 | 43.11 | 116.86 | 685.5 | log · json | |
| ci-2 | ubuntu-24.04-oldpc | base-q4_k_m | 8.54 | 21.8 | 66.19 | 685.7 | log · json | |
| ci-2 | windows-2022-cuda | base-q4_k_m | 11.25 | 41.26 | 135.64 | 655.6 | log · json | |
| ci-2 | windows-2022-cuda | fail-gpulayers-no-gpu | 10.06 | 46.85 | 98.7 | 619 | log · json | |
| ci-2 | windows-2022-cuda | fail-usecuda-on-nocuda | 10.13 | 33.5 | 1 | log · json | ||
| ci-2 | windows-2022-cuda | fail-usevulkan-no-gpu | 10.11 | 33.9 | 1 | log · json | ||
| ci-2 | windows-2022-nocuda | api-all | 5.55 | 618.9 | log · json | |||
| ci-2 | windows-2022-nocuda | api-multiuser-1 | 5.56 | 619.3 | log · json | |||
| ci-2 | windows-2022-nocuda | api-parallel-2 | 5.53 | 624.4 | log · json | |||
| ci-2 | windows-2022-nocuda | api-password | 6.07 | 616.9 | log · json | |||
| ci-2 | windows-2022-nocuda | base-default-args | 9.57 | 45.8 | 151.39 | 764 | log · json | |
| ci-2 | windows-2022-nocuda | base-fp16 | 5.56 | 32.7 | 194.07 | 1396.1 | log · json | |
| ci-2 | windows-2022-nocuda | base-q2_k | 6.19 | 48.56 | 123.59 | 582.8 | log · json | |
| ci-2 | windows-2022-nocuda | base-q4_k_m | 6.56 | 45.81 | 135.96 | 655.6 | log · json | |
| ci-2 | windows-2022-nocuda | base-q8_0 | 6.54 | 44.55 | 168.11 | 831 | log · json | |
| ci-2 | windows-2022-nocuda | bench-q4_k_m | 45.5 | 16.1 | 116.75 | 0 | log · json | |
| ci-2 | windows-2022-nocuda | bench-q8_0 | 60.6 | 15.75 | 80.08 | 0 | log · json | |
| ci-2 | windows-2022-nocuda | conf-export | 0 | log · json | ||||
| ci-2 | windows-2022-nocuda | conf-load | 5.55 | 45.28 | 131.03 | 644 | log · json | |
| ci-2 | windows-2022-nocuda | ctx-131072-q4 | 6.06 | 46.33 | 135.71 | 2113.8 | log · json | |
| ci-2 | windows-2022-nocuda | ctx-2048 | 5.19 | 45.98 | 97.69 | 595.1 | log · json | |
| ci-2 | windows-2022-nocuda | ctx-32768 | 6.54 | 47.23 | 95.48 | 956.7 | log · json | |
| ci-2 | windows-2022-nocuda | ctx-32768-fa | 6.05 | 45.05 | 148.44 | 956.7 | log · json | |
| ci-2 | windows-2022-nocuda | ctx-32768-fa-q4kv | 6.07 | 45.12 | 131.94 | 679.1 | log · json | |
| ci-2 | windows-2022-nocuda | ctx-32768-fa-q8kv | 6.07 | 47.65 | 146.72 | 776.5 | log · json | |
| ci-2 | windows-2022-nocuda | ctx-8192 | 6.04 | 46.28 | 132.4 | 667.9 | log · json | |
| ci-2 | windows-2022-nocuda | fail-ctx-too-big | 5.06 | 30.5 | 2 | log · json | ||
| ci-2 | windows-2022-nocuda | fail-failsafe | 11.57 | 5.62 | 6.51 | 619 | log · json | |
| ci-2 | windows-2022-nocuda | fail-gpulayers-no-gpu | 5.04 | 47.74 | 139.71 | 619.2 | log · json | |
| ci-2 | windows-2022-nocuda | fail-missing-model | 7.57 | 36.5 | 2 | log · json | ||
| ci-2 | windows-2022-nocuda | fail-noavx2 | 5.54 | 29.65 | 63.44 | 619.3 | log · json | |
| ci-2 | windows-2022-nocuda | fail-not-gguf | 7.54 | 36.4 | 3 | log · json | ||
| ci-2 | windows-2022-nocuda | fail-truncated | 7.59 | 37.7 | 3 | log · json | ||
| ci-2 | windows-2022-nocuda | fail-usecuda-on-nocuda | 6.06 | 44.63 | 134.75 | 619.6 | log · json | |
| ci-2 | windows-2022-nocuda | fail-usevulkan-no-gpu | 5.05 | 33.9 | 1 | log · json | ||
| ci-2 | windows-2022-nocuda | feat-multimodal | 6.04 | 741.9 | log · json | |||
| ci-2 | windows-2022-nocuda | mmap-on | 5.57 | 43.84 | 150.79 | 530.2 | log · json | |
| ci-2 | windows-2022-nocuda | threads-1 | 5.54 | 33.35 | 79.96 | 655.9 | log · json | |
| ci-2 | windows-2022-nocuda | threads-2 | 5.55 | 46.11 | 129.07 | 655.6 | log · json | |
| ci-2 | windows-2022-nocuda | threads-4 | 5.55 | 56.74 | 155.15 | 655.3 | log · json | |
| ci-2 | windows-2022-oldpc | base-q4_k_m | 12.23 | 29.75 | 72.61 | 655.8 | log · json | |
| ci-2 | windows-2025-cuda | base-q4_k_m | 12.29 | 34.33 | 102.02 | 659.1 | log · json | |
| ci-2 | windows-2025-cuda | fail-gpulayers-no-gpu | 15.12 | 43.35 | 116.92 | 622.6 | log · json | |
| ci-2 | windows-2025-cuda | fail-usecuda-on-nocuda | 12.59 | 36.4 | 1 | log · json | ||
| ci-2 | windows-2025-cuda | fail-usevulkan-no-gpu | 14.08 | 41.83 | 118.01 | 625.8 | log · json | |
| ci-2 | windows-2025-nocuda | api-all | 6.55 | 622.2 | log · json | |||
| ci-2 | windows-2025-nocuda | api-multiuser-1 | 6.55 | 622.5 | log · json | |||
| ci-2 | windows-2025-nocuda | api-parallel-2 | 6.04 | 627.9 | log · json | |||
| ci-2 | windows-2025-nocuda | api-password | 6.56 | 620.2 | log · json | |||
| ci-2 | windows-2025-nocuda | base-default-args | 6.57 | 42.02 | 99.22 | 768 | log · json | |
| ci-2 | windows-2025-nocuda | base-fp16 | 7.09 | 26.02 | 169.47 | 1399.8 | log · json | |
| ci-2 | windows-2025-nocuda | base-q2_k | 7.75 | 45.02 | 103.98 | 586.4 | log · json | |
| ci-2 | windows-2025-nocuda | base-q4_k_m | 15.06 | 40.87 | 102.66 | 659.3 | log · json | |
| ci-2 | windows-2025-nocuda | base-q8_0 | 9.08 | 42.62 | 146.43 | 834.5 | log · json | |
| ci-2 | windows-2025-nocuda | bench-q4_k_m | 56 | 14.58 | 91.75 | 0 | log · json | |
| ci-2 | windows-2025-nocuda | bench-q8_0 | 44.9 | 14.18 | 126.48 | 0 | log · json | |
| ci-2 | windows-2025-nocuda | conf-export | 0 | log · json | ||||
| ci-2 | windows-2025-nocuda | conf-load | 11.08 | 34.33 | 85.59 | 647.4 | log · json | |
| ci-2 | windows-2025-nocuda | ctx-131072-q4 | 6.05 | 37.13 | 93.83 | 2116.8 | log · json | |
| ci-2 | windows-2025-nocuda | ctx-2048 | 6.04 | 43.46 | 106.15 | 598.4 | log · json | |
| ci-2 | windows-2025-nocuda | ctx-32768 | 6.03 | 43.35 | 95.72 | 960.1 | log · json | |
| ci-2 | windows-2025-nocuda | ctx-32768-fa | 6.52 | 43.52 | 120.25 | 960 | log · json | |
| ci-2 | windows-2025-nocuda | ctx-32768-fa-q4kv | 6.05 | 40.34 | 124.59 | 682.8 | log · json | |
| ci-2 | windows-2025-nocuda | ctx-32768-fa-q8kv | 6.55 | 41.61 | 104.11 | 779.7 | log · json | |
| ci-2 | windows-2025-nocuda | ctx-8192 | 6.04 | 43.6 | 108.88 | 671.6 | log · json | |
| ci-2 | windows-2025-nocuda | fail-ctx-too-big | 5.03 | 31.2 | 2 | log · json | ||
| ci-2 | windows-2025-nocuda | fail-failsafe | 11.1 | 5.24 | 6.04 | 622.5 | log · json | |
| ci-2 | windows-2025-nocuda | fail-gpulayers-no-gpu | 6.05 | 44.02 | 102.43 | 622.6 | log · json | |
| ci-2 | windows-2025-nocuda | fail-missing-model | 10.08 | 39.6 | 2 | log · json | ||
| ci-2 | windows-2025-nocuda | fail-noavx2 | 7.02 | 26.51 | 62.6 | 623.1 | log · json | |
| ci-2 | windows-2025-nocuda | fail-not-gguf | 7.54 | 39.5 | 3 | log · json | ||
| ci-2 | windows-2025-nocuda | fail-truncated | 7.55 | 40.9 | 3 | log · json | ||
| ci-2 | windows-2025-nocuda | fail-usecuda-on-nocuda | 6.54 | 43.52 | 103.83 | 623.1 | log · json | |
| ci-2 | windows-2025-nocuda | fail-usevulkan-no-gpu | 6.56 | 43.7 | 116.92 | 625.5 | log · json | |
| ci-2 | windows-2025-nocuda | feat-multimodal | 13.61 | 745.5 | log · json | |||
| ci-2 | windows-2025-nocuda | mmap-on | 6.53 | 39.62 | 100.53 | 533.3 | log · json | |
| ci-2 | windows-2025-nocuda | threads-1 | 7.07 | 25.39 | 62.63 | 659.5 | log · json | |
| ci-2 | windows-2025-nocuda | threads-2 | 6.05 | 43.36 | 114.32 | 659.2 | log · json | |
| ci-2 | windows-2025-nocuda | threads-4 | 6.57 | 48.45 | 129.69 | 659.4 | log · json | |
| ci-2 | windows-2025-oldpc | base-q4_k_m | 21.38 | 22.75 | 60.77 | 659.2 | log · json | |
| run1-2026-11-13-local | lab-lnx | api-all | 4.01 | 657.1 | log · json | |||
| run1-2026-11-13-local | lab-lnx | api-multiuser-1 | 3.51 | 642.1 | log · json | |||
| run1-2026-11-13-local | lab-lnx | api-parallel-2 | 4.01 | 643.7 | log · json | |||
| run1-2026-11-13-local | lab-lnx | api-password | 4.02 | 638.9 | log · json | |||
| run1-2026-11-13-local | lab-lnx | base-default-args | 4.01 | 18.18 | 104.68 | 784.3 | log · json | |
| run1-2026-11-13-local | lab-lnx | base-fp16 | 4.51 | 17.69 | 107.45 | 1426.8 | log · json | |
| run1-2026-11-13-local | lab-lnx | base-q2_k | 3.51 | 23.47 | 116.52 | 612.8 | log · json | |
| run1-2026-11-13-local | lab-lnx | base-q4_k_m | 4.01 | 18.49 | 112.86 | 685.6 | log · json | |
| run1-2026-11-13-local | lab-lnx | base-q8_0 | 4.01 | 19.73 | 136.04 | 860.8 | log · json | |
| run1-2026-11-13-local | lab-lnx | bench-q4_k_m | 50.9 | 16.25 | 97.78 | 0 | log · json | |
| run1-2026-11-13-local | lab-lnx | bench-q8_0 | 43 | 16.06 | 121.93 | 0 | log · json | |
| run1-2026-11-13-local | lab-lnx | conf-export | 0 | log · json | ||||
| run1-2026-11-13-local | lab-lnx | conf-load | 4.01 | 19.29 | 101.6 | 664.2 | log · json | |
| run1-2026-11-13-local | lab-lnx | ctx-131072-q4 | 5.01 | 22.11 | 123.78 | 2122.7 | log · json | |
| run1-2026-11-13-local | lab-lnx | ctx-2048 | 3.51 | 18.08 | 114.11 | 615.7 | log · json | |
| run1-2026-11-13-local | lab-lnx | ctx-32768 | 4.01 | 21.63 | 124.18 | 975.6 | log · json | |
| run1-2026-11-13-local | lab-lnx | ctx-32768-fa | 4.01 | 18.75 | 134.75 | 975.8 | log · json | |
| run1-2026-11-13-local | lab-lnx | ctx-32768-fa-q4kv | 4.01 | 19.31 | 121.79 | 697.5 | log · json | |
| run1-2026-11-13-local | lab-lnx | ctx-32768-fa-q8kv | 4.01 | 18.87 | 127.52 | 794.3 | log · json | |
| run1-2026-11-13-local | lab-lnx | ctx-8192 | 4.01 | 18.61 | 114.11 | 688.3 | log · json | |
| run1-2026-11-13-local | lab-lnx | fail-ctx-too-big | 2 | 9.7 | 2 | log · json | ||
| run1-2026-11-13-local | lab-lnx | fail-failsafe | 7.51 | 5.85 | 9.13 | 639.4 | log · json | |
| run1-2026-11-13-local | lab-lnx | fail-gpulayers-no-gpu | 4.01 | 20.01 | 121.41 | 639.6 | log · json | |
| run1-2026-11-13-local | lab-lnx | fail-missing-model | 6.01 | 47.1 | 2 | log · json | ||
| run1-2026-11-13-local | lab-lnx | fail-noavx2 | 4.01 | 14.77 | 64.08 | 639.6 | log · json | |
| run1-2026-11-13-local | lab-lnx | fail-not-gguf | 6.01 | 47 | 3 | log · json | ||
| run1-2026-11-13-local | lab-lnx | fail-truncated | 6.01 | 62.5 | 3 | log · json | ||
| run1-2026-11-13-local | lab-lnx | fail-usecuda-on-nocuda | 4.51 | 19.54 | 102.98 | 639.9 | log · json | |
| run1-2026-11-13-local | lab-lnx | fail-usevulkan-no-gpu | 4.01 | 18.85 | 91.79 | 642 | log · json | |
| run1-2026-11-13-local | lab-lnx | feat-multimodal | 3.01 | 772.3 | log · json | |||
| run1-2026-11-13-local | lab-lnx | mmap-on | 3.51 | 19.61 | 125.83 | 639.6 | log · json | |
| run1-2026-11-13-local | lab-lnx | threads-1 | 3.51 | 11.03 | 66.9 | 685.4 | log · json | |
| run1-2026-11-13-local | lab-lnx | threads-2 | 3.51 | 18.85 | 117.2 | 685.7 | log · json | |
| run1-2026-11-13-local | lab-lnx | threads-4 | 4.01 | 24.55 | 201.39 | 685.4 | log · json |
Screenshots
| lab-w25-launcher-quick-01 | LAB ci-1 launcher screenshot | lab-w25 | 1.124.2 | 2026-11-13 | The KoboldCpp launcher on first open: Quick Launch tab with Backend, the default checkboxes and the 16384 context slider. |
| lab-w22-launcher-quick-01 | LAB ci-1 launcher screenshot | lab-w22 | 1.124.2 | 2026-11-13 | Same launcher on Windows Server 2022: identical layout and defaults. |
| lab-lnx-lite-welcome-01 | run1 Playwright | lab-lnx | 1.124.2 | 2026-11-13 | First visit to KoboldAI Lite: the UI-style picker (Classic, Aesthetic, Corpo) appears before you can chat. |
| lab-lnx-lite-firstprompt-01 | run1 Playwright | lab-lnx | 1.124.2 | 2026-11-13 | A prompt typed before choosing a UI style stays in the box: the welcome dialog must be closed first. |
| lab-lnx-api-swagger-01 | run1 Playwright | lab-lnx | 1.124.2 | 2026-11-13 | The built-in API documentation at /api: OpenAI-compatible endpoints listed first. |
| lab-lnx-lite-mobile-01 | run1 Playwright | lab-lnx | 1.124.2 | 2026-11-13 | KoboldAI Lite on a phone-sized screen (390 px wide). |
| gha-w25-launcher-quick-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Quick Launch tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-hardware-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Hardware tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-context-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Context tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-files-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Loaded Files tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-network-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Network tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-horde-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Horde Worker tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-imagegen-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Image Gen tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-audio-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Audio tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-agent-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Agent tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-admin-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Admin tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-w25-launcher-extra-01 | T-GUI-01 (lab2-1, win_launcher_tabs.py) | gha-w25 | 1.124.2 | 2026-11-13 | The Extra tab of the KoboldCpp launcher with its default values, on Windows (no GPU, so the backend shows "Use CPU"). |
| gha-u24-sd-sdxs-01 | T-MM-01 (lab2-1, media_probe.py) | gha-u24 | 1.124.2 | 2026-11-13 | An image KoboldCpp generated on a CPU-only machine in 8.4 seconds (sdxs-512 tiny model, 1 step, 512×512). |
| gha-u24-sd-openai-01 | T-MM-01 (lab2-1) | gha-u24 | 1.124.2 | 2026-11-13 | The same model called through the OpenAI-compatible images endpoint. |
| gha-st-firstrun-01 | T-INT-01 (lab2-2, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | First start of SillyTavern: you must choose a persona name before connecting anything. |
| gha-st-home-01 | T-INT-01 (lab2-2, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | SillyTavern after the welcome dialog, not yet connected. |
| gha-st-connect-01 | T-INT-01 (lab2-2, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | The plug icon opens API Connections. |
| gha-st-connect-02 | T-INT-01 (lab2-2, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | Text Completion with API Type set to KoboldCpp: the API key is optional and the example URL is http://127.0.0.1:5001. |
| gha-st-url-history-01 | T-INT-01 (lab2-4) | gha-u24 | 1.124.2 | 2026-11-13 | SillyTavern lists previously tried URLs in a dropdown that covers the Connect button; press Escape first. |
| gha-st-textcomp-correct-01 | T-INT-01 (lab2-6, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | Connected: the green dot shows the model KoboldCpp has loaded. |
| gha-st-textcomp-v1-suffix-01 | T-INT-01 (lab2-6, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | A URL ending in /v1 also connects in KoboldCpp mode; the red "Missing key" marker appears although no key is needed. |
| gha-st-textcomp-localhost-name-01 | T-INT-01 (lab2-6, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | http://localhost:5001 works the same as 127.0.0.1 on the same computer. |
| gha-st-textcomp-wrong-port-01 | T-INT-01 (lab2-6, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | A wrong port: "No connection..." and the plug icon turns red. |
| gha-st-textcomp-https-on-http-server-01 | T-INT-01 (lab2-6, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | https:// does not work against a normal (http) KoboldCpp server. |
| gha-st-chat-01 | T-INT-01 (lab2-6, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | A first reply through SillyTavern from KoboldCpp. |
| gha-st-chatcomp-missing-v1-01 | T-INT-01 (lab2-6, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | Chat Completion, Custom (OpenAI-compatible): the base URL validated and listed the model even without /v1. |
| gha-st-chatcomp-correct-01 | T-INT-01 (lab2-6, st_connect.js) | gha-u24 | 1.124.2 | 2026-11-13 | Chat Completion with the documented /v1 base URL. |
| gha-u24-lite-settings-general-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite settings, General tab: usage mode, UI style, instruct tag preset and responses. |
| gha-u24-lite-settings-samplers-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite settings, Samplers tab: context size, max output, sampler preset and the default sampler values. |
| gha-u24-lite-settings-media-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite settings, Media tab: image backend (found the loaded image model by itself), image options and text to speech. |
| gha-u24-lite-settings-tools-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite settings, Tools tab: MCP tool calling and KoboldCpp web search. |
| gha-u24-lite-settings-tokens-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite settings, Tokens tab: stop sequences, logit biases, phrase bans, regex replace and thinking tags. |
| gha-u24-lite-settings-gui-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite settings, GUI tab: themes, colour schemes, markdown and extra UIs. |
| gha-u24-lite-settings-misc-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite settings, Misc tab: autosave, advanced settings such as EOS token ban, and experimental options. |
| gha-u24-lite-context-textdb-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite Context dialog, TextDB tab: provider choice, document storage and search settings. |
| gha-u24-lite-context-worldinfo-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite Context dialog, World Info tab. |
| gha-u24-lite-context-memory-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite Context dialog, Memory tab. |
| gha-u24-lite-classic-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite in Instruct mode with the classic theme, a fresh session. |
| gha-u24-lite-welcome-mobile-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite welcome screen at phone width. |
| gha-u24-lite-classic-mobile-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | Lite chat screen at phone width. |
| gha-u24-stableui-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | StableUI, the image page served at /sdui. |
| gha-u24-musicui-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | MusicUI, the music page served at /musicui. |
| gha-u24-lcpp-ui-01 | lab3 run 3 (scripts/lab3_screens.mjs, real run with text, image, TTS, Whisper and embeddings models) | gha-u24 | 1.124.2 | 2026-11-14 | The bundled llama.cpp web page served at /lcpp. |
| gha-u24-lite-tts-settings-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | Lite Media tab with Text to speech set to KoboldCpp TTS API. |
| gha-u24-lite-addfile-menu-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | Lite Add File menu: generate image, generate speech, capture, upload. |
| gha-u24-lite-image-generated-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | A picture made inside Lite with the Automatic button, shown under the last reply. |
| gha-u24-lite-upload-image-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | Lite after uploading a test picture through Add File. |
| gha-u24-lite-vision-reply-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | Lite with a vision model: the picture and the answer Triangles. |
| gha-u24-lite-textdb-provider-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | TextDB set to KoboldCpp Embeddings with two short documents pasted in. |
| gha-u24-lite-textdb-answer-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | Lite answering a question from the TextDB documents. |
| gha-u24-lite-stt-settings-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | Lite Media tab with Speech to text set to Detect Voice. |
| gha-u24-lite-stt-result-01 | lab4 run 2 (scripts/lab4_lite_actions.mjs, real run with fake microphone) | gha-u24 | 1.124.2 | 2026-11-14 | Lite after Detect Voice: spoken phrases appear as user turns and get replies. |
Why outputs differ between machines
Same model, same prompt, same seed, temperature 0: on Linux SmolVLM-256M answered “The shapes are circles and squares.” (twice, in two runs), but on Windows Server 2022, Windows Server 2025 and the macOS virtual machine it produced a LaTeX-like fragment F33. Small floating-point differences between processors change which word wins when two are close. For our method that means: quality and model tests are run on a named real machine and reported for that machine only; a seed is not portable between computers; and speeds vary, which is why we take three runs. A second example: on GitHub’s macOS virtual machine the Metal path was slower than the CPU F29, so lab numbers are never real-world speed.
Lab numbers versus real hardware
None of our lab machines has a graphics card. Their speeds are CPU numbers for a 0.5B model. They prove behaviour (which message, which file, the relative effect of a flag) and relative differences, and never the speed you will see with a real model on a real PC. GPU, VRAM and speed-by-model claims need real hardware, or the software Vulkan device, which shows exactly where memory goes but says nothing about speed. Every lab chart carries a badge: “CPU lab, 0.5B model: relative differences only”.
What we have not tested yet
GPU offload, layer splits, VRAM needs per model/quant/context, KV and compute buffers, --lowvram, MoE placement | Software Vulkan (lavapipe) allocation logs for models up to 12B (proven: F35–F37) | — | planned |
| Real-GPU speed, CUDA vs Vulkan speed, partial-offload slowdown, autofit on a real card, CUDA/Metal OOM wording, old NVIDIA arch errors | Optional cloud-GPU budget (testing guide §3.5) | Users report (with links) until then | planned |
| Windows security prompts | Defender scans on runners (lab2) | SmartScreen and Smart App Control dialogs: Microsoft docs + users report | planned |
| Image, TTS, music, video | CPU runs with small models (lab2 media job), template loads and allocation logs for big ones | Generation speed on GPUs: users report or cloud-GPU | planned |
| SillyTavern, Open WebUI, n8n, Home Assistant, LangChain, Agnaistic, RisuAI | Scripted on GitHub runners (Playwright / Docker / APIs) | Account-only cloud apps (Janitor AI, Chub): their public docs | planned |
| Remote tunnel, LAN, phone browser | Docker networks, tunnel reachability from a second runner, mobile emulation, Android emulator | — | planned |
| Android/Termux, Colab, RunPod/Vast, Docker | Termux Docker image, Android emulator, notebook replay on a runner, Docker image on a runner | Phone speed/thermals and Colab UI: users report | planned |
| Launcher on Linux/macOS, High-DPI | Xvfb, macOS runner screen capture, Windows DPI scaling on runners | Wayland specifics: users report | planned |
| Model quality | 3B–12B (and MoE) on CPU runners with the published prompt set and rubric (T-MODEL-02) | Models that need > 16 GB: metadata only | planned |
Items leave this list when their test is done and recorded. Several rows in the table have since been partly covered: see the findings from F59 onward.
How we pick and test models
A model appears on the models page only if its licence allows personal use, it comes from its original publisher or a named, widely used quantizer, and the current KoboldCpp supports its architecture. Sizes are read from the real file headers. “Tested by us” means we loaded the model in our lab and recorded what happened; so far that is two models. We have not scored any model on answer quality, so no model is ranked or called best. When we do, the prompts and the written scoring rules will be published here, and scores will be given only for the named machine they were run on.
Re-testing every release
Within a week of a new KoboldCpp release the lab runs again for the new version, the flags and the API routes are compared with the previous version, the key real-hardware tests are repeated where we have the hardware, and the pages whose findings changed get a new test date. The release list is on the releases page.
Tools are tested too
Our own tools have automated tests that run before every build, with fixtures built from our recorded runs. Last passed on 14 November 2026:
| Download picker | Every OS, GPU and CPU combination, old file names, the "wrong file" cases, links go to GitHub only | tests/tools/download-picker.test.ts, qa:download |
| VRAM calculator | KV sizes equal the logged sizes, 13 GGUF header fixtures, predicted memory within 10% of measured allocations, share link, no requests in local mode | tests/fit.test.ts, gguf.test.ts, hf.test.ts, memory.test.ts, qa:fit |
| Command builder | 20 configurations run through the real binary with --exportconfig, 30 checker fixtures, quoting for three shells, share link without secrets | tests/tools/config-binary.test.ts, tests/command.test.ts, qa:config |
| Log analyzer | 212 fixture logs from our runs, redaction, speed, edge cases, no network during analysis | tests/tools/log-analyzer.test.ts, qa:tool-log |
| Pages | Numbers on pages equal the recorded data | tests/tune-pages.test.ts and others |
The API tester and the template finder are not built yet.
Found something different?
Send your result (KoboldCpp version, your machine, the log) to the address on the contact page. We re-test, then update the finding or publish a correction on the corrections page.
Frequently asked questions
Do you test on real GPUs?
Not yet. Our lab has CPU machines and a software Vulkan device that shows where memory goes, but no graphics card, so we publish no real-GPU speeds. Pages that need them say so, or label the number as users' reports.
Why are some numbers from machines without a GPU?
They prove behaviour and relative effects: which message appears, which file loads, how much a flag changes speed. They are never the speed you will see with a real model on a real PC, and every lab chart carries a badge saying so.
How often do you re-test?
After every KoboldCpp release we re-run the lab, compare the flags and endpoints with the previous version, and update the pages whose findings changed. The newest version we have run is 1.124.2.
Can I download your data?
Yes. The evidence browser below has a CSV download, and every row links to the raw log and result file. The raw files are not indexed by search engines but anyone can open them.
Why can the same prompt give a different answer on my PC?
A small model can pick a different word when two choices are almost equally likely, and floating-point arithmetic differs slightly between processors and operating systems. We saw exactly that: the same model, prompt and seed gave different answers on Linux and on Windows.