Build or check a KoboldCpp launch command

Selects the NVIDIA CUDA backend (or hipBLAS/ROCm in ROCm builds). Optionally give a GPU index or 'all'; legacy sub-options (mmq/nommq/lowvram/rowsplit) are still parsed and translated into the newer flags.

Selects the Vulkan backend, which works on NVIDIA, AMD and Intel GPUs. Optionally list device indices; leave empty to auto-detect.

Forces CPU-only inference with no GPU backend. Replaces the old --noblas.

Loads a .kcpps settings file (or .kcppt template, also from a URL). Anything you also pass on the command line wins over the file.

Maximum context window in tokens to allocate (256-524288). Bigger values need more RAM/VRAM; default 16384 in 1.124.

How many model layers to offload to the GPU. -1 (default) turns on autofit; 0 keeps everything on the CPU.

IP address to bind the HTTP server to. Empty (default) listens on all interfaces; use 127.0.0.1 for local-only.

Opens your web browser on the UI once the model has loaded.

For CUDA/Vulkan multi-GPU: space-separated proportions for how much of the model each GPU gets, e.g. `7 3`.

CPU threads used for generation. Default is computed from your physical core count (the value shown in --help is machine-specific).

TCP port for the web UI and all APIs. Default 5001.

Prints the KoboldCpp version and exits.

Hidden legacy flag: old way to disable BLAS; now mapped to --usecpu.

Starts the bundled KoboldCpp Agent (tool-using assistant for code and tasks) in a new terminal window, or in the current one when combined with --cli. Forces Jinja and raises context/gen defaults.

Resumes a saved KoboldCpp Agent session from its JSON save file when the agent starts.

Prints metadata, tensor names and weight types of a GGUF or safetensors file, then exits.

Forces automatic fitting of the model across GPU(s)/CPU (layers, MoE overrides, splits), overriding manual layer settings.

Spare VRAM in MB that autofit leaves free as a safety margin (default 1024).

Logical batch size for prompt processing (default 512). Larger is faster but uses more memory; -1 turns batching off. Formerly --blasbatchsize.

Physical micro-batch size inside each batch. -1 (default) means same as --batchsize; lower it to save memory.

Runs a benchmark instead of starting the server; optionally appends results to a CSV file.

Thread count used during batched prompt processing; defaults to the --threads value.

Instruct-template adapter (JSON file or bundled name such as ChatML, Llama-3) used to format /v1/chat/completions prompts. Default AutoGuess detects it from the model.

Runs an interactive chat in the terminal with no HTTP server. With --agent, runs the agent in the same terminal.

Extra diagnostic output: -1 quiet (used for Horde), 0 normal, 1 verbose, higher for more detail.

Tokens to generate when a request does not specify a length (default 2048). Most frontends send their own value.

llama.cpp-style comma-separated device list (e.g. Vulkan0,Vulkan1) that overrides the normal GPU selection.

Folder where models given as URLs are downloaded to and looked up in (default: working directory).

Number of tokens the draft model proposes per step in speculative decoding (default 4).

GPU layers for the draft model (default 999 = fully offloaded).

Multi-GPU split ratios for the draft model; only applies when the main model is split with --tensor_split.

Small model with the same vocabulary as the main model, used for speculative decoding to speed up generation.

Turns on classifier-free guidance so negative prompts work for text; costs speed and memory.

Writes the current arguments to a .kcpps file (then exits) for reuse with --config.

Writes the current arguments as a shareable .kcppt template with machine-specific settings stripped.

Maximum-compatibility mode for very old CPUs (no AVX); implies --noavx2. Very slow.

Windows only: brings the console window to the front on each new generation to avoid idle slowdowns.

JSON object of default generation fields added to API requests that don't set them, e.g. {"temperature":0.7}.

Makes --gendefaults values override what clients send instead of only filling gaps.

Hard cap on generated tokens for every request (0 = no cap). Formerly --promptlimit.

Raises the process CPU priority. Experimental; can cause system lag.

Skips optional files (LoRA, mmproj, etc.) that cannot be found instead of aborting.

Uses the model's embedded Jinja chat template (including tool calls) for chat completions, falling back to KoboldCpp's universal formatting if it fails. Since 1.123 also covers the former --jinjatools.

Extra JSON variables passed into the Jinja template renderer (llama.cpp's --chat-template-kwargs).

Path to a custom .jinja chat template that replaces the model's own.

Quick switch for thinking/reasoning in Jinja templates: default, true or false. Setting it enables Jinja.

Applies a text LoRA (GGUF) on top of the base model.

Strength multiplier for the text LoRA (default 1.0).

Keeps the KV cache in system RAM instead of VRAM (llama.cpp -nkvo). Fits more layers but is slow.

In multi-GPU setups, index of the GPU used as the main device (-1 = default).

Largest HTTP request body accepted, in MB (default 32); bigger requests are dropped.

Path to a Claude-Desktop-format mcp.json listing MCP tool servers (HTTP or stdio) to expose to the UI and agent.

Multimodal projector GGUF that adds image (and for some models audio) understanding to a compatible text model.

Keeps the vision/audio projector on the CPU even when the main model is on GPU.

Controls on-demand reading of large architecture-specific tensors (e.g. per-layer embeddings): auto (>4 GiB), on, or off.

Experimental VRAM cache (MiB) for MoE experts that otherwise live on the CPU; use with --moecpu or --autofit.

Keeps the MoE expert weights of the first N layers (all if no number) in system RAM, freeing VRAM for big MoE models.

Like --moecpu but for dense FFN weights of the first N layers.

Overrides how many experts are active per token in MoE models (-1 = model default).

Hosts a shared Kobold Lite session that several users can join and edit together.

Disables SSL certificate verification for outgoing connections (e.g. the Horde worker). Insecure.

Disables fast-forwarding, so every prompt is fully reprocessed; also disables context shift.

Turns off flash attention, which is on by default since 1.107.

Uses the slower non-AVX2 library for older CPUs (with Vulkan if requested).

Stops the BOS token being prepended to prompts. Rarely needed.

CUDA only: disables quantized matrix-multiplication (MMQ) kernels.

Starts the server/UI without loading any model (e.g. to use the UI with Horde or a remote API).

Disables ContextShift (KV-cache shifting when the context is full).

Uses a full-size KV cache instead of sliding-window attention on models that support SWA. Needed if you want context shifting on such models.

Shell command to run once the model has loaded (e.g. start a tunnel). Ignored when it comes from a config file unless --allow-config-onready is set.

Lets configs loaded after launch (admin swaps) set --onready. Only for trusted configs.

Overrides GGUF metadata keys, format name=type:value, comma-separated (llama.cpp --override-kv).

Pretends the model was trained on this context length when computing RoPE scaling.

Places tensors matching a regex on a chosen backend buffer (llama.cpp -ot), e.g. to push experts to CPU.

Number of generation slots that can run at once (experimental continuous batching). Default 1.

Total KV budget in tokens shared by all parallel slots; defaults to --contextsize.

API key that clients must send to use text endpoints (image endpoints are not protected). Also settable via KCPP_PASSWORD.

Serves a saved story JSON at /api/extra/preloadstory so Kobold Lite opens with it.

Runs a single prompt, prints the reply to stdout and exits (can be combined with --benchmark).

KV-cache data type: f16 (default), bf16, q8_0, q5_1 or q4_0. Quantizing saves memory; without flash attention only K is quantized. Old numeric values 0-3 still accepted.

Maximum number of requests allowed to wait in the queue (default 10).

Hides prompts and generated text from the console (auto-on for Horde workers).

Per-IP rate limit: each IP may start a new generation only every N seconds (0 = off).

Default reasoning effort for thinking models (none/low/medium/high/xhigh); API requests can override it.

Downloads cloudflared and opens a trycloudflare.com tunnel so the instance is reachable from the internet without port forwarding.

Manual RoPE scaling: frequency scale then frequency base (e.g. 0.25 10000). Without it, scaling is set automatically.

Server-side save database (.jsondb) so users can save/load stories to the server.

Lets a newly started instance on the same port shut this one down, avoiding port clashes.

Keeps KV-cache snapshots in RAM (optionally N slots) so switching between chats avoids full reprocessing. Requires fast-forwarding.

Legacy context-reuse trick that reserves part of the context; superseded by ContextShift and not recommended.

How the model is split across GPUs: layer (default) or tensor (experimental). 'row' is still accepted but, since row split was removed in v1.118, it prints a warning and is treated as tensor split (splitmode_choices_to_int @ v1.124.2).

Serves HTTPS using an unencrypted certificate and key PEM file.

Extra tokens of padding for the SWA KV cache, which controls how far you can rewind before forced reprocessing.

Extracts the bundled files of the single-file binary into a folder (for running from source/unpacked).

Uses a model's built-in multi-token-prediction layers as the draft for speculative decoding, if present.

Locks the model in RAM so the OS cannot swap it out.

Memory-maps the model file instead of reading it fully. Off by default since 1.82.

Loads GGUF files with direct I/O, which can speed up cold loads on some disks.

Maximum image resolution fed to the vision projector (512-2048, default 1024).

Overrides the maximum number of embedding tokens per image (-1 = model default).

Overrides the minimum number of embedding tokens per image (-1 = model default).

Enables the built-in web search proxy (DuckDuckGo) used by Kobold Lite and the agent.

Shows the GUI launcher even when a config was passed, pre-filled with those settings.

Never shows the GUI launcher (headless start); overrides --showgui.

Hidden legacy flag from when mmap was the default.

Model name your AI Horde worker advertises.

Name of your AI Horde worker.

Your AI Horde API key; setting it (with names) starts the embedded Horde worker.

Largest context your Horde worker accepts (0 = same as --contextsize).

Most tokens your Horde worker will generate per job.

Audio VAE used by LTX 2.3 video generation (video with sound).

Shared-server safety: caps image steps and size (optional max side length, min 512).

Caps total image area (e.g. 640 allows 640x640 or 512x768) to limit memory use.

First CLIP text encoder file (e.g. CLIP-L) if not baked into the image model. Formerly --sdclipl.

Second CLIP encoder (e.g. CLIP-G) or tokenizer file. Formerly --sdclipg.

Where image text encoders/LLM run: GPU index, main, or CPU (default CPU).

Conv2D direct mode for image generation: off (default), vaeonly or full. May save memory or crash on some backends.

Uses flash attention in image/video generation.

Image LoRA files (or a folder for runtime LoRAs) to apply.

Strength multiplier(s) for the image LoRAs.

GPU (index, main or CPU) that holds the image-generation weights.

Image/video model (safetensors or GGUF) to enable image generation: SD1.5, SDXL, SD3, Flux, Qwen Image, WAN, etc.

Keeps image weights in RAM and moves them to VRAM only when needed.

PhotoMaker model for face cloning with SDXL models.

Quantize the image model on load to save memory: 0 off, 1 q8, 2 q4.

Text encoder or LLM used by newer image models (T5-XXL, Qwen, etc.). Replaces --sdt5xxl.

CPU threads for image generation (default: same as --threads).

Image size above which VAE decoding is tiled to save memory (default 512; 0 disables).

ESRGAN upscaler model for enlarging generated images.

Custom VAE that replaces the one inside the image model.

Uses the bundled tiny TAESD VAE: very fast, also fixes broken VAEs.

Where the image VAE runs: GPU index, main (default) or CPU.

Caps VRAM use of image generation to this many MB.

Whisper .bin model for speech-to-text, or a SheetSage2 GGUF for music-to-ABC transcription.

Text-to-speech GGUF model (Kokoro, OuteTTS, Qwen3-TTS, Parler, Dia, etc.).

WavTokenizer/codec GGUF required by some TTS models (OuteTTS, Qwen3-TTS).

Runs the TTS model on the GPU.

Maximum audio tokens generated per TTS request (default 4096).

CPU threads for TTS (default: same as --threads).

Folder of reference voice samples for TTS voice cloning.

Music language model (ACE-Step LM or YuE2) for music generation.

Text embedding model used by music generation (e.g. Qwen3-Embedding-0.6B).

Music diffusion (DiT) model, e.g. ACE-Step v1.5 turbo.

Music VAE / audio decoder model.

Unloads music models when not generating to save VRAM.

GGUF embedding model to serve /v1/embeddings and /api/extra/embeddings.

Overrides the maximum context of the embeddings model.

Offloads the embeddings model to the GPU.

RPC GPU sharing over the network: disabled, connect (use remote GPUs) or host (share yours).

Port the RPC server listens on in host mode (default 5551).

IP the RPC server binds to in host mode (default 0.0.0.0).

Comma-separated devices exposed by the RPC host.

Host mode: folder where tensors sent by clients are cached so reloading the same model is faster. Never cleaned; trust all clients.

Connect mode: comma-separated host:port list of remote RPC servers.

Enables admin mode: models/configs can be unloaded and swapped at runtime from the UI or admin API.

Password protecting admin functions (strongly advised on public instances). Also via KCPP_ADMINPASSWORD.

Folder of .kcpps/.kcppt/.gguf files that admin mode may switch between.

Seconds of idleness after which the current model is unloaded automatically (0 = never).

Reverse-proxy router that swaps to the model named in each OpenAI-API request, then answers it. Requires --admin.

HTTP request timeout in seconds (default 600; 0 = library default).

On top of router mode, automatically switches between the model types defined in one config (text, image, audio...) as requests arrive.

In autoswap mode, model families up to this size (MB, default 256) stay loaded; only one larger family is resident at a time.

Base .kcpps applied during admin model swaps when no other base is chosen.