Run KoboldCpp on a rented cloud GPU
Use the KoboldCpp RunPod image linked from the project's README, set the model link in KCPP_MODEL and extra launch flags in KCPP_ARGS, pick a GPU with enough VRAM for your model, and open port 5001 when the download finishes. You pay per hour while the pod runs, so stop or delete it when you are done. We tested the same Docker image on our own machine, but we did not rent a pod, so we have no prices or GPU speeds.
Checked against KoboldCpp 1.124.2 · updated 13 Nov 2026
- lab-lnx: Linux test VM · 4 vCPU · 15 GB · no GPU
On this page
Which service: KoboldCpp RunPod and the others
| RunPod | The README links a KoboldCpp RunPod image (koboldai.org/runpodcpp); it is built on the official Docker image. | Not rented; the image itself was run locally <EvidenceBadge id="F95" /> |
| SimplePod | The README suggests it for smaller models (koboldai.org/simplepod). | Not rented |
| Vast.ai and others | Any service that runs a Docker image can use koboldai/koboldcpp ([Docker](/docker/)). We know of no official template there. | Not rented |
The project says its Docker image is meant mainly for cloud GPU rentals. Its entrypoint uses /runpod-volume as the working folder when that folder exists, which is where RunPod mounts persistent storage (from the reference script).
Step 1: Deploy the template
Open the RunPod image from the README and deploy it on a GPU with enough VRAM (will it fit). We did not deploy one, so we have no screenshots of the template’s fields or the GPU choice.
Step 2: Set the model and flags
The image reads KCPP_MODEL (a link to the GGUF file, or several for a split model), KCPP_ARGS (all launch flags), and also KCPP_IMGMODEL, KCPP_MMPROJ, KCPP_WHISPERMODEL, KCPP_TTSMODEL and KCPP_EMBEDMODEL (from the reference script). Build KCPP_ARGS in the command builder: for example --usecuda --gpulayers 99 --contextsize 8192. Give the disk at least the model size plus a margin. Users report the web page is unavailable until the model finishes downloading (issue 29): the container downloads KoboldCpp, then the model, before it serves. On our CPU run it was ready 12 s after start with a local model and a ready file F95.
Step 3: Connect
Open the pod’s HTTP link for port 5001. For apps, see the API page and SillyTavern.
If several people share a pod, requests queue: by default the second waits for the first F26.
Which GPU for which model, and what it costs
We rented nothing, so we publish no price or speed per GPU. Prices change by the hour and differ between on-demand and community offers. To work out your own cost per 1,000 generated tokens:
| 1. Measure your speed | Generate a reply and read tokens per second from the console. |
| 2. Read the price | The hourly price shown by the service when you deploy. |
| 3. Divide | Cost per 1,000 tokens = hourly price ÷ (tokens per second × 3,600) × 1,000. |
For which GPU fits which model use the calculator with the pod’s VRAM.
Stop paying
Before you log off
Troubleshooting pods
- The container stalls for three minutes and exits: the script does this on an error so you can read it (Docker).
- Template broken or health checks failing: users report both (issues 1199 and 1114); we did not reproduce them.
- AMD template: users report problems (issue 1319); not tested.
- OpenAI API through RunPod’s proxy: users report trouble (issue 973); not tested.
Frequently asked questions
How much does it cost to run KoboldCpp on RunPod?
You pay by the hour for the GPU you pick, and the price changes, so we publish none. Divide the hourly price by 3,600 times your measured tokens per second to get the price per token. Stop the pod when you are done.
Which GPU do I need for a 24B or 32B model?
One with enough VRAM for the file plus context: use the calculator with the pod's VRAM. We did not rent a pod, so we have not measured speed on rented GPUs.
Is there an official KoboldCpp RunPod template?
The project's README links a KoboldCpp RunPod image and a SimplePod option. We did not open them.
How do I keep my pod private?
Set a password in KCPP_ARGS. It protects generation, not everything, and the image opens a public tunnel unless KCPP_DONT_TUNNEL is true. Treat the pod's address as public.