koboldcpp.dev

Embeddings, documents, web search and tools in KoboldCpp

Start KoboldCpp with --embeddingsmodel and a GGUF embedding model to get vectors from /v1/embeddings. Without that flag the endpoint still answers, with an empty vector, so check the length. For documents in Lite use TextDB, for live information start with --websearch, and for tools give KoboldCpp an MCP config with --mcpfile. We tested embeddings and the empty-vector trap; web search returned nothing in our lab; MCP, TextDB and the agent are untested here.

Checked against KoboldCpp 1.124.2 · updated 13 Nov 2026
  • lab-lnx: Linux test VM · 4 vCPU · 15 GB · no GPU
  • gha-u24: Ubuntu 24.04 test machine · 4 vCPU · 16 GB · no GPU
How we test
On this page
  1. Which feature do you need?
  2. KoboldCpp embeddings
  3. The empty-vector trap
  4. TextDB in KoboldAI Lite
  5. Web search
  6. Tool calling
  7. MCP tools
  8. The KoboldCpp Agent

Which feature do you need?

Features and what turns them on
Vectors for your own app--embeddingsmodelTested <EvidenceBadge id="F99" />
Lite uses your notes or documentsTextDB in LiteNot run
Current information from the web--websearchReturned nothing in our lab <EvidenceBadge id="F99" />
The model calls tools--mcpfileRoute tested, no server <EvidenceBadge id="F99" />
A bundled tool-using assistant--agentNot run

/api/extra/version shows what is on. In our run, with an embedding model and web search loaded: embeddings true, websearch true, mcp false F99.

KoboldCpp embeddings

  1. Load a GGUF embedding model

    Download one (Hugging Face). In the launcher it goes in the Embeds Model field, or use (since 1.87). (default 4096) limits the text length and puts it on the GPU.

  2. Check the flag

    /api/extra/version must say "embeddings": true.

  3. Ask for vectors

    POST to /v1/embeddings with input as a string or a list.

We ran all-MiniLM-L6-v2 (Q8_0) F99:

Embeddings with all-MiniLM-L6-v2 Q8_0, KoboldCpp 1.124.2, CPU
Six inputs in one request200, six vectors of 384 numbers, each of length 1.0, in 0.04 s
The same text twiceIdentical vectors
cat and kitten (cosine)0.786
cat and stock market0.197
A cat sentence and a kitten sentence0.621
A cat sentence and an earnings sentence-0.002
3000 words with --embeddingsmaxctx 512200 and a normal vector; 511 tokens used, so the text was cut, with no error

Related things are close and unrelated ones are not. The numbers depend on the model file, so compare only within one file: an earlier run with another copy of the model gave 0.659 for cat and kitten F27. Routes: /v1/embeddings (tested), /api/extra/embeddings (answered with data, model, object, usage) and Ollama’s /api/embed (answered with embeddings) F99. For RAG, KoboldCpp only makes vectors: storing and searching them is your app’s job (LangChain). A reranker is not offered (upstream).

The empty-vector trap

/v1/embeddings with no embedding modelF99 · KoboldCpp 1.124.2 · status 200
{"object": "list", "data": [{"object": "embedding", "index": 0, "embedding": []}], "model": "inactive", "usage": {"prompt_tokens": 0, "total_tokens": 0}}1Inactive Modules: … VectorEmbeddings …2
  1. {"object": "list", "data": [{"object": "embedding", "index": 0, "embed: An empty list and the model name inactive.
  2. Inactive Modules: … VectorEmbeddings …: The start-up log says it too.

Detect it by checking the vector’s length, or the embeddings flag before you index F72.

TextDB in KoboldAI Lite

TextDB is retrieval inside Lite (Context, then the TextDB tab). Its provider box offers four choices, read from Lite’s own help text F104: Disabled; Local TextDB, which searches in your browser with strings; KoboldCpp Embeddings, which uses the embeddings model you loaded with --embeddingsmodel; and OpenAI API Embeddings, which sends text to a remote API. Defaults are 3 search results, 300 tokens of added history and a snippet size of 800. It can cause the prompt to be processed again, so give it enough context (context). We ran a search with the KoboldCpp Embeddings provider and two short documents pasted into the TextDB box: Lite sent three POST /api/extra/embeddings (the question and each document) before generating, and Qwen2.5-0.5B answered “Orsolya Brandt is the pastry chef at the Falcon Hotel.” from the stored text F105. We did not test the Local TextDB or OpenAI providers, or large documents.

TextDB set to KoboldCpp Embeddings with two short documents pasted in.gha-u24-lite-textdb-provider-01 · gha-u24 · KoboldCpp 1.124.2 · 14 Nov 2026
Lite TextDB tab with the provider KoboldCpp Embeddings and a document text box holding two short documents
Lite answering a question from the TextDB documents.gha-u24-lite-textdb-answer-01 · gha-u24 · KoboldCpp 1.124.2 · 14 Nov 2026
Lite chat where the question about the Falcon Hotel pastry chef is answered from the stored documents
Lite Context dialog, TextDB tab: provider choice, document storage and search settings.gha-u24-lite-context-textdb-01 · gha-u24 · KoboldCpp 1.124.2 · 14 Nov 2026
Lite Context Data dialog TextDB tab with a provider dropdown set to Disabled, search results 3 and snippet size 800

Web search

(the Network tab’s Enable WebSearch; since 1.81) lets Lite and the agent search the web through the server; the route is POST /api/extra/websearch with q. Search queries leave your PC for the search provider. In Lite the switches are in Settings, Tools tab: “Enable Websearch” and “Use multiple passes” under KoboldCpp Classic Websearch, and above them MCP tool calling with “Enable Toolcalling”, “Automatically Execute Tools”, custom tools and MCP server URLs F104.

Lite settings, Tools tab: MCP tool calling and KoboldCpp web search.gha-u24-lite-settings-tools-01 · gha-u24 · KoboldCpp 1.124.2 · 14 Nov 2026
Lite Settings Tools tab with MCP tool calling, MCP server URLs and the Enable Websearch option

Tool calling

OpenAI-style tools exist since 1.70 and automatic tool choice since 1.87; --jinja gives templates that support tools. With a 0.5B model the tool call came back parsed with the default adapter and as plain text with --jinja, so change one thing at a time F72. See the chat API and chat templates.

MCP tools

takes a Claude-Desktop-style mcp.json listing servers (since 1.106). The /mcp route answers JSON-RPC: a GET returned 405, and tools/list with no servers returned an empty tool list F99. We did not run an MCP server or the bundled UI’s MCP support.

The KoboldCpp Agent

(since 1.122) starts a bundled tool-using assistant in a new terminal, or in the current one with ; resumes a saved session (1.123). The launcher has an Agent tab. untested Third-party agents use the OpenAI-style API: see other apps.

Frequently asked questions

Does KoboldCpp support embeddings?

Yes, with --embeddingsmodel and a GGUF embedding model. In our test a small model returned 384-dimension vectors from /v1/embeddings.

Why are my embeddings empty?

No embedding model is loaded. The endpoint still answers 200 with an empty vector and the model name inactive. Check /api/extra/version for embeddings true, or the length of the vector.

Is TextDB a vector database?

Not by default. Lite offers four providers: Disabled, Local TextDB (keyword search in your browser), KoboldCpp Embeddings (uses the embeddings model you loaded) and OpenAI API Embeddings. We ran a search with the KoboldCpp Embeddings provider and a small model answered correctly from two stored documents.

How do I enable web search?

Start with --websearch (or tick Enable WebSearch in the launcher). The route /api/extra/websearch answers an empty list whether the flag is on or off, so test with a real query. Search queries leave your PC.

Does KoboldCpp support MCP?

Yes: --mcpfile with an MCP config, and a /mcp route that answers JSON-RPC. With no servers configured, tools/list returned an empty list in our test. We did not run an MCP server.

What is the KoboldCpp Agent?

A bundled tool-using assistant started with --agent (since 1.122), in a new terminal or the current one with --cli. We did not run it.

Can KoboldCpp do RAG on its own?

KoboldCpp serves the vectors. Storing and searching them is your app's job: use LangChain or similar.