KoboldCpp Setup — Skales Guide

1 · Download KoboldCpp

1

Go to github.com/LostRuins/koboldcpp and download the latest release for your OS.

2

Windows: Download koboldcpp.exe — no installation needed, double-click to run.

3

macOS / Linux: Download the binary, make it executable: chmod +x koboldcpp, then run it.

💡 Which model should I use?

Any GGUF instruct model works. Good starting points: Llama 3.1 8B Instruct, Mistral 7B Instruct, or Qwen2 7B Instruct. Download from Hugging Face — search for "Q4_K_M" quantization for a good speed/quality balance.

2 · Launch KoboldCpp

A

GUI launcher: Double-click koboldcpp.exe, select your GGUF model file from the file picker, then click Launch.

B

CLI: Run from your terminal:

koboldcpp model.gguf --port 5001

Once running, you'll see a message like: Starting Kobold API on port 5001. The web UI is at http://127.0.0.1:5001.

3 · Configure Skales

1

Open Skales → Settings → AI Providers. The page opens on the provider grid: one tile per provider, with badges for what each one can do.

2

Switch on the Custom (OpenAI-compatible) tile. KoboldCpp has no tile of its own — it is reached through this one, the same way llama.cpp and vLLM are.

3

Open the card and set the endpoint URL:

http://127.0.0.1:5001/v1

Use 127.0.0.1 instead of localhost — this avoids IPv6 resolution issues on some systems.

4

Leave the API Key field empty. Custom endpoints are keyless in Skales — the card asks for an address, not a credential.

5

Click Fetch Models. If KoboldCpp is running, the model name will appear in the model dropdown.

6

Tool Calling is ON by default. KoboldCpp handles JSON fixing internally, so tools work even with models that don't natively support structured output.

✅ Verification

Click Test Connection on the card. The connection map checks address, server, model, context, tools and a real test turn, turns green only when the test turn comes back, and names the first link that breaks with what to do about it. If "Fetch Models" shows nothing, check that KoboldCpp is running and the port is correct.

4 · Vision (Optional)

KoboldCpp supports multimodal / vision models (LLaVA, BakLLaVA, etc.).

1

Load a vision-capable GGUF model in KoboldCpp (must include a mmproj projector file).

2

In Skales Settings → Vision, set the Vision URL to the same endpoint:

http://127.0.0.1:5001/v1

3

Select the vision model in the Vision model dropdown. You can now send images in chat.

5 · TTS (Optional)

KoboldCpp has a built-in TTS server (also compatible with XTTS-API-Server).

1

Launch KoboldCpp with a TTS model, or enable the built-in TTS in the launcher UI.

2

In Skales Settings → TTS, set the TTS URL to:

http://127.0.0.1:5001

3

Select KoboldCpp / XTTS as the TTS provider and enable voice responses.

Performance Tips

GPU Acceleration Enable GPU layers in the KoboldCpp launcher for 5–20× faster inference.

Timeout Setting Increase the request timeout to 120 s+ in Skales Settings for large models on slow hardware.

Context Size Lower context (2048–4096) speeds up inference significantly on CPU-only setups.

Quantization Q4_K_M is a good balance. Q5_K_M if you have RAM to spare. Q2_K for very constrained systems.

Tool Call Streaming KoboldCpp withholds streaming during tool calls to fix JSON — this is normal, not a bug.

Use 127.0.0.1 Always use 127.0.0.1 instead of localhost — avoids IPv6 resolution issues on Windows.

Troubleshooting

Problem	Fix
"Ollama is not installed"	You're not using Ollama. Switch on Custom (OpenAI-compatible) in Settings → AI Providers, not the Ollama tile.
Fetch Models returns empty	Ensure KoboldCpp is running and listening on the same port you entered. Try opening `http://127.0.0.1:5001` in a browser — you should see the KoboldCpp UI.
Tool calls not working	Click Test Connection: the Tools link says whether the model accepts tool calls. If it does not, Skales still chats with this model, without tools; for files, the browser and the other tools, load a model with tool support. If you added KoboldCpp as an extra endpoint under Custom Endpoints, its Enable tool calling tick must be on (it is by default). KoboldCpp handles JSON fixing automatically.
Very slow responses	Normal for large models on CPU. Try a smaller model (3B–7B), reduce context size, or enable GPU layers in the KoboldCpp launcher.
Connection refused	KoboldCpp isn't running. Launch it and wait for the "Kobold API on port 5001" message before trying again.
The card is green but the chat fails	Click Test Connection and read the connection map: the first red link is where it breaks. A chat error from KoboldCpp also offers Check connection, which opens the same map. KoboldCpp running on another computer or inside WSL2: see Windows with WSL2, the same rules apply.
"Model not found" error	Click Fetch Models in Skales Settings to refresh the model list after KoboldCpp loads a model.