Was no llama-swap template on CA and needed it so here we go!

llama-swap is a lightweight proxy that sits in front of llama-server and allows for hot swap models. You can serve many models from a single OpenAI-compatible endpoint without manually restarting anything.

Features

Hot model loading and unloading

Aliases and groups for multi-model concurrency

OpenAI-compatible API

GPU Support

The template includes tags for NVIDIA (cuda), AMD (rocm), Vulkan (AMD/Intel iGPU), and CPU-only. Each tag description includes the required Extra Parameters for that GPU type.

Setup

config.yaml Must be manually created example config is linked to below. You can also view llama-swap github for other examples

Use Unraid terminal to make edits so filesystem watchers can see files been changed for Hot Reload I.E nano /mnt/user/appdata/llama-swap/config/config.yaml

"Waiting for Approval" Community Applications Install

Search for llama-swap in the Apps tab

Set your Models and Config directory paths

Create a config.yaml in your Config directory (example config linked in the Overview)

Reference your models in config.yaml as /models/yourmodel.gguf

Select the tag matching your GPU and update Extra Parameters accordingly

Start the container and access the UI at http://your-server-ip:8080/ui

Manual Install

1. Download the template XML from the link below

2. Place the XML file in /boot/config/plugins/dockerMan/templates-user/ on your Unraid server

3. In the Docker tab click Add Container and select llama-swap from the template dropdown

4. Set your Models and Config directory paths

5. Select the tag matching your GPU and update Extra Parameters accordingly

6. Start the container and access the UI at http://your-server-ip:8080/ui

Template download: https://raw.githubusercontent.com/PikkonMG/unraid-docker-templates/main/templates/llama-swap.xml

Links

GitHub: https://github.com/mostlygeek/llama-swap

Example config: https://github.com/PikkonMG/unraid-docker-templates/blob/main/examples/llama-swap/example-llama-swap-config.yaml

Support: https://github.com/mostlygeek/llama-swap/issues

My Unraid Template Repo: https://github.com/PikkonMG/unraid-docker-templates

Edited March 30Mar 30 by PikkonMG

Thanks for setting this up, it's working great for me so far

36 minutes ago, opticon said:

Thanks for setting this up, it's working great for me so far

🫡 Np.

I'm getting - "did not find expected key"

The config file is in place, wondering if the template needs an update?

1 hour ago, i_max said:

I'm getting - "did not find expected key"

The config file is in place, wondering if the template needs an update

That error is a YAML parse error I think, right now I can’t reproduce it from the current template working fine on my end. Please paste your current config.yaml and the full log line showing "did not find expected key" so I can check and see where it might be coming from.

Edited June 14Jun 14 by PikkonMG

On 6/14/2026 at 12:02 PM, i_max said:

I'm getting - "did not find expected key"

The config file is in place, wondering if the template needs an update?

Wanted to check back, did you figure it out if not please share your config.