Hello,

I'm oromis95 and have worked closely with Henk717 to provide you a community application for KoboldCpp. KoboldCpp is a lightweight but powerful AI backend, bundled with KoboldAI Lite front-end, meant to provide a user interface to easily interact with LLM models.
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models, inspired by the original KoboldAI. It's a single self-contained distributable from Concedo, that builds off llama.cpp and adds many additional powerful features.

You can get an idea of KoboldAILite by visiting https://huggingface.co/spaces/KoboldAI/KoboldAI-Lite. 

You can find the original project at: https://github.com/LostRuins/koboldcpp.

Or Henk's amazing docker at: https://hub.docker.com/r/koboldai/koboldcpp.

 

 

Quick Start:

 

Just install it from the Community Applications. Everything should already be set up for you. It comes with a very small llm to get you started. You can automatically place a bigger one in the appdata/koboldcpp/ folder once you figure out how it works by adding

 

'--model https://MYWEBSITE/MYMODEL.gguf'

 

to the KARGS variable.

Just right click the docker once it starts and open the WebUI.

Make sure '/mnt/user/appdata/koboldcpp' is in the 'Workspace' variable, assuming that is your appdata path.

 

If running.it as cpu only, please remove '--gpus all' from Extra Parameters.

Edited April 2, 20251 yr by oromis95

my heroes,  you did precisely what i wanted to do . It work perfectly with my intel arc a750  when i add    --usevulkan to KCPP_ARGS

Thank you for adding your solution, really glad to hear!

i'm not quite sure but i think this would also work with amd gpus

if you want to specify the amount of gpu layers simply add --gpulayers number to KCPP_ARGS