oromis95

Can confirm, the container was created incorrectly and must be modified to work, lest you risk losing all your data on the next docker update.

For anyone that is seeing this years down the road, the secrets files don't actually sit in the boot directories, but are only accessible within the docker containers and are deleted at each boot. As far as a solution for anyone that sees this, I don't have one yet.

Thank you for adding your solution, really glad to hear!

Hello, I'm oromis95 and have worked closely with Henk717 to provide you a community application for KoboldCpp. KoboldCpp is a lightweight but powerful AI backend, bundled with KoboldAI Lite front-end, meant to provide a user interface to easily interact with LLM models. KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models, inspired by the original KoboldAI. It's a single self-contained distributable from Concedo, that builds off llama.cpp and adds many additional powerful features. You can get an idea of KoboldAILite by visiting https://huggingface.co/spaces/KoboldAI/KoboldAI-Lite. You can find the original project at: https://github.com/LostRuins/koboldcpp. Or Henk's amazing docker at: https://hub.docker.com/r/koboldai/koboldcpp. Quick Start: Just install it from the Community Applications. Everything should already be set up for you. It comes with a very small llm to get you started. You can automatically place a bigger one in the appdata/koboldcpp/ folder once you figure out how it works by adding '--model https://MYWEBSITE/MYMODEL.gguf' to the KARGS variable. Just right click the docker once it starts and open the WebUI. Make sure '/mnt/user/appdata/koboldcpp' is in the 'Workspace' variable, assuming that is your appdata path. If running.it as cpu only, please remove '--gpus all' from Extra Parameters.