Hey @Holaf.
Just wanted to say well done mate. Project looks fantastic, the ability to use different interfaces just rocks. Really impressive stuff.
I've got one question, perhaps it's obvious, but I can't find the solution to my inconvenience.
Is there a way to unload the model from the GPU memory if not being used for some time, e.g. 15 minutes? A command or a script, or perhaps a line in the config to modify?
I use the same GPU for LLM, and I'm limited to 12 GB (RTX A2000) which doesn't allow me to run both of those things simultaneously. Which is fair
Whenever I use llama, it loads the model to the GPU memory, and after a few minutes of inactivity, llama release the memory and I can use SD without any more actions from my side. If I reverse the scenario, SD will load the model into GPU memory, but it will never clear it (see attached photo). I can't run llama without restarting the SD docker container.
I use Easy Diffusion as my preferred interface.
Keep up the good work!
Thanks.