2 hours ago2 hr Hey guys,dont know if this is the right sub, but I am thinking about using my own local AI....I actually have Ollama docker + OpenWebUI + SearXNG and use my CPU to test models (capped at 32GB dockersize for Ollama) like Gemma4 or qwen 3.8 locally. Buying a 32GB AI card is expensive, the Intel B65 or B70 is a very cheap option, but it´s not AMD or Nvidia. So is it fully supported in Unraid 7.3.4 and can I run my LLM´s inside docker on it? Or is it better to buy a AMD R9700 which would be 2400 €. I dont wanna know what a Nvidia card with 32GB is actually....lol....so main target is fur sure to free my system-ram and use the llm inside the gpu-ram and speed it up. some answers take 10 minutes right now :PSo I read some posts about Vulkan and slow token and so on....so who can give a suggestion? Edited 2 hours ago2 hr by Tralien
1 hour ago1 hr You should read, and ask in the support thread for the container you're planning to use. Unraid doesn't have anything to do about it, it's what the container supports that matters.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.