Summary: A support thread for the apps that I've written or just added to Unraid

 

Current Apps:

 

• DailyNotes
• EPlusTV
• Genmon
• OpenRA Server
• ollama
• ollama-webui
• ComfyUI-Magic
• Refact
• Tabby
• Stable Diffusion

The ollama container seems to have been blacklisted. Is this permanent? 

On 12/10/2023 at 12:34 AM, mechxero said:

The ollama container seems to have been blacklisted. Is this permanent? 

I'm not sure. I don't even know how that happened

I have ollama and the website working, but my gpu (P2000) stays idle while my cpu spikes, any chance the HW acceleration is not fully working? Or am I missing something? How do i go about troubleshooting that?

Hello... following up... I don't see @m0ngr31 Ollama template in CA.

 

I am trying @Joly0 template and I can get it running using the Ollama-WebUI docker. I do NOT think it is using the NVIDIA GPU. I have a GT1050 and the NVIDIA drivers installed.

 

 

Is there a way to get my GPU to work with Ollama?

 

Thank you!

 

 

Hey guys, could you try going into advanced mode while editing the container, remove both variables for nvidia devices and replace the argument in "extra paramters" with this "--gpus=all"

That worked for me. If you guys get it to work aswell, i will update the template

 

Atleast with these settings my gpu is used 

 

Also when i created that template, the settings where working, so looks like ollama changed something

Edited March 28, 20242 yr by Joly0

Thank you @Joly0. I have not tried what you described above. But I did a little more research as well.

 

After installing your Docker... I them went into the console and followed this guide:

https://www.jeremymorgan.com/blog/generative-ai/run-llm-locally-ubuntu/

 

I had to install the nvidia-cuda-toolkit, by simply entering:

apt install nvidia-cuda-toolkit

 

I think it's working but can't really tell the difference. I only have a 1050 GPU w/ low RAM. Here are the GPU stats being displayed before a question, and then while it is "thinking" . I would have thought everything would be pegged at maximum.

 

Before Asking a question:

 

While it is processing my question:

 

It is working.... and it is providing some nice answers.

 

I added the LLAVA module and it is nice that I upload an image, and it can tell me what it sees. Neat, but not sure how useful it is.

 

My real goal is to use a Text to Image module. Trying to investigate how to do this...  There is also a Docker for this called "Stable-Diffusion WebUI Docker". But you have to compile it yourself (mine froze in the middle of it): https://github.com/AbdBarho/stable-diffusion-webui-docker/wiki/Setup 😜

 

** EDIT **

I just realized that @m0ngr31 has a template for this!! Trying it out now. Big download.

 

** EDIT #2 **

My nVidia card has too little memory. I guess it was too much to hope for. Not sure if investing on a better GPU is worth it. 😕

 

It is not easy figuring out the whole AI thing in general terms. OpenAI, Hugging Face, Tensor Flow, Conda, Elon Musk lawsuit, Grok, what is free vs. what you pay for ... this is just stuff I am seeing without diving into the topic.

 

It would be nice to have a chunk of popular AI dockers... Maybe there should be a Forum section for AI.

 

Anyway, excited about this...

 

Thanks,

 

H.

 

Edited March 29, 20242 yr by hernandito

Hello,

 

I installed and run the ollama container and everything is working fine.

Except, that after some time, I get warnings about my docker.img filling up.

After taking a look at the container sizes, I see that ollama takes up 11,5GB of space.

11 GB are in the container in the /tmp directory.

After recreating the container, the size is ~500MB again for some hours, the grows back up to 11,5GB.

 

Anybody else experiencing such a behavior?

Any advise what to do instead of recreating the container once a day?

 

Thank you and best regards,

Armin

On 5/14/2024 at 3:50 PM, amueller said:

Hello,

 

I installed and run the ollama container and everything is working fine.

Except, that after some time, I get warnings about my docker.img filling up.

After taking a look at the container sizes, I see that ollama takes up 11,5GB of space.

11 GB are in the container in the /tmp directory.

After recreating the container, the size is ~500MB again for some hours, the grows back up to 11,5GB.

 

Anybody else experiencing such a behavior?

Any advise what to do instead of recreating the container once a day?

 

Thank you and best regards,

Armin

If you havent found a solution yet, i would advice adding a volume from the containers /tmp directory to somewhere on your host

I had this issue and it came down to where the appdata location was.  It needs a big place to store the models .. The container needs work in my opinion. 

Is there a proper way to set a path for RAG documents to be loaded into a model ? I would like to store all the RAG files on my array vs my appsdata which is on my limited cache drive. ideally, I would like all my different models stored on the array as well as opposed to the config location which is on the cache pool ?