Ideal docker setup for LLMs

[Pandemic]

I'm looking for help setting up docker in the most ideal way. I've made other posts but they died out and rather than trying to modify what I have I'll just ask for the best way and do that.

 

I've got an array of about 10 disks, one cache drive which is a 10k rpm raptor 150gb, and (for now) a 512gb NVME. I want to set up docker to run plex and ollama containers. Because ollama (and the UI) will be running LLMs I'm hoping to set this up to run as fast as possible.

 

I need to figure out how to remove the raptor drive from the mix, I'll figure that out later.

 

I'll run the 512gb NVME as the cache drive, I expect. Is a pool a mix of multiple drives (ex: two NVME drives?)

I think running docker from the NVME will be best, keeping it outside the array.

 

What should the docker size allocation be? I had it set to about 300gb just to keep space on reserve for the future but I'm not sure if that's necessary.

 

Let me know your suggestions so I can try to understand how to future-proof this setup.

[Gragorg]

300gb is way to big start at 10gb.  You can make it larger if you need to.  My 21 dockers use about 14gb so I currently have it set for 20gb.  I would swap the 150gb for 512gb.  First i recommend running appdata backup so you have your appdata saved to the array.  I basically disabled the docker service and set the mover up to move all the data off the old cache drive to the array.  Once the old drive was empty I put the new cache drive in place and then reversed the mover direction back to cache drive and run the mover again.  At this point you can always restore from appdata backup if something wasn't right.

[Pandemic]

I used appdata backup (I think I did it right) and then stopped the array, selected no disk for cache, started the array, stopped the array, and then selected the 512gb NVME. I kept the 300gb docker size just so I don't have to redo that aspect. I know it's way more than I need but with some LLMs being 20+ gigs I figured I'd try to future proof. 

Can I just reduce the size without having to rebuild everything?

 

Funny thing though, as I'm moving data to the UNRAID box, my speeds are about 10-20MB/sec slower than they were with the raptor drive. Everything I'm seeing suggests NVME would have a faster write speed so I'm not sure what's happening.

 

edit: this may be an issue originating from the box containing the files to transfer to the unraid box. That's about the only logical conclusion I can think of.

Edited August 23 by Pandemic