Installing Dockers on SSD (cache?)

[SliMat]

Hi Guys

 

I was wondering whether anyone can let me know if there is a tut on best practices to set UnRAID up.

 

I currently have a number of dockers installed and running and have no cache drive... so /appdata etc is installed on the main array discs, which I suspect is keeping them spun up most of the time.

 

So, I am after some pointers on how to make the system more efficient. I am guessing that I need to move my dockers onto a cache SSD and off the array. Obviously as this disk will be unprotected I imaging that it should be backed up too for redundancy.

 

Any assistance will be gratefully appreciated... I am sure that this must have been covered before - but appreciate some help!

 

Thanks

[apandey]

You can add a cache pool with ssd drive, then mark the appdata share as cache only. Then stop docker service and run mover and all appdata will be moved to cache drive. Your appdata needs to fit on your cache ssd

 

You can add a 2nd ssd of same size to the pool if you want a raid1 redundebcy on your pool

 

If running with a single ssd in cache pool, you can setup CA backup plugin to maintain a daily backup of cache on the array

[SliMat]

Thanks @apandey 

 

I will give this a go in the next few days and post back that it all goes OK.

 

Should my 20Gb docker image need to be moved too as having had a search in Midnight Commander I see docker located at

 

/mnt/disk3/system

/mnt/user0/system

/mnt/user/system

 

Which I believe all are actually the same location.

 

Thanks

[apandey]

Usually it's a good idea to keep system, appdata and domains on cache. You should stop docker and VM services before you move data, as open files cannot be moved

[SliMat]

Thanks - when I move system * appdata I will obviously need to modify something in to dockers so they know where these are now located. I have about 10 dockers which will need updating... what do I need to modify with the new locations?

[apandey]

1 minute ago, SliMat said:

Thanks - when I move system * appdata I will obviously need to modify something in to dockers so they know where these are now located

You don't have to, as long as you have pointed to containers to /mnt/user/appdata, which is the share path. You just have to have the correct cache setup in respective share settings

[SliMat]

On 3/28/2023 at 10:09 AM, apandey said:

You can add a cache pool with ssd drive, then mark the appdata share as cache only. Then stop docker service and run mover and all appdata will be moved to cache drive. Your appdata needs to fit on your cache ssd

 

You can add a 2nd ssd of same size to the pool if you want a raid1 redundebcy on your pool

 

If running with a single ssd in cache pool, you can setup CA backup plugin to maintain a daily backup of cache on the array

 

OK, I have finally added a cache disk, formatted it, stopped docker service and set the appdata share to Only:Cache - I have invoked mover, but it finishes instantly and doesnt seem to be moving.  The cache disk shows 191 reads and 56 writes... but nothing more and its not going up. When I look at the appdata location it is still showing as on the array. 

 

I have attached diags - can anyone help?

 

Thanks

unraid-diagnostics-20230405-1301.zip

[SliMat]

I also just received this error message from "Fix Common Problems" which seems to be related;

 

[SliMat]

Should I manually copy the folders & files from /mnt/user/appdata on the array to /mnt/user/appdata on the cache disk and then delete /mnt/user/appdata from the array disks?

[itimpi]

To get mover to transfer the files set Use Cache=Prefer.   Mover ignores shares with the Only setting.

[SliMat]

Ah, OK thanks @itimpi 

 

Should I then leave it as Cache: Prefer as @apandey said to set it to Cache: Only?

 

Guessing once mover is complete, change to Cache: Only and then in future it will only use the cache disk?

 

Edited April 5, 2023 by SliMat

[itimpi]

5 minutes ago, SliMat said:

Ah, OK thanks @itimpi 

 

Should I then leave it as Cache: Prefer as @apandey said to set it to Cache: Only?

 

Guessing once mover is complete, change to Cache: Only and then in future it will only use the cache disk?

 

The advantage of the Prefer setting is that if the pool fills up it can overflow to the array whereas the Only setting will give an error.   On the other hand the Only setting might be more performant.

[SliMat]

It is looking like mover has stalled moving some of my dockers over... there are a few odd files which wont move and I believe I can delete them but would like to check...

 

I have a pihole docker which says that most files are in /mnt/user/cache, but there is one file left on disk 2 - /mnt/disk2/appdata/pihole/pihole/!macvendor.db (size 13 bytes)

My Nginix Proxy Manager has a file left on disk 3 - /mnt/disk3/appdata/NginixProxyManager/!logs (size 3 bytes)

 

In Midnight Commander the above filenames are coloured red and I am thinking mover cant move them and they may be orphaned files - can I just delete them, or should I copy them to /mnt/user/appdata?

 

I do also have a blueiris docker, which seems to be unable to move thousands of files which are in the /mnt/disk3/appdata/blueiris/drive_c/ & /mnt/disk3/appdata/blueiris/dosdevices/ folders... this might be because the files in Nginix Proxy Manager and Pihole are stopping docker before it has completely moved BlueIris... but either way once the Nginix Proxy Manager and Pihole issue is solved - is there anything to stop me moving BlueIris' files in MC to the new SSD /appdata/ folder?

 

Thanks

[itimpi]

Did you disable the docker service before running mover?   This will be needed or some files will not be moved.

[SliMat]

Yes, it is disabled @itimpi

[SliMat]

OK... I thought as all mover does is moves the files, it wouldnt hurt to move them using MC... so I moved the ones which seemed stuck onto the cache disk and now it shows as all my dockers & VMs on the SSD cache  And its all working perfectly again

Edited April 5, 2023 by SliMat

[dcuellar44]

On 3/28/2023 at 5:09 AM, apandey said:

If running with a single ssd in cache pool, you can setup CA backup plugin to maintain a daily backup of cache on the array

I know this is a bit older but I am doing the same thing, except I do not want to run the two nvme's in raid 1. I'm going to set the balance to single mode. With that said I will want to back it up to the array. So I was looking for CA Backup Plugin but it seems it has been deprecated. What is the alternative?

[ConnerVT]

search CA for "appdata".  There are several versions of the backup plugin, due to changes in the maintainer of the plugin and Unraid updates.

[dcuellar44]

1 hour ago, ConnerVT said:

search CA for "appdata".  There are several versions of the backup plugin, due to changes in the maintainer of the plugin and Unraid updates.

Got it. Thank you. I'm not sure this would be the solution I need then. I asked the support if that plugin backs up the entire cache and it's just meant to back up the appdata or the docker containers. Not a big deal but also I have 800GB+ of data in appdata so I'm also concerned that what it does is replace the backup each time instead of updating the changes. That would take a long time each time. I also don't know if you can set it up to where you only keep one backup. Having multiple backups would take a ton of space.

 

I think my best option would be to fork up the money to buy two larger nvme SSDs and put it in raid1. I'll have to study up on ZFS as I hear that's the option to go with if I'm going to do this raid option. Not sure what snapshots are but I heard they are useful as well and available as part of ZFS. Lots to review.

[ConnerVT]

The Appdata pluging backs up the appdata share and flash drive.  It is a very configurable plugin, where you can select which docker appdata folders to back up, any additional folders, and exclude files/folders in any specific docker appdata folder.

 

The Docker templates are stored in your flash drive, and (obviously) the Dockers are stored in their repositories (Github and such).

 

Realize that the "cache" pool is just another pool.  If you have data outside of your appdata share, you can use the same backup strategy you would use to back up any other data.

[dcuellar44]

Ok. Got it. Thanks for your help!

[JonathanM]

6 hours ago, dcuellar44 said:

put it in raid1

Remember, RAID is not a backup, it replicates all changes in realtime, which means corruption, deletion, etc are all faithfully replicated. Backup allows you to step back in time to the moment the backup was taken if something bad happens.

 

Redundant RAID levels allow for device failure without downtime.

[dcuellar44]

2 minutes ago, JonathanM said:

Remember, RAID is not a backup, it replicates all changes in realtime, which means corruption, deletion, etc are all faithfully replicated. Backup allows you to step back in time to the moment the backup was taken if something bad happens.

 

Redundant RAID levels allow for device failure without downtime.

So if one of the two nvme cache drives fails my data is safe until I replace the drive, right? What it won't protect against is corruption in the data. Am I understanding correctly?

[JonathanM]

1 hour ago, dcuellar44 said:

So if one of the two nvme cache drives fails my data is safe until I replace the drive, right? What it won't protect against is corruption in the data. Am I understanding correctly?

In theory, yes. You just don't have the ability to go back and undo like a real backup. In practice it seems when a device fails the RAID duplication doesn't always take over seamlessly, so a backup is the better use of a second drive in my opinion.

 

YMMV, my opinion only, others disagree, etc, etc.

[dcuellar44]

Just now, JonathanM said:

In theory, yes. You just don't have the ability to go back and undo like a real backup. In practice it seems when a device fails the RAID duplication doesn't always take over seamlessly, so a backup is the better use of a second drive in my opinion.

 

YMMV, my opinion only, others disagree, etc, etc.

I get it. I plan to do the raid1 as well as a backup. My issue is that it's a lot of data to back up from scratch each time. I need to see if there is a sync that updates changes to the backup instead of doing a backup from scratch. luckyBackup may do what I'm looking for. If you know of any let me know. 800GB is just too much to expect a fresh backup each time.