Fixing newbie mistakes: Moving to tiered cache pools and fixing hard links to my (unassigned) appdata nvme disk. Does this approache make sense?

[Geekd4d]

I've been running my unraid server for years, and when I set it up, unraid did not (officially) support nvme's in cache pools (As far as I understood at the time). So i set it up as an unassigned device, mounted it as nvme_docker and moved ALL my appdata there by changing all the default paths in my docker containers to /mnt/disks/nvme_docker/appdata/<app>    I know (now) that this is a mistake.
 

I am trying to undo all my newbie mistakes without losing any data. I have a dozen docker containers I really don't want to have to redo, but I really want to correct this.
 

The way I have things setup is as follows:

• 12x 8TB data drives (working on upgrading these to 6x16tb's to reduce power draw)
• 2x 8TB Parity (upgrading to 16tb as well)
• 2x 4TB enterprise ssd's as my raid 1 cache pool (for Downloads only)
• 1x 1TB ssd for appdata backups (USB 3.2 as an unassigned disk. replacing this weekend with 4tb ssd for longer term backups)
• 1x 500gb NVME for everything docker as an unassigned device. /mnt/disks/nvme_docker (ALL appdata and docker.img)
• 1x 500gb spare usb 3.1 external (unassigned If I need to move data around, going to toss things here)

 

The way I set it up at the time, I have every docker container's appdata mounted to /mnt/disks/nvme/appdata/<app name> - yes, I know that's not ideal/correct, but at the time I set this up years ago, It was how I could make it all work, and I didn't fully understand the cache pool. so before you chastise me, I'm trying to fix it! I want to be able to upgrade any cache pool in the future easily, which this solution is not sustainable like it is.

-------------------------------------------------------------------------------------------------------------

I want to move this to a tiered Cache pool setup:

 

• 4tb ssd cache drive: cache_appdata (for containers that use up 50+gb of data but need to be fast accessible such as nextcloud, librephotos, plex metadata)
• 4tb ssd cache drive: cache_ssd (for normal downloads, move to array after a week etc)
• assign the nvme to a new cache pool: nvme_appdata and this will be where my docker.img (unless I can move to directory based.. I don't know how to do that yet though. Haven't researched it yet) as well as my appdata for the rest of my docker containers that do not eat up a lot of storage but I prefer them to be fast accessible.

 

The Way I am approaching this is as follows, and this is where I am not sure if I am doing this correctly. I know this is a lot of work, and I appreciate if you have read this far and have some advice.

 

• Stop everything first obviously.
• Copy all files from /mnt/disks/nvme to my temporary drive using either MC or rsync.
• create all my cache pools the way I want them. (assign nvme to new cache pool, and break up the existing into 2 single drive pools)
• start array so that the appropriate appdata folders are created on cache_appdata, cache_ssd and nvme.
• Move files back to where I think they should be /mnt/nvme/appdata/ for docker containers /mnt/cache_appdata/appdata/ for larger ones.
• Edit all docker containers - set them back to /mnt/user/appdata/ and then set the cache to the correct cache for the container. Don't start them.
• Move docker img file to /mnt/nvme/ (system/docker?) and then re-point the docker settings to this file. Edit the default docker appdata settings to /mnt/user/appdata (its currently /mnt/disks/nvme_docker/)
• start things back up and see what breaks?

 

Now I know I cant just use mover to acomplish all this, that's what I want to get to.. so In the future, if I need to upgrade a cache drive, I can use mover to move everything from one cache drive to another, upgrade a drive, then use mover to move them back.. Or.. am I WAY off here?

 

Thanks for sticking through this. I can explain anything in more detail if needed, But I think this is the correct approach.. Anyone want to let me know if I'm completely wrong? Have you done the same at some time?

Edited July 29, 2023 by Geekd4d

[itimpi]

21 minutes ago, Geekd4d said:

I can use mover to move everything from one cache drive to another,

Mover does not support moving files between pools.

[Geekd4d]

1 minute ago, itimpi said:

Mover does not support moving files between pools.

good to know, am I way overthinking this and I should just leave well enough alone? hah! Mainly I am trying to make this so I don't have the hard paths in all my docker containers, and since nvme is supported in a pool.. I would assume this would be a lot easier to manage.

[itimpi]

35 minutes ago, Geekd4d said:

good to know, am I way overthinking this and I should just leave well enough alone?

Not sure to be honest!

 

One thing to remember is that UD devices cannot participate in user shares whereas pools can.   Whether that will affect your workflow I am not sure.

it is possible the mover restriction might disappear in the 6.13 release, but until we actually get visibility of that release that is just a guess.

 

[Geekd4d]

36 minutes ago, itimpi said:

Not sure to be honest!

 

One thing to remember is that UD devices cannot participate in user shares whereas pools can.   Whether that will affect your workflow I am not sure.

Yeah that's what I am trying to get away from. the UD devices, as I have to mount them with /mnt/disks/<device> in all the containers.