Location of system files - libvrt and docker

[wayner]

My system share is set to Prefer my cache pool, which is an SSD.  But the libvrt and docker folders on the cache drive are empty and all of the files are on disk1 of my array which is a spinning hard drive. 

 

Since the system is set to prefer cache, shouldn't the files remain on the cache drive forever, unless it runs out of space?  My cache drive has lots of space at 53GB used and 446GB free.  How do I force them to move to the cache drive?   And why would they have moved to the array any way?

[SimonF]

Have you run the mover? Also do you have it running on a schedule?

 

You will need to disable docker and vms for it to move the files as will not move if in use.

Edited January 5, 2022 by SimonF

[wayner]

Yes and yes.  Here is the log when I just invoked mover manually.
 

Jan  5 16:05:43 Hoylak root: mover: started
Jan  5 16:05:43 Hoylak move: skip: /mnt/cache_nvme/isos/Win11_English_x64v1.iso
Jan  5 16:05:43 Hoylak move: skip: /mnt/disk1/system/docker/docker.img
Jan  5 16:05:43 Hoylak move: skip: /mnt/disk1/system/libvirt/libvirt.img
Jan  5 16:05:43 Hoylak root: mover: finished

For some reason it is skipping.  Do you have to stop the docker and VMs to have the move take place?  I could try that, but why did they not go onto the cache in the first place.

[SimonF]

2 minutes ago, wayner said:

Yes and yes.  Here is the log when I just invoked mover manually.
 

Jan  5 16:05:43 Hoylak root: mover: started
Jan  5 16:05:43 Hoylak move: skip: /mnt/cache_nvme/isos/Win11_English_x64v1.iso
Jan  5 16:05:43 Hoylak move: skip: /mnt/disk1/system/docker/docker.img
Jan  5 16:05:43 Hoylak move: skip: /mnt/disk1/system/libvirt/libvirt.img
Jan  5 16:05:43 Hoylak root: mover: finished

For some reason it is skipping.  Do you have to stop the docker and VMs to have the move take place?  I could try that, but why did they not go onto the cache in the first place.

Yes they will need to be stopped. What is the name of tbe cache drive on tbe system share?

Edited January 5, 2022 by SimonF

[wayner]

The Cache drive is Cache_nvme - if that is what you mean.

[wayner]

Thanks, that seems to have worked.  But I wonder why they ended up on the array in the first place? 

 

There was one weird thing where the cache pool showed as "cache" despite the fact that I don't have a pool with that name, it should have been "Cache_nvme".

[SimonF]

7 minutes ago, wayner said:

Thanks, that seems to have worked.  But I wonder why they ended up on the array in the first place? 

 

There was one weird thing where the cache pool showed as "cache" despite the fact that I don't have a pool with that name, it should have been "Cache_nvme".

Cache is the default name and as you named your pool cache_nvme I think you have to manually set name on the share. As cache didnt exist files where put on the array.

Edited January 5, 2022 by SimonF

[bonienl]

You can have multiple pools with any name, but it is highly recommended to have at minimum a pool named “cache”.

Lots of applications assume the existence of this pool, and you would be obliged to change the application settings each time, if the pool is named otherwise.

 

[wayner]

I only used a different name as I was following a SpaceInvader One video and he set up multiple cache pools on multiple cache drives.  I may add a second cache drive as well.

 

Would you get better performance if you had dockers and appdata on one NVME drive, and VMs on a different NVME drive?

[bonienl]

I have never done performance comparisons, but my system is set up with a dedicated pool (cache) for Docker, and a dedicated pool for VMs (on nvme), works like a charm.

 

[BKG]

Hello, I am new to unRAID. I have an array of 4 disks and one SSD for cache pool. After reading the above thread I have two questions. I have noticed the same happening in my case(docker and VM ended up on Disk 1). As I was reading more about it I removed(uninstalled) the dockers and VM and cleaned the /mnt/user/appdata with rm -rfv. I then ran the MOVER but still have two folders on Disk 1.

 

Question 1. Can I manually delete those two folders on Disk 1?

Question 2. I want to have a cache pool made from two SSD drives formated in BTRFS. Based on the below quote, should I rename the pool to "cache" after I add the second SSD?

 

Thanks,

On 1/5/2022 at 5:05 PM, bonienl said:

You can have multiple pools with any name, but it is highly recommended to have at minimum a pool named “cache”.

Lots of applications assume the existence of this pool, and you would be obliged to change the application settings each time, if the pool is named otherwise.

 

[Squid]

The default settings for those 2 folders are use cache: prefer and cache pool: cache.  If you don't have a cache pool named "cache" then you'll see what you're indeed seeing.

 

Yes, lots of apps assume a pool named "cache" in the templates, but the apps tab automatically adjusts things to reflect your system when installing.  IE: the template references /mnt/cache/blah and you don't have a cache pool, but do have a pool name "ssd", the references will be automatically adjusted to /mnt/ssd/blah

 

[trurl]

6 minutes ago, BKG said:

removed(uninstalled) the dockers and VM

Not enough. You have to disable Docker and VM Manager in Settings so files in system are not open before they can be moved or deleted. 

[BKG]

Perfect! Thank you Squid and trurl.  I didn't know that. I disabled the Docker and VM and  ended up having only isos on Disk 1 which is normal since the share is set to CACHE/YES. Now how do I make the isos share reside specifically on Disk 3? Do I just copy the folder on Disk 3 and then place the ISO image there? Also, I made the following settings in the share:

 

Thanks

[trurl]

2 minutes ago, BKG said:

Do I just copy the folder on Disk 3

How are you working with files on disks instead of user shares? You must not mix user shares and disks when moving/copying since you could accidentally specify the same file as source and destination and lose it.

[BKG]

I don't work with disk shares and do work with user shares.  I guess I used the wrong wording. At the moment isos share is on - /mnt/disk1 and want it to reside on - /mnt/disk3

 

[trurl]

You seemed to be using linux commands earlier. Is that what you propose to do? I just wanted to warn you about not specifying a user share path such as /mnt/user/isos and a disk path such as /mnt/disk1/isos when moving or copying. Fine if you specify disk paths only such as /mnt/disk1/isos and /mnt/disk3/isos

[BKG]

I am trying to understand what its the way to make the system (without the use of CLI) to create and keep ISO image share on the chosen disk. For some reason and as a default the system created the ISO share on Disk 1, even though I already had the cache pool when I created the unRAID. So, here is my case represented graphically (see attached PDF). I hope I am not overcomplicating things 🙂

 

case use (1).pdf

[trurl]

Your settings look fine, but any settings you make only affect new files. You will have to move the files from disk1 to disk3 yourself.

 

Another way you can do this without cli

1. Disable VM Manager in Settings to insure no isos are open
2. Set isos share to cache-prefer
3. Run mover to get isos moved to cache
4. Set isos share to cache-yes
5. Run mover to get isos moved to the specified included disk.

[trurl]

11 hours ago, BKG said:

For some reason and as a default the system created the ISO share on Disk 1, even though I already had the cache pool when I created the unRAID.

If you had not already set isos to cache-yes, and not already set it to include only disk3, then it would not have used cache, and would have chosen the first disk since all disks were included.

 

If you already had set isos share to cache-yes, then it got created on cache. If you had not specified any included disk yet, when mover moved it from cache, it went to the first disk.

 

7 minutes ago, trurl said:

any settings you make only affect new files

 

[BKG]

Thanks trurl. makes perfect sense! It is all buttoned up now!