Cache Disk use for Docker & VMs


Recommended Posts

Hi, I am setting up a brand new server with Unraid.

 

I am a bit confused about docker containers and vm's. I want these to run of my SSD Cache and no from the array so that they have better performance.

 

Should i set the APPDATA & DOMAIN shares Cache Settings to ONLY or PREFERED please. I am confused about this setting.. 

 

I want docker containers and vm's to always run from the SSD cache.. but at the same time maybe possibly make a copy to the protected array once a day if possible..

 

Thanks in advance

  • Like 1
Link to comment

@RyanBorh If you already have installed some dockers or VMs, disable docker and VMs first in the settings, set both shares to PREFERED, run the mover so all existing data is moved to the cache. Check if the files are moved over to the cache and check if the shares don't exists on any array drive. Simply click the folder icon on the far right behind the drive in Unraids main tab. After that change both shares to ONLY and startup the docker and VMs again in unraids settings. Every new Docker or VM you install now will always been stored on the cache drive now and newer will be moved by the Mover to the array.

 

For daily backups for the Dockers you can use the "CA Backup / Restore Appdata" plugin from the community applications.

 

For backing up your VMs there are a couple ways to do it. You can use the following script for example and configure a daily schedule for it with the "User Scripts Plugin"

 

Or in case you wanna use the BTRFS snapshot feature you can use a script to do this. This only works if the source and target drive is using BTRFS

 

  • Like 1
Link to comment

Thanks so if its a new install, if i set docker and vm option to ENABLED after i set the appdata and domain folder to ONLY there is no need to do the mover bit.. and they will alway be on the cache ssd without array copying.

 

I will use the plugin and script for backups

 

What i didn't understand is the below.. why did you say that the new ones will be moved to the array.. i want all docker and vms to always stay on the cache ssd.

29 minutes ago, bastl said:

Every new Docker or VM you install now will always been stored on the cache drive now and newer will be moved by the Mover to the array.

 

Link to comment
1 hour ago, RyanBorh said:

What i didn't understand is the below.. why did you say that the new ones will be moved to the array.. i want all docker and vms to always stay on the cache ssd.

I think it was just a mistake.

 

It sounds like you should be good to go.

 

The main problem happens when people enable docker and VM services before they even install a cache drive. Then those are installed to the array and, though not difficult, a number of steps are required to get them moved to cache where they belong since open files can't be moved.

Link to comment
4 hours ago, RyanBorh said:

What i didn't understand is the below.. why did you say that the new ones will be moved to the array.. i want all docker and vms to always stay on the cache ssd.

4 hours ago, bastl said:

Every new Docker or VM you install now will always been stored on the cache drive now and newer will be moved by the Mover to the array.

sry, a typo "will never moved to the array by the mover"

Link to comment
  • 2 years later...
On 12/6/2019 at 10:30 AM, RyanBorg said:

Thanks so if its a new install, if i set docker and vm option to ENABLED after i set the appdata and domain folder to ONLY there is no need to do the mover bit.. and they will alway be on the cache ssd without array copying.

 

I will use the plugin and script for backups

 

What i didn't understand is the below.. why did you say that the new ones will be moved to the array.. i want all docker and vms to always stay on the cache ssd.

 

Please, a doubt.
 

1) Does the cache move all files to array every day?
2) How to leave the files (Vms, Dockers...) in the SSD cache?

 

My VMs are in the UD, every week I make backup with CA Backup.
In the cache they are at risk of corruption (that's what I understood)? It doesn't seem safe to me.

Link to comment
49 minutes ago, Braulio said:

Please, a doubt.
 

1) Does the cache move all files to array every day?
2) How to leave the files (Vms, Dockers...) in the SSD cache?

 

My VMs are in the UD, every week I make backup with CA Backup.
In the cache they are at risk of corruption (that's what I understood)? It doesn't seem safe to me.

 

You set the shares you never want to move to ONLY on your share settings, so it leaves all files on the SSD.   Appdata, ISO, Domain

 

Link to comment
38 minutes ago, kizer said:

 

You set the shares you never want to move to ONLY on your share settings, so it leaves all files on the SSD.   Appdata, ISO, Domain

 

Yes, I verified that´s possible.

The cache works the same as the UD (as a disk outside the array). But I thought the cache files needed to be moved to the server to not risk losing it. I thought they will stay in memory until moved.

So, in short, does the cache just send files from one side to the other (cache / array)?

Link to comment
18 hours ago, Braulio said:

Yes, I verified that´s possible.

The cache works the same as the UD (as a disk outside the array). But I thought the cache files needed to be moved to the server to not risk losing it. I thought they will stay in memory until moved.

So, in short, does the cache just send files from one side to the other (cache / array)?

 

Yes the cache works like UD does. Keep in mind it was always the intention of the SSD/Cache to house your VM's, Appdata and shares. UD came along as a hot swappable plugin so you could backup things to a USB mounted drive with some scripting. It was later expanded to do other things. I can't think of a reason at the moment to say you shouldn't house your VM's, Appdata there. I'm sure there are Technical reasons why should do one or the other, but if it works for your work flow then so be it. 

 

I'm running two 1TB SSD's in Raid 1 for my Cache/SSD to insure my Appdata is "more" protected. I don't know if Unassigned Devices can do that.

Link to comment
3 hours ago, kizer said:

 

Yes the cache works like UD does. Keep in mind it was always the intention of the SSD/Cache to house your VM's, Appdata and shares. UD came along as a hot swappable plugin so you could backup things to a USB mounted drive with some scripting. It was later expanded to do other things. I can't think of a reason at the moment to say you shouldn't house your VM's, Appdata there. I'm sure there are Technical reasons why should do one or the other, but if it works for your work flow then so be it. 

 

I'm running two 1TB SSD's in Raid 1 for my Cache/SSD to insure my Appdata is "more" protected. I don't know if Unassigned Devices can do that.

I understood perfectly how the Cache works. It was just how I imagined.
The only thing that confused me is this part.


Do you mirror (raid) the cache to have a copy of the files. Wouldn't it be easier copy to the array?


As I understood, the cache files are safe there.
The only risk is when executing the "mover". They can be lost (power out, crash, among others).
If you don't run "mover", they are safe, as in a HD/SSD.
 

Is this corretly?

 

Thanks all for the anwsers @trurl @kizer

Link to comment

I have 2 pools, one named "cache" and one named "fast". Cache is 2x500G btrfs raid1 mirror, and is mostly for caching writes for user shares (cache:yes). Fast is 256G XFS where my docker / VM files are kept (cache:only), and these get backed up to the array.

 

Actually, my use cases are a little more complicated than that, but that is the basic idea.

Link to comment

@trurl

35 minutes ago, trurl said:

I have 2 pools, one named "cache" and one named "fast". Cache is 2x500G btrfs raid1 mirror, and is mostly for caching writes for user shares (cache:yes). Fast is 256G XFS where my docker / VM files are kept (cache:only), and these get backed up to the array.

 

Actually, my use cases are a little more complicated than that, but that is the basic idea.

In my case I'd like to leave the VMs and Dockers to cache.
I selected (cache:only). The files will not be sent to array, then I need to make a backup through "Settings -> Backup/Restore Appdata" and "Settings -> Vm Backup".
Correct?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.