NVME cache pool for VM - cow or not to cow ?

[zeus83]

Hi,

 

Need a bit of advice. I got two spare nvme drives in unassigned devices from which I want to create a secondary cache pool for VM images solely.

 

I will probably store few Windows 10 images and few linux images. For gaming VM I will use another nvme drive passed through directly, but I still prefer the system itself to be in the image file on the cache pool. I like the idea of doing easy backups and copy images to create another VM instances rapidly.

 

So far I've read through many guides over here and there on btrfs and decided to make a raid0 pool for maximizing the performance.

I still however have something to clarify:

1. Do I understand correctly that if doing proper backups of vm images and libvirt I'll be on a safer side wtih raid0 and no reason to go with raid1. Given I'm ok to pool be temporarily out of service in case of failure until I recover the data. Is it correct to assume raid0 btrfs give the maximum raw read+write performance ?
2. I've read that CoW enabled for VM image files leads to heavy defragmentation. But i'ts not clear how critical is this for SSD drives. Should I turn off CoW for VM image files ? If so and I disable CoW what is the reason then to use btrfs ? For instance I won't be able to do this for free right ?

   cp --reflink=always vdisk.img vdisk_2.img

    

       

[testdasi]

1. Yes. A good backup beats mirroring. And RAID-0 gives the best performance, up to a limit that is generally determined by your CPU single-core performance. It's also paramount that you do not access the pool through /mnt/user so you can bypass shfs.

 

2. BTRFS allows RAID-0 pool (you can use the ZFS plugin to do something similar with ZFS too). I don't think you can create a RAID pool with xfs.

[JorgeB]

1. If you have good backups, raid0 is fine.

 

2. IIRC you can now reflink with NODATACOW, you do still lose checksums, I use COW for all my VMs, and they perform fine, though would not be recommended if they were on a spinner, I think it's fine for SSDs.

[zeus83]

Thank you guys for quick response, still some questions arise

Quote

It's also paramount that you do not access the pool through /mnt/user so you can bypass shfs

Can you elaborate more on this bit ? Why should I bypass shfs ? Will it be ok to access through /mnt/disks ?

Quote

 IIRC you can now reflink with NODATACOW, you do still lose checksums, I use COW for all my VMs, and they perform fine, though would not be recommended if they were on a spinner, I think it's fine for SSDs.

When I perform an ordinary cp command in btrfs (no reflink) will it copy all file chunks physically to a new location ? Will the copy be defragmented then as a result?

[testdasi]

5 minutes ago, zeus83 said:

Can you elaborate more on this bit ? Why should I bypass shfs ? Will it be ok to access through /mnt/disks ?

Anything you access over the network through the GUI share settings will go through /mnt/user.

So /mnt/disks (i.e. unassigned devices) generally will bypass shfs.

But if, let's say, you create a symlink from a cache only share to your /mnt/disks mount points and then access the mounts indirectly through the symlink via the share SMB then you will be accessing it through shfs.

[JorgeB]

12 minutes ago, zeus83 said:

When I perform an ordinary cp command in btrfs (no reflink) will it copy all file chunks physically to a new location ?

Yes.

 

12 minutes ago, zeus83 said:

Will the copy be defragmented then as a result?

Yes, if there is available continuous space on the filesystem.

[zeus83]

Thank you guys, you helped a lot!