[Solved] BTRFS Copy-on-Write (COW) New Share Question

[Evolze]

Hi everyone,

 

I have a quick question regarding the btrfs Copy-on-Write option available in the share creation menu. It is a little bit lengthy, but I hope it makes sense. 🤞

 

Background: I'm in the process of moving all of my files and homelab projects to Unraid. The pool, called backup_cache is formatted as btrfs. I want to create a new share called backups which will be configured with the prefer cache pool setting, as shown here.

 

 

The goal of both the cache pool and share is to grab data from my other Unraid shares (i.e., appdata, domains, files, projects, system, etc.) and create a backup of them on this pool & share. 

 

Once the data has been backed up to the cache drive (1 on array, 1 copy on cache drive/share), I plan on using Duplicati or a different utility to have everything on the backups share uploaded to OneDrive. From there, the schedule would repeat once a week, where OneDrive would keep about 2-3 months of backups. I've attached a rough depiction of what I am attempting to accomplish. 

 

 

The files that will be copied/duplicated to this backups share will include virtual machine vDisks backed up & compressed as .zst (via VM Manager Plugin), Docker app data, and operating system backups (i.e., Windows Backup & Restore, Linux OS via rsync). It will also include some other files, such as Office documents, photos, videos, and other files that I will be reading and writing too often.

 

Question: As mentioned earlier, the cache pool is formatted as btrfs. Thus, the option for enabling copy-on-write (COW) is available. Given how I plan to utilize this for backups only and then to OneDrive, should I enable Copy-on-Write for this share?

 

I ask as I know that vDisks/domains and Docker/system shares are recommended to have COW turned off, but in my case, these files will be copied/backed up in a compressed format. I feel like I have a good understanding of btrfs, but I'm wanting to be extra sure as switching between the COW and NOCOW file property can't be done easily for individual files.

 

I've been scouring the forums for the last few hours to see if anyone had a similar configuration or question and could not find anything. If this has been answered before, my apologies in advance! 

 

I'm super excited to begin using my newly built Unraid host to do some benchmarking and homelab projects! 

 

Any insight or additional recommendations on the best way to configure this share would be greatly appreciated. 😀

 

TL;DR: I am creating a backups share with both compressed and uncompressed files from other Unraid shares. Data will exist on the array, backup_cache pool, and then OneDrive. Given usage, asking if btrfs's copy-on-write feature should be enabled based on this share's purpose and its file contents.

Edited March 19, 2023 by Evolze
Edited post title to include Solved.

[JorgeB]

Recommend using COW even if using a single device pool, pretty much required with multi device pools.

[Evolze]

On 11/26/2022 at 1:23 AM, JorgeB said:

Recommend using COW even if using a single device pool, pretty much required with multi device pools.

 

@JorgeB Thanks so much! Even though I do not have a multi device pool in this instance, out of curiosity, why would COW be considered required? Is it because the two drives are assisting one another to create and manage the btrfs snapshots or data copies for redundancy?

[JorgeB]

1 hour ago, Evolze said:

why would COW be considered required?

With NOCOW cheksums are disabled, so there's no way for btrfs to reconstruct the data if one device fails, possibly worse than that, if one device drops offline for some time and comes back online btrfs has no way of knowing which device has the correct data, and as it reads from both devices it will corrupt existing data.

[Evolze]

11 hours ago, JorgeB said:

With NOCOW cheksums are disabled, so there's no way for btrfs to reconstruct the data if one device fails, possibly worse than that, if one device drops offline for some time and comes back online btrfs has no way of knowing which device has the correct data, and as it reads from both devices it will corrupt existing data.

 

That makes sense, thanks so much for the information.