Apple Time Machine, BTRFS and disabling(?) Copy-on-Write (CoW)


bland328

Recommended Posts

Should I disable Copy-on-Write (CoW) for a Time Machine share on a BTRFS disk?

 

Given that I can't conceive of how or why I'd benefit from any BTRFS-level snapshotting on Time Machine file sets, and given the admittedly little bit I understand about potential CoW performance penalties, I suspect I should disable it.

 

Or, at the very least, that there's probably not a downside to disabling it.

 

Can anyone with a deeper-than-mine understanding of Time Machine and/or BTRFS Copy-on-Write provide any insight?

 

Thanks very much for any help!

Link to comment

I understand how Time Machine works with destinations that are on network shares but I don't know much about the pros and cons of CoW. If it helps, I can tell you that a Time Machine backup over the network involves a double mount and very many small files. Firstly there's a the mounting of a network share, traditionally using the AFP protocol but with increasing support for SMB. Secondly a disk image stored on the network share is mounted. This disk image is of an HFS+J filesystem, just like a local Time Machine volume and internally identical. The disk image is created as a sparse bundle, which consists of a containing folder, inside which is a plist, a database and many thousands of 8 MiB "band" files, each of which contains a fragment of the actual image data. The idea is that when Time Machine performs its next incremental backup, instead of modifying a monolithic image file, only the necessary band files that are affected need to be modified. In practice, sparse bundles have shown themselves to be rather fragile. Does that help you answer your question?

Link to comment

First of all, that's a great response, and I really appreciate it--it does give me some more insight.

 

Secondly, I feel sorta dumb about my question, because I was a little short on sleep and not properly considering (or remembering ;) ) that only my cache drive is BTRFS--my actual data drives are XFS, so I think that makes my question pretty much moot.

 

That said, I'm currently drowning in hints and tips here and there about getting Time Machine working over SMB between HIgh Sierra and Unraid 6.5.3, so if you have any more to say about "increasing support for SMB," I'd love to hear it.

 

It looks to me like Samba 8.0 tidies Time Machine issues up quite a lot, but I realize Unraid doesn't yet include it, so I'm open to suggestions.

 

And perhaps I should be starting a new thread ;)

 

EDIT: In fact, I have jumped over to join the conversation at:

 

 

Edited by bland328
Guilt over going off-topic ;)
Link to comment

Ah! I had assumed you were asking because you have btrfs formatted data disks in your array. Most people use the default XFS but btrfs is certainly an option - and you can mix and match if you want to.

 

Increasing support for SMB shares as Time Machine destinations: see this thread from Apple.

 

I remember looking at the thread you linked some time ago. There seems to be a lot of confusion over APFS formatted volumes there. Here's a quick summary:

  • Of course, Time Machine can be used to backup APFS formatted volumes because Time Machine works at the file level and isn't concerned about the underlying format.
  • Local APFS formatted volumes can't be used as Time Machine destinations. If you try Time Machine will offer to reformat them with HFS+J.
  • An APFS formatted volume directly connected to another Mac can be used as a Time Machine destination if network share is created.
  • Network shares residing on APFS volumes can only be shared using the SMB protocol.

Hopefully that clears up APFS.

 

Time Machine over SMB should "just work" with Sierra and High Sierra BUT you must configure a new network destination from scratch using SMB - you can't convert an existing AFP share to use SMB. So if you have an existing AFP Time Machine destination you need to retire it and create a new one using SMB.

 

I used to backup a couple of old Macs that can't run anything more modern than Lion to a WD MyCloud over AFP and, while it worked, I had trouble with the fragility of the sparse bundle images. Often if a lot of files had been changed since the previous hourly backup the next backup would fail and Time Machine would have to start from the beginning again - delete all the band files, clear the database and start with a full backup, losing all the previous ones. This was with a cabled network and is far worse and much more likely to happen if you use a wireless connection. So now I dedicate a separate USB drive to each Mac and as long as the USB lead doesn't accidentally get unplugged during a backup it works reliably. Accidentally unplug the USB lead and the Time Machine disk is essentially trashed so badly that Disk Utility can't repair it. The only way out is to reformat it and start again.

 

As a matter of interest, what is your objection to using AFP? I find that it works pretty well with unRAID (and with older versions of OS X it's faster than SMB) provided you relocate the Netatalk database. The .AppleDB folder is by default located in the root of the user share but it's a really bad idea to keep it on the parity protected array as updates are slow and it gets fragmented across multiple physical disks. I move it to the cache disk and AFP works so much better. See here:

 

 

Link to comment

Thanks again for a great and thorough response!

 

I get the APFS bit--I think a lot of people were initially so confused that they managed to confuse other people with their postings on the topic.

 

Interestingly, you say:

1 hour ago, John_M said:

Time Machine over SMB should "just work" with Sierra and High Sierra BUT you must configure a new network destination from scratch using SMB - you can't convert an existing AFP share to use SMB. So if you have an existing AFP Time Machine destination you need to retire it and create a new one using SMB.

.I'd sure like for that to be true, and perhaps under the right circumstances, it is, but none of my new SMB shares have been recognized by Time Machine, and even when I've used the macOS tmutil command along these lines...

sudo tmutil setdestination -ap smb://[email protected]/timemachineshare

...I've had no luck. If you know of some magic, I'd love to hear it!

 

have managed to make a sparsebundle locally, using a command like this...

hdiutil create -size 2000g -fs HFS+j -volname "Time Machine" ComputerName.sparsebundle

...then copy that out to an SMB share on my Unraid server, then mount that from the share, then hand the resulting mountpoint (e.g. '/Volumes/Time Machine') over to Time Machine using the tmutil command. That all seems to work, except it looks to me like Time Machine has then been fooled into thinking it is backing up to a local disk.

 

Which is either a problem, or it isn't. I don't know if Time Machine uses any more fault-tolerant approaches or different read/write patterns when it knows it is dealing with a network destination, so I'm not sure whether to consider this approach a high-quality solution or a dangerous stunt.

 

As for your question about why I'm trying to get away from AFP--well, at the moment, I'm honestly not really trying to solve a particular problem. So perhaps I shouldn't even be attempting it! But, since I was making some changes and interested in dropping (again, for no particular reason) dependence on that aging protocol, I figured I'd take a shot.

 

That all said, it looks like Samba version 4.8 has robust Time Machine support, as discussed here:

 

https://macosx.com/threads/smb-samba-for-time-machine-backup.324958/

 

and a variety of places online, thanks to the addition of (what I understand to be) new Global configuration options such as:

fruit:time machine = yes
fruit:advertise_fullsync = true

I don't know if there's an easy/safe/friendly way to inject Samba 4.8 into Unraid 6.5.3, but given that Unraid 6.6.0-RC1 already has Samba 4.8.4 in it, I imagine I'll just be patient.

 

Unless, of course, someone wants to tell me about that easy/safe/friendly way to inject Samba 4.8 into Unraid 6.5.3 😅

Link to comment
2 hours ago, itimpi said:

One point I have seen regarding Time Machine support is that it seems to require the User Share used to store the files be restricted to a single disk (or to use a Disk share which is effectively the same thing).   I have no personal experience of using Time Machine but thought it was worth mentioning.

 

I think it's safer to use a single disk but it isn't by any means a requirement. Everything that Time Machine does or needs is within the sparse bundle disk image. The only requirement of the file system that contains the disk image is that it can support a sparse bundle (i.e. a folder containing sub-folders, which contain many small files). The complicated stuff that Time Machine needs to work, such as hard-linked folders, is a feature of the file system contained within the disk image.

 

6 hours ago, bland328 said:

it looks to me like Time Machine has then been fooled into thinking it is backing up to a local disk.

 

That's essentially how Time Machine works anyway. The backup engine "is fooled" into thinking that it's backing up to a local HFS+J disk by the process that mounts the network share and then the sparse bundle image each hour. The image looks identical to a local file system. It's the mounting process that has had to change to accommodate SMB.

 

A little of what I know about Time Machine I have learnt from experimenting. Most of what I have learnt comes from this site. It's all old stuff but it's still mostly relevant and well worth reading. This site is in fact archived and no longer maintained as the original author (James Pond) died a number of years ago and the original domain (pondini.org) registration has expired.

 

I haven't had the chance to play with unRAID 6.6 yet. I'm about to go away for several weeks and didn't want to install it just before I leave.

 

Another thing that used to trip people up was the fact that encrypted backups to network destinations didn't work. I don't know whether that has been fixed. I expect it has but I haven't been able to find a definitive statement from Apple.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.