Jump to content

Time Machine (with zfs) Best Practices


fplanque

Recommended Posts

Posted

I've spent several days moving my time machine backups from old Apple Time Capsules to Unraid. (MacOS Ventura + Unraid 6.12.4)  (I specifically wanted to do this before upgrading to Sonoma, will give Sonoma feedback later)
 

Here's what I learnt:

 

1) When backing up to a network share, Time Machine creates a virtual drive (.sparsebundle) on the network and then proceeds to manage that whole drive remotely. This is inefficient as hell (but this is how TM manages to save space by hard-linking unchanged files between one backup and the next). This makes network backups an order of magnitude slower than backing up to a drive that's directly attached to the mac.

2) It seems it may be even more inefficient if the backup is encrypted, i-e: when the .sparsebundle is encrypted.

 

3) Because of that inefficiency, I decided to compensate by backing up to an SSD drive in my unraid. No need to use nvme drives, it will be totally overkill in regard to the speed at which the mac will send backup data. I opted for the cheapest high capacity SATA SSD drive(s) I could find on Black Friday.

4) I am using that SSD as a standalone drive/pool, not protected by anything for 2 reasons:

  1. my array parity is not SSD and would slow down writes
  2. my time machine strategy is to back up to 2 different unraid boxes in 2 different parts of the building. (This gives better protection than 2 drives mirrored in the same box + allows to back up whenever one of the 2 unraid boxes is down because I'm playing with its configuration...)

 

5) I am using zfs for the SSD (with LUKS encryption). I also use zfs compression. This only makes sense if time machine does not encrypt its backups by itself. If TM encrypts, then zfs can no longer compress it (for example because strings of nulls will be garbled), It tested this. I get about 20 to 25% compression when letting zfs compress on the server compared to when trying to compress an encrypted backup (which results in close to 0% compression).

6) I observed no slow down due to zfs compression (but I observed more space for backing up more history ;). I think I do see a slower back up with a mac/tm-encrypted backup to uncompressed zfs but I need to test this more.

7) If the backup is not encypted by TM but encrypted by zfs, it will be encrypted at rest but will be clear in transit between mac and unraid. If that is not desirable, it is possible to enable encryption at the SMB level.

To do this, first type this in mac terminal: `smbutil statshares -a` . You should see SMB traffic is not encrypted by default:

 

```

                              SMB_NEGOTIATE                 SMBV_NEG_SMB1_ENABLED
                              SMB_NEGOTIATE                 SMBV_NEG_SMB2_ENABLED
                              SMB_NEGOTIATE                 SMBV_NEG_SMB3_ENABLED
                              SMB_VERSION                   SMB_3.1.1
                              SMB_ENCRYPT_ALGORITHMS        AES_128_CCM_ENABLED
                              SMB_ENCRYPT_ALGORITHMS        AES_128_GCM_ENABLED
                              SMB_ENCRYPT_ALGORITHMS        AES_256_CCM_ENABLED
                              SMB_ENCRYPT_ALGORITHMS        AES_256_GCM_ENABLED
               >               SMB_CURR_ENCRYPT_ALGORITHM    OFF
                              SMB_SHARE_TYPE                DISK
                              SIGNING_SUPPORTED             TRUE
                              EXTENDED_SECURITY_SUPPORTED   TRUE
                              UNIX_SUPPORT                  TRUE
                              LARGE_FILE_SUPPORTED          TRUE
                              OS_X_SERVER                   TRUE
                              FILE_LEASING_SUPPORTED        TRUE
                              MULTI_CREDIT_SUPPORTED        TRUE
```

 

Then go to Unraid > Settings > SMB > Extra Config and enter:

 

```
server signing = mandatory 
smb encrypt = mandatory

```

 

Then check on the mac again:

 

```

                              SMB_NEGOTIATE                 SMBV_NEG_SMB1_ENABLED
                              SMB_NEGOTIATE                 SMBV_NEG_SMB2_ENABLED
                              SMB_NEGOTIATE                 SMBV_NEG_SMB3_ENABLED
                              SMB_VERSION                   SMB_3.1.1
                              SMB_ENCRYPT_ALGORITHMS        AES_128_CCM_ENABLED
                              SMB_ENCRYPT_ALGORITHMS        AES_128_GCM_ENABLED
                              SMB_ENCRYPT_ALGORITHMS        AES_256_CCM_ENABLED
                              SMB_ENCRYPT_ALGORITHMS        AES_256_GCM_ENABLED
              >                SMB_CURR_ENCRYPT_ALGORITHM    AES-128-GCM
                              SMB_SHARE_TYPE                DISK
                              SIGNING_SUPPORTED             TRUE
              >                SIGNING_REQUIRED              TRUE
                              EXTENDED_SECURITY_SUPPORTED   TRUE
                              UNIX_SUPPORT                  TRUE
                              LARGE_FILE_SUPPORTED          TRUE
                              OS_X_SERVER                   TRUE
                              FILE_LEASING_SUPPORTED        TRUE
                              MULTI_CREDIT_SUPPORTED        TRUE
              >                ENCRYPTION_REQUIRED           TRUE
              >                SIGNING_ON                    TRUE
```

 

This will give you encryption in transit (for all SMB traffic, not just Time Machine).

Additional considerations:

8. Using an SSD on Unraid will be faster for backing up and also for restoring compared to a spinner on Unraid (or Time Capsule) of course, but it will also be faster than a spinner attached directly to the mac. I am now considering dropping the spinner I have attached directly to the mac.

 

9. Time Machine is a b!tch for reconnecting to your backup when you need it the most (i-e when you need to restore a file). Here is a shortlist of what to do if it doesn't want to play nice:
 

  • Disconnect all network drives in the finder, then reconnect, then open the .sparsebundle volumes manually (will prompt for TM password if encrypted). Once this is done, enter time machine again.
  • If that doesn't help, you can also enter time machine history directly from a .sparsbundle that you have opened in the finder (there will be an "open with time machine" button in the Finder. This will show you only the history that is backed up to this specific bundle (if you have several) but it can be a time saver.
  • If that doesn't help, reboot your mac before trying time machine restores. A freshly booted mac tends to connect much more easily to all its shares (may be a bug in the OS network layer)
  • Occasionally you may need to restart SMB on Unraid, or even restart the array, or even restart the server. 
  • I always ended up being able to access my full file history after those steps.

 

10. However when a backup disk gets errors, it will frequently invalidate your whole .sparsebundle. This is "TimeMachine over network share"'s biggest weakness. Using mirrored zfs raid-z drives may help with that but not only did I not invest that much in SSD drives, I also don't have enough SATA ports left ;) I might try it in the future though.

  • Like 1

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...