fplanque Posted November 21, 2023 Share Posted November 21, 2023 I've spent several days moving my time machine backups from old Apple Time Capsules to Unraid. (MacOS Ventura + Unraid 6.12.4) (I specifically wanted to do this before upgrading to Sonoma, will give Sonoma feedback later) Here's what I learnt: 1) When backing up to a network share, Time Machine creates a virtual drive (.sparsebundle) on the network and then proceeds to manage that whole drive remotely. This is inefficient as hell (but this is how TM manages to save space by hard-linking unchanged files between one backup and the next). This makes network backups an order of magnitude slower than backing up to a drive that's directly attached to the mac. 2) It seems it may be even more inefficient if the backup is encrypted, i-e: when the .sparsebundle is encrypted. 3) Because of that inefficiency, I decided to compensate by backing up to an SSD drive in my unraid. No need to use nvme drives, it will be totally overkill in regard to the speed at which the mac will send backup data. I opted for the cheapest high capacity SATA SSD drive(s) I could find on Black Friday. 4) I am using that SSD as a standalone drive/pool, not protected by anything for 2 reasons: my array parity is not SSD and would slow down writes my time machine strategy is to back up to 2 different unraid boxes in 2 different parts of the building. (This gives better protection than 2 drives mirrored in the same box + allows to back up whenever one of the 2 unraid boxes is down because I'm playing with its configuration...) 5) I am using zfs for the SSD (with LUKS encryption). I also use zfs compression. This only makes sense if time machine does not encrypt its backups by itself. If TM encrypts, then zfs can no longer compress it (for example because strings of nulls will be garbled), It tested this. I get about 20 to 25% compression when letting zfs compress on the server compared to when trying to compress an encrypted backup (which results in close to 0% compression). 6) I observed no slow down due to zfs compression (but I observed more space for backing up more history ;). I think I do see a slower back up with a mac/tm-encrypted backup to uncompressed zfs but I need to test this more. 7) If the backup is not encypted by TM but encrypted by zfs, it will be encrypted at rest but will be clear in transit between mac and unraid. If that is not desirable, it is possible to enable encryption at the SMB level. To do this, first type this in mac terminal: `smbutil statshares -a` . You should see SMB traffic is not encrypted by default: ``` SMB_NEGOTIATE SMBV_NEG_SMB1_ENABLED SMB_NEGOTIATE SMBV_NEG_SMB2_ENABLED SMB_NEGOTIATE SMBV_NEG_SMB3_ENABLED SMB_VERSION SMB_3.1.1 SMB_ENCRYPT_ALGORITHMS AES_128_CCM_ENABLED SMB_ENCRYPT_ALGORITHMS AES_128_GCM_ENABLED SMB_ENCRYPT_ALGORITHMS AES_256_CCM_ENABLED SMB_ENCRYPT_ALGORITHMS AES_256_GCM_ENABLED > SMB_CURR_ENCRYPT_ALGORITHM OFF SMB_SHARE_TYPE DISK SIGNING_SUPPORTED TRUE EXTENDED_SECURITY_SUPPORTED TRUE UNIX_SUPPORT TRUE LARGE_FILE_SUPPORTED TRUE OS_X_SERVER TRUE FILE_LEASING_SUPPORTED TRUE MULTI_CREDIT_SUPPORTED TRUE ``` Then go to Unraid > Settings > SMB > Extra Config and enter: ``` server signing = mandatory smb encrypt = mandatory ``` Then check on the mac again: ``` SMB_NEGOTIATE SMBV_NEG_SMB1_ENABLED SMB_NEGOTIATE SMBV_NEG_SMB2_ENABLED SMB_NEGOTIATE SMBV_NEG_SMB3_ENABLED SMB_VERSION SMB_3.1.1 SMB_ENCRYPT_ALGORITHMS AES_128_CCM_ENABLED SMB_ENCRYPT_ALGORITHMS AES_128_GCM_ENABLED SMB_ENCRYPT_ALGORITHMS AES_256_CCM_ENABLED SMB_ENCRYPT_ALGORITHMS AES_256_GCM_ENABLED > SMB_CURR_ENCRYPT_ALGORITHM AES-128-GCM SMB_SHARE_TYPE DISK SIGNING_SUPPORTED TRUE > SIGNING_REQUIRED TRUE EXTENDED_SECURITY_SUPPORTED TRUE UNIX_SUPPORT TRUE LARGE_FILE_SUPPORTED TRUE OS_X_SERVER TRUE FILE_LEASING_SUPPORTED TRUE MULTI_CREDIT_SUPPORTED TRUE > ENCRYPTION_REQUIRED TRUE > SIGNING_ON TRUE ``` This will give you encryption in transit (for all SMB traffic, not just Time Machine). Additional considerations: 8. Using an SSD on Unraid will be faster for backing up and also for restoring compared to a spinner on Unraid (or Time Capsule) of course, but it will also be faster than a spinner attached directly to the mac. I am now considering dropping the spinner I have attached directly to the mac. 9. Time Machine is a b!tch for reconnecting to your backup when you need it the most (i-e when you need to restore a file). Here is a shortlist of what to do if it doesn't want to play nice: Disconnect all network drives in the finder, then reconnect, then open the .sparsebundle volumes manually (will prompt for TM password if encrypted). Once this is done, enter time machine again. If that doesn't help, you can also enter time machine history directly from a .sparsbundle that you have opened in the finder (there will be an "open with time machine" button in the Finder. This will show you only the history that is backed up to this specific bundle (if you have several) but it can be a time saver. If that doesn't help, reboot your mac before trying time machine restores. A freshly booted mac tends to connect much more easily to all its shares (may be a bug in the OS network layer) Occasionally you may need to restart SMB on Unraid, or even restart the array, or even restart the server. I always ended up being able to access my full file history after those steps. 10. However when a backup disk gets errors, it will frequently invalidate your whole .sparsebundle. This is "TimeMachine over network share"'s biggest weakness. Using mirrored zfs raid-z drives may help with that but not only did I not invest that much in SSD drives, I also don't have enough SATA ports left I might try it in the future though. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.