ZFS + Time Machine - Hints (SLOG)

[scot]

After a ton of attempts and failures, just wanted to give up some basic advice. TLDR: Use a SLOG with ZFS to make TimeMachine work, may have some Mac based benefits as well.

 

Issue: Using Time Machine over SMB writing to a ZFS based storage system works at first, but then starts to take longer and longer as time goes on.  Eventually it gets so slow that a failure occurs.  My guess:  If you let it continue it will seemingly start trying to backup when a previous backup is ongoing causing a corruption in the backup.  Your only option at that point is to delete and start over.

 

This only happens when you have a long history, maybe a few months with a lot of changing data backed up and a large space dedicated (5tb in my case, failures start at around 2tb in stored space used).

 

Initial solution: Tried every setting under the sun and eventually switched to mortizf's Time Machine Unraid app.  This has its own dedicated samba system and worked initially (like every time I restarted my backups) but eventually failed.  

 

Increased system speed and added 2.5gbe ports, no effect.  

 

Then a thought occurred:: Time Machine over SMB forces Write Sync.  This is important and relates to ZFS.  If the issue is the system binding up, would this be the one place an SLOG device would actually help?

 

Popped in an SLOG device, did it wrong (command line, causing Unraid heartburn) but ZFS worked and accepted the SLOG as its own.  See another post I made about fixing Unraid's visual of the SLOG device.  

 

Backups went from ~11hrs per single daily update, down to about 20-30 minutes or less if doing hourly.  This is just due to how ZFS works and inefficiencies in the overall system including SMB and the like, but it no longer gets hung up on the millions of small write and commit requests.  I can't say for sure this will solve all of your issues, and probably won't make a huge difference otherwise, but in my case *everything* works faster from Mac, including directory listing.  I have no idea why directory listing would involve a synchronous write, but... I cant argue the difference.  

 

In my case I am using a 6 drive array (4+2 raidz2) 20tb per disk, with a single nvme with 128gb in the SLOG.  Eventually I will change that to a mirrored pair and only 64gb.  

 

I make no claims about XFS and other file systems or the fuse style file system or anything else.  This is purely a ZFS based file system directly shared.