I'm having the exact same problems, with a new (trial) install of unraid 7.2.0 with a freshly created 4x8tb ZFS raidz1 pool with no compression or encryption. I've tried a bunch of different smb options, even going to far as even trying leaving the share public. For the smb share, it's exported as Yes/Time Machine, size limit of 2000000MB (2tb), case sensitivity auto, and security i've tried everything, it's currently public. For the SMB global settings, hide dot is off, multichannel is off, enhanced mac interoperability is on, and netbios is off. Mac is a M1 macbook pro running Tahoe 26.1, using encrypted backups on the mac side. What's interesting is that it's clear that the mac is definitely mounting and writing to the smb share, as a terminal on the unraid server shows the creation of: Creates a directory in the share named: mymac\ 2025-11-17-154930.incomplete/ with files: drwxrwxrwx 4 jeff users 8 Nov 17 15:49 ./
drwxrwxrwx 3 nobody users 4 Nov 17 15:49 ../
-rwxrw-rw- 1 jeff users 575 Nov 17 15:49 Info.bckup*
-rwxrw-rw- 1 jeff users 575 Nov 17 15:49 Info.plist*
drwxrwxrwx 2 jeff users 7 Nov 17 15:52 bands/
-rwxrw-rw- 1 jeff users 0 Nov 17 15:49 lock*
drwxrwxrwx 2 jeff users 7 Nov 17 15:52 mapped/
-rwxrw-rw- 1 jeff users 1092 Nov 17 15:49 token*
and when it fails a couple of minutes later with the error: "The backup disk image could not be created." Once Time Machine fails, it leaves these files in the share; drwxrwxrwx 2 jeff users 5 Nov 17 15:52 ./
drwxrwxrwx 3 nobody users 4 Nov 17 15:52 ../
-rwxrw-rw- 1 jeff users 575 Nov 17 15:49 Info.plist*
-rwxrw-rw- 1 jeff users 0 Nov 17 15:52 com.apple.TimeMachine.sync.6DB6F447-3C43-4456-A130-5016634CAD83*
-rwxrw-rw- 1 jeff users 1092 Nov 17 15:49 token*
The same macbook is backing up to a synology nas just fine. I've tried rebooting everything multiple times. I'm completely stuck at this point, any suggestions? It sounds identical to the OP's issue.