Time Machine incredibly slow 10GB in 24 hours Unraid 6.8.3


Recommended Posts

so i had the problem with catalina not seeing the TM share. I edited the smb.servce file but that gave me the share isn't capable error. so i did the 2nd thing in that thread and created the timemachine.service and that worked!

 

i'm able to select the time machine share and started a backup

 

BUT.....

 

it's been 24 hours and i've only copied 10GB out of 500GB. this will take almost 2 months to do a full backup? i've tried a few things i read about to speed it up like changing the capital letters or whatever to yes. that didn't improve anything. i've tried deleting the share and creating a new one. same thing. TM is super slow.

 

any other options i'm missing?

 

i'm running a 2019 MBP 16" i9 2.3ghz 16gb ram 1tb HD - macOS 10.15.7 - i do not want to update to big sur just yet but if that fixes TM then i would. 

 

thx

Link to comment

A few observations from my testing.

 

SMB random read/write performance in MacOS stinks. I'm using Big Sur and it's still worse than the old AFP method. Still you're getting relatively terrible performance. My initial ~150GB backup over wireless took about half a day.

 

One of the recommended SMB tweaks is disabling signing. Have you done that? It may require a reboot to take effect:

$ more /etc/nsmb.conf
[default]
signing_required=no

 

I think? the spaceinvader tutorial sets an unassigned device as the time machine share. That's good because it bypasses the cache/disk abstraction layer and every bit helps.

 

If you're backing up to an array share that uses BTRFS cache (I run a dedicated cache-only pool in beta35) setting Copy-on-write to No seems to help.

 

The new APFS formatting for Time Machine images in Big Sur (previous versions used HFS+) also seems to help. And I may be imagining it but my backups seem smaller. Note: I don't think it's possible to convert existing HFS+ images to APFS so you have to start fresh.

Edited by CS01-HS
Link to comment

i am running BTRFS - i don't know where you would turn copy-on-write to no? where is this option?

 

i did more research and i did all the smb editing on my macOS and unraid that i could find. it has improved. i'm up to 102GB now.

 

I did turn off the signing. i have no idea what helped? maybe all of it? i tried all the suggestions i could find online and they did seem to help. 

 

at this rate i should be backed up in a few days which is acceptable for a 500gb initial backup. hopefully subsequent backups will be much quicker.

Edited by [email protected]
Link to comment
8 hours ago, [email protected] said:

i am running BTRFS - i don't know where you would turn copy-on-write to no? where is this option?

Assuming that's an array share and not an unassigned device, go to Shares then click the share and you'll see the field Enable Copy-on-write.

 

Disabling it when creating a new share works (which means you'd have to start over) but I don't think changing it for an existing share with existing directories will, although there may be a way to manually convert it.

 

In my experience though these are all marginal tweaks, I'm not sure there is a way to get fast time machine backups. I switched from a Time Capsule to unRAID hoping to speed things up. If it has it's not by much. 

 

The app TimeMachineEditor lets you reduce the backup frequency so at least you get fewer slow backups.

Edited by CS01-HS
Link to comment

same here. it is running in few bytes/kilobytes per second while with Blackmagic disk speed test i can do 70 MB/s writes

also FTP can do the same speed writes

i did all the smb signing off with no help

turned off hard links no help

same to ssd drive

same on Mojave latest updates and Catalina latest updates

 

i went to unraid over synology as i read that the TM is working but now i read that in the version 6.8.3 it is not working to many users and there seems to be no real support just guessing on the forums which is very frustrating

 

Link to comment
1 hour ago, theruck said:

i went to unraid over synology as i read that the TM is working but now i read that in the version 6.8.3 it is not working to many users and there seems to be no real support just guessing on the forums which is very frustrating

6.8.3 should work. The Spaceinvader tutorial (did you follow it?) uses that or an earlier version.

 

Max throughput with SMB is fine - I can max out my wireless uploading to a share. It's small files or whatever Time Machine does under the hood that's painful. Try this benchmark for random read/write:

https://www.katsurashareware.com/amorphousdiskmark/

 

I won't blame Limetech for slowness unless I see a network implementation that isn't slow and I haven't. Apple really dropped the ball on a clever and unique feature.

Link to comment
  • 2 weeks later...

i did a series of tests. the server is a HP Microserver Turion N40L. I was running windows server on it for years but never had such issues with SMB shares like i have with Unraid. Encryption does not seem to have much effect here. It has performance trouble even when copying between internal drives.

 

immagine.thumb.png.6114a38a16d081e1f33a9a5fdeb1fbd9.png

 

Also the stats showing throughputs on the network side seem to show big BS while it is copying over files.

on MAC some serious throttling occurs

 

immagine.thumb.png.81a646b70ca372c9b6ff6f7d1d545097.png

 

and windows has some ups/downs too

immagine.png.0d9aa96425ee3c1b797fd3a72dabe325.png

 

I am curious to see if i upgrade the hardware for some SATA3 mainboard and better CPU if i get better results with unraid or if i boot xpenology if i get the same results on the same hardware. Will keep you posted but there are tons of people complaining about SMB performance on Macs but most of them are treatable on the server side.

Anyway could someone please explain me the meaning of SSD cache drive existance if the parity drive is HDD?Is the parity being written when i write to the cache or not? Coz when i run timemachine i can hear the parity drive going nuts so i wonder if it is the reading due to the time machine or some uncontrolled writing. Anyway using SSD cache or no SSD cache does not make a difference on a Timemachine backup performance.

 

Anyway using an unraid SMB share for accessing photos is a real PITA as the photos being 1.5 MB in size are basically not browsable as it takes minutes just to list the directory where the photos are and previewing them with finder takes many seconds too. And yes i have disabled the previews genration in finder and the .DS_STORE file generation on SMB client. Finder just does not get the data while the unraid appliance goes nuts just by visiting the folder with photos probably finder generating the thumbnails

immagine.thumb.png.e990289f5448376dce78f024e405efef.png

Edited by theruck
Link to comment
On 12/18/2020 at 11:18 AM, [email protected] said:

ok my backup just finished today. 

 

850gb in 14 days.

That's extreme. I think my initial backup over wireless averaged 7MB/s or at worst 5MB/s. 5MB/s (if my math's right) puts 850GB at around 2 days run time.

 

On 12/18/2020 at 11:33 AM, theruck said:

Anyway could someone please explain me the meaning of SSD cache drive existance if the parity drive is HDD?Is the parity being written when i write to the cache or not? Coz when i run timemachine i can hear the parity drive going nuts so i wonder if it is the reading due to the time machine or some uncontrolled writing. Anyway using SSD cache or no SSD cache does not make a difference on a Timemachine backup performance.

Cache will function differently depending on the share's setting.

 

If it's cache-yes, share data is written to cache and will remain there until mover's run at which point it's transferred to the (parity-protected) array. If new files are then written they'll be written to the cache and go through the same process. 

 

If it's cache-only, all share data, assuming it can fit, will remain permanently on the cache. It shouldn't touch the array.

 

If it's cache-prefer, all share data will remain on the cache and if any's on the array, maybe due to an earlier, different cache setting, it will be moved from array to cache when mover runs, assuming it can fit.

 

If you edit a share and click help on Use cache pool you'll see a more thorough explanation.

 

NOTE: Data on a single-drive cache is not parity-protected which is why unRAID offers the option to run two cache drives in RAID1 - I use this.

 

The way I've set it up Time Machine never touches the array. That seemed like a bad idea performance-wise and in terms of wear on a critical disk. If my cache pool were large enough I'd have set the share to cache-prefer, run mover and that'd be that. Because it's not, and I'm running RC2 (which supports multiple cache pools), I set up a new cache pool just for time machine (with cache-prefer.)

 

I can confirm I saw little if any difference in Time Machine speed between SSD and the time-machine pool (7200rpm drives in RAID5) but I don't how much of that was due to RAID5 and/or 7200rpm.

Link to comment
  • 2 weeks later...
On 12/5/2020 at 2:49 PM, CS01-HS said:

One of the recommended SMB tweaks is disabling signing. Have you done that? It may require a reboot to take effect:


$ more /etc/nsmb.conf
[default]
signing_required=no

 

Is that tweak done on the Unraid or Osx via terminal?

Link to comment
  • 4 weeks later...

so i have upgraded the hardware of my server and did some new tests. some interesting data came out. check it out.

source table

immagine.thumb.png.0e734fc656f21228c21f56f5a3910475.png

 

 

comparing operating system - the Linux means unraid server itself disk to disk copy

immagine.png.69888c1b96db999e39b9970df44d2668.png

 

conclusion: AFP protocol performs much better than SMB protocol on a MAC. And yes i did all the recomended tweaks on MAC side.

 

comparing throughputs when cache was used vs cache is not used

immagine.png.c77a5beb2854a84f34c106bed0f93824.png

 

conclusion: enabling the cache on your shares has negative performance effect on my writing speeds note: encryption is used and the parity disk is HDD not SSD.

 

comparing CPU performance impact

immagine.png.e240e7c21ef1724a3b6f3ecb02a4d2d1.png

 

conclusion: the better CPU had positive efect on both AFP and SMB performance with large file while it has negative effect on thousands of small files (strange)

 

 

comparing: OS

immagine.png.93b02b1fd2e3b4d94405fac125ab69dc.png

 

conclusion: the best performing client OS for SMB is Windows, on MAC AFP is the winner. For many small files FTP is the best option (the linux is the local copy)

 

 

CPU impact on drive encryption

immagine.png.3755fbe4d563bd7dcbea7e2058ee412a.png

 

conclusion: not sure if the perofrmance is due to the encryption or the fact that the higher spec CPU is able to better saturate the NIC

.

 

 

 

immagine.png

immagine.png

Edited by theruck
corrections
  • Thanks 1
Link to comment

Wow that's comprehensive.

 

1 hour ago, theruck said:

conclusion: AFP protocol performs much better than SMB protocol on a MAC. And yes i did all the recomended tweaks on MAC side.

 

It's unfortunate Apple chose to deprecate it.

 

I was surprised to see in your tests with small files that Windows wasn't much better than Mac. I assumed Mac was the biggest culprit but this suggests it's SMB itself or unraid's implementation. I guess to narrow that down you could benchmark a non-unRAID SMB share.

Link to comment
5 hours ago, CS01-HS said:

Wow that's comprehensive.

 

 

It's unfortunate Apple chose to deprecate it.

 

I was surprised to see in your tests with small files that Windows wasn't much better than Mac. I assumed Mac was the biggest culprit but this suggests it's SMB itself or unraid's implementation. I guess to narrow that down you could benchmark a non-unRAID SMB share.

i did a test on the same hardware with synology implementation here

and it was bbetter performing but it is a different style of parity so not much comparable. Anyway i stick with AFP for now. My new 4 port Intel GbE is on the way and will perform new tests with that to see if the problem can be mitigated with NIC.

  • Thanks 1
Link to comment
  • 1 month later...

FWIW, I had horrible SMB unRaid R/W speeds with my Mac workstations including Time Machine backups until I did this one weird trick...

 

 

 

 

🤣 ok, now that you fell for the click bait... after trying the various suggestions from others (I.e. turn off SMB signing, turn Case sensitivity on, forcing the min/max protocol to SMB2, etc.) nothing seemed to consistently work. However, someone suggested on the Mac unchecking "Put hard disks to sleep when possible" in the Energy Saver settings and then reboot. It seems to have 'fixed' the issue for me. Haven't had R/W problems since, even after upgrading to 6.9.0. Fwiw, Time Machine is still a bit slow, which is probably because it's setup as a direct write inside the array, but at least it's bearable for me now. At some point along the way I undid all the other suggestions. Maybe reenabling them would improve things more... IDK. Anyways, hope this helps.

Edited by Joseph
missed a step or 2
Link to comment
  • 3 weeks later...
On 3/10/2021 at 6:36 PM, Joseph said:

However, someone suggested on the Mac unchecking "Put hard disks to sleep when possible" in the Energy Saver settings and then reboot. It seems to have 'fixed' the issue for me. Haven't had R/W problems since, even after upgrading to 6.9.0.

 

Has this fix stuck? Approximately how long does it take to complete a backup and of what size?

Mine is about 20 minutes for 700MB but I wiped and re-started my backup set a few days ago so it might slow down over time.

 

You can run this command in an OS X terminal to see details as time machine runs:

log stream --style syslog  --predicate 'senderImagePath contains[cd] "TimeMachine"' --debug

 

I think it would help if we had a baseline - what's the fastest we can expect with networked backups whether that's to unRAID, synology, Time Capsule, a shared Mac, etc.

 

So far my unRAID pool beats Time Capsule.

Link to comment
On 3/31/2021 at 11:23 AM, CS01-HS said:

 

Has this fix stuck? Approximately how long does it take to complete a backup and of what size?

Mine is about 20 minutes for 700MB but I wiped and re-started my backup set a few days ago so it might slow down over time.

So far so good for several months now.

 

 

Quote

You can run this command in an OS X terminal to see details as time machine runs:

 

 I like the idea of establishing a baseline so we all know. Unfortunately, that command doesn't seem to be working. Could it be because the backup box turned off in order to do a HW upgrade? If that is the case, I'll let you know after I get it up and running and try the command again.

 

fwiw,

 

Production Box:

  • A DnD transfer of a 9.6GB file from the Mac to the cache pool now takes about 2mins when it used to take hours (I should note, I've just upgraded to 10GBe, but haven't seen any noticeable improvement; which is another topic for discussion)
  • a DnD transfer of an 8GB file from a VM which is stored on an SSD to the cache drive seems to be faster than the file transfer on the Mac. (see attached)

Backup Box:

  • If memory serves, it now takes TM about 30 mins to backup about 7GB or so, plus cleanup if any. (For whatever reason it's always backing up a minimum of about 7GB, which is annoying, but I digress). I'll double check that once I get the backup box back up and running, but it used to take hours, if not days in some cases to finish.
  • The backup box is multi use, and I have VMs backing up crucial data from the production box to the backup throughout the day, so I'm sure that affects performance as well.

Additional Thought(s):

  • Does anyone actually trust the ability of TM on unRaid to be able to successfully complete a Bare Metal Restore?

 

note: these screen captures show different files copied, but you get the idea:

Screen Shot 2021-03-31 at 4.42.19 PM.png

Screen Shot 2021-03-31 at 4.54.05 PM.png

Edited by Joseph
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.