Mac 10.14 SMB > 10gbe > Unraid 6.7 = Slow Write Speeds

[BrianIsSecond]

A couple things...

1. Was wondering if anyone else was having speed issues with combination SMB, Unraid 6.7, and 10gbe (mine is an adapter).
2. Would anyone mind helping to explain the 2 Mac related settings when upgrading to Unraid 6.7:

   1. smb: when Enhanced OS X interoperability set, include "fruit:nfs_aces = no" to be compatible with Unraid security model (I'm assuming this just means typing "fruit:nfs_aces = no" in the blank under Settings > SMB > Samba extra configuration, correct?)

   2. smb: disable samba auto-register with avahi for now (No idea what this means.)

 

Current Setup

• Unraid 6.7
• SMB > Enhanced macOS interoperability: YES
• Mac directly connected to Unraid server through 10gbe adapter (Thunderbolt 2 Akitio 10gbe)
  • For example, server is 192.168.11.220 and Mac computer is 192.168.11.1. Therefore, I just "connect to server" via smb://192.168.11.220 (no issues connecting)
• Proper Updated Driver Install for Thunderbolt 2 Akitio 10gbe (And, yes, I removed all old drivers and reinstalled new version.)
• MacOS 10.14.5
• Mac Network Settings > Advanced > Hardware
  • Configure: Manually
  • Speed: 10Gbase-T
  • Duplex: full-duplex, flow-control (only option)
  • MTU: Jumbo (9000)

 

What Works:

• MacOS 10.13 - smb://192.168.11.220 - Thunderbolt 2 10gbe Akitio adapter - Unraid 6.7 = Good Speeds
• MacOS 10.14.5 - smb://192.168.11.220 - No 10gbe Adapter (built in 1gbe) - Unraid 6.7 = 109MB/s read/write (as expected for 1gbe)

 

What Doesn't Work:

MacOS 10.14.5 - smb://192.168.11.220 - Thunderbolt 2 10gbe Akitio adapter - Unraid 6.7 = 50MB/s write (problem) & 550MB/s read

 

I am struggling trying to determine the exact issue is. It would seam to me like this has to be an MacOS 10.14.05 issue, but I know there were lots of changes to Unraid 6.7 to make things like TimeMachine work over SMB, which I am not using. I also have no idea what "disable samba auto-register with avahi" means in the Unraid 6.7 instructions. Plus I believe MacOS 10.14 supports a higher SMB version, so there still might be a setting on the Unraid server that needs adjustment.

 

Any help would be great, greatly appreciated! Thanks so much!

[Abzstrak]

any cache involved?  or is it writing straight to a single/double parity array?

The only way I foresee getting decent speeds would be to a btrfs pool.

[1812]

I'm inclined to believe it is an osx problem just because I've had things break due to upgrade as they "tweak" things here and there and don't specifically list what they do all the time. That doesn't answer your question. I have a high Sierra running a solar flare 10gbe with 5-600MB/s read/writes. If I have time today, I'll move the card over to a Mojave vm and see what happens.

[BrianIsSecond]

Thanks so much for your help and reply! I'll reply with quotes below. 

[BrianIsSecond]

1 hour ago, Abzstrak said:

any cache involved?  or is it writing straight to a single/double parity array?

The only way I foresee getting decent speeds would be to a btrfs pool.

 

Cache:

Yes, cache is involved. Sorry, I should have mentioned that. I'll include screen shots of all my settings in a corresponding message, but I've got 2 of the 250GB NVMe 970 EVO SSDs using btrfs for cache. Before the upgrades (MacOS 10.14.5 and Unraid 6.7), the speed was fantastic. Probably, 250MB/s - 350 MB/s. It's been so long since I've had issues, I forgot the exact write numbers. Definitely not the 50MB/s I'm getting now. The 1gbe get's 109 MB/s, as expected with 1gbe, so it's definitely something to do with 10gbe and SMB. In past versions of MacOS I had to put "max protocol = SMB3_10" in the Unraid SMB Extra Config settings, but I don't think that's required anymore. 

 

Again, can't thank you guys enough for your input.

[Abzstrak]

1 hour ago, 1812 said:

I'm inclined to believe it is an osx problem just because I've had things break due to upgrade as they "tweak" things here and there and don't specifically list what they do all the time. That doesn't answer your question. I have a high Sierra running a solar flare 10gbe with 5-600MB/s read/writes. If I have time today, I'll move the card over to a Mojave vm and see what happens.

yeah, this is quite possible...

 

Can you test from another machine?  or temporarily just boot it into ubuntu or something and try from there?

[BrianIsSecond]

1 hour ago, 1812 said:

I'm inclined to believe it is an osx problem just because I've had things break due to upgrade as they "tweak" things here and there and don't specifically list what they do all the time. That doesn't answer your question. I have a high Sierra running a solar flare 10gbe with 5-600MB/s read/writes. If I have time today, I'll move the card over to a Mojave vm and see what happens.

 

I'm agree with you. I'm inclined to believe it's MacOS as well. I'm just not sure what they changed. 

 

That would be wonderful if you could test it with MacOS 10.14.5 to verify. I really appreciate you testing that. I myself had an old laptop laying around running High Sierra and it worked well using same server and same 10gbe adapter, but as soon as I updated it to MacOS 10.14.5, it also slowed the write speeds down to 50MBs. So both machines at 10.14.5 write at 50MBs. Reads are around 500MB/s - 600MBs.

 

Do you know what Unraid meant by the following options in their upgrade announcement?

• smb: when Enhanced OS X interoperability set, include "fruit:nfs_aces = no" to be compatible with Unraid security model (I'm assuming this just means typing "fruit:nfs_aces = no" in the blank under Settings > SMB > Samba extra configuration, correct?)
• smb: disable samba auto-register with avahi for now (No idea what this means.)

Oh.... here are my configurations settings:

 

[BrianIsSecond]

1 hour ago, Abzstrak said:

yeah, this is quite possible...

 

Can you test from another machine?  or temporarily just boot it into ubuntu or something and try from there?

 

This is a great question. I just replied the following to user 1812:

Quote

I myself had an old laptop laying around running High Sierra and it worked well using same server, cord, and same 10gbe adapter, but as soon as I updated the laptop to MacOS 10.14.5, it also slowed the write speeds down to 50MBs. So both machines (iMac and MacBook Pro) at 10.14.5 write at 50MBs. Reads are around 500MB/s - 600MBs

... which are where they need to be. It's the writes that are the issue.

 

Therefore, that should verify that it's not the machine. Unfortunately, I don't have another 10gbe adapter lying around or I would have swapped that out too. However, I don't think that's the case, because the adapter works great with MacOS 10.13. 

 

Also still not sure what Unraid means when it says to: "disable samba auto-register with avahi for now."

 

As a last resort, I am also thinking about downgrading the Unraid back down to 6.6.6 for now to see if there is a difference with that. Never downgraded Unraid before. I'm assuming that's safe, since they give a nice clean button for it. Is that a bad assumption?

 

Edited June 9, 2019 by BrianIsSecond

[1812]

3 hours ago, BrianIsSecond said:

 

I'm agree with you. I'm inclined to believe it's MacOS as well. I'm just not sure what they changed. 

 

That would be wonderful if you could test it with MacOS 10.14.5 to verify. I really appreciate you testing that. I myself had an old laptop laying around running High Sierra and it worked well using same server and same 10gbe adapter, but as soon as I updated it to MacOS 10.14.5, it also slowed the write speeds down to 50MBs. So both machines at 10.14.5 write at 50MBs. Reads are around 500MB/s - 600MBs.

 

Do you know what Unraid meant by the following options in their upgrade announcement?

• smb: when Enhanced OS X interoperability set, include "fruit:nfs_aces = no" to be compatible with Unraid security model (I'm assuming this just means typing "fruit:nfs_aces = no" in the blank under Settings > SMB > Samba extra configuration, correct?)
• smb: disable samba auto-register with avahi for now (No idea what this means.)

Oh.... here are my configurations settings:

 

 

I won't get to it today, but, my smb extras text is:

 

max protocol = SMB2_02

#unassigned_devices_start
#Unassigned devices share includes
   include = /tmp/unassigned.devices/smb-settings.conf
#unassigned_devices_end

 

Looking at this, I remember when I was tuning it a year or so ago, I read somewhere that limiting to smb2 was a best practice for macOS 10gbe...  You might give it a try.

 

Also, my osx enhanced operability is set to no, not sure why I never enabled it.

Edited June 9, 2019 by 1812

[BrianIsSecond]

Thanks so much for your help! SMB2_2 was definitely worth a try. Unfortunately, that didn't work either. No difference. What's in: /tmp/unassigned.devices/smb-settings.conf  ?

[BrianIsSecond]

Ok... so here is what I've done.

 

SMB:

I downgraded back to 6.6.6 and tested SMB. The speed was still 50MB write and 450ish read. Therefore, I think it's safe to say that the speed issue is not Unraid.

 

AFP:

Although there are settings for AFP in Unraid 6.7, it no longer works. That's actually the reason why I was trying to use SMB in the first place. Downgrading to 6.6.6 does allow me to use AFP properly again, which gives me speeds of 397MBs write and 790MB/s read.

 

I think I heard that Apple was releasing more support for SMB in 10.15 this Fall. If that's the case, I might just use AFP via Unraid 6.6.6 until 10.15 comes out.

 

No rush, but I would still be curious how your MacOS 10.14 connects to an Unraid SMB share with 10gbe. Especially for those who might find this conversation online in the future.

 

Again... thank you so, so much for your help!

Edited June 9, 2019 by BrianIsSecond

[1812]

1 hour ago, BrianIsSecond said:

Thanks so much for your help! SMB2_2 was definitely worth a try. Unfortunately, that didn't work either. No difference. What's in: /tmp/unassigned.devices/smb-settings.conf  ?

 

unassigned devices placed it there. should make no difference

 

1 hour ago, BrianIsSecond said:

No rush, but I would still be curious how your MacOS 10.14 connects to an Unraid SMB share with 10gbe. Especially for those who might find this conversation online in the future.

 

 

I'll try and get some time tomorrow

 

 

[1812]

2 hours ago, BrianIsSecond said:

Ok... so here is what I've done.

 

SMB:

I downgraded back to 6.6.6 and tested SMB. The speed was still 50MB write and 450ish read. Therefore, I think it's safe to say that the speed issue is not Unraid.

 

AFP:

Although there are settings for AFP in Unraid 6.7, it no longer works. That's actually the reason why I was trying to use SMB in the first place. Downgrading to 6.6.6 does allow me to use AFP properly again, which gives me speeds of 397MBs write and 790MB/s read.

 

I think I heard that Apple was releasing more support for SMB in 10.15 this Fall. If that's the case, I might just use AFP via Unraid 6.6.6 until 10.15 comes out.

 

No rush, but I would still be curious how your MacOS 10.14 connects to an Unraid SMB share with 10gbe. Especially for those who might find this conversation online in the future.

 

Again... thank you so, so much for your help!

 

Good news bad news

 

Good news: I got to test this already

 

Bad news: it works fine for me on 10.14 using solar flare 10gbe card.  This is about a 3GB video file.

 

 

 

 

So either it is a problem with the adapter, or thunderbolt.

 

ALSO reset your MTU back to 1500 on both the Mac and the 10gbe port on the server. Worst case is that its slower. Best case is you had a MTU mismatch killing your write speed.

 

 

 

 

[John_M]

3 hours ago, BrianIsSecond said:

Although there are settings for AFP in Unraid 6.7, it no longer works.

AFP works fine with High Sierra 10.13.6 and Unraid 6.7.0 for me. I can't test with Mojave as none of my Macs are new enough to run it - it doesn't seem as though I'm missing much though, except for a load of hassle and incompatibility.