[Workaround] How to get MacOS SMB transfers at 1GB/s on 10GBE

[1812]

Firstly, credit to @testdasi for sharing this info in a similar thread by @mgutt for windows. (edited to fix attribution)

 

I looked around the board a bit to try and figure out why mac's get such horrible smb performance on unRaid, the os we all know and love. I wanted to setup a faster network scratch drive, but seemed to hit a wall at 400-600ish MB/s. 

 

Setup:

M1 MacBook Pro running Big Sur 11.1, connected to an OWC thunderbolt pro dock with 10GBE <--Cat6a--> MikroTik CRS305-1G-4S+IN <--DAC--> ML350p Gen 8 with a cheap mellanox connectx-2, running unRaid 6.9.0 RC2, with the share hosted on a pool of 4 Samsung EVO 860 1tb drives in RAID 10, mtu 1500 on both network adapters.

 

With this setup, perf3 yields:

 

 

 

This equates to 1.17GB/s, which is great for zero tuning. But I've never achieved anywhere near that on smb transfers on Unraid, until today.

 

 

Original directions here: 

 

 

Following those as a guide almost worked, but it required manual entry via finder>go>connect to server every time I wanted to connect. Yes, I know you can have it auto connect at launch, but for security reasons, I don't store passwords to servers on my laptop. And I hate getting nagged when I'm away from the house about not being able to find/connect to the server. So my smb-custom.conf looked like this:

 

[Scratch-Fast]
	path = /mnt/netdrive/Scratch
	comment =
	browseable = yes
	Force User = nobody
	valid users = banannajoe
	writeable = yes
	write list = admin
	vfs objects = fruit

 

For me, my pool wasn't named cache, it was named netdrive.  I made the share browsable so it was accessible via the finder and added "fruit" to the vfs objects for better macOS compatibility (or so we're told.) I also needed to add "writeable = yes" otherwise the share was readonly for me. Per the instructions, I made the share name different than the original share which was titled "Scratch". So when I use the finder to view the folders on the server, I use the one titled "Scratch-Fast".

 

The resulting smb single large file transfer speeds:

 

 

My receive speed is essentially the same now as iperf3, with the send being about 15% slower (which also matches an unposted iperf3 result.) Again, this is on mtu 1500.

 

Since this for me intended as a fast,  "cache" only share, I don't know the ramifications towards using this method of accessing a share and having the mover do it's thing later. It probably shouldn't matter, but I haven't tested it. So you assume the risk if you do this workaround.

 

Since I still have access to the original folder via the normal user share (shfs) way, here is another comparison

 

Normal user share:

 

 

Workaround:

 

 

 

There may be room for improvement on the write side, but this is good enough for me, for now.... maybe...  

Edited January 3, 2021 by 1812
Redundant title was redundant

[Supacon]

Interesting, thanks for the post. I've had some serious performance issues with SMB on macOS, but my "workaround" is just to not use SMB, as I can access the data in other ways for most of what I am doing. I mainly only use SMB for occasional archival, but when I do use SMB, especially with lots of small files, it's extremely painful, almost slow to the point of  having to question if it's even working at all and restart my mac because some low-level process gets hung up.

 

Not sure if this workaround will help my issue, but it's something I'll look into the next time I try to do some performance tuning.

[mgutt]

Don't use Black Magic for testing. Depending on the benchmark file size you are testing only the RAM of your NAS.

 

You get the real value if you transfer a huge file to the NAS which is bigger than vm.dirty_ratio and after that clear the RAM Caching on your NAS with the following command:

sync; echo 1 > /proc/sys/vm/drop_caches

 

And then download the huge file again (now it will be served through your disk).

 

By that you can see how huge your RAM is, too. At uploading it should drop after x GB. Or it does not because your SSD is fast enough, which is the perfect situation.

[1812]

4 hours ago, mgutt said:

Don't use Black Magic for testing. Depending on the benchmark file size you are testing only the RAM of your NAS.

 

You get the real value if you transfer a huge file to the NAS which is bigger than vm.dirty_ratio and after that clear the RAM Caching on your NAS with the following command:

sync; echo 1 > /proc/sys/vm/drop_caches

 

And then download the huge file again (now it will be served through your disk).

 

By that you can see how huge your RAM is, too. At uploading it should drop after x GB. Or it does not because your SSD is fast enough, which is the perfect situation.

 

As I wrote above, I did use a large file for file transfer testing. This did exceed my dirty ram percentage but I did not include that bit of information. And retesting using the ram clearing command shows the same resulting speeds as before, which is expected because I exceed the dirty ram percentage.

 

But I will disagree with you on using Black Magic. Those results demonstrate and reinforce the point of this thread, that SMB transfers are bottlenecked using regular user shares on Unraid on macOS, and that high rate SMB transfers are possible using this workaround. Your implied point that transferring to RAM is faster is valid, and your point about SSD's possibly not being as fast or fast enough is also valid. But not so much as to dismiss the results of the Black Magic test. Each person using the workaround it will have different results based on their setup, which is why I detailed mine along with the results achieved.  

[mgutt]

11 minutes ago, 1812 said:

Your implied point that transferring to RAM is faster is valid, and your point about SSD's possibly not being as fast or fast enough is also valid. But not so much as to dismiss the results of the Black Magic test.

I don't have a Mac, but you can verify my statement:

 

Settings -> Global Share Settings -> Enable Disk Shares

Optional: Shares -> Disk Shares -> disk1 -> set your user in SMB Security

 

Now benchmark \\tower\disk1 with Blackmagic Disk Speed. If your result is higher than your maximum HDD speed, than you are benchmarking your RAM and not the disk. This is my test result of disk1 with Crystal Disk Mark:

 

Those values are impossible for an HDD.

 

 

[1812]

1 hour ago, mgutt said:

I don't have a Mac, but you can verify my statement:

I don't think you understand, I am not contradicting your statement regarding Ram vs HDD speeds. I am saying that it is not as relevant to the topic as you attest. The topic is not Ram vs HDD speeds in unRaid via SMB. It is about working around the bottleneck of user shares to achieve greater SMB performance.

 

As a side note, Disk Shares don't work for pools as of 6.9.0 rc 2, although the same effect of bypassing the user shares is achieved.

Edited January 3, 2021 by 1812
more thoughts....

[mgutt]

1 hour ago, 1812 said:

It is about working around the bottleneck of user shares to achieve greater SMB performance.

That's absolutely fine. I only wanted to inform you that SSD/HDD Benchmarking Tools do not work for SMB shares. I mean, who copies files to the server and directly downloads the same file again.

[1812]

3 hours ago, mgutt said:

That's absolutely fine. I only wanted to inform you that SSD/HDD Benchmarking Tools do not work for SMB shares. I mean, who copies files to the server and directly downloads the same file again.

People testing network throughput  ¯\_(ツ)_/¯

Edited January 3, 2021 by 1812