SMB Performance Tuning


mgutt

Recommended Posts

 

At the moment I try to enable SMB Multichannel when my Client has 10G LAN and my server has two 1G LAN ports. What I tried:

 

Enabled Multichannel and added speed capabilities per adapter/ip:

image.png.65d0a3fd93c9b5da75b494ef3effb7e6.png

 

Checked on the client if both IPs have been found and selected for Multichannel:

image.png.f080e8d28d97e4e74807f3295b07c6d4.png

 

But finally it does not work as eth1 is not used:

1928259348_2021-08-2308_16_04.png.67ae8e3f3e21913de29f8943c66b8568.png

1705787933_2021-08-2308_16_20.png.9a0ef51f4c9283544c7051f642e91016.png

 

Then I reminded that it is not possible to mix RSS and non-RSS scenarios:

https://docs.microsoft.com/en-us/archive/blogs/josebda/the-basics-of-smb-multichannel-a-feature-of-windows-server-2012-and-smb-3-0

Quote

Sample Configurations that do not use SMB Multichannel

The following are sample network configurations that do not use SMB Multichannel:

  • Single non-RSS-capable network adapters. This configuration would not benefit from multiple network connections, so SMB Multichannel is not used.
  • Network adapters of different speeds. SMB Multichannel will choose to use the faster network adapter. Only network interfaces of same type (RDMA, RSS or none) and speed will be used simultaneously by SMB Multichannel, so the slower adapter will be idle.

 

So the first step was to disable RSS on the clients adapter:

image.png.34f7ee56a9d931b034809903706228b5.png

 

But still no activity on eth1:

image.png.aba0578c624ca0466cdd3c0ced33f0e3.png

 

Then I did:

- disabled IPv6 on the server and on the client

- rebooted server and client

 

And now it works:

image.png.b5722d6c195f6a9776f42888b0696b79.png

 

Was it because of disabling RSS? No, after re-enabling IPv6 and rebooting, it does not work anymore:

image.png.4f07740a936937515fe10443766b35cb.png

 

As you can see "thoth" resolves to an IPv6 address. I tried to copy to both IPv4 addresses of the server, but it does not enable SMB Multichannel:

image.png.af58817484216d626d574899150c8c0c.png

 

image.png.f824f17b1892d1db5d9157465e89954c.png

 

This is strange as for both IPs both target server adapters were found:

image.png.917896c9519100bcf3fe7c5b7135e6d7.png

 

Maybe SMB multichannel works only for SMB server names? Let's try it out by adding "tower" as a new server name for .8:

image.png.d626a10144cd4527f85c40d5de5bea87.png

 

again no success after copying to "tower":

image.png.7a9dc706053ad73da61507a962d17d98.png

 

Next step was to disable IPv6 in the network adapter properties:

1883849231_2021-08-2309_14_19.png.997da4aee18589d648bc579c6fe50f0c.png

 

Even rebooting the client does not help... 

 

I did a little bit research and on this blog I found someone who gets IPv6 addresses if he executes Get-SmbMultichannelConnection:

https://blog.chaospixel.com/linux/2016/09/samba-enable-smb-multichannel-support-on-linux.html

image.png.2a1f3973141f6c72548d52b1351994e1.png

 

So I think my problem is that this command returns in my case only IPv4 addresses even if the clients network adapter has IPv6 enabled. But why 🤔

 

The smb service on Unraid listens to IPv6:

netstat -lnp | grep smb
tcp        0      0 0.0.0.0:139             0.0.0.0:*               LISTEN      30644/smbd          
tcp        0      0 0.0.0.0:445             0.0.0.0:*               LISTEN      30644/smbd          
tcp6       0      0 :::139                  :::*                    LISTEN      30644/smbd          
tcp6       0      0 :::445                  :::*                    LISTEN      30644/smbd

 

And the client resolves with "ping" the smb server name with an IPv6 address, too....

 

Just for fun I added the IPv6 addresses to the smb conf:

interfaces = "fd00::b62e:99ff:fea8:c72c;speed=1000000000" "fd00::b62e:99ff:fea8:c72a;speed=1000000000" "192.168.178.8;speed=1000000000" "192.168.178.9;speed=1000000000"

 

SMB listens now only to these specific IPs:

netstat -lnp --wide | grep smb
tcp        0      0 192.168.178.8:139       0.0.0.0:*               LISTEN      24774/smbd          
tcp        0      0 192.168.178.9:139       0.0.0.0:*               LISTEN      24774/smbd          
tcp        0      0 192.168.178.8:445       0.0.0.0:*               LISTEN      24774/smbd          
tcp        0      0 192.168.178.9:445       0.0.0.0:*               LISTEN      24774/smbd          
tcp6       0      0 fd00::b62e:99ff:fea8:c72a:139 :::*                    LISTEN      24774/smbd          
tcp6       0      0 fd00::b62e:99ff:fea8:c72c:139 :::*                    LISTEN      24774/smbd          
tcp6       0      0 fd00::b62e:99ff:fea8:c72a:445 :::*                    LISTEN      24774/smbd          
tcp6       0      0 fd00::b62e:99ff:fea8:c72c:445 :::*                    LISTEN      24774/smbd

 

And ironically the Windows client now reaches the server through one (?!) IPv6:

Get-SmbMultichannelConnection

Server Name Selected Client IP                             Server IP                 Client Interface Index Server Interface Index Client RSS Capable Client RDMA Capable
----------- -------- ---------                             ---------                 ---------------------- ---------------------- ------------------ -------------------
THOTH       True     192.168.178.21                        192.168.178.9             13                     11                     False              False
THOTH       True     2003:e0:a71d:5700:e988:c813:e0d4:a16a fd00::b62e:99ff:fea8:c72c 13                     12                     False              False

 

But transfer speed is still capped to one....

 

Next try. Disable IPv6 on the client, force SMB to listen only to IPv4, set 192.168.178.8 as the IP of "THOTH" through the windows hosts file and reboot the client, but still no success 🙈

# SMB Conf:
interfaces = "192.168.178.8;speed=1000000000" "192.168.178.9;speed=1000000000"
bind interfaces only = yes
netstat -lnp --wide | grep smb
tcp        0      0 192.168.178.8:139       0.0.0.0:*               LISTEN      31758/smbd          
tcp        0      0 192.168.178.9:139       0.0.0.0:*               LISTEN      31758/smbd          
tcp        0      0 192.168.178.8:445       0.0.0.0:*               LISTEN      31758/smbd          
tcp        0      0 192.168.178.9:445       0.0.0.0:*               LISTEN      31758/smbd

 

So do it reverse. This time IPv6 is enabled on the client, but server gets IPv6 fully disabled...

image.png.7dbaf4551c7ebb3d27e8959242d7368c.png

 

Not sure if this is important, but the server resolves to the second ethernet port. Don't know why:

1499090554_2021-08-2310_32_02.png.8f3612f8eb8a6596e413f122ac810d65.png

 

Server and client reboot... does not work.

 

... after several additional tests I found out, that it is unreliable. For example if I disable IPv6 only in the router and reboot the client, then SMB Multichannel works. But only for several minutes?! Then it fails again.

 

Next test is to disable IPv4 in the router and reboot all devices incl switches. Maybe thats the reason?!

Link to comment
  • 3 weeks later...

In the last "All things Unraid" there was this Blog Thread: https://unraid.net/blog/how-to-beta-test-smb-multi-channel-support

Basically the sum up of your start post ;-)

BUT two more arguments have been added to smb-extra "aio read size" and "aio write size"

As i had no clue what that meant i asked my pal Google and found at Samba.org a detailed explanation (you also have this link in your post https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html)

Besides the fact that (if i understand it right) "aio write size" and "aio read size" should be already "1" on default i also found "aio max threads" which description sounds interesting.

But you wrote none of the other arguments had any noticable effect on speed. So you tested thatspecific argument as well?

Multihreaded Samba should give, in theory, a big performance boost - for weak single core perfomance CPUs at least. Also couldn't find any hints which Samba version might be required for this.

Link to comment
2 minutes ago, jj1987 said:

But you wrote none of the other arguments had any noticable effect on speed. So you tested thatspecific argument as well?

 

Yes, I tested everything and the only difference I found was "aio write size" must be enabled and "write cache size" should be not zero. But write cache size has been already removed in Samba 4.12 because io_uring became the default:

https://wiki.samba.org/index.php/Samba_Features_added/changed#REMOVED_FEATURES_3

 

PS Unraid 6.9.2 uses already Samba 4.12.14

 

8 minutes ago, jj1987 said:

"aio write size" and "aio read size" should be already "1" on default

Yes.

 

And another important change with Samba 4.13 is the auto detection of RSS. So maybe since Unraid 6.10 we don't need this line anymore:

interfaces = "10.10.10.10;capability=RSS,speed=10000000000"

 

So finally we only need to enable SMB Multichannel and everything should be perfectly running.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.