10gbe SMB additional settings

[therapist]

I am working to improve SMB performance and have found a collection of settings that work well.  I would like to deploy to all interfaces, but I get strange delays / drops when applied as written:

 

server multi channel support = yes
aio read size = 1
aio write size = 1
interfaces = "192.168.1.248;capability=RSS,speed=10000000000" "192.168.40.2;capability=RSS,speed=10000000000"

 

with both interface characteristics specified the main vm on 192.168.1.x hangs on file transfer and transfers dreadfully slow

if i remove the 2nd interface (192.168.40.2) main vm on 192.168.1.x works OK, but VM on that interface does not exceed ~175mb/s transfer

 

are the interfaces listed correctly?

 

[therapist]

I have hit a bit of a speed limit on my setup

 

iperf3 tests indicate 8+gb/s of available bandwidth between unraid and my test PC, but I cant transfer over SMB faster than 490-520 mb/s

 

unraid v6.11.5 on

EPYCD8-2T w/ 7551

eth0 10gb rj45 --> sfp+ @ mikrotik CRS305-1G-4S+

test share/disk = INTEL SSDPF2KX038TZ

 

test pc = unraid VM w/ mobo eth1 passed through

24c 24gb memory

disk = Seagate FireCuda 530

 

diagnostics attached

crunch-diagnostics-20240129-1604.zip

[itimpi]

Are you transferring to a User Share or direct to the drive?   Asking as the Fuse layer used to support User Shares can impose that sort of speed limit.   If you are transferring to a User Share then if it is all one one device/pool so it can become an Exclusive share (which bypasses Fuse) you can get the same performance as transferring directly to the physical device.

[therapist]

3 hours ago, itimpi said:

Are you transferring to a User Share or direct to the drive?   Asking as the Fuse layer used to support User Shares can impose that sort of speed limit.   If you are transferring to a User Share then if it is all one one device/pool so it can become an Exclusive share (which bypasses Fuse) you can get the same performance as transferring directly to the physical device.

 

I have been doing my testing with both

I have a EVO 860 SSD as my main cache disk...so current performance saturates that read/write

I have an intel NVME disk installed for VM VHDs and can test full 10gbe bandwidth through that

 

Copying a testfile from RAID1 "cache-protected" to nvme gets speeds I would expect from RAID1 SAS SSD --> NVME

 

Benchmarking the VHD for the VM (which is on the nvme) gets results I expect

 

iperf shows bandwidth is there

 

But when I transfer to DISK share using SMB the speed doesnt translate

 

 

I thought that it was a networking issue At first because originally my VM was coming out through br0 on unraid. I was able to pass through one of the 10ge ports from my motherboard directly to the VM and improved speed but not to where I think it should be

Edited January 30 by therapist

[JorgeB]

7 hours ago, therapist said:

iperf shows bandwidth is there

This looks like a dual stream test, try single stream, that should get you the closest result to a single transfer. 

[therapist]

On 1/30/2024 at 3:21 AM, JorgeB said:

This looks like a dual stream test, try single stream, that should get you the closest result to a single transfer. 

 

this is indicating nothing better than gigabit...

all links are indicating 10gb

any idea where to look into what i am doing wrong?

 

 

[JorgeB]

Low iperf results are usually related to NICs (or NIC driver/settings), cables, switch, client PC, etc

[therapist]

On 1/31/2024 at 1:54 PM, JorgeB said:

Low iperf results are usually related to NICs (or NIC driver/settings), cables, switch, client PC, etc

so i have a VM on this unraid box that is on a vlan (192.168.40.253)
it connects to unraid SMB at 192.168.40.2

 

file transfer rates are close to reported bandwidth

 

there are no wires, just the virtual 10gbe adapter and unraid

i have reset the VM network settings to default, adjusted max rss to 6 to match cores on VM

unraid has all default network settings except for buffers through tips & tweaks plugin...which shouldnt matter for a VM using br0.x, no?

 

 

I get better than gigabit, but something isnt right since im not seeing anywhere near proper speeds

 

 

 

 

 

 

[JorgeB]

In my experience a VM is not a good way to test, the virtual NIC usually performs very far from 10GbE line speed.