SuberSeb

Can you provide the whole specs of your servers? And also firmware version on NICs?

What firmware version installed on your NICs? Did you try to upgrade/downgrade firmware? It's the only idea that I have at the moment. I will try to test too.

What NICs do you have?

.13 has the same issue.

I have MCX311A. Did you try to resolve it somehow without rollback? I tried .12 with no luck, will try .13 today.

Thanks for the reply and for the help, but in my case I'm not using ZFS at all... By pools are BTRFS pools. Anyway, thanks for checking and for help.

After downgrade to 6.12.10 speed improves instantly: pv /mnt/user/Test/image.img > /mnt/remotes/192.168.1.1_Test/image.img 51.9GiB 0:02:17 [ 385MiB/s]

I made some tests. First one is SMB, second one is NFS. Very poor speeds for NVMe to NVMe transfer. NFS is slow even for HDD. pv /mnt/user/Test/image.img > /mnt/remotes/192.168.1.1_Test/image.img 51.9GiB 0:04:30 [ 195MiB/s] [======================================================================================================>] 100% pv /mnt/user/Test/image.img > /mnt/remotes/192.168.1.1_Test/image.img 51.9GiB 0:05:22 [ 164MiB/s] [======================================================================================================>] 100% So seems not to be related to SMB. Here is iperf3 result: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.05 GBytes 8.99 Gbits/sec 0 495 KBytes [ 5] 1.00-2.00 sec 1.09 GBytes 9.39 Gbits/sec 0 639 KBytes [ 5] 2.00-3.00 sec 1.09 GBytes 9.37 Gbits/sec 0 515 KBytes [ 5] 3.00-4.00 sec 1.09 GBytes 9.39 Gbits/sec 0 481 KBytes [ 5] 4.00-5.00 sec 1.09 GBytes 9.41 Gbits/sec 0 427 KBytes [ 5] 5.00-6.00 sec 1.09 GBytes 9.37 Gbits/sec 0 475 KBytes [ 5] 6.00-7.00 sec 1.09 GBytes 9.38 Gbits/sec 0 433 KBytes [ 5] 7.00-8.00 sec 1.09 GBytes 9.40 Gbits/sec 0 438 KBytes [ 5] 8.00-9.00 sec 1.09 GBytes 9.40 Gbits/sec 0 481 KBytes [ 5] 9.00-10.00 sec 1.09 GBytes 9.38 Gbits/sec 0 424 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 10.9 GBytes 9.35 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 10.9 GBytes 9.34 Gbits/sec receiver

I will try to test with a NFS mounted share, also I will try to test with a Windows PC but it's unlikely that it will hit 10G due to PC hardware limitations.

Can it be related to SMB and/or UD plugin?

Well, I don't see what can affect it either. There were no any configuration or hardware updates for months (or even years), servers just working and I do not touch anything. But the strange thing that I had this issue twice and both times right after OS updates... Any ideas what else I should to check?

I'm using Mellanox ConnectX-3 MCX311A.

I've checked multiple things. Network seems totally ok (almost 10G between servers in both directions), SMB/Network settings was not changed for years and seems ok too. Router, switches are totally ok. So this thread is my last chance to resolve this issue. Thank you anyway.

I forgot to mention that I got 3 servers on Unraid and problem exists on the all of them.

Already done this before opening this thread, speed improves instantly.