xxx59712 Posted July 16, 2020 Share Posted July 16, 2020 Hi, Windows workstation and Unraid server are connected directly by a DAC Copper cable (I ruled out the cable issue by using 3 differents brands and cable length). XA520-DA2 Cards and Driver 4.1.197.0 - 27/11/2019. Windows 10 Pro / Unraid (latest version) The problem is that Bandwith results are below expectations and not symmetrical and that they are well below what I get on Ubuntu. Using two Ubuntu live USB keys and doing a Iperf on both machines, I get a constant robust symmetrical 9.49 Gbits/sec between the two workstations (see below). I ruled out the PCIe ports restrictions by leaving just the NIC on the 16x port of each machine and by using a Remote Desktop to connect to Windows. Here are the results on Windows. I tried every possible settings on the Device manager Driver tab. The only thing that seems to improve things a bit are Disabling Flow Control and setting the Interrupt Moderation Rate to Off. The other settings, jumbo frame and other Buffers have zero impact on the performance. Anybody with experience with the Intel X520 cards would have an idea how to improve things? >iperf3 -c 10.10.10.3 (Windows Machine) Connecting to host 10.10.10.3, port 5201 [ 4] local 10.10.10.2 port 9356 connected to 10.10.10.3 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 604 MBytes 5.07 Gbits/sec [ 4] 1.00-2.00 sec 678 MBytes 5.69 Gbits/sec [ 4] 2.00-3.00 sec 731 MBytes 6.13 Gbits/sec [ 4] 3.00-4.00 sec 745 MBytes 6.25 Gbits/sec [ 4] 4.00-5.00 sec 747 MBytes 6.27 Gbits/sec [ 4] 5.00-6.00 sec 742 MBytes 6.22 Gbits/sec [ 4] 6.00-7.00 sec 769 MBytes 6.45 Gbits/sec [ 4] 7.00-8.00 sec 764 MBytes 6.41 Gbits/sec [ 4] 8.00-9.00 sec 758 MBytes 6.36 Gbits/sec [ 4] 9.00-10.00 sec 788 MBytes 6.61 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 7.16 GBytes 6.15 Gbits/sec sender [ 4] 0.00-10.00 sec 7.16 GBytes 6.15 Gbits/sec receiver iperf Done. iperf3 -c 10.10.10.2 (Unraid) Connecting to host 10.10.10.2, port 5201 [ 4] local 10.10.10.3 port 49799 connected to 10.10.10.2 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 324 MBytes 2.72 Gbits/sec [ 4] 1.00-2.00 sec 334 MBytes 2.80 Gbits/sec [ 4] 2.00-3.00 sec 350 MBytes 2.93 Gbits/sec [ 4] 3.00-4.00 sec 327 MBytes 2.74 Gbits/sec [ 4] 4.00-5.00 sec 324 MBytes 2.72 Gbits/sec [ 4] 5.00-6.00 sec 276 MBytes 2.31 Gbits/sec [ 4] 6.00-7.00 sec 321 MBytes 2.69 Gbits/sec [ 4] 7.00-8.00 sec 323 MBytes 2.71 Gbits/sec [ 4] 8.00-9.00 sec 336 MBytes 2.82 Gbits/sec [ 4] 9.00-10.00 sec 336 MBytes 2.82 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 3.17 GBytes 2.73 Gbits/sec sender [ 4] 0.00-10.00 sec 3.17 GBytes 2.73 Gbits/sec receiver iperf Done. Ubuntu Live USB Key (20.04 version) Connecting to host 10.10.10.3, port 5201 [ 5] local 10.10.10.2 port 59388 connected to 10.10.10.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.09 GBytes 9.32 Gbits/sec 85 1.77 MBytes [ 5] 1.00-2.00 sec 1.10 GBytes 9.42 Gbits/sec 7 1.77 MBytes [ 5] 2.00-3.00 sec 1.10 GBytes 9.42 Gbits/sec 10 1.77 MBytes [ 5] 3.00-4.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.77 MBytes [ 5] 4.00-5.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.77 MBytes [ 5] 5.00-6.00 sec 1.09 GBytes 9.41 Gbits/sec 0 1.77 MBytes [ 5] 6.00-7.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.77 MBytes [ 5] 7.00-8.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.77 MBytes [ 5] 8.00-9.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.77 MBytes [ 5] 9.00-10.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.77 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 11.0 GBytes 9.41 Gbits/sec 102 sender [ 5] 0.00-10.00 sec 10.9 GBytes 9.40 Gbits/sec receiver iperf Done. Connecting to host 10.10.10.2, port 5201 [ 5] local 10.10.10.3 port 46954 connected to 10.10.10.2 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.08 GBytes 9.32 Gbits/sec 37 1.36 MBytes [ 5] 1.00-2.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.36 MBytes [ 5] 2.00-3.00 sec 1.09 GBytes 9.41 Gbits/sec 0 1.36 MBytes [ 5] 3.00-4.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.36 MBytes [ 5] 4.00-5.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.36 MBytes [ 5] 5.00-6.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.36 MBytes [ 5] 6.00-7.00 sec 1.09 GBytes 9.41 Gbits/sec 0 1.37 MBytes [ 5] 7.00-8.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.37 MBytes [ 5] 8.00-9.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.37 MBytes [ 5] 9.00-10.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.38 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 10.9 GBytes 9.40 Gbits/sec 37 sender [ 5] 0.00-10.00 sec 10.9 GBytes 9.40 Gbits/sec receiver iperf Done. Thank you. Quote Link to comment
JorgeB Posted July 16, 2020 Share Posted July 16, 2020 Install the Tips and Tweaks plugin and try changing some of the NIC settings, there are some reports that performance with those NICs is better with NIC Offload disable. Quote Link to comment
xxx59712 Posted July 16, 2020 Author Share Posted July 16, 2020 10 hours ago, johnnie.black said: Install the Tips and Tweaks plugin and try changing some of the NIC settings, there are some reports that performance with those NICs is better with NIC Offload disable. Thank you for the suggestion, I will try but my feeling that it comes from the Windows side, not the unraid side. Anybody has experience with that card under Windows? Any idea on the Windows side? Quote Link to comment
xxx59712 Posted July 16, 2020 Author Share Posted July 16, 2020 Just tried an Ubuntu 20.04 "iperfing" Unraid and bingo 9.49 GBits/sec. So Unaid is not the limiting factor, Windows is! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.