MostHated Posted April 18, 2021 Share Posted April 18, 2021 (edited) Hey there, I setup a 10gb network the other day and everything has been working great so far, except for my Windows VM. It has been giving rather confusing results. The setup: NAS - Qnap TVS-951x NIC 1gb - 192.168.50.200 NIC 10gb - 172.254.6.200 (255.255.0.0 / no gateway) Desktop PC - Pop_OS 20.10 - Gigabye Aorus Elite x570 Onboard 1gb NIC - 192.168.50.225 Asus XG-C100C - 172.254.6.201 (255.255.0.0 / no gateway) Unraid Server - MSI Gaming Plus x470 Onboard 1gb - 192.168.50.230 Windows 10 VM via 1gb Virtio-Net - 192.168.50.233 (both VM NIC show 10gb in network settings) Asus XG-C100C - 172.254.6.202 (255.255.0.0 / no gateway) Windows 10 VM via 10gb Virtio - 172.254.6.204 (255.255.0.0 / no gateway) (Used Virtio-Net originally for 2nd NIC, but VM was even slower, so tried Virtio and it was a bit faster) MikroTik 5 port 10gb switch for the 172.x.x.x devices Shows 10GB Fullduplex across the board Now here is what I am seeing. From my desktop to the NAS via iperf3 : (about expected and consistent) Connecting to host 172.254.6.200, port 5201 [ 5] local 172.254.6.201 port 57692 connected to 172.254.6.200 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 771 MBytes 6.47 Gbits/sec 1625 494 KBytes [ 5] 1.00-2.00 sec 748 MBytes 6.27 Gbits/sec 1605 358 KBytes [ 5] 2.00-3.00 sec 766 MBytes 6.43 Gbits/sec 2144 255 KBytes [ 5] 3.00-4.00 sec 782 MBytes 6.56 Gbits/sec 1892 322 KBytes [ 5] 4.00-5.00 sec 786 MBytes 6.60 Gbits/sec 1654 296 KBytes [ 5] 5.00-6.00 sec 786 MBytes 6.60 Gbits/sec 1946 320 KBytes [ 5] 6.00-7.00 sec 759 MBytes 6.36 Gbits/sec 1821 274 KBytes [ 5] 7.00-8.00 sec 779 MBytes 6.53 Gbits/sec 1850 345 KBytes [ 5] 8.00-9.00 sec 781 MBytes 6.55 Gbits/sec 1795 399 KBytes [ 5] 9.00-10.00 sec 722 MBytes 6.06 Gbits/sec 1880 264 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 7.50 GBytes 6.44 Gbits/sec 18212 sender [ 5] 0.00-10.00 sec 7.50 GBytes 6.44 Gbits/sec receiver My Desktop to the Unraid server: (Beautiful, love it, consistent) Connecting to host 172.254.6.202, port 5201 [ 5] local 172.254.6.201 port 41536 connected to 172.254.6.202 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.10 GBytes 9.43 Gbits/sec 0 1.77 MBytes [ 5] 1.00-2.00 sec 1.10 GBytes 9.42 Gbits/sec 0 2.05 MBytes [ 5] 2.00-3.00 sec 1.10 GBytes 9.42 Gbits/sec 0 2.15 MBytes [ 5] 3.00-4.00 sec 1.09 GBytes 9.41 Gbits/sec 0 2.32 MBytes [ 5] 4.00-5.00 sec 1.10 GBytes 9.42 Gbits/sec 0 2.43 MBytes [ 5] 5.00-6.00 sec 1.10 GBytes 9.42 Gbits/sec 0 2.43 MBytes [ 5] 6.00-7.00 sec 1.10 GBytes 9.42 Gbits/sec 5 1.07 MBytes [ 5] 7.00-8.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.40 MBytes [ 5] 8.00-9.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.45 MBytes [ 5] 9.00-10.00 sec 1.10 GBytes 9.42 Gbits/sec 0 1.49 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 11.0 GBytes 9.42 Gbits/sec 5 sender [ 5] 0.00-10.00 sec 11.0 GBytes 9.41 Gbits/sec receiver -------------------------------------------------- Here is where the issues start: My Desktop to the VM: Run 1: Desktop Side iperf client Connecting to host 172.254.6.204, port 5201 [ 5] local 172.254.6.201 port 38202 connected to 172.254.6.204 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 232 MBytes 1.95 Gbits/sec 0 220 KBytes [ 5] 1.00-2.00 sec 250 MBytes 2.10 Gbits/sec 0 220 KBytes [ 5] 2.00-3.00 sec 170 MBytes 1.43 Gbits/sec 0 220 KBytes [ 5] 3.00-4.00 sec 229 MBytes 1.92 Gbits/sec 0 220 KBytes [ 5] 4.00-5.00 sec 54.4 MBytes 456 Mbits/sec 0 220 KBytes [ 5] 5.00-6.00 sec 54.8 MBytes 460 Mbits/sec 0 220 KBytes [ 5] 6.00-7.00 sec 172 MBytes 1.45 Gbits/sec 0 220 KBytes [ 5] 7.00-8.00 sec 211 MBytes 1.77 Gbits/sec 0 220 KBytes [ 5] 8.00-9.00 sec 54.9 MBytes 460 Mbits/sec 0 220 KBytes [ 5] 9.00-10.00 sec 55.3 MBytes 464 Mbits/sec 0 220 KBytes [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.45 GBytes 1.25 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 1.45 GBytes 1.24 Gbits/sec receiver VM running as iperf server : Accepted connection from 172.254.6.201, port 38200 [ 5] local 172.254.6.204 port 5201 connected to 172.254.6.201 port 38202 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 231 MBytes 1.94 Gbits/sec [ 5] 1.00-2.00 sec 250 MBytes 2.10 Gbits/sec [ 5] 2.00-3.00 sec 170 MBytes 1.43 Gbits/sec [ 5] 3.00-4.00 sec 229 MBytes 1.92 Gbits/sec [ 5] 4.00-5.00 sec 54.3 MBytes 455 Mbits/sec [ 5] 5.00-6.00 sec 54.5 MBytes 457 Mbits/sec [ 5] 6.00-7.00 sec 172 MBytes 1.45 Gbits/sec [ 5] 7.00-8.00 sec 211 MBytes 1.77 Gbits/sec [ 5] 8.00-9.00 sec 54.9 MBytes 460 Mbits/sec [ 5] 9.00-10.00 sec 55.0 MBytes 461 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.00 sec 1.45 GBytes 1.24 Gbits/sec receiver ------------------ Run 2: Desktop to VM Connecting to host 172.254.6.204, port 5201 [ 5] local 172.254.6.201 port 43478 connected to 172.254.6.204 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 126 MBytes 1.05 Gbits/sec 0 217 KBytes [ 5] 1.00-2.00 sec 57.8 MBytes 485 Mbits/sec 0 217 KBytes [ 5] 2.00-3.00 sec 57.3 MBytes 481 Mbits/sec 0 217 KBytes [ 5] 3.00-4.00 sec 58.3 MBytes 489 Mbits/sec 0 217 KBytes [ 5] 4.00-5.00 sec 57.8 MBytes 485 Mbits/sec 0 217 KBytes [ 5] 5.00-6.00 sec 57.8 MBytes 485 Mbits/sec 0 217 KBytes [ 5] 6.00-7.00 sec 57.8 MBytes 485 Mbits/sec 0 217 KBytes [ 5] 7.00-8.00 sec 82.2 MBytes 690 Mbits/sec 0 217 KBytes [ 5] 8.00-9.00 sec 65.8 MBytes 552 Mbits/sec 0 217 KBytes [ 5] 9.00-10.00 sec 58.3 MBytes 489 Mbits/sec 0 217 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 679 MBytes 569 Mbits/sec 0 sender [ 5] 0.00-10.00 sec 678 MBytes 568 Mbits/sec receiver VM Side: Accepted connection from 172.254.6.201, port 43476 [ 5] local 172.254.6.204 port 5201 connected to 172.254.6.201 port 43478 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 125 MBytes 1.04 Gbits/sec [ 5] 1.00-2.00 sec 57.5 MBytes 483 Mbits/sec [ 5] 2.00-3.00 sec 57.7 MBytes 484 Mbits/sec [ 5] 3.00-4.00 sec 57.9 MBytes 486 Mbits/sec [ 5] 4.00-5.00 sec 58.2 MBytes 488 Mbits/sec [ 5] 5.00-6.00 sec 57.7 MBytes 484 Mbits/sec [ 5] 6.00-7.00 sec 57.9 MBytes 486 Mbits/sec [ 5] 7.00-8.00 sec 81.9 MBytes 687 Mbits/sec [ 5] 8.00-9.00 sec 65.9 MBytes 553 Mbits/sec [ 5] 9.00-10.00 sec 58.3 MBytes 489 Mbits/sec [ 5] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.00 sec 678 MBytes 568 Mbits/sec receiver -------------------------------------------- Now, the other way around, initiating the client from the VM to my Desktop PC: Connecting to host 172.254.6.201, port 5201 [ 4] local 172.254.6.204 port 60680 connected to 172.254.6.201 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 112 MBytes 935 Mbits/sec [ 4] 1.00-2.00 sec 113 MBytes 947 Mbits/sec [ 4] 2.00-3.00 sec 113 MBytes 945 Mbits/sec [ 4] 3.00-4.00 sec 113 MBytes 949 Mbits/sec [ 4] 4.00-5.00 sec 113 MBytes 948 Mbits/sec [ 4] 5.00-6.00 sec 113 MBytes 945 Mbits/sec [ 4] 6.00-7.00 sec 112 MBytes 944 Mbits/sec [ 4] 7.00-8.00 sec 113 MBytes 945 Mbits/sec [ 4] 8.00-9.00 sec 113 MBytes 948 Mbits/sec [ 4] 9.00-10.00 sec 113 MBytes 948 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 1.10 GBytes 946 Mbits/sec sender [ 4] 0.00-10.00 sec 1.10 GBytes 946 Mbits/sec receiver Then from the VM to NAS: Connecting to host 172.254.6.200, port 5201 [ 4] local 172.254.6.204 port 60640 connected to 172.254.6.200 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 317 MBytes 2.66 Gbits/sec [ 4] 1.00-2.00 sec 358 MBytes 3.01 Gbits/sec [ 4] 2.00-3.00 sec 359 MBytes 3.01 Gbits/sec [ 4] 3.00-4.00 sec 371 MBytes 3.12 Gbits/sec [ 4] 4.00-5.00 sec 378 MBytes 3.17 Gbits/sec [ 4] 5.00-6.00 sec 381 MBytes 3.19 Gbits/sec [ 4] 6.00-7.00 sec 385 MBytes 3.23 Gbits/sec [ 4] 7.00-8.00 sec 393 MBytes 3.29 Gbits/sec [ 4] 8.00-9.00 sec 387 MBytes 3.25 Gbits/sec [ 4] 9.00-10.00 sec 389 MBytes 3.27 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 3.63 GBytes 3.12 Gbits/sec sender [ 4] 0.00-10.00 sec 3.63 GBytes 3.12 Gbits/sec receiver Then the Unraid Server itself to the NAS: Connecting to host 172.254.6.200, port 5201 [ 5] local 172.254.6.202 port 51496 connected to 172.254.6.200 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 741 MBytes 6.22 Gbits/sec 244 305 KBytes [ 5] 1.00-2.00 sec 774 MBytes 6.49 Gbits/sec 0 297 KBytes [ 5] 2.00-3.00 sec 772 MBytes 6.48 Gbits/sec 0 308 KBytes [ 5] 3.00-4.00 sec 726 MBytes 6.09 Gbits/sec 505 342 KBytes [ 5] 4.00-5.00 sec 770 MBytes 6.46 Gbits/sec 14 328 KBytes [ 5] 5.00-6.00 sec 712 MBytes 5.98 Gbits/sec 4140 342 KBytes [ 5] 6.00-7.00 sec 702 MBytes 5.89 Gbits/sec 1359 308 KBytes [ 5] 7.00-8.00 sec 742 MBytes 6.23 Gbits/sec 57 305 KBytes [ 5] 8.00-9.00 sec 781 MBytes 6.55 Gbits/sec 0 303 KBytes [ 5] 9.00-10.00 sec 785 MBytes 6.58 Gbits/sec 0 303 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 7.33 GBytes 6.30 Gbits/sec 6319 sender [ 5] 0.00-10.00 sec 7.32 GBytes 6.29 Gbits/sec receiver ---- So, all in all, it just seems like the VM is having issues, does anyone have any recommendations of things I might look into or try? Edited April 18, 2021 by MostHated Quote Link to comment
Vr2Io Posted April 18, 2021 Share Posted April 18, 2021 Best you could disconnect the 1G network, traffic may go through there. Quote Link to comment
MostHated Posted April 18, 2021 Author Share Posted April 18, 2021 (edited) The 1gb network is used for internet access. The 10gb network has no outside physical connection. The switch is only connected to the three devices, which are using static IP's with no gateway or dns. Edited April 18, 2021 by MostHated Quote Link to comment
Vr2Io Posted April 18, 2021 Share Posted April 18, 2021 (edited) I know that, but some how traffic may route thr between both subnet in Unraid or NAS .... And you said Windows attach br0 or br1 but all test under 172.254.x.x. You don't need change any setting, just before start iperf test, temporary disconnect all 1G link, there are no hurt. Edited April 18, 2021 by Vr2Io Quote Link to comment
MostHated Posted April 18, 2021 Author Share Posted April 18, 2021 (edited) I am not quite sure I follow. The VM does have both br0 (the 192.x.x.x) and br1 (the 172.x.x.x) added. All tests were performed using 172.x.x.x, but that is because that is what I typed in, in order to run the test through the 10gb connection. (200-204 whichever I wanted to test) iperf3 -c 172.254.6.200 That said, I can't remove the 1gb, otherwise there would be no internet connection. I don't have any static routes setup within the switch that would allow traffic from the 192.x.x.x subnet. Is there some sort of network segregation that I might need to perform within Unraid itself? Edited April 18, 2021 by MostHated Quote Link to comment
Vr2Io Posted April 18, 2021 Share Posted April 18, 2021 (edited) This is temporary for test, not disconnect forever. Best should be use one subnet only, one solution is use a 10G sfp+ to 1G RJ45 to connect both 10G and 1G switch, then all client use 1 NIC only. Edited April 18, 2021 by Vr2Io Quote Link to comment
MostHated Posted April 18, 2021 Author Share Posted April 18, 2021 I understand, I can give that a try. My hope was to keep the 10gb for connection between the three devices only, but, I suppose it would not be the end of the world if it only had the one connection. The switch has 4 SPF rj45 (currently using 3) but also has 1 built-in 1gb rj45 which can be used to connect it to the network. If nothing else, as you mentioned, it would be worth it just to see if it helps and narrow things down. Quote Link to comment
Vr2Io Posted April 18, 2021 Share Posted April 18, 2021 (edited) 15 minutes ago, MostHated said: narrow things down. Yes I misunderstand 5port 10G switch was five 10G sfp+ port, it is four 10G + one 1G. Why don't design in 1 subnet 1 NIC ? Edited April 18, 2021 by Vr2Io Quote Link to comment
MostHated Posted April 18, 2021 Author Share Posted April 18, 2021 The original idea was primarily for access control and maximum throughput. The 172 network has my NAS and all of my development projects, and also my home automation server and devices, it was just how I envisioned the setup being when I started putting it together. I suppose though, that giving a Windows VM access to both 192.x and 172.x at the same time negates any security in the first place. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.