10gbe virtual network connection between unRAID and Ubuntu VM

[sgibbers17]

I have been searching for a way to enable a faster virtual network connection between my Ubuntu VM and my unRAID server than the 1gig connection that it is currently using.  All my searches either are not for VMs or for windows or for how to pass through a network card.  Maybe I'm just searching the wrong terms.  When I set up my windows VM it just worked as a 10g connection.  If a faster than 10g connection is possible that would be better as I am using nvme drives for my VMs and Cache drive.

[ghost82]

Interesting matter!

To connect a vm to the host I can think about different ways:

- passthrough (vt-d) a high speed network card to the vm, like infiniband and have a second network card of the same high speed type on the host, connected to the same switch

- use a bridge connection between the host and the vm: you can have a bridge connection between a host and a vm having a physical nic on the host, or even without having a physical nic, by creating a dummy network interface in the host (this is something I've just learnt now)

- mount filesystems in the vm in a different way than the more standard network protocols (ntfs or smb), depending on the guest/available drivers/compatibility: for example 9p fs

 

about bridge connections and virtio/virtio-net: virtio and virtio-net are para-virtualized drivers and the operating systems know that it's not a real physical device; there isn't a real "link speed" applied, if you read one in a guest it's only cosmetic.

This is true also if you setup a bridge connection bound to a real nic: in my case for example I have a virtio bridge connection bound to an intel 82574L 1GbE physical nic in the host, but an iperf3 test host<->guest shows speeds up to 16-17 Gbits/s.

 

As far as I know virtio/virtio-net is the fastiest and easiest way to connect vms.

Edited June 22, 2022 by ghost82

[MAM59]

10 hours ago, ghost82 said:

, if you read one in a guest it's only cosmetic.

Sadly, you are wrong 😞

There are many programs on the guest that read that fake speed and allocate ressources accordingly. And, if the "cosmetic value" is far off from reality, they may produce hangups, slowdowns or pumping effects (buffer overruns and so on).

 

It would be very nice and wise if these speeds would match.

 

Edited June 23, 2022 by MAM59

[ghost82]

52 minutes ago, MAM59 said:

There are many programs on the guest that read that fake speed and allocate ressources accordingly

Hi, do you have any example for this?

[MAM59]

Just now, ghost82 said:

Hi, do you have any example for this?

almost any major app like apache, mysql, SMB and even TCP/IP itself have a builtin oder plugin "autotuning feature". Those incorporate the network speed into their calculations.

 

Usually this is not visible to the user or even the admin. Its hidden inside the code.

 

And, what is helpful in most cases, can be very bad if feeded with the wrong values.

 

But QEMU is not alone with this problem, Hyper-V for instance presented a 10G connection to the guests from the beginning on. The backening host mostly ran on 1G only. This resulted in new transfers starting blazing fast, then stopping (buffer overflow), then pacing down.

TCP/IP by design is able to adopt to slower speeds quite fast, but reacts quite slowly if speed is going up again. It's a builtin safety feature to prevent too many ups and downs.

So the Hyperv 10 to 1 is visible, but rather harmless, the 1 to 10 on the other hand is very noticable and annoying. It takes quite long until you get to the highest possible speed. Short transfers never get there.

(but of course, at the end, everything works, its just a feature of optimization)

 

You may read more details of this if you look for instance of the "Nagle - Algorithm". A very old and simple way to increase network througput. It takes the link speed into account to predict the needed timeouts for sending a packet or to wait for more data to create bigger packets. If the base speed is wrong, obviously the algorithm produces wrong timeouts too. (which are corrected later on after it came to a emergeny situation)