Jumbo Frames

[mikeyosm]

Hi

 

For months now I have set jumbo frames 9000 on my unraid host vm bridge interface and Windows 10 VMs so that I have a true 10Gbps internal (winthin the unraid host) network.

Reason for doing this is so I make use of the max transfer rate when moving files between XFS shares sat on multiple SSDs capable of 500MB/s. Generally, this works well and I do see speeds over 200MB/s when moving large files around. I do however have very random network drops on my W10 VM and I suspect it may have something to do with enabling jumbo frames.

 

Question, is Jumbo Frames really needed to achieve speeds of over 150MB/s? Can I still get whopping speeds >200MB/s with the MTU set to 1500 and the VM NICs set to 10Gbps?

My understanding is that the MTU set at 1500 was limiting the max transer rate?

 

Please explain. Thank you.

[pwm]

The network interface can manage way over 200 MB/s even without jumbo frames.

 

Jumbo frames was extremely important for early 1gbit network interfaces because of the large cost of handling all the individual interrupts to pick up all packets.

But all newer decent network cards support coalesce where the card doesn't signal every received packet. So the OS can pick up lots of packets at the same time. And the OS can also post lots of packets to be sent.

 

Jumbo frames will still reduce the amount of CPU load, because it's fewer packet headers to look at for the TCP stack and the firewall when deciding if the packet should be accepted and which stream it belongs to.

 

With or without jumbo frames, it's still required that the receiving machine is fast enough to extract the received data in time - before the receive buffers overflows in the network card. If they overflow, then you get lost packets and the transfer stalls until the sending side times out waiting for acknowledge and resends the unacknowledged packets.

 

Have you checked the packet counters on the two machines, to see if you get any errors?

Are you connecting directly between the machines, or do you have a switch? If so - does the switch produce any port statistics?