10gbe ethernet issue, only one way

[frubsen]

Hi all,

 

Built an unraid server with a 10gbe NIC. The Asus XG-C100C.  I also have the same card in my main Windows 10 machine.  I have them hooked into a Netgear GS110MX switch.  Both are connected to the 10gb ports.

 

When doing some speed tests using iperf, If I set up my Win10 computer as the server, and run the command from the console in Unraid, I get any where from 9.7gb/s -9.9gb/s transfer speed which is great.

 

However, if I set the Unraid server as the iperf server, and run the command from my Win10 computer, It mazes out right around 6gb/s.

So I can't figure out which side the issue is on.  Clearly the switch and cables are fine, because I am able to max everything out going the one way.  But I can't tell if its a receive side issue on the Unraid side, or if it's a transmit issue on the Win10 side.

 

Both have the MTU set at 9014 (Unraid is 9000).  

 

What other settings should I be looking at to troubleshoot this?

 

Thanks

[frubsen]

Interesting, if I do do 2 or more streams in iperf I can saturate the full 10G link.

 

Any reason why going the one way I can saturate the 10G link with 1 stream, yet the other way I have to use 2 streams?

[JorgeB]

9 hours ago, frubsen said:

and run the command from my Win10 computer, It mazes out right around 6gb/s.

Try with another computer if available, or a clean OS.

 

5 hours ago, frubsen said:

Any reason why going the one way I can saturate the 10G link with 1 stream, yet the other way I have to use 2 streams?

This is normal, that's why you should test with a single stream for best results.

[frubsen]

Hi everyone,

 

I just wanted to follow up on this and ask for any more advice or things to look at.  Unfortunately it's not really easy for me to test the connection with another computer as the only 2 computers that have 10gb NICs are my main PC and my unraid server. 

 

I did another test today and I can transfer a file from my NVME cache on my Unraid server to my main PC and I get a sustained 1.15GB/Sec transfer which is awesome.

 

However sending that file back to the cache drive, I max out around 900MB/sec...usually fluctuates between 700-850MB/sec.  Both drives are Gen3 NVME capable of read/write @ 3500/3000MB/sec

 

I guess my question is...is this a Transmit issue on my main PC, or a receive issue on my Unraid server?  Clearly the network is capable of 10GB since I can max out the connection transferring files one way.  Are there settings network settings I should check/change on my Unraid server?

 

I've played around with almost every possible combination of NIC settings on the windows side with no improvement.

 

Thanks