Issues with 10G Network Transfer Speed


pish180

Recommended Posts

  • 3 weeks later...
  • 4 weeks later...

I was able to do some more testing as I ordered a new server.   Not 100% sure what happened during this whole process but the Windows 10 system listed "L" in my drawing is having some major issues.  I do know the issues is software or OS related not Hardware.   I'm not able to determine what is causing the massive slowdown nor do I have the patience to deal with it.  

 

My plan is to get the UnRaid working ASAP and transfer over all the services to my new server and wipe the Windows 10 machine.

 

I was able to transfer between my 2 servers using all of the exact same components at 7.5 - 9.5Gbps using iperf3. 

Link to comment
  • 6 months later...
  • 2 weeks later...
On 9/27/2020 at 12:39 AM, SuberSeb said:

Hello @pish180, do you solved that issue? I have similar problem, only 3.5Gbps under windows 10. If I load from Ubuntu live CD, I got 9.9Gbps. Clean install of windows 10 is not an option right now.

Not on the Win10 machine.  I blew it away after I built up Unraid and moved to containers.  The Win 10 machine was running so much software it would have been really hard to find out what was causing it.  I tried the obvious things like drivers, AV, etc.   Gl.

Link to comment
  • 1 month later...
On 10/11/2020 at 1:34 AM, pish180 said:

Not on the Win10 machine.  I blew it away after I built up Unraid and moved to containers.  The Win 10 machine was running so much software it would have been really hard to find out what was causing it.  I tried the obvious things like drivers, AV, etc.   Gl.

Any luck with a solution?  I seem to have the same problem, but with an Aquantia 10GbE PCIex4 single SFP+ card in my unraid server, connected to a MikroTik 10GbE switch (CRS305) and then to my main Win10 PC with an identical Aquantia 10GbE card on the same switch, all with SFP+ DAC connections (1m).

 

iperf3 with one stream gets about 2.7 to 3.2Gbits/s, but can get total throughput of 8.2Gbs with 4 streams (-P4) and 8.94Gbs with 5 streams (-P5).

 

I can teracopy from a Win10 NVMe SSD to my unraid SSD cache drive (in btrfs, two SSD cache pool) at 280-300MBs.

 

I've disconnected my two gigabit unraid ethernet connections to ensure that all is routed through the 10GbE NIC.

 

Appreciate any thoughts!

Link to comment
  • 1 month later...
On 11/23/2020 at 6:19 PM, pasgener said:

Any luck with a solution?  I seem to have the same problem, but with an Aquantia 10GbE PCIex4 single SFP+ card in my unraid server, connected to a MikroTik 10GbE switch (CRS305) and then to my main Win10 PC with an identical Aquantia 10GbE card on the same switch, all with SFP+ DAC connections (1m).

 

iperf3 with one stream gets about 2.7 to 3.2Gbits/s, but can get total throughput of 8.2Gbs with 4 streams (-P4) and 8.94Gbs with 5 streams (-P5).

 

I can teracopy from a Win10 NVMe SSD to my unraid SSD cache drive (in btrfs, two SSD cache pool) at 280-300MBs.

 

I've disconnected my two gigabit unraid ethernet connections to ensure that all is routed through the 10GbE NIC.

 

Appreciate any thoughts!

Wish I knew but there was something going on with the Windows host to the UnRaid host.  I had tried swapping cards, cables, ports, etc.  I also tried putting the 10G card in a FreeNas (BSD) box (completely separate 2u Dell server) and then ran Iperf3 on there and it worked fine.  I tried driver settings and networking config settings (MTU, etc) and nothing worked.  It had all the same networking hardware that was just in the Windows machine.   I didn't have the time or hardware to install Windows on the Dell server to try it out there.  If you have some time and the hardware it would be a good test to see if its a single Windows system or if its just something between Windows and Linux that is causing the slowdown.  I know for a fact that I did a Windows to FreeNas with 10G and it saturated the 10G connection.  I was looking for screenshots because I swear I took one (but I can't find it).  FreeNas was in Raid-Z2 with 8 drives and the Windows was Hardware Raid6 with 10 drives.  And at that time I had moved stuff from Windows to FreeNas at blistering speeds.  

I found it really odd that I had different speeds in different directions between Unraid and Windows.  No other systems exhibited this behavior.  

I really put the 10G card in the Unraid box for 2 reasons.
1. The main reason... So I could transfer files from my old Windows server to my new Unraid box faster.  10g Windows host to 10g UnRaid host (Raid 6 on WIndows could easily saturate 10G).  TBH... unless you are reading/writing it to an SSD or an Array setup you will max out the speed of the single drive.  3.2Gbps is much faster than sustained speed of the mechanical drive).  

2.  The lesser reason... So I can serve multiple clients at the same time and the network won't be the issue.  Theoretically I have a 1Gbps unlink + internal Plex offerings.  I could saturate that link and several home players before networking is an issue.  That being said 3.2Gbps is still probably way more than you could ever saturate unless you have a > 1Gbps internet uplink.  


I'm not super satisfied with the outcome of this either but I just set the transfers to go and waited for them to be done with slow speed.  I also ran into another issue with share settings and caching.  Feel free to look up that thread I created.   I did a lot of research there as well, essentially the sustained speed of the transfer would drop to double digits if you used most free with parity.  Guessing it doesn't listen to the Parity write flag when it changes drives.  I'm guessing but I think it started writing parity and slowing down everything.  Long story but I figured out a work around in that thread for that problem.  

Link to comment
46 minutes ago, pish180 said:

Guessing it doesn't listen to the Parity write flag when it changes drives

Not sure what you mean by this as there is no such thing as a Parity Write flag?    Any time any sector on any drive is written, the corresponding sector on the parity drive(s) is written as part of the same operation and this happens immediately before the write is deemed to have finished.    If you are constantly changing drives this can lead to excessive head movement slowing things down.

Link to comment
18 hours ago, itimpi said:

Not sure what you mean by this as there is no such thing as a Parity Write flag?    Any time any sector on any drive is written, the corresponding sector on the parity drive(s) is written as part of the same operation and this happens immediately before the write is deemed to have finished.    If you are constantly changing drives this can lead to excessive head movement slowing things down.

 

Link to comment
  • 3 months later...

I am experiencing a similar issue to this thread. Running Aquantia 10GbE on my desktop PC (64gb ram) and in the Unraid server, (32gb ram), then my NAS is a QNap TVS-951x (32gb ram). The three are connected using a MikroTik switch.

 

iperf from my desktop to the unraid server itself (and vice versa) is solid full speed on a single thread, but iperf to the Windows 10 VM running on the server, I can't get over 3.5gb (using virtio, only 1.5gb if using virtio-net).

 

My desktop has a pcie4 Sabrent Rocket nvme ssd, and the VM on the Unraid server is running on dual Samsung 970 Evo+ nvme drives. The NAS has 4x Samsung 860 SSD in raid 5 for cache/QTier. To/From everything is pretty much max/or at least expected speed minus the Windows VM, no matter which device I try to test.

 

Not sure what, if anything, can be done at this point?

 

 

Edited by MostHated
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.