Hi all, does anyone here have 2.5Gbe working well in unraid?  I have 2.5G Nic's and 2.5G switch.  When i copy a file from desktop box with a 2.5G nic over to the unraid server that also has a 2.5G nic the file transfer starts off fast at ~280MB/s but then quickly plumits down to ~60MB/s.  What settings am i missing to have this file transfer stay closer to the ~280MB/s and remain at that speed for the entire transfer?

 

thank you

This is usually not an issue with network at all.

The slowdowns are a combination of to few RAM, a missing or too slow or not sustainable cache SSD and finally disk speed of the target drives.

Can be tricky to find out who is to blame.

 

What happens is that transfers start fast, RAM fills up. During this time the optional cache SSD is written. This usually has a RAM cache too (the better ones), If this cache fills up too because too much data is coming in, write speed drops to real flash speed which is likely around 60/Mbs.

There are very few and usually more expensive SSDs that can keep up with high speed for a longer period.

For Samsung for instance, the PRO line can do, the EVO line not. But even the price or the name is no guarantee, you should consult recent speed tests. The manufacturers wont tell you the "sustained write rate". Marketing only stresses out peak rates (and do not tell for how long they will be kept).

 

 

That looks like you are being limited by the device/pool write speed, where are you writing to?

On 3/14/2025 at 8:33 PM, JorgeB said:

That looks like you are being limited by the device/pool write speed, where are you writing to?

in a test setup i am pulling the data from a sata 2.5" SSD in the windows desktop and writing to the unraid device pool that consists of 1 sata 2.5" ssd for parity and 1 sata ssd for data.

30 minutes ago, crizz1111 said:

unraid device pool that consists of 1 sata 2.5" ssd for parity and 1 sata ssd for data.

for speed tests use a single ssd with an "exclusive mode" share (no array, no cache)

This will bypass all the UNRAID filesystem layers and go directly to the drive.

 

create a pool with ONE ssd, create (and share) a share on this ssd. This will automatically generate "exclusive mode" for this share.

 

Edited March 15, 20251 yr by MAM59

44 minutes ago, crizz1111 said:

of 1 sata 2.5" ssd for parity and 1 sata ssd for data.

Post the diags, mostly to see the device models, not all SSDs can sustain high write speeds.

24 minutes ago, MAM59 said:

for speed tests use a single ssd with an "exclusive mode" share (no array, no cache)

This will bypass all the UNRAID filesystem layers and go directly to the drive.

 

create a pool with ONE ssd, create (and share) a share on this ssd. This will automatically generate "exclusive mode" for this share.

 

Yeah, does not look very good.

Should be like

"G" instead of "M"...

make sure all your attached devices (computers / switches and so on) have turned on "flow control" and are capable of handling it correctly

else I am running out on ideas.

Sorry

 

does the second highlight mean i cant have flow control on, on any of my NIC's?

no no. all fine. Flow control is ON. Just dont turn it off.

But as I said, all other devices need to support and honor it too.

This is because "2,5G does not exist". It is an "illegal" speed, it uses 10G encodings and speed. Just it introduces "10G with pauses", from 4 different windows it only uses one (to allow recovery for the cable and to allow cheaper and lower quality chips to be sold). Flow control is needed to tell the partner which one is to look at and which three need to be ignored.

(simply spoken, the real encoding stuff is heavily complicated, you would not sleep well anymore if you try to understand it).

So without flow control, they talk but many packets are lost and need to be repeated. This seriously drops the throughput rate.

 

But your 400mbs? naah, flow control problems would go down to 50-60Mbs...

 

Why is your CPU "glowing" (100% usage) on some cores? Does not sound very good too...

 

Port 3 in the below screen grab is my unraid server with a TP-Link TX201 in it.  Maybe the driver for this card in linux does not support flow control?

 

 

dunno... but at least this screenshot does not look right. You WANT FC but the card does not grant it... danger ahead!

Port 1 looks ok... I guess this is your PC?

 

 

19 minutes ago, MAM59 said:

dunno... but at least this screenshot does not look right. You WANT FC but the card does not grant it... danger ahead!

Port 1 looks ok... I guess this is your PC?

 

 

yes port 1 is desktop pc with a Dell Intel X550-T2 card in it.

try to swap the cards. X550 sounds much better for a server...

That TP Link has a realtek 8125 chipset, I'm not sure, but I have seen a lot of driver problems here in this forum. Maybe you should search and read them?

 

Warning for the Intel card in Unraid: I faintly remember it will only offer 1G and 10G speeds (you will notice it if you connect and get only 1G LAN). This is perfectly correct, as I said before already: 2,5G does not exist!

But I also remember there is a special driver for it in the Apps section (or somewhere, search the forum) that will allow it to operate at 2,5 or 5G speeds too.

 

 

Edited March 15, 20251 yr by MAM59

4 hours ago, crizz1111 said:

writing to the unraid device pool that consists of 1 sata 2.5" ssd for parity and 1 sata ssd for data.

If you mean the Unraid parity array, SSDs in the array cannot be trimmed. I know this is just a test, but thought I would mention it.