Transfer speeds dropping to 25mb/s or less

[Pixel5]

Hello People,

 

im still waiting for some cables so as of now im only using my unraid system with two NVME SSD´s installed.

Im using one as a volume without parity and one mounted as an unassigned device.

 

Now im playing around a bit to see what kind of transfer speeds i can get and how the system performs in general with unraid and im running into some issued with transfer speeds and general responsiveness of the system.

So im transferring some big ISO files from an NVME SSD on my PC via 10G Ethernet to the NVME SSD in the NAS.

The transfer starts out at a solid 900 - 1000mb/s which lasts for a few seconds before it rapidly drops to below 50mb/s and finally usually under 25mb/s

At the same time the unraid GUI becomes very unresponsive and doing anything else becomes basically impossible regardless to which of the SSD´s i transfer my files.

Checking the CPU usage i can see sudden spikes to 100% on individual CPU cores while the other ones are basically at idle.

 

in the reverse direction i usually get a solid 900-1000mb/s without any major slow downs.

The SSD temperature in the NAS never exceeds 65°C and i checked some reviews where they said this WD Black model doesnt throttle until it hits 80°C and even if it does its still well over 100mb/s

 

I also tested a slower transfer where i copy from my PC´s HDD to the SSD on the NAS which is capped at 170mb/s due to my HDD but even this one slows down to under 25mb/s.

 

Does anyone have any idea whats going on here exactly?

 

thevault-diagnostics-20201209-0908.zip

[JorgeB]

Is it the same transferring to both devices or just the array one?

[Pixel5]

if i transfer to the unassigned device its slightly faster but there is a still a huge drop ins transfer speed without the device being hot enough to thermal throttle.

[JorgeB]

That suggests a device issue, see if you try with a different device, even a regular HDD.

[Pixel5]

i just tested all drives with Diskspeed docker and while the SSD´s do thermal throttle after a while as expected the never drop below 200mb/s during the test so i would say its unlikely that its a problem with the device itself but im also currently transferring some files to my computer to so i can copy something again with cache disabled.

[Pixel5]

i now also completed a file transfer with Cache disabled and it was going as fast as the HDD in the NAS would allow.

Now the question still is why this only happens with Cache enabled but when i check the cache drive its all good and not showing any issues until it thermal throttles at 80°C

[JorgeB]

31 minutes ago, Pixel5 said:

i just tested all drives with Diskspeed docker and while the SSD´s do thermal throttle after a while as expected the never drop below 200mb/s during the test

That's only a read test, and the current problem is with writes.

 

8 minutes ago, Pixel5 said:

i now also completed a file transfer with Cache disabled

What cache? There was no cache device assigned.

[Pixel5]

2 minutes ago, JorgeB said:

That's only a read test, and the current problem is with writes.

 

What cache? There was no cache device assigned.

right i forgot that its only a read test

 

I just received the cable i needed to install all my HDD´s so now i have one of the SSD´s assigned as a cache drive and still one as an unassigned device.

When i copy to a share that has cache disabled the transfer is actually faster than going to the cache drive.

Ill try to pass through an SSD to a windows VM and see what kind of results i get with the tests i can run on windows.

[JorgeB]

Just now, Pixel5 said:

When i copy to a share that has cache disabled the transfer is actually faster than going to the cache drive.

This suggests like mentioned above that it's a device problem.

[Pixel5]

1 minute ago, JorgeB said:

This suggests like mentioned above that it's a device problem.

but both SSD´s were fine in the system they were in before and both only have like ~20TB written to them.

Also i now installed another 8TB drive which currently holds most of my data and im using krusader to copy these files to my main array.

The share im currently copying to has cache enabled so currently everything is going onto my cache drive without throttling.

So far the slow transfer speed is only there if i transfer via an SMB share.

[JorgeB]

9 minutes ago, Pixel5 said:

So far the slow transfer speed is only there if i transfer via an SMB share.

By the way you described it it doesn't look like an SMB problem:

 

7 hours ago, Pixel5 said:

The transfer starts out at a solid 900 - 1000mb/s which lasts for a few seconds before it rapidly drops to below 50mb/s and finally usually under 25mb/s

SMB/LAN problem would usually be evident from the beginning, usually.

[Pixel5]

2 minutes ago, JorgeB said:

By the way you described it it doesn't look like an SMB problem:

 

SMB/LAN problem would usually be evident from the beginning, usually.

thats exactly why im wondering what is going on, i just transferred 250GB to the Cache drive at 200mb/s without any issue but again the moment i do the same via 10G lan it will slow down to below 50mb/s which makes no sense at all.