Jump to content

Slow r/w speeds with NVMe cache drive [Resolved]


Recommended Posts

Edit: What ultimately solved my problem was reinstalling Windows - now I'm back on 1GbE but at least get around 100-110MB/s. The Windows installation was about a year old and only had seen new RAM and some new HDD and SSD storage. Let's hope the upgrade to 10GbE later this week goes well.

 

I recently installed a 1TB WD Blue M.2 NVMe SSD as cache, as well as a DeLock 2.5GbE network card and directly connected my PC (via its onboard 2.5GbE port to my server). Read and write speeds are down compared to before the update.

 

Here are some metrics:

  • 20-40MB/s write and 20-80MB/s read (the same as via 1GbE but more stable)
  • read and write speed are pretty much the same for array and cache (verified using the web UI)
  • HDDs are capable of 80-170MB/s r/w (according to DiskSpeed docker) - averages around 120MB/s
  • cache SDD is cabable of 1GB/s-1.1GB/s r/w (according to DiskSpeed docker)
  • 2.5GbE is direct connect Cat.7 (1m), 1GbE is via two switches and a router (total of 25m of Cat.6)
  • <1ms latency via 2.5GbE instead of 2-3ms latency via 1GbE
  • previous no-name 240GB M.2 SATA SSD was a lot faster

 

Overall, the web UI feels a lot snappiere since accessing it via 2.5GbE - before traffic needed to go through a switch, to my router, and back to a switch - now it's direct. Windows 10 shows me 2.5GbE speeds. The M.2 SSD is connected via PCIe 3.0 x4. I really don't know where the bottleneck is here. Internally (e.g., copying from array to cache) is faster but not as fast as I expect from this hardware.

 

I'd be grateful if someone else could take a look at this. I'm happy to provide more information - I just don't know what more at the moment.

 

My system:

  • ASRock X399 Taichi
  • Threadripper 1900X
  • 2x16GB DDR4-3600 (@3200 1.25V) - survived a memBench run
  • 1x WD Blue 1TB as cache
  • 2x WD Se 4TB as main array drives (no parity atm)

The system I connect from is a Ryzen 9 system with a B550 Mainboard with onboard 2.5GbE - plenty of power on that end.

syslog.txt

Edited by DesertCookie
1) Added info. 2) Added my solution.
Link to comment
30 minutes ago, JorgeB said:

[...] are both using the same switch or is the 2.5Gbe link direct? Also try with a different source computer if available.

The 2.5GbE link is direct, as I wrote. I only have a phone via WiFi 4 available as second client (or a VM in the server but that's obviously useless here).

 

I didn't change MTU as I didn't expect that to be relevant for only 2.5GbE. Could that cause an improvement?

Edited by DesertCookie
Added info
Link to comment

I've retested today - after having made no changes, not even shutting down the computers since the last test. Now I get the following results:

1GbE:   412 Mb/s (spread of 277 to 859 Mb/s)
2.5GbE: 508 Mb/s (spread of 432 to 567 Mb/s)

While this is better it still doesn't represent what I would expect from this set up.

 

Edit: A little while later I found that 1GbE is mroe reliable. Whether it's problem of the ethernet cards or one of Unraid with the ethernet cards, I don't know. I'll be adding the name of the cards to the original post as soon as I stop my server for the next time.

Edited by DesertCookie
Link to comment
  • DesertCookie changed the title to Slow r/w speeds with NVMe cache drive [Resolved]

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...