Hi again,

I'm trying to figure out if this is normal or not. I upgraded the CACHE from a 512gb to a 1TB nvme drive. After changing the share to go to Cache and then Array, moving files to the NVME is great, I'm getting 140MB/s and then slows to about 113MB/s over time. Now, when the NVME is full, I stop the transfer and manually start the Mover in the Scheduler. The speed varies, 130kb/s, then 0, then goes to max 9MB/s then 0, back to 130ish... over and over.

So, question to anyone out there, where do you think the bottleneck is? I would suspect that the array would happily take the files and move them at a consistent speed, especially from the NVME. The files range from 200MB to 1.3GB in size. Again, I tried many.... many steps including disabling the Parity check (which I don't think works), changing the disk to Reconstruct Write and things I forget. Is this data copy slowness normal??

Thanks,
Jacques.

Edited November 11, 2025Nov 11 by Ayanefan

You are likely to get better informed feedback if you attach your system’s diagnostics (with everything in the one zip file) to your next post in this thread. It is always a good idea when asking questions to supply your diagnostics so we can see details of your system, how you have things configured, and the current syslog.

discovery-diagnostics-20251108-1612.zip

I've attached the diagnostics. I did a quick run-through but I'm not sure what I'm looking for.

J

I notice some SMR drives. I'm guessing that is at least a part of why slower than you expect.

I will pick up a couple of IronWolf 4tb drives today and replace the 8tb Parity with a 4TB, no worries, the largest disk is 4TB. They are on sale now for CAD $120 which is about $85 USD. Even Grok is telling me that SMR could be my weakness but it told me to get a 8TB SSD instead (money much?). I only have 1 CMR in the whole setup at the moment. I'll let you know how it goes.

From Google's AI:

You never want to assume that a HD is CMR unless it is specifically stated that it is. If any doubt exists, go to the manufacturer's website and look up the spec's for the drive model and capacity. You need to check both because there were instances where one capacity was CMR and another capacity was SMR!!! (And beware of purchasing 'shucked' drives as you never know what is inside until you get it open. Manufacturers seem to use external drive enclosures to move excess inventory!!!)

Popped in a 4tb WD ST40000VN006, CMR for the Parity. It is running at 120MB/s for the Parity rebuild. I guess I will let you know tomorrow if things are much better.

a couple of hours in and still solid.

Ok, testing complete, bulletpoints:

• DO NOT use SMR drives as a parity drive. Things go well until a few Gigabytes get written then the constant read/write/check will kill the speed if copying directly to the array. This can be from 140MB/s then down to 2MB/s in my case. Switching to CMR has fixed that issue, the writes are consistent once the disks speeds from copying on the array saturate and slow the array down. I am surprised that this is not in BIG, BOLD, RED letters anywhere.

• Your array will be only as fast as your slowest disk when accessed. So in my case, I have 2 parity CMR drives, 1 CMR in the Array and 5 SMR drives. When initially copying to the CMR, speed stays consistent around 120MB/s. When the disk hit the half-way full mark, it moved to the next SMR drive. After 20 minutes, the speed slowed down to around 20 - 40MB/s . Significant drop. Mine is a budget build using hardware I had on hand. I will replace the SMR drives with CMR slowly over time.

• SMR drives are perfectly fine if you are using it as long term storage and do not need to do constant writes to them. Read times are fine, write actions are not.

• Definitely use a NVMe drive or other fast sata drives to cache your copies, set your mover to run every night if you are not moving this over all the time. Use the system idle time to do things like trimming, moving or parity checks. If you are not there, you won't care.