My monthly Parity Check started yesterday morning and is running at nearly a third or its regular speed (55MB/s vs 140MB/s). The only real change that I can think of is that over the weekend I added a 2nd NVME drive so that I could covert my cache drive to a pool to avoid any issues if a drive died. Not sure if this could be related to running out of PCIe lanes and causing the slower performance or unrelated. I have attached a diagnostic report as well. Parity is still running and has about 17 hours to go, so I haven’t had a chance to go into the BIOS and look around or pull the extra NVME drive out to see if that fixes the problem either.

homeserver-diagnostics-20260203-0934.zip

Do any of your drives show SMART warning (👎) on the DASHBOARD page?

Looks like Disk 14 has 36 CRC errors. I believe those are from a little while ago when I had a bad power connection to the drive.

Just now, bisk said:

those are from a little while ago when I had a bad power connection to the drive

Nothing in current syslog about that so probably not the problem.

Maybe this would give some insight:

30 minutes ago, trurl said:

Maybe this would give some insight:

Thanks. I’lll give that a try once the parity check completes over night.

I installed the Disk Speed docker and ran the benchmark on all of the drives and all reported the correct full speeds. So I decided to remove the two Optane drives that are on PCIe x1 to NVME cards and now the parity calculation runs at the 140MB/s speed that I was previously seeing. So I’m assuming this means that I don’t have sufficient PCIe lanes to support all of the devices at full speeds?

That doesn't make much sense. Did you run the controller tests as well?

I’m finding it odd as well. I did some benchmarking on the controllers. The only difference between the two sets are with and without the Optane drives installed in the PCIe x1 cards.

These first two screenshots are without the two Optane drives installed

These next two screenshots are with the Optane drives installed

Edited February 4Feb 4 by bisk

Have this "problem" since many years - disks can handle 250-100MB/sek but when i copy i get only 70MB/sek constant "max" even on the beginning of an empty disk..

Must be an internal unraid "thing"...

Edited February 4Feb 4 by Zonediver

2 hours ago, Zonediver said:

when i copy

Copy to where?

3 minutes ago, trurl said:

Copy to where?

From a SSD on my PC over a 10GBit-LAN direct to an empty Disk...

Copying to the Cash-SSD gives me 1185MB/sek...

Its not LAN-realted and the disk can handle up to 230MB/Sek...

The process moving files from the Cash-SSD to a disk is also slow - 50-60MB/Sek max.

Maybe its parity related but the parity disk itself can also handle up to 276MB/sek.

All HDDs are connected to the LSI 9207.

Edited February 4Feb 4 by Zonediver

1 hour ago, Zonediver said:

moving files from the Cash-SSD to a disk is also slow - 50-60MB/Sek max.

Maybe its parity related

That is actually pretty good for write to the parity array unless you have Turbo on.

https://docs.unraid.net/unraid-os/using-unraid-to/manage-storage/array/overview/#array-write-modes

1 hour ago, trurl said:

That is actually pretty good for write to the parity array unless you have Turbo on.

https://docs.unraid.net/unraid-os/using-unraid-to/manage-storage/array/overview/#array-write-modes

Thanks for the hint - will give them a try 👍

That's very strange; HBA still has ful bandwith

I’m leaning toward the issue being with the PCIe to NVME cards (https://www.amazon.ca/dp/B09P3HY3P3?ref=ppx_yo2ov_dt_b_fed_asin_title) since the performance improves when they’re no longer installed and as soon as I add even one the performance tanks. I’ve ordered different ones from Amazon to try. Fingers crossed they work. If they don’t, then I’ll try a SATA to NVME cards instead.

Edited February 5Feb 5 by bisk

So none of the replacement cards have worked. I’ve tried putting the Optane drives in as well as NVME drives and they all kill the parity calculation speeds. I even installed the cards without a drive installed and that even had the same affect. I’m at a loss as to why this is happening. The BIOS on my motherboard has also been updated to the most recent version. Any other thoughts on what I can try or what might be causing the issue?

Have you tried changing the PCIe slots the cards are plugged into? I've ran into scenarios on my DiskSpeed dev box where a card wouldn't have the expected speeds if it was plugged in x slot vs y slot, even though the slots had enough PCI lanes for it and the bios hadn't changed the number of lanes utilized (that may also be an issue, check your MB docs).

Awesome to see my DiskSpeed app being used to troubleshoot issues.

If you click on the "Rescan Controllers", your 10TB & 14TB drives will now have images. Also, if you want your other drives to have a uniform capacity overlay, you can edit just one of them and then set the checkbox to apply the same changes to all like-model drives. That typically happens from different people uploading the first image for the drive and they had differing opinions of the overlay text.

So I think I’ve got everything sorted out. I initially had the graphics card in PCIEX16_1 and the LSI HBA in PCIEX16_2. As soon as I added anything into any of the PCIEX1_x slots, then PCIEX16_2 will change from x4 to x1. Since I’m only using the video card for transcoding, I swapped the slots that it and the HBA are installed in and now my parity calculation is running nicely at about 170-180MB/s with the NVME risers in PCIEX1_1 and PCIEX1_3 (PCIEX1_2 is blocked since the video card is a double height card).

Hmm, in that case the container was not showing the correct link and width speeds.