JorgeB

Sync errors are in completely different sectors and appear to be all together, very strange, I assume your RAM is ECC (and it's enable) since it's a Xeon, if not run memtest, other than that if you have one try using an add-on controller.

Do you mean from one array drive to another? That will always be slow, and write mode will auto change from reconstruct to r/m/w.

Even the short SMART test is failing: "read failure" SMART Extended Self-test Log Version: 1 (1 sectors) Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed: read failure 90% 1047 5661095093 # 2 Short offline Completed: read failure 90% 1033 5661095093 # 3 Short offline Completed without error 00% 50 - # 4 Extended offline Aborted by host 50% 50 -

This error is what's causing the problem: Apr 11 07:23:26 Tower emhttpd: error: mkmbr, 1976: Device or resource busy (16): ioctl BLKRRPART: /dev/sdc No idea why this error though, please try wiping the device first: blkdiscard /dev/sdX Replace X with correct letter, currently it's sdc. P.S. not that it matters for this but the 860 EVO is a SATA device, not NVMe.

Yes, that's a known issue, or better yet to known issues, most free should never be used for best performance, even with older releases where it didn't auto switch to R/M/W there was still a very noticeable performance impact when parity writes overlapped, this performance impact is much worse with the new behavior, and one I'm against as I've already posted multiple times, IMHO that should only happen if write mode is set to auto, when I enable turbo write I want turbo write always.

Rebuild appears to have stalled, don't see a reason for that, there are for some reason spin downs logged for disk1, but that should be a reason for stalling. Try rebooting, it will re-start rebuilding at array start, if it stalls again post new diags.

System load is very high: 06:41:34 up 14:32, 3 users, load average: 47.50, 51.20, 52.77 Cores: 32 But can't see what's causing it, reboot, stop all dockers/VMs, check load, if low start dockers one by and keep checking.

From the diags when first rebuilt disk1 it was unmountable, but instead of checking filesystem you formatted the disk, so all data is gone, format is never part of a rebuild, do you still have the old disk?

It's OK, if I rename it to zip I can open it, but lots of other users with Macs can post a zip, but don't ask me how, I don't use them.

We need the generated zip file, upload that.

With reconstruct write enable you should get write speeds around the max speed of the slowest disk in use at any point, so if you were writing to an empty disk max speed should be >100MB/s, if you were writing for example to one disk with about 2.8/2.9TB used speed should be around 60MB/s, since it would be limited by the 3TB WD green, other than that you can have one or more disks with slow sector zones, was the parity check speed always normal, without slow periods? If you have dynamix stats installed you can check the graph (before rebooting).

Possibly I wasn't clear, do a couple of consecutive correcting parity checks without rebooting and post the diags.

Please post the diagnostics: Tools -> Diagnostics

SMART tests confirm disk5 is failing, new cables/port won't help.

Are you sure about that? Pretty sure that won't work and you'll get this unhelpful error.

Do a couple of consecutive correcting parity checks and post the diags.

It's up to you, but note that raid0 will only use 250GB from each device, so to keep both single profile is the best option, it just depends if those extra 250GB are worth it.

Get an LSI HBA, there are 8 and 16 port models, also BIOS can be removed so they don't increase boot time.

Any LSI with a SAS2008/2308/3008/3408 chipset in IT mode, e.g., 9201-8i, 9211-8i, 9207-8i, 9300-8i, 9400-8i, etc and clones, like the Dell H200/H310 and IBM M1015, these latter ones need to be crossflashed. And yes, they should work fine on an X8SIL.

Look for CA backup and restore.

Looks like the typical SASLP problem, errors on multiple disks: Apr 9 12:59:14 unRAID kernel: ata7.00: status: { DRDY } Apr 9 12:59:14 unRAID kernel: ata8.00: status: { DRDY } Apr 9 12:59:14 unRAID kernel: ata9.00: status: { DRDY } Apr 9 12:59:14 unRAID kernel: ata7: hard resetting link Apr 9 12:59:14 unRAID kernel: ata8: hard resetting link Apr 9 12:59:14 unRAID kernel: ata10.00: status: { DRDY } Apr 9 12:59:14 unRAID kernel: ata9.00: failed command: READ FPDMA QUEUED Apr 9 12:59:14 unRAID kernel: ata11.00: status: { DRDY } Apr 9 12:59:14 unRAID kernel: ata10.00: failed command: READ FPDMA QUEUED Apr 9 12:59:14 unRAID kernel: ata9.00: cmd 60/10:00:60:c7:0d/00:00:00:00:00/40 tag 28 ncq dma 8192 in Apr 9 12:59:14 unRAID kernel: res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout) Apr 9 12:59:14 unRAID kernel: ata10.00: cmd 60/10:00:60:c7:0d/00:00:00:00:00/40 tag 20 ncq dma 8192 in Apr 9 12:59:14 unRAID kernel: res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout) Apr 9 12:59:14 unRAID kernel: ata9.00: status: { DRDY } Apr 9 12:59:14 unRAID kernel: ata9.00: failed command: READ FPDMA QUEUED You should replace them with LSI HBAs, also change onboard SATA from IDE to AHCI.

Parity looks more like a connection problem, replace both cables.

Disk5 appears to be failing, you can confirm by running an extended SMART test or running another parity sync to see if it goes better this time, note that current parity isn't 100% valid because of the read errors on disk5 during the sync.

Then something else happened, just updating/unassigning it wouldn't cause an invalid partition error, see if it mounts with UD, if yes backup then format on cache slot and restore the data.

If it was a single device cache just re-assign it.