JorgeB

Forgot to mention, data corruption is being detected in the docker pool you should run a scrub, also see this.

Looks more like a power/connection problem.

No, but a parity check is basically the same as an extended SMART test.

You can disable loading the i915 driver, but since the current log has this: Feb 13 00:03:25 Orcrist kernel: macvlan_broadcast+0x10e/0x13c [macvlan] Feb 13 00:03:25 Orcrist kernel: macvlan_process_broadcast+0xf8/0x143 [macvlan] The problem is likely this one: Macvlan call traces are usually the result of having dockers with a custom IP address, upgrading to v6.10 and switching to ipvlan might fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enable, top right)), or see below for more info. https://forums.unraid.net/topic/70529-650-call-traces-when-assigning-ip-address-to-docker-containers/ See also here: https://forums.unraid.net/bug-reports/stable-releases/690691-kernel-panic-due-to-netfilter-nf_nat_setup_info-docker-static-ip-macvlan-r1356/

You're using a controller with SATA port multiplier(s), these are not recommended and known to drop devices without a reason, that would be the #1 suspect, #2 would be a power/connection issue.

Please use the existing plugin support thread:

This means the SMART test failed with a read error.

Enable the syslog server and post that after a crash, hopefully it catches something.

Cache device appears to be failing, sorry I forgot to mention that before, the reason I asked to swap controllers is that a failing device shouldn't drop and crash the controller, like it happens with the mvsas driver, you can run an extended SMART test to confirm, if it failing try to copy everything you can, either manually or using for example ddrescue.

It is, see here for a recent discussion about that.

Check that the default filesystem (settings -> disk settings) is not set to an encrypted one.

Power cycle the serve and/or replace/swap cables on the cache device, if it still doesn't come up after that it's likely dead.

Run it again without -n or nothing will be done.

You'll need to force a re-start, if the parity sync didn't finished before it crashed it will just start over.

I don't use that but all the info should be in the above linked thread, or ask there if there are any doubts.

Both pools are currently the same filesystem, using both devices, this means the device wasn't correctly removed from the pool before assign it to the new pool, to fix this you can try this: Stop the array, disable docker/VMs services, unassign the devices from both pools, start array, stop array, assign both devices to the same pool (there can't be a "all data on this device will be deleted" red warning for any pool device) , start array, stop array, unassign one the pool devices, start array so it can be removed from the pool, stop array, you can now assign the removed device to a new pool, start array, format new pool, all should be good now, if it isn't post new diags, you can also re-enable docker/VMs services.

Unlikely, once you fix/re-format that filesystem shares should come back, there could be some data corruption though if there really is bad RAM or other hardware issue.

Looks like a board compatibly issue, probably not much more you can do other than using a different board or different model HBA.

This would be for the future, when Unraid supports TRIM for array devices, for now it doesn't, so you can use any tipe of SSD, they will never be trimmed, so write performance might degrade over time, depending on the use case and also on the SSDs used. Better performance since parity can slow down writes, specially if not using turbo write or doing simultaneous writes to multiple array devices, also parity SSD will take a beating compared to the other ones, since it's updated when there are writes to any of the other array devices, i.e., if you have 10 array devices parity will have 10 times the number of writes of the other ones, and because of that it will also suffer more from the lack of trim. Nothing special about that, just assign one disk, or even a flash drive, as an array device and then you can run one or more cache pools, these can be trimmed and can also use different raid profiles for better performance.

SAS cables work with SATA devices, for SAS devices you need to use SAS cables only, so they will work, of course you also need to connect the power to them, seems obvious but it wouldn't be the first time someone didn't do it.

There's a problem with the cache fileystem, and since data corruption was being detected suggest running memtest before re-formatting it.

Grab the diagnostics when the transfer speed slows down and post them here.

Please post the diagnostics: Tools.

There are simultaneous issues with multiple disks: Feb 12 03:42:03 Tartarus kernel: ata5.00: exception Emask 0x10 SAct 0xc080003f SErr 0x90200 action 0xe frozen Feb 12 03:42:03 Tartarus kernel: ata5.00: irq_stat 0x00400000, PHY RDY changed Feb 12 03:42:03 Tartarus kernel: ata5: SError: { Persist PHYRdyChg 10B8B } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/08:00:68:44:e2/00:00:83:01:00/40 tag 0 ncq dma 4096 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/a8:08:70:44:e2/02:00:83:01:00/40 tag 1 ncq dma 348160 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/c8:10:18:47:e2/01:00:83:01:00/40 tag 2 ncq dma 233472 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/d8:18:e0:48:e2/00:00:83:01:00/40 tag 3 ncq dma 110592 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/08:20:b8:49:e2/00:00:83:01:00/40 tag 4 ncq dma 4096 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/c0:28:70:4c:e2/01:00:83:01:00/40 tag 5 ncq dma 229376 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/08:b8:c0:49:e2/00:00:83:01:00/40 tag 23 ncq dma 4096 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/e0:f0:88:43:e2/00:00:83:01:00/40 tag 30 ncq dma 114688 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5.00: failed command: WRITE FPDMA QUEUED Feb 12 03:42:03 Tartarus kernel: ata5.00: cmd 61/a8:f8:c8:49:e2/02:00:83:01:00/40 tag 31 ncq dma 348160 out Feb 12 03:42:03 Tartarus kernel: res 40/00:00:c8:49:e2/00:00:83:01:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:03 Tartarus kernel: ata5.00: status: { DRDY } Feb 12 03:42:03 Tartarus kernel: ata5: hard resetting link Feb 12 03:42:05 Tartarus kernel: ata5: SATA link down (SStatus 0 SControl 310) Feb 12 03:42:05 Tartarus kernel: ata5: hard resetting link Feb 12 03:42:06 Tartarus kernel: ata9.00: exception Emask 0x10 SAct 0x400001c0 SErr 0x90200 action 0xe frozen Feb 12 03:42:06 Tartarus kernel: ata9.00: irq_stat 0x00400000, PHY RDY changed Feb 12 03:42:06 Tartarus kernel: ata9: SError: { Persist PHYRdyChg 10B8B } Feb 12 03:42:06 Tartarus kernel: ata9.00: failed command: READ FPDMA QUEUED Feb 12 03:42:06 Tartarus kernel: ata9.00: cmd 60/00:30:90:8e:5e/01:00:f5:02:00/40 tag 6 ncq dma 131072 in Feb 12 03:42:06 Tartarus kernel: res 40/00:00:90:8e:5e/00:00:f5:02:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:06 Tartarus kernel: ata9.00: status: { DRDY } Feb 12 03:42:06 Tartarus kernel: ata9.00: failed command: READ FPDMA QUEUED Feb 12 03:42:06 Tartarus kernel: ata9.00: cmd 60/00:38:20:7a:01/01:00:69:04:00/40 tag 7 ncq dma 131072 in Feb 12 03:42:06 Tartarus kernel: res 40/00:00:90:8e:5e/00:00:f5:02:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:06 Tartarus kernel: ata9.00: status: { DRDY } Feb 12 03:42:06 Tartarus kernel: ata9.00: failed command: READ FPDMA QUEUED Feb 12 03:42:06 Tartarus kernel: ata9.00: cmd 60/00:40:20:7b:01/01:00:69:04:00/40 tag 8 ncq dma 131072 in Feb 12 03:42:06 Tartarus kernel: res 40/00:00:90:8e:5e/00:00:f5:02:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:06 Tartarus kernel: ata9.00: status: { DRDY } Feb 12 03:42:06 Tartarus kernel: ata9.00: failed command: READ FPDMA QUEUED Feb 12 03:42:06 Tartarus kernel: ata9.00: cmd 60/00:f0:90:8d:5e/01:00:f5:02:00/40 tag 30 ncq dma 131072 in Feb 12 03:42:06 Tartarus kernel: res 40/00:00:90:8e:5e/00:00:f5:02:00/40 Emask 0x10 (ATA bus error) Feb 12 03:42:06 Tartarus kernel: ata9.00: status: { DRDY } Feb 12 03:42:06 Tartarus kernel: ata9: hard resetting link Feb 12 03:42:06 Tartarus kernel: ata5: SATA link down (SStatus 0 SControl 310) Feb 12 03:42:07 Tartarus kernel: ata5: hard resetting link Feb 12 03:42:08 Tartarus kernel: ata5: SATA link down (SStatus 0 SControl 310) Feb 12 03:42:08 Tartarus kernel: ata5.00: disabled This is usually a power/connection problem.

Start with this.