JorgeB

Thanks for the diags. I was able to see the replacement in the syslog-previous and reproduce the issue, it's a corner case that can happen if the missing device has the same identifier as the remaining one. I created a bug report https://product.unraid.net/b/unraid-os-bugs/p/7-3-1-boot-pool-replacement-can-leave-stale-missing-mirror-member

Look for the existing support thread for that plugin.

Unraid doesn't support spinning down SAS drives, and the plugin doesn't work correclty with every model, suggest posting in the existing plugin support thread.

Try booting in safe mode and/or use a different browser

Still seems to me like some kind of bottleneck; definitely it's not an Unraid limitation, with a fast enough CPU and devices I basically get line speed with 25GbE (with exlusive shares, FUSE always add some overhead, but this is typically noticeable with 10GbE or higher only)

You can try downgrading to the previous known good release with your server to make sure the issues are related to the update.

Please name the shares you are trying to copy from/to.

Docker folder with native zfs driver is not recommended and known to have multiple issues, I would recommend recreating that as a docker image, even other docker folders can have strange issues in my experience, but if you prefer to keep the folder, recreate it using the overlay2 driver. https://docs.unraid.net/unraid-os/troubleshooting/common-issues/docker-troubleshooting/#re-creating-the-docker-image-file Then: https://docs.unraid.net/unraid-os/troubleshooting/common-issues/docker-troubleshooting/#reinstalling-docker-applications Also see below if you have any custom docker networks: https://docs.unraid.net/unraid-os/troubleshooting/common-issues/docker-troubleshooting/#restoring-docker-custom-networks

Since there's nothing relevant logged, one thing you can try is to boot the server in safe mode with all docker containers/VMs disabled, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one, including the individual docker containers.

Just that pool

If it's a kernel regression it will need to be fixed upstream in the kernel, then Unraid will inherit it

If you have the diags from the replacement would be interested in seeing them, but to fix the issue, open a terminal window and type: zpool detach flash 14843927469164253652

That suggests to me a CPU performance issue, I would not expect to see so much difference with those numbers. Make sure the CPU is throttling up; also check Settings - Power Mode is set to Best Performance.

Please post the diagsotics during a atransfer, also post the results of a single stream iperf test in one direction, then the other one (-R) to rule out a network problem.

That should not cause issues other than the spam.

Try copying to a disk share (or exclusive share)

Probably, but should be harmless.

That one should be OK

Jun 22 23:31:04 Tower webgui: /boot/config/plugins/dynamix/dynamix.cfg corrupted or missing Check if the file exists: cat /boot/config/plugins/dynamix/dynamix.cfg If it doesn't or looks corrupt, delete it and then make a dummy change to a setting that will recreate this file, for example, change the theme

Nothing relevant was logged that I can see in the last lines before the crash, except maybe the Docker Compose lines; they don't look normal, but it may be worth retesting with the Docker engine disabled, or trying to run half the containers you have; if it still crashes, try the other half. Jun 22 07:00:16 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 07:05:44 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 07:30:19 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 07:33:49 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 07:35:47 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 08:00:22 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 08:03:51 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 08:04:05 NakUnraid emhttpd: spinning down /dev/sdb Jun 22 08:05:50 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 08:30:25 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 08:35:53 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 08:40:48 NakUnraid compose.manager: [INFO] [nchan] publisher starting Jun 22 16:51:43 NakUnraid rc.rsyslogd: Syslog server daemon... Started.

I assume you mean disk4? Errors with multiple disks, and you are using a controller with SATA multipliers, which are not recommended; it should be replaced when possible: https://forums.unraid.net/topic/102010-recommended-controllers-for-unraid/ Jun 22 18:31:37 BattleStar kernel: ata8: found unknown device (class 0) ### [PREVIOUS LINE REPEATED 1 TIMES] ### Jun 22 18:31:37 BattleStar kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Jun 22 18:31:41 BattleStar kernel: ata8.00: failed to IDENTIFY (I/O error, err_mask=0x100) Jun 22 18:31:41 BattleStar kernel: ata8.00: revalidation failed (errno=-5) Jun 22 18:31:42 BattleStar kernel: ata8: hard resetting link Jun 22 18:31:42 BattleStar kernel: ata7.02: SATA link up 6.0 Gbps (SStatus 133 SControl 330) Jun 22 18:31:47 BattleStar kernel: ata8: link is slow to respond, please be patient (ready=0) Jun 22 18:31:48 BattleStar kernel: ata7.02: qc timeout after 5000 msecs (cmd 0xec) Jun 22 18:31:48 BattleStar kernel: ata7.02: failed to IDENTIFY (I/O error, err_mask=0x4) Jun 22 18:31:48 BattleStar kernel: ata7.02: revalidation failed (errno=-5) Jun 22 18:31:48 BattleStar kernel: ata7.15: hard resetting link Jun 22 18:31:49 BattleStar kernel: ata7.15: SATA link up 6.0 Gbps (SStatus 133 SControl 300) Jun 22 18:31:49 BattleStar kernel: ata7.00: hard resetting link Jun 22 18:31:49 BattleStar kernel: ata7.00: SATA link up 6.0 Gbps (SStatus 133 SControl 330) Jun 22 18:31:49 BattleStar kernel: ata7.01: hard resetting link Jun 22 18:31:50 BattleStar kernel: ata7.01: SATA link down (SStatus 0 SControl 330) Jun 22 18:31:51 BattleStar kernel: ata7.02: SATA link up 6.0 Gbps (SStatus 133 SControl 330) Jun 22 18:31:51 BattleStar kernel: ata7.00: configured for UDMA/133 Jun 22 18:31:51 BattleStar kernel: ata7.02: model number mismatch 'ST6000VX001-2BD186' != 'ST6000VX001' Jun 22 18:31:51 BattleStar kernel: ata7.02: revalidation failed (errno=-19) Jun 22 18:31:51 BattleStar kernel: ata7.02: limiting SATA link speed to 3.0 Gbps

I would suspect the change in stability were more caused by the other changes than the downgrade, but you can try reupgrading in the near future while leaving everything else intact

If it only mounted read only, yes, you will need to copy the data someplace else and then reformat.

Jun 22 15:05:05 Towerx2 kernel: mpt3sas_cm0: diag reset: FAILED Jun 22 15:05:05 Towerx2 kernel: mpt3sas_cm0: failure at drivers/scsi/mpt3sas/mpt3sas_scsih.c:12349/_scsih_probe()! HBA was failing to initialize; the BIOS update may have helped with that

Since the drive is encrypted, you need to decrypt it first, one of the reasons I don't recommend encryption unless absolutely necessary; it just complicates things, assuming you are using a keyfile and it's loaded: cryptsetup luksOpen /dev/sdf1 sdf1 --key-file=/root/keyfile