JorgeB

That's correct but since you're not running the test from the GUI you need to disable spin down, or SMART test will be interrupted.

I've been doing the same, for several years now, I snapshot the VMs daily with a script without shutting them down, but since this results in a crash consistent state, I also turn then all off once a week and run another script for offline snapshots, if I need to go back I try to use an offline snapshot but have restored to online snapshots some times without issues, and this way I have ore options, since it's not very convenient for me to shutdown the VMs every day. Interesting, I've been using send/receive for all my backups, including disk by disk full server backups to another Unraid server, and it's been working fine for me.

Please post the diagnostics: Tools -> Diagnostics

If it happens with all ports it can be a controller problem, lots of fake LSI out there.

Because it's not always obvious, and the errors can be intermittent, on WD disks like the one you posted this attribute ideally would be 0: ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE 1 Raw_Read_Error_Rate POSR-K 200 200 051 - 161 Though never a good sign, a non zero value by itself doesn't mean the disk is failing, but that together with the recent UNC @ LBA errors indicates the disk has seen better days.

Once a disk gets disable it needs to be rebuilt, either to a new disk or the same, if you want to use the same see below, but like mentioned that disk isn't looking very healthy. https://wiki.unraid.net/Troubleshooting#Re-enable_the_drive

Yes Use -x instead of -a, more info.

You don't usually get replies to feature requests, except sometimes by other users if they want the same (like +1), but requests are still checked and considered by LT.

No, it will check all disks.

Disk is not unmountable, it's disabled, two different things, unmountable is usually a corrupt filesystem, a disk gets disabled when there's a write error, sometimes cable/connection related, sometimes a failing disk, that disk is showing UNC @ LBA erros, i.e., bad sectors, and if it's not the first time it gets disabled you might want to consider replacing it. Also, disk6 has filesystem corruption you want to check that, with the array in maintenance mode: https://wiki.unraid.net/Check_Disk_Filesystems

Scrub is used to check data integrity, not the disk, something is corrupting the data, even if the disk is failing it should never return corrupt data, though like mentioned it's happened before, running a SMART extended test is a good idea, alternatively you can also run a parity check (non correct), that will also read the entire disk.

Not sure why that would happen, outside of my wheelhouse.

No, it's using the megaraid driver: 01:00.0 RAID bus controller [0104]: Broadcom / LSI MegaRAID SAS 2008 [Falcon] [1000:0073] (rev 03) Subsystem: Dell PERC H310 [1028:1f78] Kernel driver in use: megaraid_sas Kernel modules: megaraid_sas BTW, I've seen the same controller in raid mode still using the HBA driver (mpt3sas), not sure why sometimes one or the other is used, but that one is using the megaraid driver, SMART might work if you set the correct options, but it would be best to flash the controller to IT mode.

If I am correctly understanding the question: Disk1 is currently being emulated, using parity plus the other drives, any data currently on the emulated disk1 is the same that will be there if you rebuild it, so answer is yes, assuming that is what the OP is asking.

Because or controller is set to raid mode and is using the raid driver, it should be re-flashed to IT mode.

Intel gigabit is detected but it's eth1, you can change it to eth0 on Settings -> Network Settings -> Interface Rules (reboot required to take effect)

That shouldn't be a reason for a clean shutdown to fail. but try.

It's more of a known issue than bug, when there's no IOMMU support it's not possible to disable VM services on the GUI, you can do it by editing domain.cfg on your flash drive (config/domain.cfg) and changing SERVICE="enable" to "disable".

It is, but there are 2 phases, it first tries a clean shutdown but it the timeout is reached (as set on Settings -> Disk Settings -> Shutdown time-out) it then tries to do a shutdown anyway, in that case diagnostics should have been saved on the flash logs folder and might provide a clue on what's hanging it up.

Log is still being spammed, first with this which I don't know where it's coming from: Nov 23 04:40:40 GrundyNAS rpcbind[15627]: connect from 192.168.1.102 to getport/addr(mountd) Nov 23 04:40:42 GrundyNAS rpcbind[15636]: connect from 192.168.1.105 to getport/addr(mountd) Nov 23 04:40:44 GrundyNAS rpcbind[15654]: connect from 192.168.1.101 to getport/addr(mountd) Nov 23 04:40:50 GrundyNAS rpcbind[15686]: connect from 192.168.1.102 to getport/addr(mountd) Nov 23 04:40:52 GrundyNAS rpcbind[15695]: connect from 192.168.1.105 to getport/addr(mountd) Nov 23 04:40:55 GrundyNAS rpcbind[15718]: connect from 192.168.1.101 to getport/addr(mountd) Also with what looks like a connection problem with disk5: Nov 24 17:36:04 GrundyNAS kernel: ata1: link is slow to respond, please be patient (ready=0) Nov 24 17:36:06 GrundyNAS kernel: ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Nov 24 17:36:06 GrundyNAS kernel: ata1.00: configured for UDMA/33 Nov 24 17:36:06 GrundyNAS kernel: ata1: EH complete Nov 24 17:36:06 GrundyNAS kernel: ata1.00: exception Emask 0x10 SAct 0x0 SErr 0x4090000 action 0xe frozen Nov 24 17:36:06 GrundyNAS kernel: ata1.00: irq_stat 0x00400040, connection status changed Nov 24 17:36:06 GrundyNAS kernel: ata1: SError: { PHYRdyChg 10B8B DevExch } Nov 24 17:36:06 GrundyNAS kernel: ata1.00: failed command: READ DMA EXT Nov 24 17:36:06 GrundyNAS kernel: ata1.00: cmd 25/00:00:b0:77:f4/00:01:72:00:00/e0 tag 31 dma 131072 in Nov 24 17:36:06 GrundyNAS kernel: res 50/00:00:5f:6d:87/00:00:75:00:00/e0 Emask 0x10 (ATA bus error) Nov 24 17:36:06 GrundyNAS kernel: ata1.00: status: { DRDY } Nov 24 17:36:06 GrundyNAS kernel: ata1: hard resetting link Again because of this spam there are several hours missing from the syslog, including when the problem starts, replace cables on disk5 and see if you can find what's causing the other spam.

Ahh, though it had finished, btrfs can't correct the errors since each disk is an independent filesystem without redundancy, you need to delete or replace the affected files, they are identified on the syslog, but if these are new files you need to find out why data is being corrupted, is corruption limited to this disk?

No, it's IT mode only.

Main issues with the SAS2LP are dropped disks and in some cases gives always 5 sync errors on a parity check after rebooting (corrupting data), but they do still work for some, though can start giving problems at any time, usually after an Unraid release update. As for LSI, any with a SAS2008/2308/3008/3408 chipset in IT mode, e.g., 9201-8i, 9211-8i, 9207-8i, 9300-8i, 9400-8i, etc and clones, like the Dell H200/H310 and IBM M1015, these latter ones need to be crossflashed.

Scrub finished without errors so everything is fine now, you can also reset disk2 stats with: btrfs dev stats -z /dev/md2 So it's much easier to see if new errors come up in the future.

Looks like a problem with the flash drive or the OS files, make a backup of the current flash (or just the config folder), re-create the flash using the Unraid flash creator tool, restore the config folder from your backup overwriting the existing one, that should hopefully do it.