JorgeB

Assuming HBA is using latest IT mode firmware: Yes Correct No There's enough bandwidth for 8 hard drives (and more, more require an expander) Yes Trim won't work. Not really, you'll lose a slot and increase power consumption a little, also make sure it's sufficiently cooled, there needs to be some airflow on/around it.

Unfortunately there's nothing relevant logged, this usually suggests a hardware issue, one thing you can try it to boot the server in safe mode with all docker/VMs disable, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one.

Unfortunately there's nothing relevant logged, this usually suggests a hardware issue, one thing you can try it to boot the server in safe mode with all docker/VMs disable, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one.

Feb 25 18:21:38 SwagServer kernel: BTRFS info (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 67623, gen 0 Feb 25 18:21:38 SwagServer kernel: BTRFS info (device sdd1): bdev /dev/sdc1 errs: wr 3341754, rd 454753, flush 228253, corrupt 0, gen 0 Feb 25 18:21:38 SwagServer kernel: BTRFS info (device sdd1): bdev /dev/sdb1 errs: wr 284514652, rd 245960197, flush 2035012, corrupt 37140664, gen 57 All pool devices show errors, read/write errors suggest two of them dropped offline in the past, suggest you backup and re-format the pool, then check here for better pool monitoring.

Diags only show the beginning of the parity sync but strange that it would finish without errors in this case, did they turn green in the end?

Should be fine, if you have mover logging enable you can remove those, or it will show the file names.

Yes, then you just need to assign the disk with the changed name.

Enable the syslog server and post that after a crash, it might catch something.

Looks like a connection problem, and CRC errors usually point to the SATA cables, start by replacing those.

Assign all disks that have a mount button in UD as data, assign the two that don't have one as parity, start the array to begin a parity sync, since the disks were encrypted Unraid will ask for a new key, enter the old one twice, the disk that was rebuilding will likely show data but some (or a lot) might be corrupt depending on when the rebuild stopped.

It's not showing up on SMART like it usually does, but looks like the drives are formatted with protection, try re-formatting the devices with sg_utils, more info below: Feb 24 10:42:18 UNRAID01 kernel: blk_update_request: protection error, dev sdb, sector 15362376248 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0

We can force Unraid to enable disk3 to rebuild disk2, but that will only work if parity is still valid, i.e., nothing was written to disk3 once it got disable, if you believe it's still valid you can try to rebuild using a new disk, even if it's not successful it won't make things worse, unless another disk fails during the rebuild.

You don't, you just need a key (without password) and for that key to exist in both servers, the key you're using has a password.

Like mentioned you don't need to input any password for rsync with SSH keys (without pass in the keys), I use it everyday.

Correct. Not possible, because you only have single parity and already have a disable disk, you can use ddrescue on disk2 to recover as much data as possible then do a new config with the clone and the remaining disks to re-sync parity, note that since it's not clear for how long disk3 has been disabled any data written to the emulated disk will be lost after the new config.

That suggests you created an SSH key with a password, if there's no password and it exists on both servers it won't ask for one. https://phoenixnap.com/kb/setup-passwordless-ssh

Yeah, but at least we still need to know which one was parity1, parity2 can be synced later if array disk order isn't known.

Diags are after rebooting, so not much to see, other than filesystem corruption on disks 4 and 9.

Happy to confirm it's finally fixed.

We can force Unraid to rebuild the same disk again, but only if you know all the old assignments, and what disk was being rebuilt, if you don't know you can still recover most of the array except the disk being replaced, maybe can still get some data from the old one if not completely dead.

Not enough info posted, do you known the disk assignments? Single or dual parity? What disk was being replaced? If you know the answer to those then also post the diagnostics.

This doesn't matter, what does is the test result: SMART Extended Self-test Log Version: 1 (1 sectors) Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 47825 - In this case it passed.

Maybe, it's worth looking at.

Unraid driver is crashing, this sometimes happens with some hardware/kernel combinations, suggest going back to v6.9.2 then try rc3 when available, since it will use a much newer kernel it might not have the same issue.