JorgeB

Yeah, also using the Marvell, and the log does appear to confirm it was a problem with that controller. You can keep using it, but it's unprotected if there's another disk failure, also avoid using disk1 since it will require using all the others one, and because of that not perform as well. Still needs to be rebuilt, but there's less data at risk. If you can get a new controller soon I would wait, if it's going to take more than a week or so probably best to try and rebuild as is. Those errors are not in the log posted, but as long as the parity check was non correct it will be fine, if it was correct there's a (very) small chance of parity corruption, that in turn would result in a corrupt disk1 rebuild.

That's usually a flash drive issue, try redoing it, this can also help sometimes: https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=713741

That could help us see the cause of the problem, my first guess would be the Marvell controller, those are not recommended, and not surprisingly it's what disks 1 and 5 are using. As for current status, disk1 looks fine, emulated disk is mounting so next step would be to rebuild on top, though I would recommend replacing that controller with one of the recommended ones.

Diags?

This suggests an issue with this controller: 04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01) Subsystem: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:1060] Kernel driver in use: ahci Kernel modules: ahci Try not using it or installing it in a different PCIe slot.

This can sometimes happens with some kernel/hardware combinations, try updating to v6.10-rc2, it uses a much newer kernel.

Like mentioned there will likely still be issues.

Could be, or a cable/power issue.

Dec 31 17:38:45 Tower kernel: attempt to access beyond end of device Dec 31 17:38:45 Tower kernel: md12: rw=4096, want=7814037064, limit=7814034952 Dec 31 17:38:45 Tower kernel: XFS (md12): last sector read failed This suggests you used a slightly smaller 4TB disk in this position, maybe this one: Model Family: Western Digital Red Device Model: WDC WD40EFRX-68WT0N0 Serial Number: WD-WCC4E1826444 User Capacity: 4,000,785,948,160 bytes [4.00 TB] Notice that it's slightly smaller than the other ones: User Capacity: 4,000,787,030,016 bytes [4.00 TB] Maybe a shucked disk?

Only by doing a new config, and you'll lose any data on that disk, though it's currently not mounting and it might be difficult to rebuild anyway because of the issues with the other disk.

https://forums.unraid.net/topic/86337-are-my-smart-reports-bad/?do=findComment&comment=800888

Yep, normal for the GUI to not show any errors since the connection is recovering, but it causes timeouts. Are there still similar errors in the log?

There's nothing obvious logged, if it's a hardware problem it's kind of expected, one thing you can try it to boot the server in safe mode with all docker/VMs disable, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one.

You can use the link above to do it using Unraid, it's usually pretty safe, there could be a problem if you lose power during the update and there's no UPS.

Yep.

Dec 31 09:19:22 Heimdall kernel: sd 7:0:16:0: Power-on or device reset occurred There are multiple errors like this for disk1, this is usually a connection/power problem.

Auto parity check is non correct, so it's normal for the next one to find the same errors, if there aren't any new ones.

Yes, but since the array was started without it it's now disabled. It was 10TB, so you need a 10TB or larger, problem is that there will likely be some read errors during the rebuild because of the other failing disk, this in turn will cause some corruption when that one is replaced later.

Could be the old firmware, try updating to latest, this is how it should look like: Dec 8 13:46:06 Tower7 kernel: mpt2sas_cm0: LSISAS2008: FWVersion(20.00.07.00), ChipRevision(0x03), BiosVersion(00.00.00.00) Dec 8 13:46:06 Tower7 kernel: mpt2sas_cm0: Protocol=(Initiator,Target), Capabilities=(TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)

Well not sure now, these contradict themselves:

When you mentioned not being detected by Unraid I assumed not detected by Linux, it's being detected by Unraid, it's in RAID mode and with a very old firmware, so it should be flashed to firmware 20.00.07.00 IT mode: Dec 31 16:29:04 Tower kernel: mpt2sas_cm0: LSISAS2008: FWVersion(07.15.08.00), ChipRevision(0x03), BiosVersion(07.02.04.00) Dec 31 16:29:04 Tower kernel: mpt2sas_cm0: Protocol=(Initiator,Target), Capabilities=(Raid,TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)

Please post the diagnostics to confirm if it's being detected by Linux or not.

Disk2 is disable, you only have one parity drive, so you needed to replace that one first.

File should be Pro.key, not ._Pro.key

Is that December 30th? Log looks normal at that time.