Apologies for the title, not sure how else I can explain it. I'm currently panicking, but I'll try to explain this to the best of my abilities.

Per diagnostics, I have a 5 disc array (4 data/1 parity). I have 1 SATA disc, and it's one of my smaller data discs, the rest are SAS drives. I know the SATA drive has been acting suspect for awhile, it can read and write decent, but during parity check it makes a beeping sound, and the speeds only display for a second and then drop to 0. I've just been putting off the monthly parity checks until I could replace the bad SATA disc with a SAS one. I had to cancel a parity check last night because it was never going to finish, but then I check today, and one of my SAS drives (disc 3) was disabled and others have some read errors. I restarted the server thinking maybe my HBA card was acting up, ran some smart tests, fixed a file system corruption on disc 3, looked over the emulated contents and saw everything looked completely correct, and tried rebuilding the disc on itself. The stupid SATA drive was beeping and making the rebuild out to take year, and then within 20 minutes the disc 3 rebuild was aborted for read errors. Am I just screwed here?

Can I use the unbalanced plugin to move everything off the SATA disc (4) onto disc 1 and 2, remove it permanently from the array, and then rebuild disc 3? I do believe disc 3 to still be in good health and it's the faulty SATA disc (4) that is causing the rebuild to fail. As I said earlier, the emulated content on the parity disc all looked and played fine from what I checked. I've attached a video of the beeping SATA drive, some diagnostics from the fresh reboot that probably aren't extremely insightful, and an older diagnostics from when I was having cache issues (Fixed that problem, but it's the only diagnostics I have on hand with a long runtime.)

Thank you so much!

diagnostics-old.zip thewired-diagnostics-20260608-1911.zip

Edited June 9Jun 9 by Friarsgate_Customs
I don't know why the video was sized so huge by default, sorry

You cannot recover the data if there are really two bad disks and one parity, SMART looks fine for the SATA disk, and it looks more like a power/connection issue. Replace both cables from that disk and post new diags after array start.

6 hours ago, JorgeB said:

You cannot recover the data if there are really two bad disks and one parity, SMART looks fine for the SATA disk, and it looks more like a power/connection issue. Replace both cables from that disk and post new diags after array start.

I'm gonna try reseating every cable on every drive again... All the cables are brand new and I haven't even touched anything on the inside in months besides replacing an m.2 last month. I tried making a manual backup of my Immich/Nextcloud data onto a local PC, and the SATA drive started beeping again, the copy task went from estimated completion of 40 minutes to 4 hours, failed, and then all of my SAS drives got read errors. I don't know if the HBA card is going out or what. All 4 of my SAS drives were working great for months, I get to a parity check day, cancel it because of the SATA drive being slow, and now it's all falling apart lol.

Gonna open up the case here in a minute. I'm probably going to try some BIOS changes as well. Changing a PCIE slot from x2 to x4, and my M.2 speed from x2 to x4 as well (at the expense of disabling 2 SATA ports I'm not using). Maybe the extra lanes to the HBA card will help things be more stable.

40 minutes ago, Friarsgate_Customs said:

Maybe the extra lanes to the HBA card will help things be more stable.

That should not make any difference, main issue I would try would be to replace the power cable for the SATA drive, to see if it helps with the noises.

Alright, here's my process:

Power down, re-seat all my SAS cables and HBA card, replace all SATA data cables (again), use a different SATA power cable for the SATA HDD. The SATA power breakout cable I'm using for my SAS drives has the 3.3v line snipped, so I used a different breakout cable with that line intact. Reboot, run extended SMART tests on all drives (disk 1 kept getting interrupted for some reason, so I ran a few short tests instead), all the disks pass the test, check filesystem on all the drives and see that they're good. I downloaded all my most important data onto my gaming pc as a backup just in case. Did a complete rebuild of disk 3 in maintenance mode, and I did get a lot of read errors, but only in the first hour. The previously beeping SATA disk 4 actually ran at full speed with no weird noises. I downloaded my diagnostics and then shut down.

I've restarted the system, started the array, and I'm going to start docker back up now. I haven't ran a parity check in months because of the SATA drive, but it seems to be properly working now, so I think my next move should be to run a correcting parity check? Does that sound right? I've ordered a 4TB SAS drive to have on standby in case it starts acting suspect again. I've attached the diagnostics prior to shutdown, and I can attach another if you'd find it helpful. Crisis might be averted for now.

Thank you so much

disc3_rebuild_6.11.26.zip

Errors on disk1 and disk 2 during the rebuild, which still look like a power/connection issue, any power splitters being used?

3 hours ago, JorgeB said:

Errors on disk1 and disk 2 during the rebuild, which still look like a power/connection issue, any power splitters being used?

Yeah I've got a sata power extension with the 3.3v wire cut that I use for the parity disk and 1-3. I can't remember why, but I know I've done over 30 days with no errors previously. Even during the rebuild, the errors all came within the first 2-3 hours and the remaining 20 had none.

3 hours ago, JorgeB said:

Errors on disk1 and disk 2 during the rebuild, which still look like a power/connection issue, any power splitters being used?

Yeah I've got a sata power extension with the 3.3v wire cut that I use for the parity disk and 1-3. I can't remember why, but I know I've done over 30 days with no errors previously. Even during the rebuild, the errors all came within the first 2-3 hours and the remaining 20 had none.

If that is a SATA to more than 2 SATA plugs, it should not be used. Also note that the rebuilt disk can have some data corruption due to the read errors on those disks, but they weren't that many, so it should be mostly OK, and not much you can do about that now.

I've got a 1-5 sata power extension, and it's the nicer crimped style variety. It hasn't given me issues previously, but I'll try to replace it. It would probably be better to get a modular PSU that has more sata connectors.

The drive does seem to have all the content intact, and I did a backup of the most important data before the rebuild (had no errors during this). In your opinion, do you think I should run an error-correcting parity check now, or just wait until my next scheduled non-correcting check?

Thanks again for helping so much.

27 minutes ago, Friarsgate_Customs said:

I've got a 1-5 sata power extension

These should never be used, one SATA plug should never be split into more than two.

28 minutes ago, Friarsgate_Customs said:

In your opinion, do you think I should run an error-correcting parity check now, or just wait until my next scheduled non-correcting check?

I think you can just wait, and ideally change that splitter before.

About a week ago I had another drive drop out and need rebuilt. I didn't rebuild it right away and just bought a new PSU and splitters. I went with the MWE Gold 850 V2 because it comes with 10 SATA power cables from factory and has factory headers for more if needed. Also went with 1>2 splitters that don't have a 3.3v line at all, so no need to cut it for my SAS drives. It's a overkill setup for my server, but it should be good for future proof reasons, and I'm very sick of instability.

I might get killed for saying this, but I was only using a 400w EVGA non modular PSU before. At peak usage, my UPS was reporting 250w draw, and that's including my inefficient Opnsense router, switch, WiFi AP, and ISP Modem. So the Unraid server itself was nowhere near the 400w limit of the PSU. Since I installed the new PSU and rebuilt the disc again, no issues. My power draw is lower as well, due to the PSU barely breaking a sweat. Been 10 days and no issues so far. Hopefully I don't have to open the case again for at least a year.

Thanks Jorge for looking out.