I have woke this morning and have found that 2 disks have failed in my array, Disk7 and 8

I'm running 7.2.4, what is my best course of action?

Where 1 disk has failed before, I've restarted the array with out it, rebuilt the array, then added it back in and then rebuilt array again.

I guess I could do this one disk at a time but I am worried as I I'll have no parity protection

See attached syslog

tower-syslog-20260509-0831.zip

Edited May 9May 9 by clevoir
typo

I've bitten the bullet and bought 2 8TB new hard drives as I could do with some more capacity

I see that the hard drives failed after a power failure, I've seen this before and then bought a UPS with AVR, so disappointed a power failure has caused drives to fail?

1 hour ago, clevoir said:

so disappointed a power failure has caused drives to fail

Are you sure that the disks have failed and not simply been disabled due a write error caused by an external factor (such as the power failure)?

Running the extended SMART test on a drive is a good indication of a drives health.

The array won't let me run any self-tests, if I click on either sefl-test nothing happens?

Probably worth posting diagnostics zip file created after a failure trying to run the self-test so we can see why it failed.

Please find attached diagnostics

tower-diagnostics-20260509-1148.zip

Good news is both emulated disks are mountable so rebuild should be OK after hardware problems are resolved.

Both disks disconnected and reconnected as Unassigned Devices. Do the extended tests on the Unassigned Devices.

SMART for those Unassigned Devices looks OK, but both disks were logging critical medium errors before they disconnected. So definitely do the extended test on both.

Thanks for the reply, I've got some new 8TB hard drives being delivered today

I'll swap them out with failed ones, then do some testing on the old ones.

,

I am doing extended test now, and I intend to preclear the new disks before use which is going to take some hours

If failed disks test OK, I guess I could rebuild the array with them or at least one of them, and if rebuild is OK I can replace them with the new disks later at leisure?

I am just concerned about the amount of time it's going to preclear the new disks and then rebuild the array, and worry if I have another another fault as I will have no parity protection.

I guess it's going to safer to rebuild the array with one disk at a time rather than 2?

8 hours ago, clevoir said:

safer to rebuild the array with one disk at a time rather than 2?

Since you have dual parity, even if the other disk has problems while you rebuild one, the rebuild should be OK because you still have enough redundancy. On the other hand, it will take longer to do them one at a time so will be longer until everything is good again.

Might be a good idea to avoid anything writing to the array (users, dockers, VMs, scripts) until you get them both rebuilt.

Both existing disks tested OK with a long SMART test, however I have stopped the array and tried to add one of them.

It's listed but when I select it as an array disk, it's immediate changed to not assigned. I have tried with the other disk and this has the same effect?

I have cleared both disks, and both are available to to preclear. I am thinking about preclearing the Disk 8 (4TB) to see if it will then let me add it to the array.

In the meantime i am preclearing the 2 new 8TB drives, which looks like it's going to take a total of 50 hours

Please find attached diagnostics

tower-diagnostics-20260510-1052.zip

11 hours ago, trurl said:

Might be a good idea to avoid anything writing to the array (users, dockers, VMs, scripts) until you get them both rebuilt.

I forgot to mention Mover, so it was busy moving things to the array while those disks were emulated. Shouldn't be a problem, that's what emulation is for. But it does mean the physical disks are more out-of-sync with the array than before.

Maybe I overlooked it, but I didn't see in syslog where you actually reassigned the disks.

24 minutes ago, clevoir said:

tried to add one of them.

I never use the work "add" when I mean "replace". "Add" means to a new slot in the array, not to the same slot that you are replacing. Did you try to "replace", or did you try to "add"?

I do see this though

May 10 11:27:28 Tower kernel: mdcmd (9): import 7 sdn 64 5860522532 0 WDC_WD60EFPX-68C5ZN0_WD-WX42D8445KLT
May 10 11:27:28 Tower kernel: md: import disk7: lock_bdev error: -2
May 10 11:27:28 Tower kernel: md: import_slot: 7 empty
May 10 11:27:28 Tower kernel: mdcmd (10): import 8 sdd 64 3907018532 0 WDC_WD40EFAX-68JH4N1_WD-WX22DB05D97H
May 10 11:27:28 Tower kernel: md: import disk8: lock_bdev error: -2
May 10 11:27:28 Tower kernel: md: import_slot: 8 empty

Not sure what that's about. A few search results from a few years ago. I'll take a closer look.

Sorry perhaps wrong choice of words, I stopped the array and I tried to reassign the previously failed disks 7 & 8 back into their original allocations

They were selectable, but as soon as I selected them, the array reported the disk allocation unassigned

If the original disks are going to cause a problem I may just hold out until the new disks have finished their preclear.

I was just trying to seek some reassurance by having one or both the original disks working again, before they were replaced by the new disks

Edited May 10May 10 by clevoir
added

The problem disks are on different controllers, but none of the other disks on either controller is having problems.

Do you have a backplane or are there separate cables going to each drive?

All disks are in a number of 5 to 3 IcyDock backplanes

New disks are have just started Preclear Post Read which is going to take 10 hours

The failed disks are shown as unassigned devices, and are available to preclear, I can't understand why I couldn't reassign them back to the array? I've been running Unraid since V3 and have not seen the behavior before.

I am looking to replace them them to the new drives once the preclear has passed, and carry out testing on the failed ones

Nothing has changed on the backplanes, disks / cabling hasn't been touched. Failed disks are showing that they have power to them.

Edited May 10May 10 by clevoir
typo

I have now swapped out Disk 8, and started an array rebuild

However the rebuild has paused after 12 mins, see attached diagnostics?

I've just resumed the rebuild

I have a spare controller card that I could swap out with the once that serves this disk

tower-diagnostics-20260511-0717.zip

It's logged as a disk error, but looks more to me like a controller or power issue

The rebuild is thankfully still ongoing, if it fails again I'll check power / data wiring and move the disk to another controller.

Should it rebuild OK I guess that can do the same.

I thought that I would add the new disks one at a time in case of problems.

After restarting rebuild i have noticed that it has logged 54 faults.

I thought that I'd resumed the rebuild after the faults were logged, but i have noticed it is doing a read check instead?

Shall I let this complete, or power down / check data & power leads + move disk to another controller?

A read check implies the dtive was disabled (red ‘x’), and if so not a lot of point in continuing the check. Can’t do more than guess without diagnostics.

This is the latest diagnostics, shall I power down / check cabling / move disk to new controller and then try and start the rebuild again?

tower-diagnostics-20260511-1305.zip

27 minutes ago, clevoir said:

shall I power down / check cabling / move disk to new controller and then try and start the rebuild again?

Worth a try, and post new diagnostics if it fails again.