Just had two drives error out and have a red X on them. Uh oh?

[flyize]

I tried to reboot, but that didn't seem to help. Attached diagnostics from before the reboot. Any ideas?

 

truffle-syslog-20240228-1751.zip

[JorgeB]

6 minutes ago, flyize said:

but that didn't seem to help

A reboot will never help with disabled devices.

 

6 minutes ago, flyize said:

Attached diagnostics from before the reboot

That is just the syslog, please also post the complete diags from now.

[flyize]

8 minutes ago, JorgeB said:

A reboot will never help with disabled devices.

 

That is just the syslog, please also post the complete diags from now.

Well that was a bonehead move on my part. Ugh

 

 

truffle-diagnostics-20240228-1318.zip

[JorgeB]

It's kind of strange the way both disks got disabled, since there aren't any controllers error before, but both look healthy, so almost certainly not a disk problem, especially with two getting disabled at the same time, check/replace cables to rule that out, especially if they share a power splitter or something else, then, and if both emulated disks mount and contents look correct, you can rebuild on top.

 

https://docs.unraid.net/unraid-os/manual/storage-management#rebuilding-a-drive-onto-itself

 

You can do both at the same time.

[flyize]

Oh crap, I powered off the server without recording the actual disabled drives. Where do I see that in the diags?

[JorgeB]

Disk2 - ST8000DM004-2CX188_ZCT250PW

Disk8 - WDC_WD80EDAZ-11TA3A0_VGK45L3G

[flyize]

Thanks! Looks like those two drives are on the same molex to sata power splitter. If I order another one and things spin up, do I have to rebuild?

[JorgeB]

Yep, but only 

43 minutes ago, JorgeB said:

if both emulated disks mount and contents look correct

 

[flyize]

Any reason I can't just bring them back online and pretend nothing happened (or maybe do a parity check)?

 

edit: FWIW, if I look at the drives, all the data seems to be there.

Edited February 28 by flyize

[trurl]

37 minutes ago, flyize said:

if I look at the drives, all the data seems to be there.

Of course, because they are being emulated by parity.

 

That does at least answer this question though:

1 hour ago, JorgeB said:

both emulated disks mount and contents look correct

 

But probably better if you post new diagnostics after you get hardware fixes done.

[trurl]

https://docs.unraid.net/unraid-os/manual/storage-management/#what-is-a-failed-disabled-drive

 

Here is a relevant quote:

Quote

When a disk is marked as disabled and Unraid indicates it is being emulated then the following points apply:

Unraid will stop writing to the physical drive. Any writes to the 'emulated' drive will not be reflected on the physical drive but will be reflected in parity so from the end-user perspective then the array seems to be updating data as normal.

So rebuilding is the only way to recover emulated writes. And at the very least the failed write that disabled the drives is being emulated. Could be a lot more.

[flyize]

7 minutes ago, trurl said:

Of course, because they are being emulated by parity.

 

That does at least answer this question though:

 

But probably better if you post new diagnostics after you get hardware fixes done.

I went in an mounted /dev/sdk. It's actually emulating the whole device? If so, how will I know when I get it fixed?

 

edit: Added diagnostics

truffle-diagnostics-20240228-1458.zip

Edited February 28 by flyize

[trurl]

1 minute ago, flyize said:

I went in an mounted /dev/sdk

How? More details please.

[flyize]

1 minute ago, trurl said:

How? More details please.

mount /dev/sdk /tempdrive

 

[trurl]

If you didn't mount it read-only, then you made it even more slightly out-of-sync with the array.

 

But since you are apparently basing your previous statement

53 minutes ago, flyize said:

if I look at the drives, all the data seems to be there

on the contents of the physical disk, we don't really know if the emulated disk is mountable after all.

 

14 minutes ago, trurl said:

probably better if you post new diagnostics after you get hardware fixes done.

 

[flyize]

Okay. So unassign the disks, mount the array. Then navigate through the emulated disks (via the Unraid GUI I guess?) and make sure I see data. Then reassign the disks and rebuild (in-place)?

[flyize]

4 hours ago, flyize said:

Okay. So unassign the disks, mount the array. Then navigate through the emulated disks (via the Unraid GUI I guess?) and make sure I see data. Then reassign the disks and rebuild (in-place)?

Well hopefully that's correct, cause I'm in the middle of the rebuild!

[flyize]

Thanks @JorgeB. All fixed up.