Need guidance on rectifying disk errors

[glazed-homecoming2581]

I'm relatively new to unRAID and I've encountered some issues recently, potentially a catastrophic 2 disk failure. I would greatly appreciate any guidance or advice you can provide.

 

Here's my current configuration:

 

• 1 parity drive
• 3 data drives
• 1 cache drive
• 2 out of 3 data disks displaying the error message "unmountable: wrong or no file system"
  • 1 of these disks is showing as green
  • The other is displaying a warning triangle with the message "device contents emulated"
  • xfs_repair scans report an I/O related error on both disks
• Currently performing a data rebuild
• Interestingly, the only healthy disk is also showing up as an unassigned device

 

 

lfnshare-diagnostics-20230921-1724.zip

[trurl]

34 minutes ago, glazed-homecoming2581 said:

Currently performing a data rebuild

Should have waited for advise 

[JorgeB]

SMART for disk2 looks real bad, disk1 looks mostly OK but the errors are reported as a disk problem, disk3 is also reporting read errors, those are not logged as a disk problem, but the disk has pending sectors, cancel the rebuild, run an extended SMART test on disks 1 and 3 and post new diags, but it doesn't look good.

[glazed-homecoming2581]

On 9/21/2023 at 6:12 PM, JorgeB said:

SMART for disk2 looks real bad, disk1 looks mostly OK but the errors are reported as a disk problem, disk3 is also reporting read errors, those are not logged as a disk problem, but the disk has pending sectors, cancel the rebuild, run an extended SMART test on disks 1 and 3 and post new diags, but it doesn't look good.

 

I am not able to cancel the rebuild for some reason. When I hit cancel, it asks me if I wish to proceed and nothing happens thereafter. Logs attached.

lfnshare-diagnostics-20230929-1031.zip

[JorgeB]

Type reboot in the CLI, if it doesn't reboot after 5 minutes you will need to force it.

[glazed-homecoming2581]

Here is the diags after running an extended SMART test on disks 1 and 3

lfnshare-diagnostics-20230929-2033.zip

[JorgeB]

Disk1 passed, but disk3 failed, disk2 is also bad so not many options, you can try to copy everything you can from those disks elsewhere, or clone the failing disks with ddresccue.

[glazed-homecoming2581]

Much appreciated @JorgeB. Although I'm not overly bothered about that data, I would like to attempt data recovery but I'm finding it strange that whenever I reboot the server, I have a different combination of disks available within the GUI. Even my brand new replacement disk is now not appearing (it did appear as unassigned previously). Any thoughts?

[JorgeB]

19 minutes ago, glazed-homecoming2581 said:

Even my brand new replacement disk is now not appearing

That suggests a power/connection problem.

[glazed-homecoming2581]

Is there any chance that all this could be server related such as the PSU in your opinion? I'm running Dell Precision Tower 7910

Edited October 1 by glazed-homecoming2581

[JorgeB]

Could be PSU, or for example the SATA cables.

[glazed-homecoming2581]

After some troubleshooting, I have concluded that this failure was due to the disks rather than the PSU or cables. Whilst I am waiting for replacement drives, my parity is showing as invalid now. Could this be also due to a bad disk or rather a configuration issue since I still have an array with 2 disks that show:

 

UNMOUNTABLE: WRONG OR NO FILE SYSTEM

 

I've re-started the array to at least get my Docker containers running for now.

lfnshare-diagnostics-20231013-1227.zip

[JorgeB]

Parity needs to be synced, but there are still read errors with disk2.

[glazed-homecoming2581]

1 hour ago, JorgeB said:

Parity needs to be synced, but there are still read errors with disk2.

Both disk 1 and 2 are showing UNMOUNTABLE: WRONG OR NO FILE SYSTEM. Disk 3 is the only drive that has been replaced so far.

[JorgeB]

Not point in trying to fix the unmountable issue with disk2 while there are read errors, disk appears to be failing you can run an extended SMART test to confirm.