Unmountable Drives - Formatting Issues

[WirralTom]

Hi all,

 

I'll keep things brief. I had a 12Tb drive that was reporting errors. I've RMA'd it and had a replacement shipped. I've popped the replacement in and the system has completed a parity rebuild taking 20-odd hours. I've just restarted the array and it's now showing that Disk 4 (the replacement disk) needs formatting as expected, however it is also showing my cache pool as requiring formatting, somerthing I obviously don't want to do. It's showing Disk 4 as "Unmountable - unsupported partition layout" but the cache drives as "Unmountable - "No file system" Does anyone have any idea what's gone wrong and what I need to do to fix it?

 

Thanks,

 

Tom

[JorgeB]

19 minutes ago, WirralTom said:

needs formatting as expected,

That's not expected, and don't format anything, please post the diagnostics.

[JonathanM]

Attach diagnostics to your next post.

 

FORMATTING is NEVER expected on a rebuild. Was the data slot emulated with all the data when you removed the old drive? The rebuild should be identical to what was showing when the drive was removed.

[WirralTom]

Diagnostics attached. I've just upgraded to 6.11.3, system is showing parity as valid but still showing the Disk 4 and cache disks as requiring formatting. The diagnostics seem to reflect this.

morpheus-diagnostics-20221110-1327.zip

Edited November 10, 2022 by WirralTom

[JorgeB]

Reboot, before starting the array unassign disk4, start array, post new diags.

[WirralTom]

Ok, rebooted, took disk 4 out of the array. Array has now restarted without issue, and is no longer asking for a format of disk 4 and the array.

 

New diagnostics attached

 

 

morpheus-diagnostics-20221110-1452.zip

[JorgeB]

First lets try and fix the pool, looks like one the NVMe devices dropped offline in the past and it was never synced, stop the array, type:

wipefs -a /dev/nvme0n1p1

then unassign both pool devices

start array

stop array

reassign only the Sabrent NVMe

start array

post new diags.

[WirralTom]

OK, steps followed. When I ran the command it's returned "8 bytes wer erased at offset 0x00010040 (btrfs): 5f 42 48 52 66 53 5f 4d"

 

Rest of the steps followed and new diagnostics attached

 

Tom

morpheus-diagnostics-20221110-1750.zip

[JorgeB]

OK, pool looks OK now, you can add the other NMVe device and also rebuild the disabled disk, just re-assign them to their correct slots and start the array, also suggest seeing here for better pool monitoring in case it drops again.

[WirralTom]

Thank you for this Jorge, I really appreciate it! So I stopped the array and re-added the two missing devices (the cache drive and Disk 4). Array has restarted fine and has started a Data Rebuild, however it's thrown up an interesting cache disk warning. Is it anything to worry about?

[JorgeB]

It's normal during the pool balance, nothing to worry about.

[WirralTom]

Brilliant, thank you. I'll mark this as solved then and let it rebuild