Disk unmountable, check filesystem missing from gui, can't find superblock

[zer0zer0]

All of a sudden one of my disks gave me the dreaded error of unmountable, no supported file system

 

Disk 4 - HUH721010AL4204_7PH0NGHC (sde)

 

Check xfs filesystem is totally missing from the gui for this disk, but there for all the other array disks?

Running a check from the CLI comes back with could not find a valid secondary superblock

 

Where do I go from here apart from just replacing the drive?

 

Diagnostics zip attached

darkstor-diagnostics-20230810-2128.zip

Edited August 11, 2023 by zer0zer0

[JorgeB]

3 hours ago, zer0zer0 said:

apart from just replacing the drive?

That won't help, start the array in maintenance mode and post the output of:

xfs_repair -v /dev/md4p1

 

[zer0zer0]

12 hours ago, JorgeB said:

That won't help, start the array in maintenance mode and post the output of:

xfs_repair -v /dev/md4p1

 

It’s going to take a while 😃

 

why /dev/md4p1 and not /dev/sde?

[JonathanM]

1 hour ago, zer0zer0 said:

why /dev/md4p1 and not /dev/sde?

The md part makes sure you keep parity valid, if you alter the sd device you would need to rebuild parity. The Xp1 part designates which disk and partition to target, if you scan sde it will miss the filesystem because it's inside the first partition.

[zer0zer0]

Same result unfortunately

 

Sorry, could not find valid secondary super block

Edited August 12, 2023 by zer0zer0

[JorgeB]

Are you sure that this was ever formatted? Kind of strange that the filesystem is set to auto, assuming it's still sde post the output of:

fdisk -l /dev/sde

 

[zer0zer0]

5 hours ago, JorgeB said:

Are you sure that this was ever formatted? Kind of strange that the filesystem is set to auto, assuming it's still sde post the output of:

fdisk -l /dev/sde

 

 

It was definitely formatted with xfs and then all of a sudden just threw that error

 

root@DARKSTOR:~# fdisk -l /dev/sde
Disk /dev/sde: 9.1 TiB, 10000831348736 bytes, 2441609216 sectors
Disk model: HUH721010AL4204
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 14CEF3CF-1F72-48D2-8C97-83C61932AE02

Device     Start        End    Sectors  Size Type
/dev/sde1      8 2441609210 2441609203  9.1T Linux filesystem

 

[JorgeB]

That start sector is wrong, should be 64, something messed with your partition/disk, you can try running testdisk to see if it finds the old partition.

 

 

[zer0zer0]

9 hours ago, JorgeB said:

That start sector is wrong, should be 64, something messed with your partition/disk, you can try running testdisk to see if it finds the old partition.

 

 

 

Hmm, all of the other disks also start at 8?

 

Device     Start        End    Sectors  Size Type
/dev/sdb1      8 2441609210 2441609203  9.1T Linux filesystem
/dev/sdc1      8 2441609210 2441609203  9.1T Linux filesystem
/dev/sdd1      8 2441609210 2441609203  9.1T Linux filesystem
/dev/sde1      8 2441609210 2441609203  9.1T Linux filesystem
/dev/sdf1      8 2441609210 2441609203  9.1T Linux filesystem

 

Edited August 14, 2023 by zer0zer0

[JorgeB]

My bad, missed that you are using 4Kn disks, with 512E disks the partition starts at sector 64, but since with 4Kn each sector is 8 times bigger, 64/8=8, it just means that's not the problem, if you are certain the disk was formatted with xfs you can try running a file recovery util like UFS explorer to see if it can recover any data, just to scan the disk you can use the free trial.

[zer0zer0]

So, it won't rebuild from parity?

 

That data is only recoverable using testdisk or similar?

[JorgeB]

11 hours ago, zer0zer0 said:

So, it won't rebuild from parity?

If parity is in sync the result will most likely be the same, but you can try, unassign the disk (keep this disk intact), start the array, see if the emulated disk mounts, if it doesn't try running xfs_repair on the emulated disk.