[SOLVED] Unmountable: No file system

[squishee]

Hi,

 

Earlier today I wasn't able to see some files on my server. I just gave it a reboot and one of my disks says "Unmountable: No file system". This is the first time this has ever happened and I'm not sure what to do. Any help would be greatly appreciated. 

server-diagnostics-20200424-1237.zip

Edited April 26, 2020 by squishee

[JorgeB]

https://wiki.unraid.net/Check_Disk_Filesystems#Checking_and_fixing_drives_in_the_webGui

[squishee]

Thanks Jonnie. I tried the -n mode when running xfs_repair and I received an error. It advised me to use the -L option, which I did, and got the output below:

Phase 1 - find and verify superblock...
        - block cache size set to 349656 entries
Phase 2 - using internal log
        - zero log...
zero_log: head block 112257 tail block 112221
ALERT: The filesystem has valuable metadata changes in a log which is being
destroyed because the -L option was used.

fatal error -- failed to clear log

I'll add that I'm hearing some odd sounds (sort of like a beeping) coming out of my server. 

[JorgeB]

Could be a disk issue, SMART looks mostly OK, run an extended SMART test.

[squishee]

So the extended SMART test completed without error but as I was running it, I got these alerts where the reallocated sector count ran up to 56. Is that a sign of a faulty disc?

[itimpi]

While reallocated sectors are not inherently bad, if the reallocated sector count is continuing to increase then that is a good indication that disk failure could be imminent.   

[JorgeB]

Try running xfs_repair again, and if the reallocated sectors keep increasing you should consider replacing that disk.

[squishee]

So I ran -nv and received the following output this time. I'm not really sure what I should do as a next step. The reallocated sectors haven't increased since my last post. 

 

https://pastebin.com/27fcbRWf

[JorgeB]

Run without -n or nothing will be done.

[squishee]

I ran -v and received this error again:

Phase 1 - find and verify superblock...
        - block cache size set to 349656 entries
Phase 2 - using internal log
        - zero log...
zero_log: head block 228037 tail block 227489
ERROR: The filesystem has valuable metadata changes in a log which needs to
be replayed.  Mount the filesystem to replay the log, and unmount it before
re-running xfs_repair.  If you are unable to mount the filesystem, then use
the -L option to destroy the log and attempt a repair.
Note that destroying the log may cause corruption -- please attempt a mount
of the filesystem before doing this.

Per the message, I tried running -L and received another fatal error:

Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
ALERT: The filesystem has valuable metadata changes in a log which is being
destroyed because the -L option was used.

fatal error -- failed to clear log

[JorgeB]

Please post current diags just to see if there's any disk read error, either that or it's a xfs_repair issue.

[squishee]

Attached

server-diagnostics-20200425-0955.zip

[JorgeB]

It does look like a disk problem, thought the ATA errors are not logged as media errors, but adding the weird noises and reallocated sectors it points to that, you could unassign that disk, then attempt to fix the filesystem on the emulated disk, if successful rebuild to a new disk.

[squishee]

To do that, is it just a matter of unassigning the disk and formatting it?

[JorgeB]

3 minutes ago, squishee said:

and formatting it?

NEVER formatting, unassign the disk, start the array, emulated disk is likely still going to be unmountable, if yes run xfs_repair on the emulated disk. though there's a small possibility the emulated disk could mount from start if the actual disk really has a problem.

 

 

[squishee]

As you suspected, the emulated disk was still unmountable. Running -v gave me this output: https://pastebin.com/N5fYa2TU

[JorgeB]

It should mount now if you start the array.

[squishee]

Thanks! I was able to mount and rebuild the disk successfully, however, throughout the process I received 3-4 alerts where the reallocated sector count was climbing. It's currently at 4568 - should I replace the disk, and if so, would you happen to know if this is something that falls under manufacturer warranty?

[itimpi]

That disk definitely needs replacing ASAP.  I would also expect there to be no problem with a RMA with that many reallocated sectors, particularly as the number keeps increasing.   It feels as if the disk is probably on its last legs

 

 

[squishee]

Thanks again. Appreciate the help!

[JorgeB]

Since you mentioned strange noises, and when you get the new disk, make sure to check/replace the power connection, bad/failing power can sometimes cause disks to develop bad sectors.