I woke up to a drive being unmountable, unsure what it is or what to do. Can anyone here help?

[sloob]

I'm very much a noob and I don't know what to do. Usually when something is wrong with a disk the green dot icon on the browser tab turns into an orange triangle but not this time.

Pardon me if this obvious.

[trurl]

If possible before rebooting and preferably with the array started
Go to Tools - Diagnostics and attach the complete Diagnostics ZIP file to your NEXT post in this thread.

[sloob]

Right, sorry about that, I should have included that in my first post.

 

Here it is. also, recently I've had TONS of issues with my server (It had previously ran more or less flawlessly for the last two years) like docker apps deleting themselves or apps not starting

unraid-diagnostics-20201218-1741.zip

[trurl]

Disk1 was already unmountable when you booted and syslog doesn't tell us anything before that. SMART attributes for disk1 look OK though.

 

Also noticed docker.img isn't within any user share. The usual convention these days is to put it in the system share. Do you know what disk it is on?

 

You will have to repair the filesystem on disk1.

https://wiki.unraid.net/Check_Disk_Filesystems#Checking_and_fixing_drives_in_the_webGui

[sloob]

Thank you for the help, I ran the test with the -n option and it found a ton of errors. The next step would be to perform the test again but with no option I imagine but I am unable to do so. it gives me the following error:

Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
ERROR: The filesystem has valuable metadata changes in a log which needs to
be replayed.  Mount the filesystem to replay the log, and unmount it before
re-running xfs_repair.  If you are unable to mount the filesystem, then use
the -L option to destroy the log and attempt a repair.
Note that destroying the log may cause corruption -- please attempt a mount
of the filesystem before doing this.

Should I go ahead and add the -L option? if worse comes to worst I can always format the drive and rebuild it using the parity right?

[trurl]

4 minutes ago, sloob said:

add the -L option

Yes, Unraid has already determined the disk can't be mounted so that is what you have to do.

 

5 minutes ago, sloob said:

if worse comes to worst I can always format the drive and rebuild it using the parity right?

ABSOLUTELY NOT!!!

 

Format is a write operation. Format means "write an empty filesystem to this disk". That is what it has always meant in every operating system you have ever used. Unraid treats that write operation exactly as it does any other, but updating parity. So after formatting a disk in the parity array, parity agrees the disk has an empty filesystem. So rebuilding a formatted disk can only result in a formatted disk.

[sloob]

Oh, I see, thanks for clarifying that. But I could always format it from another computer and use the parity to rebuild it right? Unraid would just treat it like a replacement drive and start rebuilding it right?

Edited December 18, 2020 by sloob

[trurl]

No point in formatting it outside the array either. Rebuild will completely overwrite the disk whether you format it or clear it or even put it in Windows and fill it up with files. It is just formatting it in the array that will make it impossible to rebuild.

 

Unfortunately, rebuilding an unmountable filesytem will almost always result in an unmountable filesystem.

[trurl]

23 minutes ago, sloob said:

I ran the test with the -n option and it found a ton of errors.

If you want us to take a look at those results do it again and post the output.

[sloob]

Thanks a lot for your help! I ran it with the -L option and rebooted. it mounted fine, I'm now running a parity check. I'm guessing it's normal that it's finding errors?

[trurl]

9 minutes ago, sloob said:

normal that it's finding errors

I assume you mean parity sync errors and not I/O errors. Depends on what happened before those diagnostics you posted. How many sync errors are you getting?

[sloob]

So far this is what I'm getting

Total size:	2 TB	
Elapsed time:	22 minutes	
Current position:	36.0 GB (1.8 %)	
Estimated speed:	24.7 MB/sec	
Estimated finish:	22 hours, 7 minutes	
Sync errors corrected:	223

 

[trurl]

That looks OK.

[trurl]

On second thought that looks extremely slow. Is that typical for your system? I would expect 2TB parity to take less than 6 hours.

[sloob]

On 12/18/2020 at 7:00 PM, trurl said:

On second thought that looks extremely slow. Is that typical for your system? I would expect 2TB parity to take less than 6 hours.

I'm sorry I never got your message, In the end it took about 8 hours total. This morning I woke up to the exact same error, Is it the drive that's faulty you think?

[trurl]

1 hour ago, sloob said:

exact same error

What error is that?

 

Post new diagnostics

[sloob]

16 minutes ago, trurl said:

What error is that?

 

Post new diagnostics

I meant the unmountable:No file system error.

 

Here is the diagnostic and just in case the smart report for that drive

unraid-diagnostics-20201224-1624.zip unraid-smart-20201224-1323.zip

[trurl]

FYI diagnostics already includes SMART for all connected disks as well as syslog since last reboot and many other things.

 

Nothing obviously wrong with the disk but you haven't done any SMART tests on it. You might run and extended SMART test.

 

Since you rebooted before getting those diagnostics nothing in syslog that might indicate what happened. You will have to repair it again.

 

Do you have backups of everything important and irreplaceable?

[sloob]

On 12/24/2020 at 5:23 PM, trurl said:

FYI diagnostics already includes SMART for all connected disks as well as syslog since last reboot and many other things.

 

Nothing obviously wrong with the disk but you haven't done any SMART tests on it. You might run and extended SMART test.

 

Since you rebooted before getting those diagnostics nothing in syslog that might indicate what happened. You will have to repair it again.

 

Do you have backups of everything important and irreplaceable?

After a few days it happened again! (unmountable:No file system error.) and now I didn't reboot. Here is the file.

unraid-diagnostics-20210103-2051.zip

[itimpi]

The syslog reports lots of read errors on disk2, and there is no SMART information for the drive suggesting it has dropped offline for some reason.

[sloob]

Just now, itimpi said:

The syslog reports lots of read errors on disk2, and there is no SMART information for the drive suggesting it has dropped offline for some reason.

It's strange because the issue is with disk1 (the unmountable:No file system error appears on disk1, not disk2)

[trurl]

34 minutes ago, sloob said:

It's strange because the issue is with disk1 (the unmountable:No file system error appears on disk1, not disk2)

So you have 2 different problems that both need to be fixed. Check connections and post new diagnostics.

[sloob]

On 1/3/2021 at 9:53 PM, trurl said:

So you have 2 different problems that both need to be fixed. Check connections and post new diagnostics.

Aight, I stopped the server and opened it up and made sure every cables were connected, started the array and this morning same problem, read errors on disk 2 and 3

unraid-diagnostics-20210109-0936.zip

[trurl]

Getting conflicting information from those diagnostics. Post a screenshot of Main - Array Devices

[sloob]

Just now, trurl said:

Getting conflicting information from those diagnostics. Post a screenshot of Main - Array Devices

Here it is