Both cache drives read "unmountable: no file system (no btrfs devices)" and appdata backup empty

[ainuke]

I am so frustrated right now I can't even see straight.

 

Inability to connect to Plex Server (docker) got me to check on unraid, and all Dockers were offline. I don't know how I got to it but found procedure to recreate docker image. However, that's when I saw that both cache drives (two SSDs mirrored) were showing No File System.

 

Attempts to "mount -o degraded,usebackuproot,ro /dev/sdb1 /x" on either drive produced "mount: /x: can't read superblock on /dev/sdb1."

 

I've had one bad SSD in the cache pool (mirrored) make both drives unreadable before, so I reformatted the newer one, expecting to restore the appdata to it and go on about my merry way. Except the appdata folder in my backup location was empty. There was a dated folder, but it contained 0bytes and no other folders, files, or anything. I have been using Backup/Restore Appdata for as long as I've used unRaid, and get Prowl notifications every morning that appdata backup has happened.

 

I don't know what happened to start this all, but my expectation was that using two cache drives meant that if one failed, the other would continue and I could replace the failed one and continue on as if nothing had happened. I also expected that the daily affirmations that my appdata was being backed up meant that  it would be there in case of total cache pool failure. Neither of these assumptions appear to have been correct.

Am I missing something?

 

So, do I have any other means to repair the remaining cache drive that is showing no file system? I can restore all the dockers to functional status, but the couple Maria databases contained in the appdata will be lost, as will the entirety of the Minecraft worlds that my kids use. The rest is just a PITA to reconfigure (Plex, Flarum, Homebridge, AMP, etc.).

 

But the even larger issue is just how reliable is this OS, anyway. I've now had two critical cache failures, and a backup reporting system that's useless. What happens when I lose a data drive? Is my expectation that the two parity drives I've invested in are going to be able to recover my array *also* a fantasy?

 

I'm seriously considering switching back to WHS2011; it wasn't as sexy as unRaid, but it also never gave me these kinds of problems. 

 

 

[JorgeB]

12 hours ago, ainuke said:

so I reformatted the newer one, expecting to restore the appdata to it

Reformatting is never part of recovering data, with an array or pool drive.

 

12 hours ago, ainuke said:

So, do I have any other means to repair the remaining cache drive that is showing no file system?

Please post the diagnostics.

[ainuke]

11 hours ago, JorgeB said:

Reformatting is never part of recovering data, with an array or pool drive.

 

Please post the diagnostics.

Thanks for the response; and sorry for the OP sounding like such a rant.

 

Understood about reformatting; it was my only path to a functional server, on which my home network depends. I had hoped that I could restore appdata from backup; but rightfully should've checked beforehand.

 

will post diagnostics this evening.

[ainuke]

On 6/1/2022 at 4:22 AM, JorgeB said:

Please post the diagnostics.

Sorry for the delay; life happened.

 

EDIT: This is the current state diagnostics, with the reformatted cache drive (one only) and just a couple dockers. I can re-install the remaining drive that provoked the "no file system" error and re-submit the diagnostics if needed.

orthanc-diagnostics-20220603-1306.zip

Edited June 3, 2022 by ainuke
added detail

[JorgeB]

Yes, install the other one but don't assign it, then post new diags.

[ainuke]

I didn't reboot; let me know if you want to.

The drive in question is dev2.

orthanc-diagnostics-20220603-2030.zip

[ainuke]

well, crap.

I rebooted, and now it's pooling the "bad" drive with the reformatted drive.

I expect the appdata on the bad one is gone, gone, gone. If it was recoverable to begin with...

 

[JorgeB]

5 hours ago, ainuke said:

now it's pooling the "bad"

Plase post new diags.

[ainuke]

well, crap.

I rebooted, and now it's pooling the "bad" drive with the reformatted drive.

I expect the appdata on the bad one is gone, gone, gone. If it was recoverable to begin with...

 

orthanc-diagnostics-20220604-1611.zip

[JorgeB]

11 hours ago, ainuke said:

I expect the appdata on the bad one is gone, gone, gone.

I'm afraid it is, device was wiped and added to the new pool.

[ainuke]

12 hours ago, JorgeB said:

I'm afraid it is, device was wiped and added to the new pool.

Is there any indication as to why it all went south in the first place?
And should I not be getting redundancy from having two cache drives?

[JorgeB]

10 hours ago, ainuke said:

Is there any indication as to why it all went south in the first place?

Can see that maybe if you saved the diags before formatting the device.

 

10 hours ago, ainuke said:

And should I not be getting redundancy from having two cache drives?

When you connected the other device a btrfs fs was found:

 

Jun  3 20:18:37 Orthanc kernel: BTRFS: device fsid 6bf8119c-9c5e-42c8-953b-925fbec0bb9b devid 4 transid 11646266 /dev/sdl1 scanned by udevd (5687)

 

So if it hadn't been added to the new pool all data should have been recoverable, assuming it was a redundant pool in the first place.