Guys, I messed something up bad...

[LakersFan]

Hey all,

Let me just start by saying I've been rolling along with Unraid for 10 years without an issue. No joke... I bought my key on 9/14/2012

But today something happened and I have no idea where it began. I ran a User Script (one that I've ran multiple times) and it froze up Unraid to the point where I got a internal server error when trying to login in. I made the mistake of hard resetting my PC and that was clearly a bad idea. When restarting it, it wouldn't boot. So I hooked up the monitor to find it was starting into the bios. Weird... I looked at my flash drive and it appears many files were missing. I tried to troubleshoot based off what I could from this website. Ultimately, I reflashed my drive and replaced the "leftover" files from my (what was once) working drive and put them on the newly installed flash drive. Some things returned, like nfo and png files from movies, but not the mp4's/mkv's. Docker's looks to have returned, but I no longer have the option to login into any of their GUI's. I did make a backup of the "fubared" flash drive before i erased it for the new. Hopefully something can be done. I made a diagnostics file of where's I'm at right now. And like I said, I have the old contents of the flash drive that I backed up "as-is" if needed. I *think* I may have a backup of my flash drive, but of course it's on my array that i can't access.

 

Any help would be appreciated. Thanks all!

tower-diagnostics-20220911-2029.zip

[Frank1940]

Try starting the server in the 'Safe' mode.  (This is one of the boot options.   )

 

If it starts, get another diagnostics file and upload that.

 

As a piece of information, do you have a printout of the disk assignments?   (If you have "Notifications" set up for the array status report, you will find this in that notice.)  

 

EDIT:  For @LakersFan and others who are making a backup of the Flash Drive to the array.   This is like storing the combination to a safe inside of the safe.  When you need it, you can't get to it.   Make a backup of your flash drive by clicking on the 'Flash" on the Main tab

Then click on "FLASH BACKUP"

This will save the backup to your PC (or whatever device you are accessing the GUI from).

Edited September 12, 2022 by Frank1940

[trurl]

13 hours ago, LakersFan said:

I *think* I may have a backup of my flash drive, but of course it's on my array that i can't access.

Don't understand this part since you seem to have your array up and working.

 

Everything about your configuration (any settings from webUI) is in the config folder of flash/old flash/flash backup

[kizer]

If your having thumbstick issues you can pull it from your server and plugin to another PC and back it up. /config folder is the most important. 

[LakersFan]

14 hours ago, Frank1940 said:

Try starting the server in the 'Safe' mode.  (This is one of the boot options.   )

 

If it starts, get another diagnostics file and upload that.

 

Attached, thank you for looking at it.

 

Quote

As a piece of information, do you have a printout of the disk assignments?   (If you have "Notifications" set up for the array status report, you will find this in that notice.)  

Where would I find the array status report?

 

 

 

9 hours ago, trurl said:

Don't understand this part since you seem to have your array up and working.

 

Everything about your configuration (any settings from webUI) is in the config folder of flash/old flash/flash backup

 

8 hours ago, kizer said:

If your having thumbstick issues you can pull it from your server and plugin to another PC and back it up. /config folder is the most important. 

Thanks guys. Attached is a pic of what's in my config folder that was on the flash drive when it stropped working... Does it look complete? What can I do with it?

 

 

tower-diagnostics-20220912-1924.zip

[trurl]

9 hours ago, LakersFan said:

config folder that was on the flash drive when it stropped working

Have you tried to access the flash backup you think is on the array? How current is it? It might be more trustworthy than files from the flash that quit working.

[Frank1940]

9 hours ago, LakersFan said:

Where would I find the array status report?

In a notification sent to another device.  It is setup here:

 

Does it look like all of the drives are present and the data is present and in the proper shares?  (I suspect everything is...)   Here is the current drives assigned (From the Diagnostics file you uploaded):

 

It is showing single parity, *six* data drives and a cache drive.   If all of this looks correct, then you have a corruption in one the files on the flash drive for one of your Plugins or Dockers.  

 

Have you made a backup of contents of the current flash drive?   IF not, do it NOW before you proceed any further!!!!

 

Have you run a chkdsk on that flash on your PC?  IF not, do so.

 

[Frank1940]

14 hours ago, LakersFan said:

 

Thanks guys. Attached is a pic of what's in my config folder that was on the flash drive when it stropped working... Does it look complete? What can I do with it?

 

 

 

Where is your    .key       file (the Unraid license file)?  It should be in this folder.  See if you can find a copy of it or the link to it.

[LakersFan]

16 hours ago, trurl said:

Have you tried to access the flash backup you think is on the array? How current is it? It might be more trustworthy than files from the flash that quit working.

Unfortunately I'm unable to locate it. Here's the issue ... had it on my PC. Then one day I decided to set up Nextcloud. Moved everything over to my Nextcloud account and completely forgot I had a backup of it (that was now moved over to Nextcloud). Haven't needed it in 10 years. Dumb mistake? Absolutely.

[LakersFan]

15 hours ago, Frank1940 said:

In a notification sent to another device.  It is setup here:

Unfortunately I do not have an array status report.

 

 

15 hours ago, Frank1940 said:

Does it look like all of the drives are present and the data is present and in the proper shares?  (I suspect everything is...)   Here is the current drives assigned (From the Diagnostics file you uploaded):

 

It is showing single parity, *six* data drives and a cache drive.   If all of this looks correct, then you have a corruption in one the files on the flash drive for one of your Plugins or Dockers.

It looks accurate from my recollection.

 

 

15 hours ago, Frank1940 said:

Have you made a backup of contents of the current flash drive?   IF not, do it NOW before you proceed any further!!!!

 

Have you run a chkdsk on that flash on your PC?  IF not, do so.

Yes, I have a backup of the flash drive when I first pulled it out. I did run a disk scan on it and there were errors that it said were corrected.

 

 

11 hours ago, Frank1940 said:

 

Where is your    .key       file (the Unraid license file)?  It should be in this folder.  See if you can find a copy of it or the link to it.

It wasn't in there when I first pulled it out. When trying to fix this on my own, I had to request a new key.

[LakersFan]

One thing I'd like to add is I noticed the only media I have seems to be on Disk 6 (and it looks like it's not all there ... but honestly can't remember how full the HDD was). Tonight I took my flash drive and replaced the config file that I initially pulled off it when this all started. I also added the new key that I received the other day when I was troubleshooting. It boot up and it appears to be doing a "Parity check" right now. The media is still mostly missing, but hopefully it'll rebuild?

 

I really just want the home movies. The rest is garbage collected over the years lol

Edited September 14, 2022 by LakersFan

[trurl]

7 hours ago, LakersFan said:

replaced the config file

Do you actually mean the config folder? There is no file named config on flash. There is a config folder that contains all of your configuration, which is any settings made in the webUI.

 

7 hours ago, LakersFan said:

It boot up and it appears to be doing a "Parity check" right now. The media is still mostly missing, but hopefully it'll rebuild?

No possible way parity check can "rebuild" any data since all it does is check parity, which contains none of your data. Automatic parity check you get from unclean shutdown doesn't even correct parity, it just checks it.

 

Looking at those earlier diagnostics, all your array disks appear to be empty or mostly so. Are you sure you didn't agree to format anything? What was the purpose of that rogue script you mention in OP?

 

You must have left out some important details since nothing you have mentioned so far should have resulted in any data loss.

 

[trurl]

Do you have backups of anything important and irreplaceable?

[LakersFan]

14 hours ago, trurl said:

Do you actually mean the config folder? There is no file named config on flash. There is a config folder that contains all of your configuration, which is any settings made in the webUI.

 

No possible way parity check can "rebuild" any data since all it does is check parity, which contains none of your data. Automatic parity check you get from unclean shutdown doesn't even correct parity, it just checks it.

 

Looking at those earlier diagnostics, all your array disks appear to be empty or mostly so. Are you sure you didn't agree to format anything? What was the purpose of that rogue script you mention in OP?

 

You must have left out some important details since nothing you have mentioned so far should have resulted in any data loss.

 

Yes sir, sorry. I meant the config folder. You’re correct in that parity had zero errors and nothing was rebuilt. I’d like to think I’m a pretty astute guy when it comes to PC’s. I know I didn’t agree to format anything for sure. I clicked the user script I created a while ago and it ran differently then it had before… it started checking for empty folders then it buckled my server. I couldn’t log into the tower either via telnet or using a browser. So I had to hard boot it. The script was a simple script to sync an EPG. I’ll see if I can find it. Nothing crazy though. No clue what happened. I’m anonymous here, I’d tell you if I did something wrong. 
 

Bottom line, is there any other troubleshooting, rebuilding or recovery I can try? I’ll send the drives off if some wizard out there can find the home videos folder on it and recovered any files that were seemingly deleted. If not, is what it is. Life moves on. I do have some backed up, but not all. Lesson learned. 

[LakersFan]

Ok, I take the “astute” part out of the above post. I reviewed the script I used and this is my fault. I had an “-mtime +7 -delete” command to remove a daily backup after seven days. Above that was the directory I was supposed to have it directed to, however I commented that line out (and another accompanying line). So I guess it deleted anything and everything over 7 days. Damn.

 

Welp, I assume they’re gone forever, but need someone to confirm it’s a lost cause so I can move on. Thanks guys, appreciate the support!

[Frank1940]

5 minutes ago, LakersFan said:

Ok, I take the “astute” part out of the above post. I reviewed the script I used and this is my fault. I had an “-mtime +7 -delete” command to remove a daily backup after seven days. Above that was the directory I was supposed to have it directed to, however I commented that line out (and another accompanying line). So I guess it deleted anything and everything over 7 days. Damn.

 

Welp, I assume they’re gone forever, but need someone to confirm it’s a lost cause so I can move on. Thanks guys, appreciate the support!

 

Be careful; at this point.  There are utilities that can recover deleted files (sometimes).  You need to stop using that server and make absolutely sure not write anything more to those data disks. 

 

Each data disk (in an Unraid array) is formatted using a standard Linux file system.  Each file is written to a single disk using the standard Linux write function for that file system.  In fact, any Unraid data disk can be read with any standard Linux installation.  

 

Look at the file system that your data disks are formatted with.  (There are only three options--- reiserfs, XFS and btrfs.)   After you find which ones you are using, You can make a an investigation as to whether you want to try to recover them yourself or get a data recovery company to do it.  (You might never get everything back but often a fair  percentage is better than nothing.)

[LakersFan]

Thanks Frank, they’re all XFS. I’ll start researching recovery. 

[trurl]

https://www.ufsexplorer.com/

 

http://www.reclaime-pro.com/

 

No experience with either

[LakersFan]

Thanks! I also found this post where someone had some luck.. say a prayer for me! lol

 

[trurl]

That was the other one I couldn't remember.

[LakersFan]

So far so good. I was able to recover several home vids from the first HDD I tried (using the link above). Thanks again guys!

Edited September 16, 2022 by LakersFan