Help with dead USB drive & data recovery


Recommended Posts

I am aware of how air-headed this all was, let me be the poster child as to how not backing up is bad.

 

My Missteps

- My USB drive holding my OS configs died. Hardware failure. Even tried a data recovery firm.

- I rearranged drives 6 months ago and forgot to reconfigure my backup schedule properly.

 

What I have to work with

- I have an old backup (6+ months) that has my old drive assignments with a different parody and such

- A screen shot of my current drive assignments before the USB failure and post-drive rearrangement.

 

What do I do? 

A) Make a new USB with my old confg, map my drives to its most current positions, pre-USB failure, and all data on the array will be recovered automagically? 🤞

B) Start over? 😢

C) Some other thing I am unaware of?

 

Hopefully someone will verify and/or change my suspicions that option A is the way to go before I try it tonight. 

 

I've been an avid Unraid'r for 5+ years and this community has always been great so thank you ahead of time.

Link to comment

Take a deep breath and decide to try things slow and easy.  First, you have not lost any data.   You may lose your plugin setups, Dockers setups and VM setups.  These will most likely have to be rebuilt.

 

First, what luck did the data recovery firm have?  The most important directory that would be needed for recovery is the  config  directory/folder.  But even without that we should be able to get the server back running with all of its data intact.

 

So begin by telling us what you remember.  How many parity drives, how many cache drives and how many data drives?   You can usually figure which drives are the cache drives as most folks uses smaller capacity drives or ssd's for them.  

 

If you have purchased your Unraid license, do you have the .key file saved somewhere where you can find it.  If you don't have the file, you can start here to get a replacement key:

 

    https://wiki.unraid.net/UnRAID_6/Changing_The_Flash_Device#Manual_Method_.28Legacy.29

 

At the bottom of the page, you will find a link to LimeTech support to discuss getting a replacement key file if none of the options on this page are applicable.

  • Like 1
Link to comment

My plugins/dockers/vms can all burn, thats ok with me.

 

As mentioned above, I have a screen shot of how my drives were properly assigned. If it helps, I had dual parody disks, 1 cache drive & 9 array drives. (All drives are encrypted)

 

Key file is backed up. Yes, will need updated with the new USB drive.

 

I do have an older config/ folder I can use to get things started. 

Link to comment

So I add all drives (1 cache, 2 parody, 9 array) to the slots exactly as it once was, this should work.

 

One more question, do I start this with my old `/config` applied onto a fresh copy of unraid or not?

 

Wouldn't it warn about starting array with a new config and force a parody rebuild? Which is fine in the case that it keeps the data already on the disks.

 

Two questions actually, do I need my old `/config` in regards to anything of my array encryption key?

Edited by MadeOfCard
Link to comment

One of the things in your old config that you don't want to bring forward is the disk assignments if they have changed since that backup of config was taken. More than one user has started up with an old backup only to find that they have made that critical mistake I mentioned, assigned a data (former parity) disk to a parity slot.

 

Your disk assignments are in config/super.dat, and if you delete that from config before booting it will make you assign your disks. Whether the rest of that old config will be exactly what you need is another question, but you should be able to work through that from the webUI without any data loss.

 

Just one more thing. If for some reason Unraid offers to FORMAT anything, DON'T DO IT! Come back here for further advice.

 

Link to comment

OK---  My recommendation is to add the cache drives as cache drives.  Then add all of the Data and Parity Drives as DATA drives!  Then start the array.  Two (and ONLY two) of the drives will be unformatted and not mounted as DATA drives.  These are your parity drives.  If there are more or fewer than two drives in this condition,   STOP!     and ask for more help!!!!

 

If there are exactly two drives unformatted, stop the array and assign one drive as parity 1 and the other drive as Parity 2.  Start the array and a parity rebuilt will start.  It should finish without a problem and you will have a protected array again. 

 

(The reason, you don't want to assign a drive as parity to begin with is that as soon as the array starts,  a parity rebuilt will begin and if that drive happens to be a data drive, you will lose all of the data instantly!) 

 

EDIT: (Order of the parity drives are unimportant as parity will have to be rebuilt in any case.  This is specfically true with Parity 2 as its calculations are based on data location in the data matrix.) 

 

 

Edited by Frank1940
  • Like 1
Link to comment
57 minutes ago, MadeOfCard said:

I have a screen shot of how my drives were properly assigned.

7 minutes ago, Frank1940 said:

Order of the parity drives are unimportant as parity will have to be rebuilt in any case

If you are absolutely sure you know which drives are parity then you can avoid rebuilding as I mentioned, but the order of parity IS important if you aren't going to rebuild.

 

Link to comment

Full array recovery!

 

NOTE: The step missing from the above method in this case was:

My disks were all XFS (encrypted), not the default XFS. Set the default formatting to the appropriate under Disk Settings then reassign them.

 

Thanks to @Frank1940@trurl and whomever Tom is from support for the very quick response on getting my registration key. I owe you guys a 🥩 dinner. 

Link to comment
  • 2 years later...

Hello,

Apologies for resurrecting an almost 3 year old thread; but I am in an identical situation:
My USB Flash Drive became corrupted and the files on it cannot be recovered.  I did not make a backup of my config. 

I do have emails of my daily status checks so I have a good idea of which drives are parity, cache and data.  

 

I was running  6.9.2 I believe.

 

I am trying to follow the suggestions in this thread but I am confused on how to recover my user shares?  I get that the data may possibly be still on the drives, but how do I get those files attributed to my shares?

 

Thanks in advance for any suggestions.

Link to comment

In addition to your disk assignments, your lost config also contained your license, and every setting you made in the webUI. So you will have to setup your dockers from scratch, but if you do them exactly as you had them before then they will find their appdata and the applications will work as before.

Link to comment
26 minutes ago, trurl said:

In addition to your disk assignments, your lost config also contained your license, and every setting you made in the webUI. So you will have to setup your dockers from scratch, but if you do them exactly as you had them before then they will find their appdata and the applications will work as before.

Thanks for that.  Fortunately I do have a backup copy of my license file.  I will have to figure out how to get the docker stuff reinstalled.... I recall it was a bit involved the last time I did it, but as long as I can get my data back it should be ok.

 

Thanks again!

Link to comment

Hello,

I wanted to update anyone interested in this topic.

Following the wonderful suggestions in this thread I was able to recover everything important.  My failing USB drive still had enough life left that I could copy the following files:
disk

super

shadow

 

I lost everything else, and after each single file copy, the USB would unmount....  Regardless, when I copied them into my newly created USB flash disk, everything just sort of worked!!
Since I had never requested a new key I was able to request a key right from the GUI and one was sent to my e-mail for the new USB disk.

I am going to set up a scheduled task to make automatic backups of my USB drive from now on.

Thanks again for your assistance trurl! 

  • Like 1
Link to comment
  • 1 year later...
On 2/6/2019 at 8:34 PM, Frank1940 said:

OK---  My recommendation is to add the cache drives as cache drives.  Then add all of the Data and Parity Drives as DATA drives!  Then start the array.  Two (and ONLY two) of the drives will be unformatted and not mounted as DATA drives.  These are your parity drives.  If there are more or fewer than two drives in this condition,   STOP!     and ask for more help!!!!

 

If there are exactly two drives unformatted, stop the array and assign one drive as parity 1 and the other drive as Parity 2.  Start the array and a parity rebuilt will start.  It should finish without a problem and you will have a protected array again. 

 

(The reason, you don't want to assign a drive as parity to begin with is that as soon as the array starts,  a parity rebuilt will begin and if that drive happens to be a data drive, you will lose all of the data instantly!) 

 

EDIT: (Order of the parity drives are unimportant as parity will have to be rebuilt in any case.  This is specfically true with Parity 2 as its calculations are based on data location in the data matrix.) 

 

 

 

On 2/6/2019 at 8:44 PM, trurl said:

If you are absolutely sure you know which drives are parity then you can avoid rebuilding as I mentioned, but the order of parity IS important if you aren't going to rebuild.

 

 

I have this problem right now.

 

My question is: I know what 2 drive are parity, but NOT in what order the data drives where before. is this important? (The data drive mount order)

Link to comment
10 minutes ago, holler said:

I know what 2 drive are parity, but NOT in what order the data drives where before. is this important? (The data drive mount order)

Parity 1 will be correct.  If you do not know the exact arrangement of the data drives, Parity 2 will not be correct.  (Parity1 is a simple XOR operation of all the data for each bit in the array.  Parity 2 is a complex matrix calculation of those bits and the result of that calculation is determined by the order of the disks in the array.)  (These statements assume that you know which drive was parity 1 and which drive was parity 2.) 

 

You now know which two disks are parity and the remaining disks are the data disks and, if you don't know the order of the assignment for each parity disk, you are left with one option,  Assign the unformatted disks as parity disks and all of the other disks as data disks.  Then start the array and allow parity to rebuilt.  (It will take 2-3hrs per TB of the largest parity disk.)

 

If you should happen to know which disk (Take a look at the 'Power on hours' in Attributes page for the parity drives for a possible clue!)  was assigned as parity1, you could follow this procedure:

 

        https://docs.unraid.net/unraid-os/manual/storage-management/#reset-the-array-configuration

 

IF you do use the 'Parity is Valid' check box, the array will only have single parity until you can run a parity check with the 'Write corrections to parity' box checked.

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.