Issue with Unraid Format/Restore After Adding Disks


Recommended Posts

Hey Everyone,

 

I recently added two new disks to my Unraid array (giving me 5 total drives-4 plus parity).  Upon starting the array, I believe I formatted all new disks by accident not realizing what I was doing.  Shortly thereafter, I used the restore function which I understand has reset my parity drive (I think I hit it two separate times). 

 

Before I really understood what was happening, I thought my issues were due to a filed drive because 1 of the 2 drives I added was red.  Therefore, I replaced one of them with a larger drive.  So I added two drives, formatted, restored, and replaced one drive with a larger drive.  My current state is that the same drive I replaced is still red, although it did recognize that I replaced it with a larger one.

 

I found some useful information on the forums but I have some questions before I do anything:

 

I want to follow the instructions posted by Joe L. here (http://lime-technology.com/forum/index.php?topic=6008.msg57379#msg57379), but within his post he is advising me to use the Trust-My-Parity process (http://lime-technology.com/wiki/index.php?title=Make_unRAID_Trust_the_Parity_Drive,_Avoid_Rebuilding_Parity_Unnecessarily).  My issue with this is this statement:

 

It should NEVER be used if you have a failed disk or are in the process of replacing a failed disk.

It should NEVER be used if you are replacing a drive with one of a larger size.

 

I feel that I fit into either one or both of those categories which means I don’t even know where to begin.  Can anyone please offer some suggestions?  I did e-mail Unraid support but have not heard anything back after a couple days so I thought I would post here.

 

I attached a screen shot.  Disk3 is the disk I replaced with a larger one, Disk4 is the other disk that was added.

Screenshot.JPG.b8521bd3f928afa603fec32c74df0408.JPG

Link to comment

Hey Everyone,

 

I recently added two new disks to my Unraid array (giving me 5 total drives-4 plus parity).  Upon starting the array, I believe I formatted all new disks by accident not realizing what I was doing.  Shortly thereafter, I used the restore function which I understand has reset my parity drive (I think I hit it two separate times). 

 

Before I really understood what was happening, I thought my issues were due to a filed drive because 1 of the 2 drives I added was red.  Therefore, I replaced one of them with a larger drive.  So I added two drives, formatted, restored, and replaced one drive with a larger drive.  My current state is that the same drive I replaced is still red, although it did recognize that I replaced it with a larger one.

 

I found some useful information on the forums but I have some questions before I do anything:

 

I want to follow the instructions posted by Joe L. here (http://lime-technology.com/forum/index.php?topic=6008.msg57379#msg57379), but within his post he is advising me to use the Trust-My-Parity process (http://lime-technology.com/wiki/index.php?title=Make_unRAID_Trust_the_Parity_Drive,_Avoid_Rebuilding_Parity_Unnecessarily).  My issue with this is this statement:

 

It should NEVER be used if you have a failed disk or are in the process of replacing a failed disk.

It should NEVER be used if you are replacing a drive with one of a larger size.

 

I feel that I fit into either one or both of those categories which means I don’t even know where to begin.  Can anyone please offer some suggestions?  I did e-mail Unraid support but have not heard anything back after a couple days so I thought I would post here.

 

I attached a screen shot.  Disk3 is the disk I replaced with a larger one, Disk4 is the other disk that was added.

 

At this point, don't even bother with the trust my parity process UNLESS you can put the exact same disks back into place as when parity was last fully calculated AND only if you have not written to any of the drives since then (formatting included, as it writes to the drives)    You can use the procedure, but parity will be useless to recover what was on the drives. 

 

I think you've basically lost whatever was on the failed drive... but if by chance it is readable or just has a bad cable connection to it, I might be wrong, but it is unlikey.  You might be able to unplug the drive now, and deal with it AFTER you get the other data on the other drives recovered as best as possible by using the rebuild-tree together with the command to scan the entire drive.

 

I am guessing you are using version 4.5.3 of unRAID.  It has a MAJOR bug which under some conditions will show all the drives as un-formatted and subsequently format them all when you press the format button.  Am I right?

 

Also... NEVER use the button labeled as "Restore" unless you are intending to initialize a new disk configuration and want to immediately invalidate the parity calculations base don the prior disk disk configuration.  The button is very poorly labeled, it has been removed from the unRAID interface in later releases of unRAID as too many users, like yourself, press it thinking it has something to do with restoring their data.  Just the opposite, it invalidates parity making re-construction impossible.

 

Oh yes, upgrade to 4.5.6 once you get stable to eliminate the bug you've encountered.  If you power up and see all the drives as "un-formatted, simply stop the array by pressing "Stop" and then re-start it by pressing "Start"

 

The drives should all them mount and not show as un-formatted unless they really are un-formatted.

 

Joe L.

Link to comment
At this point, don't even bother with the trust my parity process UNLESS you can put the exact same disks back into place as when parity was last fully calculated AND only if you have not written to any of the drives since then (formatting included, as it writes to the drives)    You can use the procedure, but parity will be useless to recover what was on the drives.

 

It is possible for me to put the exact same disks back to the last operational setup and I have not written anything to any drives since this happened.  What benefit will I get from doing this?  You mention that the parity will still be useless to recover what was on the drives...

 

I think you've basically lost whatever was on the failed drive... but if by chance it is readable or just has a bad cable connection to it, I might be wrong, but it is unlikey.  You might be able to unplug the drive now, and deal with it AFTER you get the other data on the other drives recovered as best as possible by using the rebuild-tree together with the command to scan the entire drive.

 

Luckily, the failed drive is one of the two new drives I was adding.  In the screen shot, disk3 and disk4 were newly added.

 

I am guessing you are using version 4.5.3 of unRAID.  It has a MAJOR bug which under some conditions will show all the drives as un-formatted and subsequently format them all when you press the format button.  Am I right?

 

I don't want to turn it on to check at this point but I am guessing your right about the version.  You're definitely right that it listed all drives as unformatted.

 

Oh yes, upgrade to 4.5.6 once you get stable to eliminate the bug you've encountered.  If you power up and see all the drives as "un-formatted, simply stop the array by pressing "Stop" and then re-start it by pressing "Start"

 

I will definitely do this whether I recover my data or not.

 

 

So, based on the very first thing you said... I can configure my system back to when it was working (2 data drives plus 1 parity drive).  No new data has been written to the array since.  What will this do for me?  I couldn't tell from your response if it would lead to potential data recovery.

 

Thanks for the response,

Matt

Link to comment

You basically need to follow the steps to run reiserfsck with the rebuild-tree and scan-entire-drive options on each of the two data drives that had data you wish to recover. 

 

Don't bother with anything else at this time.

 

Once you've un-did as much of the damage the formatting caused you can then deal with the new drive that is un-writable (for whatever reason)

Don't bother with trusting parity at this time as it is not involved in getting you back the data on the failed drive. (As you said, there's nothing on that drive)

 

Since you've formatted the drives you will need to rebuild their respective file-trees.  (Their directory structures pointing to the data blocks containing your files)

 

Whatever you do, DO NOT WRITE ANY NEW FILES TO THE DRIVES YOU ACCIDENTALLY FORMATTED.

 

You will need to perform this set of steps for each of the drives you accidentally formatted (I'll use disk1 and disk2 as an example, change the number for each of your other disks in turn):

 

Turn off any spin-down timers.  We do not want the disks to be spun down in the middle of the repair.

Turn off any sleep/suspend screen savers on your PC if doing this via telnet.  Don't want it shutting down in the middle of a repair step.  It might be better if the following commands are typed on the unRAID system console directly, but as long as your PC does not shut itself down, you'll be fine using telnet.

 

Stop samba (you will not be able to un-mount the disk if it is busy... and samba will keep it busy)

/root/samba stop

un-mount the drive. 

umount /dev/md1

Rebuild the file-tree scanning the entire disk for files.

reiserfsck -–scan-whole-partition -–rebuild-tree /dev/md1

Respond with "Yes" when prompted... Upper-case "Y", lower case "es"  The scan can take many hours for a large drive. 

 

Re-mount the drive:

mount /dev/md1 /mnt/disk1

 

un-mount the next drive. 

umount /dev/md2

Rebuild the file-tree scanning the entire disk for files.

reiserfsck -–scan-whole-partition -–rebuild-tree /dev/md2

Respond with "Yes" when prompted... Upper-case "Y", lower case "es"  The scan can take many hours for a large drive. 

 

Re-mount the drive:

mount /dev/md2 /mnt/disk2

 

Do the above steps for each of your disks, then....

when all done, you should be able to re-start samba

/root/samba start

and see if your files have been restored.  Files in the top-most directory will probably end up in a lost+found directory created by the restore process.  You'll need to re-name them to their correct names.  Files lower in the directory hierarchy will probably have their correct names.

 

Link to comment
un-mount the drive.

umount /dev/md1

 

When I type this, I get this message: 

umount: /dev/md1: not found

 

Rebuild the file-tree scanning the entire disk for files.

reiserfsck -–scan-whole-partition -–rebuild-tree /dev/md1

 

Thinking that the "umount: /dev/md1: not found" error might have meant is was already un-mounted (I don't know any better), I skipped to the next step.

 

When I type this command, then type "Yes", I get this:

Failed to open the device '/dev/md1' : No such file or directory

 

It then goes back to where it started:

root@matts-server :~#

 

 

Any suggestions for this?

Link to comment

Well it's looking like almost all of my files have been restored!  Thanks for the advice!  Hard to tell for sure at this point because so many are in random folders in the "lost+found" directory.

 

I have a couple more questions now:

 

First, I'm scared to touch anything... can I re-create my shares and begin to move files to where I want them?  Currently I see 4 folders (disk1, disk2, disk3, & disk4).  If I create a share, can I drag from the disk folders to the share folder?

 

Second, I still have a disk that is red.  It's disk3 but I still see disk3 when I am viewing my server in Windows Explorer.  Should I figure out how to address this problem before I begin moving files around?  What do you think I should do here?

 

Thanks again!

Link to comment

Well it's looking like almost all of my files have been restored!  Thanks for the advice!  Hard to tell for sure at this point because so many are in random folders in the "lost+found" directory.

 

I have a couple more questions now:

 

First, I'm scared to touch anything... can I re-create my shares and begin to move files to where I want them?  Currently I see 4 folders (disk1, disk2, disk3, & disk4).  If I create a share, can I drag from the disk folders to the share folder?

 

Second, I still have a disk that is red.  It's disk3 but I still see disk3 when I am viewing my server in Windows Explorer.  Should I figure out how to address this problem before I begin moving files around?  What do you think I should do here?

 

Thanks again!

You want to first enable user-shares.  That will automatically create user-shares for all the top level directories.  Many of use create just a few  (I have Movies, Pictures, Mp3, and "data")  I organize my files under those main folders.

 

disk3 is being emulated from parity and the other data disks.  It sounds as if the disk itself is defective or the cabling to it or the port it is connected to is defective. 

 

On each of disk1, disk2, disk3, you can move the files as you desire.  You currently need to get disk3 working before you can have parity re-calculated and be protected from another disk failure.

 

Are there any files on disk3?  (at one point you said it was empty... Is it still empty?)

 

Joe L.

Link to comment
Are there any files on disk3?  (at one point you said it was empty... Is it still empty?)

 

Yes, disk3 is still empty.  I am not sure what is going on with this disk.  When I started this whole mess, I added disk3 and disk4... disk3 was a 1TB drive originally (disk4 was a 1.5TB drive and works fine).  I thought disk3 was a failed disk (appeared red), so I replaced it with a new, 1.5TB drive.  It is still red.  Maybe this means the cabling or port is defective?  I think I have one more SATA port available on the motherboard, should I shut down and plug disk3 into another port?

 

On each of disk1, disk2, disk3, you can move the files as you desire.

 

So you are saying I can move from a disk folder to a share folder, right (after I create my shares)?  My question/concern here is that the disk folder is one hard drive, but the share folder may be multiple hard drives... Unraid will figure this part out?

Link to comment

You can limit user shares to as many or as few disks as you prefer. When I first started a few weeks ago, I was anal about where data went, because I was used to having to manage every disk. Now I have my user shares set up the way I want them, and pretty much let the chips fall where they may. I generally do try to keep things sorted so I'm minimizing disk spinup, but other than that, I'm not too fussed.

Link to comment

So I was able to restore almost all of my files and re-organized them back into the shares they were in.  Thanks for all the help.

 

However, now I have upgraded to 4.5.6 and when I plugged my flash drive back in all of my settings were lost.  The server name has gone back to "Tower" and I don't have the ability to create user shares.  When I go to the Shares section, there is nothing but white space below the Export Settings box.

 

The Main page looks like it did before (I had to go in and re-assign the drives like the readme instructed) and all my files are viewable in the disk folders.  I just can't create shares...  I attached my syslog in case that will be any help.

syslog.txt

Link to comment

Nevermind on the shares questions.  I see now that I needed to enable them.  I'm still curious why it reset all my settings but if it's not causing any issues I guess it's not a big deal.

 

After the issues I had, if anyone wants to browse my syslog I posted previously and check for potential problems I would really appreciate it.

 

Thanks!

Link to comment

Nevermind on the shares questions.  I see now that I needed to enable them.  I'm still curious why it reset all my settings but if it's not causing any issues I guess it's not a big deal.

 

After the issues I had, if anyone wants to browse my syslog I posted previously and check for potential problems I would really appreciate it.

 

Thanks!

To upgrade you only needed to copy the bzroot and bzimage file from the new release to the root of your flash drive and to reboot.  It sounds as if you copied the config folder too.

 

It is not an issue, but that would explain why you needed to re-assign the disks, enable user-shares, and set the server name.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.