This isnt the end of the world, but considering I have an extensive movie collection, I dont know what happened to my data. 

 

The succession of events:

 

1.  Wanted to add a new drive to my array. So I plugged that in yesterday and began the preclear disk function on it.  It finished this morning.

2.  This afternoon, I then tried to add it to my array - Plugged it in, identified it in the devices page, and selected it. 

3. When I clicked onto the main page, it showed it correctly and it was listed as UNFORMATTED.  This was the only drive that said UNFORMATTED.  The only option I believe I had was to click on Format. 

4.  Anyway, I clicked on that, and the next thing you know, all drives were being labeled as "Formatting".  I didnt know what to do and just let it run for about 10 minutes.  Once it was done, all data was gone. 

 

Did I basically lose all of the info on all drives? 

Since you've formatted the drives you will need to rebuild their respective file-trees.  (Their directory structures pointing to the data blocks containing your files)

 

Whatever you do, DO NOT WRITE ANY NEW FILES TO THE DRIVES YOU ACCIDENTALLY FORMATTED.

 

You will need to perform this set of steps for each of the drives you accidentally formatted (I'll use disk1 and disk2 as an example, change the number for each of your other disks in turn):

 

Turn off any spin-down timers.  We do not want the disks to be spun down in the middle of the repair.

Turn off any sleep/suspend screen savers on your PC if doing this via telnet.  Don't want it shutting down in the middle of a repair step.  It might be better if the following commands are typed on the unRAID system console directly, but as long as your PC does not shut itself down, you'll be fine using telnet.

 

Stop samba (you will not be able to un-mount the disk if it is busy... and samba will keep it busy)

/root/samba stop

un-mount the drive. 

umount /dev/md1

Rebuild the file-tree scanning the entire disk for files.

reiserfsck -–scan-whole-partition -–rebuild-tree /dev/md1

Respond with "Yes" when prompted... Upper-case "Y", lower case "es"  The scan can take many hours for a large drive. 

 

Re-mount the drive:

mount /dev/md1 /mnt/disk1

 

un-mount the next drive. 

umount /dev/md2

Rebuild the file-tree scanning the entire disk for files.

reiserfsck -–scan-whole-partition -–rebuild-tree /dev/md2

Respond with "Yes" when prompted... Upper-case "Y", lower case "es"  The scan can take many hours for a large drive. 

 

Re-mount the drive:

mount /dev/md2 /mnt/disk2

 

Do the above steps for each of your disks, then....

when all done, you should be able to re-start samba

/root/samba start

and see if your files have been restored.  Files in the top-most directory will probably end up in a lost+found directory created by the restore process.  You'll need to re-name them to their correct names.  Files lower in the directory hierarchy will probably have their correct names.

 

While this is all happening, write to lime-technology describing how the "unformatted" bug has caused you to lose all your data and request it be fixed as soon as possible. 

See this thread for another user's experiences http://lime-technology.com/forum/index.php?topic=6084.0

 

Send an e-mail to lime-technology.  Tell them you lost all your data because of the "unformatted" bug.

 

We've never heard of all the disks being formatted unless they were showing as un-formatted on the display. (in error, but showing un-formatted because they were not mounted)  I'm guessing the disks were all showing as un-formatted and you did not notice.

email to: [email protected]

 

I am about 99.9% sure it was just the one drive...but at the same time, I trust your guys' knowledge as well.  I only remember this because I specifically read your exact recommendation right before this happened, and I sat there and verified that it was just the one drive that was showing unformatted.  As soon as I clicked on the button, it immediately showed all drives being formatted.  I remember reading about the bug, so I THOUGHT that the bug was what I was seeing - in that it was showing as formatting, but really it wasnt. 

 

With respect to this process of rebuilding the structure - does this have to be done one drive at a time?  Or can you use multiple screens to do all drives at once.  because we are talking about 5 large drives, I dont know if it will take just as long to put my collection back on myself? I only say this because I have had this running on MD1 for about an hour, and it is doing something, but its still stuck at 0%.  Does that sound right?  Thanks fellas

Presuming this does not recover anything, or if I got to the point where it was just taking too long, to start from square one - would each drive need to be precleared once again, like I did at the beginning?  Before I started the fsck recovery attempt, all my drives were being reported as OK, with parity valid, but data 100% gone - like it was never there.  Was that inaccurate? 

 

It is quite concerning to me,despite the fact that I did press the button, that one little click and 10 minutes later, all data is gone.  Would that process have been longer if it were full of data?  I have 6 drives, 8TB of capacity, with about 15-20% data on them.  Or i should say 'had' that much. 

 

 

Presuming this does not recover anything, or if I got to the point where it was just taking too long, to start from square one - would each drive need to be precleared once again, like I did at the beginning?  Before I started the fsck recovery attempt, all my drives were being reported as OK, with parity valid, but data 100% gone - like it was never there.  Was that inaccurate? 

 

It is quite concerning to me,despite the fact that I did press the button, that one little click and 10 minutes later, all data is gone.  Would that process have been longer if it were full of data?  I have 6 drives, 8TB of capacity, with about 15-20% data on them.  Or i should say 'had' that much. 

 

I encountered the same exact problem you have and have been running restores on all my disks for the last few days.  My understanding is that when unraid reformatted the drives it didn't overwrite the whole drive just the index which contained a pointer to all the files.  I followed Joe L's steps in recovering my data and while I will have to do some moving around of files it sure beats trying to re-rip all my movies.

I am about 99.9% sure it was just the one drive...but at the same time, I trust your guys' knowledge as well.  I only remember this because I specifically read your exact recommendation right before this happened, and I sat there and verified that it was just the one drive that was showing unformatted.  As soon as I clicked on the button, it immediately showed all drives being formatted.  I remember reading about the bug, so I THOUGHT that the bug was what I was seeing - in that it was showing as formatting, but really it wasnt. 

 

With respect to this process of rebuilding the structure - does this have to be done one drive at a time?  Or can you use multiple screens to do all drives at once.  because we are talking about 5 large drives, I dont know if it will take just as long to put my collection back on myself? I only say this because I have had this running on MD1 for about an hour, and it is doing something, but its still stuck at 0%.  Does that sound right?  Thanks fellas

 

This is the second time I've added capacity to my server so I wasn't really paying attention to my array when I clicked through the steps of adding a new hard drive.  However, I do side with you.  I'm pretty sure only one drive showed up as being un formatted.  Had all my drives said un formatted I would have probably shit myself in that step opposed to after I saw all my drives being formatted after clicking the format button.

Hey Jkukow....thanks for sharing your experience.  How many drives do you have?  And of the ones that have completed, whats your recovery rate look like?  Are you running just one disk at a time?  

 

Wish you the best in your recovery process....

Hey Jkukow....thanks for sharing your experience.  How many drives do you have?  And of the ones that have completed, whats your recovery rate look like?  Are you running just one disk at a time?  

 

Wish you the best in your recovery process....

 

I've got two 1 TBs, a 1.5 TB, and 750GB for data.  I don't know the exact time it takes but I've been able to do a new drive every 12 hours, or at least that's when I check the server and the recovery is complete.  I took a peak at a couple of the drives and my movies retained their names, however they are not in their original folders so I will have to reorganize everything.  I had all my photos stored on the drive as well and that's not quite as pretty since I've lost the folder structure for all those as well.  However, being paranoid about my photos I keep an off site backup (parity doesn't work if your house burns down) so I'm going to recover all my photos that way.

 

I wish you speedy and 100% recovery process as well. 

Ok, sounds like we have similar drive sizes so at least I know what to expect.  I guess the safe way is to run this one drive at a time, rather than one on the host, and others via telnet. 

 

Thanks again.

I am about 99.9% sure it was just the one drive...but at the same time, I trust your guys' knowledge as well.  I only remember this because I specifically read your exact recommendation right before this happened, and I sat there and verified that it was just the one drive that was showing unformatted.  As soon as I clicked on the button, it immediately showed all drives being formatted.  I remember reading about the bug, so I THOUGHT that the bug was what I was seeing - in that it was showing as formatting, but really it wasnt. 

You are the first to report it formatted all your drives even though you think it only showed one drive as un-formatted.  For this reason, I suspect the others showed un-formatted too.  we'll never know, not unless you have not rebooted and can post the original syslog.

With respect to this process of rebuilding the structure - does this have to be done one drive at a time?  Or can you use multiple screens to do all drives at once.  because we are talking about 5 large drives, I dont know if it will take just as long to put my collection back on myself? I only say this because I have had this running on MD1 for about an hour, and it is doing something, but its still stuck at 0%.  Does that sound right?  Thanks fellas

Without doing anything at all other than reading, at 100MB/s it will take about 5 1/2 hours just to read a 2TB drive.  Expect the rebuild-tree to take longer.

 

You can run the rebuild-tree in parallel.  On the system console you can use ALT-F1 through ALT-F6 to switch between 6 different virtual consoles.  You can log in individually on each, and run one file-system check on each.

 

You can use ALT-F1 through ALT-F6 to switch between them to view their output as they progress.

 

If you do use telnet, make sure you disable any screen saves, or sleep functions on the PC. You do not want it going to sleep since it will not detect any keyboard activity.

Presuming this does not recover anything, or if I got to the point where it was just taking too long, to start from square one - would each drive need to be precleared once again, like I did at the beginning?  Before I started the fsck recovery attempt, all my drives were being reported as OK, with parity valid, but data 100% gone - like it was never there.  Was that inaccurate? 

It will take many hours... remember, every sector of every disk needs to be scanned.  Do not get impatient.  It will take MUCH longer to re-rip and re-write all the files to the disks if you resort to that.

 

Yes, if you re-formatted the disks you basically just overwrote the pointers to the files.  While that happened, parity was kept in sync, therefore no need to pre-clear, if you were starting from scratch all you would need do is copy the replacement files into place.

It is quite concerning to me,despite the fact that I did press the button, that one little click and 10 minutes later, all data is gone.  Would that process have been longer if it were full of data?

It did not matter if the disks were empty or full.  Formatting a drive basically writes the bitmaps and structures in place that will later track the data files.  You just overwrote those existing structures.  Basically, your data is still there, just no directory entries pointing to it.

  I have 6 drives, 8TB of capacity, with about 15-20% data on them.  Or i should say 'had' that much.

Hopefully, most all will be recovered.  You'll find it in the lost+found folder on the drive, so you'll need to move it to its correct top level directory name, but moves are fast as long as you are moving files on the same disk.  I suggest you use the "disk" shares when doing this and not use the user-shares.  (It only adds overhead and you really don't want things slowing down when moving around that much data)

 

Joe L.

I only remember this because I specifically read your exact recommendation right before this happened, and I sat there and verified that it was just the one drive that was showing unformatted.  As soon as I clicked on the button, it immediately showed all drives being formatted.  I remember reading about the bug, so I THOUGHT that the bug was what I was seeing - in that it was showing as formatting, but really it wasnt. 

 

That is really worrying.  Even knowing about the unformatted bug, he still lost data because of it.  Fix this LimeTech!

Ok all, here is an update.  Basically saw this as an opportunity to get away and let unraid recovery do its thing - so took the family away for a couple of days while it worked away. I was able to remote access to one of my pcs and telnet into my unraid box to see progress.  I went ahead and had a couple of sessions going - recovering each drive one at a time.  It took a little over a day and a half for each - some went  quicker than others.  It finished sometime yesterday, after which point I mounted each drive individually, then restarted samba per Joe L's instructions.  Once that was done, a parity check started and I let it go do its thing.  Parity completed in a few hours, and when I logged back into the unmenu page, I was relieved to see that the "% available" was NOT the same number as the "% capacity" which at least meant that some data was recovered.  I went through each share and saw that approx 50% of the data was recovered.  I think Joe mentioned something about some data moving into a 'Lost and Found' folder but I am not quite sure where that would be in the file structure.  How would I navigate to that folder? 

 

Based on my initial review of the %s of the drives available, it SEEMS like most if not all data was recovered but not in their original location.  In other words, I expected to see more than what I saw - so I am guessing that the other missing data is there, but I just am trying to figure out how to navigate to this 'Lost and Found' folder.  I havent gone into each share and respective folder to inspect its contents yet  -  I am doing that now. 

 

Once I find this lost/found folder, I can give a more accurate report of how the recovery process went. 

 

More to come soon - and hopefully, some insight as to the whereabouts of this hodgepodge folder...

 

 

 

Hopefully, most all will be recovered.  You'll find it in the lost+found folder on the drive, so you'll need to move it to its correct top level directory name, but moves are fast as long as you are moving files on the same disk.  I suggest you use the "disk" shares when doing this and not use the user-shares.  (It only adds overhead and you really don't want things slowing down when moving around that much data)

 

 

I cannot figure out how to navigate to this lost/found folder.  I think by this you mean I need to export this under the drive settings in the config gui (Export Disk shares under the Shares tab- since I currently do not do this, I only export User Shares)  So once I export Disk shares, I would then be able to locate this folder. 

 

Post back shortly.

Ok all, here is an update.  Basically saw this as an opportunity to get away and let unraid recovery do its thing - so took the family away for a couple of days while it worked away. I was able to remote access to one of my pcs and telnet into my unraid box to see progress.  I went ahead and had a couple of sessions going - recovering each drive one at a time.  It took a little over a day and a half for each - some went  quicker than others.  It finished sometime yesterday, after which point I mounted each drive individually, then restarted samba per Joe L's instructions.  Once that was done, a parity check started and I let it go do its thing.  Parity completed in a few hours, and when I logged back into the unmenu page, I was relieved to see that the "% available" was NOT the same number as the "% capacity" which at least meant that some data was recovered.  I went through each share and saw that approx 50% of the data was recovered.  I think Joe mentioned something about some data moving into a 'Lost and Found' folder but I am not quite sure where that would be in the file structure.  How would I navigate to that folder? 

 

Based on my initial review of the %s of the drives available, it SEEMS like most if not all data was recovered but not in their original location.  In other words, I expected to see more than what I saw - so I am guessing that the other missing data is there, but I just am trying to figure out how to navigate to this 'Lost and Found' folder.  I havent gone into each share and respective folder to inspect its contents yet  -  I am doing that now. 

 

Once I find this lost/found folder, I can give a more accurate report of how the recovery process went. 

 

More to come soon - and hopefully, some insight as to the whereabouts of this hodgepodge folder...

 

log onto unRAID via telnet.  Change directory to each drive's mount point, as an example:

cd /mnt/disk1

 

type

ls -l

the lost+found directory will be there, parallel to your top level "user-share" directories.    

I'd stay away from using user-shares until you get things more stable, so I'd not try to get to it through a user-share myself.

 

You can use

mc

to browse what is in the folders and move things around. (mc is midnight commander)

 

Joe L.

OK, I exported Disk Shares and was able to see the Lost/Found folders, like joe mentioned.  Large folders - lots of data in them.  Attached screenshot of one of them.  I guess these are just 'misplaced' folders that need to moved to their respective shares.  I only have 3 main shares - Movies, Photos and Work - so shouldnt be too hard to drag and drop accordingly. 

 

[/img]

Ok, I got your message after I posted my last one regarding how I navigate to the Lost Found folder.  So if I am reading you correctly, your suggestion is to NOT use user shares via windows explorer...rather to use mc (i am familiar with that program) via telnet and get things properly placed/mapped that way. 

 

I am finding that, depending on how user level was split, using explorer is going to be more difficult that I originally thought.  If I had split level to 999, I could potentially find a lot of various files in all sorts of places.  For movies, I think I should be OK because I had that split to 3.  Photos might be a different story. 

 

So, following your recommendation, use MC - and dont export user shares at all...for the time being.  Ill go with that.  Ill run parity once again after I get through a whole disk.  Might be a few hours or more. 

 

 

This is embarassing to someone if you ask me. And I don't mean the gborrillo or jkukow. Poor guys, my thoughts go out to you.

 

Joe, what would unRaid be without your support ? OK, maybe please don't answer that...

This is embarassing to someone if you ask me. And I don't mean the gborrillo or jkukow. Poor guys, my thoughts go out to you.

 

Joe, what would unRaid be without your support ? OK, maybe please don't answer that...

 

Joe L. truly does earn the hero member.

There are a bunch of others too!!

Ok, I got your message after I posted my last one regarding how I navigate to the Lost Found folder.  So if I am reading you correctly, your suggestion is to NOT use user shares via windows explorer...rather to use mc (i am familiar with that program) via telnet and get things properly placed/mapped that way.   

If you use user-shares, unRAID may decide the best place for a directory is a different drive than it is currently on.  That will take a LOT longer than moving a directory on the same drive (basically instant)

 

For that reason I do not think using user-shares to move things from the lost+found folder is a good idea.  Use the "disk" shares. You'll get everything back into place a lot more quickly.

This is embarassing to someone if you ask me. And I don't mean the gborrillo or jkukow. Poor guys, my thoughts go out to you.

 

Joe, what would unRaid be without your support ? OK, maybe please don't answer that...

 

Yes, it sure is. I try to be supportive of unRAID but it becomes hard when a major issue like this goes completely unaddressed for 7 weeks now...

 

It really is a good thing that someone has Joe to help out customers....

 

And I really hope you guys get most if not all of your data back.

 

Peter

 

This is embarassing to someone if you ask me. And I don't mean the gborrillo or jkukow. Poor guys, my thoughts go out to you.

 

Joe, what would unRaid be without your support ? OK, maybe please don't answer that...

 

Yes, it sure is. I try to be supportive of unRAID but it becomes hard when a major issue like this goes completely unaddressed for 7 weeks now...

 

It really is a good thing that someone has Joe to help out customers....

 

And I really hope you guys get most if not all of your data back.

 

Peter

 

 

I don't think it's unaddressed, I did see something about Tom working on a fix.

It should be communicated better.

So - just want to be completely sure.  I understand what is being recommended as far as using disk shares.  Does that mean I should go into the config and actually disable user shares?  Or simply do not export them?  I dont know what the ramafications are of disabling user shares - since my file structure was based on user shares originally. 

 

A bit confusing because each drive now has a Lost/Found folder.  If I look at my user shares, it shows approximately 50% of original data in its original location.  So take disk1 for example, I have a Lost/Found folder there with tons of randomly named folders.  By opening each one, I can see what data is inside - Movie/Photo/Music.  So say its a Movie file, I dont know what drive it originally was in, because it was based on user shares.  It could have been any of them.  I could point it towards the user share, but thats what I think Joe was referring to as far as taking much more time and not dealing with user shares for the time being.  I dont have a problem with that, but I think I must be missing somethign because there is no way to determine what disk a folder/file should be on, unless a completely new remap is in order. 

 

The only thing I could think of is to actually DISABLE user shares, and go with Disk Shares, and actually remap everything acccording to disk shares. ie: Ill say I want all movies to be on disk1, create a folder on disk1, and then move everything I find there.  Then create a folder for Music on Disk2, or whatever.  Essentially a complete remap based on disk shares.  Is that correct or am I completely missing something here.