I just upgraded one of the 750GB discs in my 14-disc 11.5TB unRAID setup with a new Western Digital 1.5TB green drive.  I was transferring a ripped Blu-Ray movie over to the array and the transfer kept getting hung up.  I'd have to go into the array and delete the partially transferred file and start over.  That last time this occurred I connected to the tower and the main page showed the parity drive, the new WD drive, and twelve unformatted drives. 

 

I attempted to stop the array so I could power it down but the system was hung and would not allow me to stop the array. The option to power down would never appear.  Like a dummy, instead of manually shutting down the server I inadvertently clicked on the format button, causing all of the "unformatted" drives to undergo a reformat with the end result being total loss of data on those drives.  Ironically, all data on the new WD drive had been previously restored by the parity drive when I first installed it so that data remained intact.

 

Anyone have a clue why my drives suddenly lost their formatting?  Are there issues with the new WD green drives I should be aware of?  I have a couple of the 750GB WD green drives in the array and never had any issues until now.

It is unlikely that the data is lost.... most likely the drives were reported as unformatted in error.

 

Try updating your USB stick to the latest version of uNRAID... to includes some fixes for a bug where drives were erroneously being reported as unformatted.

 

It is unlikely that the data is lost.... most likely the drives were reported as unformatted in error.

 

Try updating your USB stick to the latest version of uNRAID... to includes some fixes for a bug where drives were erroneously being reported as unformatted.

 

You might be able to get back the data on those drive by using the rebuild-tree option of reiserfsck and scanning the entire drive.

 

It will take it a while to scan the entire disk, but the files should be able to be recovered.

 

See the wiki where it talks about recovering "deleted' files.

 

Joe L.

Wouldn't the format option have wiped out the data?  The array is currently in the process of running a new parity check. When the drives were reformatted I swapped out two more of the drives with the assumption that I had already lost the data and that I had essentially lost parity so it would simply see the drives as new and rebuild the array from that point.

 

Would it be possible to recover my data by doing the following:

 

1.  Stop the parity check and shut down the array.

2.  Reinstall the other two original drives that had data.

3.  Upgrade the software.

4.  Restart the array and run a new parity check.

5.  When parity has finished, upgrade the two drives individually.

 

I checked the release notes in the latest version (4.5.3) and didn't see instructions for upgrading the software.  Do I still simply replace the bzimage and bzroot files on the USB flash drive with the newer versions?

Wouldn't the format option have wiped out the data?

 

Yes.... I missed the part where you said you pressed the format button.

Wouldn't the format option have wiped out the data?

 

Yes, but if it was a typical format, you can rescan for what looks like files to be recovered. As Joe L said, you may be able to recover "by using the rebuild-tree option of reiserfsck and scanning the entire drive."

I'm trying the recovery method on the first drive.  We'll see how it goes so wish me luck. 

 

The format process didn't take very long to accomplish so I'm guessing it just does a quick format similar to what takes place on a Windows drive where it just rebuilds the FAT table (or whatever they use for NTFS) rather than doing a read/write of every block for a full format.  If that's the case then the data could very well still be intact on the drive.

Try updating your USB stick to the latest version of uNRAID... to includes some fixes for a bug where drives were erroneously being reported as unformatted.

 

Odd point. I upgraded to 4.5.3. when I rebooted, all my drives said they were Unformatted.

There was some kind of mount error.

 

This really needs to be fixed. If there is some signature to tell if the drive is formatted vs unmounted then this needs to be used. It's too easy for someone to make a mistake.

 

I stopped the array, then started it and the drives came up mounted.

 

Wouldn't the format option have wiped out the data?

 

Yes, but if it was a typical format, you can rescan for what looks like files to be recovered. As Joe L said, you may be able to recover "by using the rebuild-tree option of reiserfsck and scanning the entire drive."

 

There have been times when people have overwritten part of the drive and the rebuild-tree saved some of the data.

 

We really need some form of md5sum and/or par2 method of checking the data and maybe even restoring data that has been corrupted.

The command would be:

reiserfsck -–scan-whole-partition -–rebuild-tree /dev/md1

reiserfsck -–scan-whole-partition -–rebuild-tree /dev/md2

etc.

 

You would need to unmount then each in turn, as described in the wiki

when prompted by reiserfsck, answer "Yes" (capital "Y" lower case "es")

 

Joe L.

Good news to report.  I just finished running the recovery program on the first drive and it appears that everything recovered intact, at least upon a cursory glance.  I haven't tried to play back any of the DVD or Blu-Ray rips yet but all the folders and files seem to be there.  Since none of the drives have been written to since I installed the upgrade drive I expect I should be able to recover everything (knock on wood). 

Good news to report.  I just finished running the recovery program on the first drive and it appears that everything recovered intact, at least upon a cursory glance.  I haven't tried to play back any of the DVD or Blu-Ray rips yet but all the folders and files seem to be there.  Since none of the drives have been written to since I installed the upgrade drive I expect I should be able to recover everything (knock on wood). 

That is very encouraging news...  Are you running the reiserfsck on the /dev/md? devices? or the raw disks /dev/sd?1 ? ? ?

 

If you are using the former (/dev/md??) then parity is being corrected as you recover the files.  If using the latter, then parity will be out of date and you will need to press the "Check" button to have it get back in sync.

 

Good luck with your other drives.  I hope you are as successful with them in recovering your files.

 

For future reference, how large are your disks? and how long did the scan-entire-drive take? ? ?

 

Joe L.

Oh yes... unformatted simply means not yet mounted...  If you just powered on after a abrupt power down the file-systems will be re-playing their journal entries and will not mount until they are done.  We heard of cases where this can take as long as 5 minutes or so.

 

Yes, it is BUG that it displays the word "Unformatted" at all, and even worse that it will attempt to format disks that held your data.

 

Report this BUG to lime-technology in an e-mail pointing to this thread.  Plead that he fix this bug so no others suffer the same potential loss.    so far there have been three or four others showing "unformatted" drives on start-up.

 

Joe L.

Good news to report.  I just finished running the recovery program on the first drive and it appears that everything recovered intact, at least upon a cursory glance.  I haven't tried to play back any of the DVD or Blu-Ray rips yet but all the folders and files seem to be there.  Since none of the drives have been written to since I installed the upgrade drive I expect I should be able to recover everything (knock on wood).  

That is very encouraging news...  Are you running the reiserfsck on the /dev/md? devices? or the raw disks /dev/sd?1 ? ? ?

 

If you are using the former (/dev/md??) then parity is being corrected as you recover the files.  If using the latter, then parity will be out of date and you will need to press the "Check" button to have it get back in sync.

 

Good luck with your other drives.  I hope you are as successful with them in recovering your files.

 

For future reference, how large are your disks? and how long did the scan-entire-drive take? ? ?

 

Joe L.

Here's the command I'm using after stopping samba and un-mounting the drive:

 

reiserfsck --rebuilt-tree -S /dev/md1

 

The entire drive is being scanned without any additional commands.  I got a couple of error messages about free space being reported improperly following the scan but it indicated that the errors were corrected.

 

I've got about an equal mix of 1.5TB and 750GB drives.  I'm gradually replacing the 750GB drives with larger ones as more space is needed or I see a great deal on a larger drive.  The 1.5TB drives are starting to drop below $100 with no rebates so they're starting to look more and more attractive.  Of course, if I'm less successful recovering the data on the remaining drives then I won't have a need to upgrade any drives for quite some time.  Here's hoping I can max this sucker out.

 

I haven't timed it but I'm guessing it takes about 2-1/2 to three hours for a 1.5TB drive, give or take.

Try updating your USB stick to the latest version of uNRAID... to includes some fixes for a bug where drives were erroneously being reported as unformatted.

 

Odd point. I upgraded to 4.5.3. when I rebooted, all my drives said they were Unformatted.

There was some kind of mount error.

 

This really needs to be fixed. If there is some signature to tell if the drive is formatted vs unmounted then this needs to be used. It's too easy for someone to make a mistake.

 

I stopped the array, then started it and the drives came up mounted.

 

So, it's not just me.  All my drives showed as Unformatted after I upgraded to 4.5.3.  Almost gave me a heart attack! 

A quick look in my syslog didn't show me anything bad, so -- in blind panic -- I stoped/started the array a few times,

then all disks came up normally mounted. 

 

At a time like that, one can reach for a hatchet and unknowingly do a lot of damage to the server.

 

Your thread title is kind of missleading...  I think "4.4.2 had an error and I accidentally wiped over 7TB of data" would be much better.

 

Pressing format on drives that have data is a user error, not really an unraid error. There are countless threads and I think it's even in the FAQ to never press the format button if the drive has data. It's pretty much the #1 cause of data loss in unRAID.

 

I hope you get your data back, but I would really recommend reading the top 10 causes of data loss on unRAID, and the unRAID FAQ. This whole thing could of been avoided, and hopefully next time you will know what to do.

Pressing format on drives that have data is a user error, not really an unraid error. There are countless threads and I think it's even in the FAQ to never press the format button if the drive has data. It's pretty much the #1 cause of data loss in unRAID.

While this may be true, I still consider it to be a serious bug that the web page would use the word "Unformatted".

 

Pressing format on drives that have data is a user error, not really an unraid error. There are countless threads and I think it's even in the FAQ to never press the format button if the drive has data. It's pretty much the #1 cause of data loss in unRAID.

While this may be true, I still consider it to be a serious bug that the web page would use the word "Unformatted".

 

 

It's also not the best idea to have a global format button near all the other management buttons. If the system thinks the drives are unformatted, it would be better to have individual format buttons next to each drive.

And also, unRAID should really "know" (perhaps within super.dat) that the particular drive was formatted in the past and instead display "error reading drive" near the drive. How can it not know that the drive was formatted in the past? 

Your thread title is kind of missleading...  I think "4.4.2 had an error and I accidentally wiped over 7TB of data" would be much better.

 

Pressing format on drives that have data is a user error, not really an unraid error. There are countless threads and I think it's even in the FAQ to never press the format button if the drive has data. It's pretty much the #1 cause of data loss in unRAID.

 

I hope you get your data back, but I would really recommend reading the top 10 causes of data loss on unRAID, and the unRAID FAQ. This whole thing could of been avoided, and hopefully next time you will know what to do.

I'll definitely check the FAQ before I take any drastic measures in the future.  Pressing the format button was definitely a dumb thing to do.  I'm still not sure why I did it, but it was probably a panic reaction since I thought the data was already lost to me.  Fortunately, I'm having success recovering the lost data as well as lots of files that had previously been deleted.  The reiserfsck function creates a Lost&Found folder where it places recovered files that had been previously deleted.

 

My previous time estimate for the recovery process was way off.  I thought the recovery process was working on a 1.5TB drive when in fact it was only a 750GB drive.  The time to process a 1.5TB drive took about five hours and maybe a bit more as I let it run overnight.  I did another 750GB drive this morning and it took about 3-1/2 hours.  I think it depends a lot on how much deleted data is on the drive as well as data that was there prior to doing the reformat.

Pressing format on drives that have data is a user error, not really an unraid error. There are countless threads and I think it's even in the FAQ to never press the format button if the drive has data. It's pretty much the #1 cause of data loss in unRAID.

While this may be true, I still consider it to be a serious bug that the web page would use the word "Unformatted".

 

 

It's also not the best idea to have a global format button near all the other management buttons. If the system thinks the drives are unformatted, it would be better to have individual format buttons next to each drive.

 

Right.  Those individual format buttons could go in the disk's 'individual' page where you set up its spinup group and stuff.

 

I had this happen as well to one of my servers. All showed as unformatted, and by refreshing the page i saw something was writing to the disks and parity drive. Needless to say i panicked and hard resetted the server thinking something was corrupting my data. If only i knew it was just re-playing their journal entries and mounting the drives when they are done, i wouldn't do such a bad thing.

 

This really needs to be fixed: 'Unformatted' is such an evil word, every new user translates it into "bye,bye data"

Both of you, send Lime-Tech an e-mail stating it needs to be fixed upon start-up.  I like the individual format button idea.

Point Tom to this thread.... let him know panic is not a good thing.

E-mail sent to Lime Tech this morning.  So far my recovery process appears to be going well.  I'm down to two 1.5TB drives and two 750GB drives left to process.  One of the 1.5TB drives is being working this morning and I expect it to be completed by the time I go home for lunch.  I'll get the other two 750GB drives done by this evening and run the remaining 1.5TB drive overnite.  I'll then have to map to each drive and delete the Lost & Found folders on each drive and then start a parity check.  

 

If all goes well I'll take another shot at replacing one more 750GB drive with a new 1.5TB unit, except this time I'll know not to panic if the drives are listed as unformatted.  The weird thing is that I've been running unRAID for about two years now and I've never seen this problem before now, which is why I reacted the way I did.

 

I don't know if I mentioned this but I had restarted the array after having difficulties completing a file transfer.  All of the drives appeared normal once I was able to connect.  It wasn't until after several more aborted attempts to transfer the file that I discovered the drives were listed as unformatted.  The array had been up and running for quite some time (at least an hour) when this occurred.

Update:  I'm down to processing my last 1.5TB disc and so far I've had mostly good results as far as I can tell.  I did have one disk that listed just a Lost & Found folder after performing the recovery.  I went through most of the L&F folders and just deleted the contents since they were mostly scattered files.  All of my Blu-Ray rips were stored as iso's and my DVD's were ripped as files.  I had some HD-DVDs and some transport stream files saved from Tivo recordings but at this point it's hard to say just how much was saved until I try to take some kind of inventory.  Considering the possible outcome I feel that I got off lucky.

 

When I get up in the morning I'll delete the final L&F folder and initiate a parity check to backup the data.  Once I've done that I'll take another crack at upgrading a second drive.  Wish me luck.

Update:  I'm down to processing my last 1.5TB disc and so far I've had mostly good results as far as I can tell.  I did have one disk that listed just a Lost & Found folder after performing the recovery.  I went through most of the L&F folders and just deleted the contents since they were mostly scattered files.  All of my Blu-Ray rips were stored as iso's and my DVD's were ripped as files.  I had some HD-DVDs and some transport stream files saved from Tivo recordings but at this point it's hard to say just how much was saved until I try to take some kind of inventory.  Considering the possible outcome I feel that I got off lucky.

 

When I get up in the morning I'll delete the final L&F folder and initiate a parity check to backup the data.  Once I've done that I'll take another crack at upgrading a second drive.  Wish me luck.

If you've been doing the rebuild-tree on the /dev/md? devices, then parity is being corrected in sync with the files being recovered.  Your additional parity-check should find very little, if anything.

I've finished the data recovery and final parity check and took inventory as best I could on the data that was recovered.  It looks like about 75 DVD and Blu-Ray rips got trashed, resulting in the loss of about 1.5 - 2TB of data.  There were two drives that were a total loss and just had the Lost & Found folder.  I just deleted whatever was in the Lost & Found folders because they were mostly a collection of disjointed and miscellaneous files and not worth the effort to try and sort them out.  There's nothing that can't be replaced so I'm not losing any sleep over it.  I try not to get upset over things like this because it's all part of the learning process.  I learn more from my mistakes than my successes (and I've learned a lot ).

 

Being the glutton for punishment that I am, I have replaced one of the remaining 750GB drives with another 1.5TB drive and the array is currently in the process of rebuilding the data.  Hopefully this upgrade will go smoother than the last one.