Accidentally assigned data disk as parity

[Monkeyair]

Thanks for that.

 

If I run a parity check with the part formatted data disk as an array device but not mounted in the array, am I going to make things worse by running a parity check?  Am I better off waiting for a new disk to replace the data drive and do a rebuild? If there is anything left to rebuild as most of my data was on disk 1 (Formatted disk)

 

I'm trying not to touch the said data disk any more in case it has to go of to a data recovery company.

 

[SSD]

You can run a non-correcting parity check. It will tell you if the data disk is out of sync with parity. Normally being out of sync is a bad thing, but if yours is out of sync it may mean that the rebuild operation would yield a more recoverable starting point than the current disk. Or at least you'd have two swings at the bat, so to speak. If the non-correcting check comes back with no sync errors, you know a rebuild will make no difference. You'll probably know pretty quickly is the non-correcting check starts showing thousands of sync errors. If so you'll have your answer and no value letting it run to the end. But I would not stop it if it is showing no or a very small number of sync errors.

 

Good luck again.

[Monkeyair]

You can run a non-correcting parity check. It will tell you if the data disk is out of sync with parity. Normally being out of sync is a bad thing, but if yours is out of sync it may mean that the rebuild operation would yield a more recoverable starting point than the current disk. Or at least you'd have two swings at the bat, so to speak. If the non-correcting check comes back with no sync errors, you know a rebuild will make no difference. You'll probably know pretty quickly is the non-correcting check starts showing thousands of sync errors. If so you'll have your answer and no value letting it run to the end. But I would not stop it if it is showing no or a very small number of sync errors.

 

Good luck again.

27 mins gone and 1931810 errors........

[SSD]

You have your answer. So much has happened I'm not sure if the reconstructed drive will contain anything of value or not. But hopefully it is pre-reiserfsck.

[Monkeyair]

Hopefully.  Thanks again for all of your help.

[garycase]

Actually it's not at all unusual that you're seeing thousands of errors, since you had written to the wrong data disk.

 

But if you have NOT written to the original parity disk at all, and if the "failed" disk (the one you originally wanted to remove) remains readable for long enough, there's a chance you MAY be able to recover most of the data from the data disk you incorrectly assigned as parity and thus "trashed" ... depending on exactly what you did here:

 

Fortunately I have managed to mount the faulty drive for now.  It must be an intermittent fault.  So I only have 1 failed drive for the moment.

 

My parity now shows as valid, but when the array starts no rebuild option is available as the data disk that I used the wrong recovery program on is unmountable.

 

The option is given to me to create a file system and format the disk.  I want the disk to mount and rebuild the parity. 

 

I initially selected format but bottled it and restarted the system.  What is the correct thing to do? Have I compromised the data on my parity drive by letting the data disk format for a while?

 

You DID compromise the parity drive with the format ... every bit that was written as part of the format also caused the corresponding parity bit to be updated, so all of those are incorrect relative to your original data.    How badly that was compromised depends on just how long the format was running.    It's not likely it was as bad as a parity check would have been, as the data drive was already overwritten on every sector that had been completed in the original parity sync when it was assigned as parity.

 

I suspect you've simply lost all of the data on that drive (unless you have good backups) ... but you can try the following to see if you can recover some of it:

 

You can do the following:

 

(a)  Do another New Config assigning ALL of your original data disks and the original parity disk with the "Trust Parity" option [i believe the actual wording is "Parity is already valid."    When you Start the array it will automatically start a parity check ==>  IMMEDIATELY CANCEL that check  (it's going to compromise a few bits at the beginning of the parity drive; but these are already compromised from your earlier actions, so if you're quick it won't make things any worse than they already are)

 

(b)  Stop the array and unassign the compromised data disk.

 

©  Start the array so that disk shows as "Missing".    You can now look at the "missing" disk with Windows Explorer (it will be emulated) and you'll be able to see what the contents will be if you rebuild the disk.

 

(d)  If the contents look worth recovering, then Stop the array; re-assign the drive; and then Start the array and it will do a rebuild.

 

 

 

 

 

 

 

 

 

 

 

 

[Monkeyair]

The compromised data disk is listed as unmountable, which is why I thought that the format option would fix the file system ready for a rebuild.  Obviously, I know that is not the case now.

 

By the time I noticed what the format was doing the system had written 4000000 writes to a 6TB parity.  The total data on the compromised disk was roughly 5TB.  How much damage can I expect 4000000 writes to have done?

 

Will the unmountable disk show in explorer?  Can I get it to mount without compromising any more data on the disk?

 

[RobJ]

By the time I noticed what the format was doing the system had written 4000000 writes to a 6TB parity.  The total data on the compromised disk was roughly 5TB.  How much damage can I expect 4000000 writes to have done?

 

Formatting with XFS must be more extensive than with ReiserFS.  A Reiser format, like an NTFS quick format, does not actually format the entire partition, just the beginning section of it, and takes only a few minutes.  It writes the basic empty file system, with a small tree and some empty leaves, then as you add data to the drive, builds out the tree, essentially formatting on the fly.  The damage from 4000000 writes depends on the write block size, if 1KB then 4GB overwritten, if 4KB then 16GB overwritten.  But you don't know where it's writing, sequentially from the beginning, or evenly spaced out writes, preparing empty leaves across the disk surface.  And the earlier reiserfsck probably corrupted more locations.

 

As said above, you have 2 versions of the disk surface, the current physical one, and the virtual one.  For highest recovery chances, you should pull the current drive, and rebuild onto a second 6TB drive, giving you 2 different drives, one which may have more data than the other, but we don't know which that is.

 

There is a tool called TestDisk (a bootable live CD), which is very good for recovering partitions and their data (I'm not experienced with it myself though).  You'll have to figure out how to get it to ignore the empty formatted partition it will find first, and go on to search the entire disk surface, building a file system from what it can find.  That's going to be very hard, because of the new fresh format.  There are also other data recovery tools, not free, but very good.  They will have the same issues though, differentiating between what is a part of the original file system and what is just part of the empty new file system.

 

Data recovery companies are happy to take your money.  It's been a long time since I checked, but it used to be $500 and up, may be more now, especially with these much larger drives.  This may be the first 6TB drive they have ever seen!  And you will have 2 of them!  And XFS which is relatively new!  They will love you!  (How much can you get if you sell your firstborn?)   

[garycase]

As Rob noted, one alternative is a professional data recovery service ... but these are NOT inexpensive.  Gillware is an excellent choice and is, by data recovery standards, relatively inexpensive ($400 and up).    They also have a "no recovery, no fee" policy, so if they can't recover any of your data there's no charge unless you've authorized specialized cleanroom work as part of the attempt.  https://gillware.com/

 

However, it seems the format you did wasn't completed, so the drive is still unmountable.  You can still try what I suggested above by doing the following:

 

(a)  Do a New Config and assign ONLY the troublesome data drive to the array as the only data drive, with no other drive assigned (i.e. no parity).    Start the array and let UnRAID format the disk.

 

(b)  Now do another New Config and do as I suggested above [i.e. "Trust Parity" and IMMEDIATELY cancel the parity check that starts when the array is started.]

 

As I noted above, you may very well have already corrupted too much for this to work; but there's a chance that you'll be able to recover some (possibly a good bit) of your data => you'll know by looking at the emulated drive (after you've unassigned the physical drive).

 

[Monkeyair]

I followed the instructions and got the disk formatted:

 

 

I then did a new config and used the valid parity, started the array and stopped the parity check.  I then stopped the array and unassigned the disk to show missing.

 

When I restarted the array I see the following and no emulated drive in windows explorer:

 

 

If I then add the device back to the array I get the option to rebuild.  Should I do this?

[garycase]

No.    The emulated disk is "unmountable", so it's not going to help to rebuild the disk -- it will still be unmountable.  Your initial format went too far, and the parity/other disks now reflect that incomplete (and unmountable) state.

 

Your best option (assuming you don't have backups)  is to either use a professional recovery service; or you can try a file system scan/repair on the corrupted disk.    With Reiser disks the reiserfsck would often work wonders;  I'm not sure how well the equivalent utilities (xfs check and xfs repair) work with XFS disks.

 

 

[Fonzarelli]

Just a suggestion, ive had realy good luck with TestDisk for recovering data from repartitioned or formatted drives. Its free and quite simple to use.

[Monkeyair]

Just a suggestion, ive had realy good luck with TestDisk for recovering data from repartitioned or formatted drives. Its free and quite simple to use.

Thanks I'll take a look....

[Monkeyair]

Just a suggestion, ive had realy good luck with TestDisk for recovering data from repartitioned or formatted drives. Its free and quite simple to use.

So, I have ran this and the TestDisk got a bit complicated with the Linux file system.  I'll come back to that and see if I can get it to repair the XFS file format.

 

In the meantime TestDisks associated program PhotoRec is in the process of recovering my data.  It has 37 hours left to run and is finding all artwork, photos and movies files from the compromised disk.

 

So good, so far 

 

Thanks

[Monkeyair]

It's even recovering my music files.  I'm hoping that this means that I may be able to recover the disk format and data back to a state that will enable Unraid to mount it and get me out of jail.

 

Worst case is that I have a lot of file sorting and some extremely valuable lessons for the future! 

[garycase]

PhotoRec's documentation says that it restricts itself to read-only access to the disk you're recovering from ... which means it won't write to the disk you're doing the recovery on.    Is that how it's working?    You should be storing the recovered files on a different disk.    Assuming that's the case (hopefully it is, as ALL good data recovery programs do this so they can do a "safe recovery").

 

If that's indeed the case, it clearly is not going to make the disk mountable ... but once you've recovered all of your data, you can then format the disk and copy the data you've recovered back to it.

 

On the other hand, TestDisk is designed to actually modify the disk to attempt to recover partition information.  But even if that succeeds, the likelihood of all the data being good at that point is fairly small ... I'd still reformat the disk and copy your data back to it.

 

Yet another case where the first sentence of my write-up here hits home:

http://lime-technology.com/forum/index.php?topic=31020.0

... perhaps that write-up will be one of your "... valuable lessons for the future." 

 

 

 

[Monkeyair]

I'm just happy to recover the data.  I don't care how long it takes to copy it back.

 

I like your backup advice and shall embrace the lessons.  Cheers.

 

I'm a happy man 

 

Oh, and yes the data is being transferred to another disk.

[Fonzarelli]

Glad to hear the data recovery is successful. I've been in the same situation when you realize the catastrophic mistake you've made seconds after you've pressed the button 

[garycase]

Glad to hear the data recovery is successful. I've been in the same situation when you realize the catastrophic mistake you've made seconds after you've pressed the button 

 

That's usually the time when you suddenly realize that perhaps backing up isn't such a bad idea after all 

[RobJ]

Wouldn't it also be prudent to update the wiki for v6, since you don't need to drop down to a telnet session to check / repair file systems?

 

Wow!  I had NO IDEA that feature was available, never saw it myself, and I don't recall anyone else ever mentioning it!  Very nice addition, Tom and bonienl!  Please pat yourselves on the back.  I checked it and it is well implemented, only visible on the appropriate drives, and tailored to the appropriate tool for the drive's format.  Needs a little more help for advanced options, but that's true for the wiki too.

 

As soon as I have a chance, I'll add it to the wiki.

Yeah, I *think* its been there since dynamix was integrated (or soon thereafter).  Thought of updating the wiki myself, but you're the expert (or one of them) on drive recovery, so would hate to put incorrect information into the wiki

 

Please don't hesitate to add anything that comes to mind to the wiki (within reason of course)!  There's no such thing as an expert in the field of computers.  What you think you know all about today, is out-of-date or incomplete tomorrow.  And what one person adds to the wiki is generally quickly corrected by another.

 

I've finally added/integrated the webGui method into the Check Disk File systems wiki page, but I'm not particularly happy with it, found it rather difficult to insert for some reason.  Happy to hear others reviews, happier still to see others editing and improving the wiki.