Accidentally assigned data disk as parity

[Monkeyair]

I've done something very silly today.....

 

I had an old disk in my array that had failed, so I removed it and ran a new config to reduce the array by one drive.  I then added the drives, but put mixed up drive 1 and the parity.

 

Once the system started I stopped the parity synch within seconds, but my data cannot be reached.

 

I've tried following guidance for reiserfsck, but have had no joy.

 

Can anybody help recover my data?

 

My Crashplan backup was part way through, so I only have some of my docs backed up.

[itimpi]

I've done something very silly today.....

 

I had an old disk in my array that had failed, so I removed it and ran a new config to reduce the array by one drive.  I then added the drives, but put mixed up drive 1 and the parity.

 

Once the system started I stopped the parity synch within seconds, but my data cannot be reached.

 

I've tried following guidance for reiserfsck, but have had no joy.

 

Can anybody help recover my data?

 

My Crashplan backup was part way through, so I only have some of my docs backed up.

Are you sure the disk was in Reiserfs format and not XFS (which is now the default for new disks in v6) as the recovery tool is different.

 

You mention trying reiserfsck, but what EXACTLY are the commands you have tried to run, and the results.

 

Also, since you started the parity sync it may also be necessary to rebuild the partition table - do you get any partitions showing if you run gdisk against the disk in question?

[SSD]

I've done something very silly today.....

 

I had an old disk in my array that had failed, so I removed it and ran a new config to reduce the array by one drive.  I then added the drives, but put mixed up drive 1 and the parity.

 

Once the system started I stopped the parity synch within seconds, but my data cannot be reached.

 

I've tried following guidance for reiserfsck, but have had no joy.

 

Can anybody help recover my data?

 

My Crashplan backup was part way through, so I only have some of my docs backed up.

 

What "following guidance"?

 

You have a good chance of recovering all or most of your data, depending on how quickly you stopped the parity build. But you have some work ahead of you.

 

Look HERE for a similar story of woe from ME in my early days of unRAID usage. You won't have the issue requiring Spinrite. The arguments to the reiserfsck have not changed, although last time I ran it I found the order of the arguments needed to be tweaked.

 

But please confirm that your disk was indeed RFS formatted. If it was formatted with XFS, obviously this won't work.

[Monkeyair]

The disks were XFS formatted.

 

This is the gdisk output:

 

 

This is the guidance that I followed:

 

So back to the first disk that had been in the parity slot for a short time.  Luckily Tom was on-line and helped me figure out what to do.  I ran “reiserfsck /dev/md1” (UPDATE 8/8/11: If you ever run on on an OS level device rather than an unRAID device, make sure to use the first partition - ie., “reiserfsck /dev/sdc1", and not the disk level device.)  (and it questioned whether this was really a reiserfs formatted drive.  It told me to run with the “--rebuild-sb option” if I was sure.  If ever you have to run with this option, it asks you a bunch of questions.  Here are the correct responses (many thanks to Tom for helping me through these):

 

1.  The version of reiserfs is 3.6.x.  This is for unRAID 4.2.1.  (This is NOT the default, so be careful)

2.  Block size is 4096 (default)

3.    “No journal device was specified.  (If journal is not available, re-run with --no-journal-available option specified)”  Is journal default?”  (Answer Y)

4.  “Do you use resizer?” (Answer N)

5.  It tells you that a new uuid has been generated. 

6.    “rebuild-sb: You either have a corrupted journal or have just changed the start of the partition with some partition table editor. If you are sure that the start of the partition is ok, rebuild the journal header. Do you want to rebuild the journal header?”  Answer Y

7.  The following info is displayed:

 

Reiserfs super block in block 16 on 0x901 of format 3.6 with standard journal

Count of blocks on the device: 73264320

Number of bitmaps: 2236

Blocksize: 4096

Free blocks (count of blocks - used [journal, bitmaps, data, reserved] blocks): 0

Root block: 0

Filesystem is NOT clean

Tree height: 0

Hash function used to sort names: not set

Objectid map size 0, max 972

Journal parameters:

        Device [0x0]

        Magic [0x0]

        Size 8193 blocks (including 1 for journal header) (first block 18)

        Max transaction length 1024 blocks

        Max batch size 900 blocks

        Max commit age 30

Blocks reserved by journal: 0

Fs state field: 0x1:

        some corruptions exist.

sb_version: 2

inode generation number: 0

UUID: <my UUID removed>

LABEL:

Set flags in SB:

Is this ok ? (y/n)[n]:

 

8.  Answer “Y”. 

9.  With amazing speed the program does its thing and ends

[Monkeyair]

Bjp that was your guidance 

[SSD]

You cannot use a program designed to recover an RFS (reiserfs) formatted disk to recover an XFS formatted disk!

 

There was a recent post by RobJ assisting a user with an XFS repair. Will try to find and post a link.

[SSD]

Here is the link to the thread discussing XFS repair.

 

http://lime-technology.com/forum/index.php?topic=41698.msg395954

[Monkeyair]

Thanks a lot

[SSD]

I just updated the ancient post to clarify that reiserfsck is only for reiserfs formatted disks.

[RobJ]

I'm too late for you, but I JUST LAST NIGHT updated the Check Disk File systems wiki page, to add support for XFS repair.  And just now added a strong warning about knowing your file system format and using the right tool.

 

I should probably add those question answers for the reiserfsck --rebuild-sb option, not that that would have helped you!  Sorry!

[Monkeyair]

I'm too late for you, but I JUST LAST NIGHT updated the Check Disk File systems wiki page, to add support for XFS repair.  And just now added a strong warning about knowing your file system format and using the right tool.

 

I should probably add those question answers for the reiserfsck --rebuild-sb option, not that that would have helped you!  Sorry!

 

Nothing to be sorry for! it's my own silly fault.  I can't believe that I did this....

 

The disks are now both running the XFS repair, so hopefully all will not be lost.

 

Thanks for replying.

[itimpi]

I'm too late for you, but I JUST LAST NIGHT updated the Check Disk File systems wiki page, to add support for XFS repair.  And just now added a strong warning about knowing your file system format and using the right tool.

 

I should probably add those question answers for the reiserfsck --rebuild-sb option, not that that would have helped you!  Sorry!

 

Nothing to be sorry for! it's my own silly fault.  I can't believe that I did this....

 

The disks are now both running the XFS repair, so hopefully all will not be lost.

Both   I thought you said it was one disk plus parity?  The Parity disk does not contain a file system so there is nothing to repair on that disk!  You will simply have to regenerate it from all the data disks.

[Monkeyair]

Every day is a school day! 1 is a data drive, the other is the parity drive that is having nothing repaired 

 

From the Wiki:

 

Very important!!! Do NOT run these tools on the parity drive. It does NOT have a file system, and running ANY file system repair tool on your parity drive can corrupt it!

[RobJ]

Would you mind if we point to your thread, as an example of what *NOT* to do?   

 

Sorry, I just could not resist writing that!  And I need to stop laughing too!         

 

On a more serious tone, dual parity would have been a life saver here, as it would have allowed you to safely rebuild the data drive.

[Squid]

I'm too late for you, but I JUST LAST NIGHT updated the Check Disk File systems wiki page, to add support for XFS repair.  And just now added a strong warning about knowing your file system format and using the right tool.

 

I should probably add those question answers for the reiserfsck --rebuild-sb option, not that that would have helped you!  Sorry!

Wouldn't it also be prudent to update the wiki for v6, since you don't need to drop down to a telnet session to check / repair file systems?

[Monkeyair]

Would you mind if we point to your thread, as an example of what *NOT* to do?   

 

Sorry, I just could not resist writing that!  And I need to stop laughing too!         

 

On a more serious tone, dual parity would have been a life saver here, as it would have allowed you to safely rebuild the data drive.

Dual parity? Sounds like a good idea to me.

 

No probs, go ahead and point all you like.  Anything that helps someone else that takes their eye off the ball.  I won't be the last I'm sure 

[RobJ]

I'm too late for you, but I JUST LAST NIGHT updated the Check Disk File systems wiki page, to add support for XFS repair.  And just now added a strong warning about knowing your file system format and using the right tool.

 

I should probably add those question answers for the reiserfsck --rebuild-sb option, not that that would have helped you!  Sorry!

Wouldn't it also be prudent to update the wiki for v6, since you don't need to drop down to a telnet session to check / repair file systems?

 

Wow!  I had NO IDEA that feature was available, never saw it myself, and I don't recall anyone else ever mentioning it!  Very nice addition, Tom and bonienl!  Please pat yourselves on the back.  I checked it and it is well implemented, only visible on the appropriate drives, and tailored to the appropriate tool for the drive's format.  Needs a little more help for advanced options, but that's true for the wiki too.

 

As soon as I have a chance, I'll add it to the wiki.

[Squid]

I'm too late for you, but I JUST LAST NIGHT updated the Check Disk File systems wiki page, to add support for XFS repair.  And just now added a strong warning about knowing your file system format and using the right tool.

 

I should probably add those question answers for the reiserfsck --rebuild-sb option, not that that would have helped you!  Sorry!

Wouldn't it also be prudent to update the wiki for v6, since you don't need to drop down to a telnet session to check / repair file systems?

 

Wow!  I had NO IDEA that feature was available, never saw it myself, and I don't recall anyone else ever mentioning it!  Very nice addition, Tom and bonienl!  Please pat yourselves on the back.  I checked it and it is well implemented, only visible on the appropriate drives, and tailored to the appropriate tool for the drive's format.  Needs a little more help for advanced options, but that's true for the wiki too.

 

As soon as I have a chance, I'll add it to the wiki.

Yeah, I *think* its been there since dynamix was integrated (or soon thereafter).  Thought of updating the wiki myself, but you're the expert (or one of them) on drive recovery, so would hate to put incorrect information into the wiki

[Monkeyair]

So, I did the XFS repair and I got this:

 

 

This is the Disk info:

 

 

And after running a new config, I see that this disk info in the unassignded devices shows a reiserfs file system:

 

 

So have I screwed this disk up by running a program designed to recover an RFS (reiserfs) formatted disk to recover an XFS formatted disk?

 

Any way back do you think?

[SSD]

There is very little experience here with this xfs repair tool.

 

I would say things don't look good. Suggest you do some Google searching. You could reach out to a data recovery service.

[Monkeyair]

Ok. Thanks anyway

[garycase]

On a more serious tone, dual parity would have been a life saver here, as it would have allowed you to safely rebuild the data drive.

 

Unfortunately that's not the case vis-à-vis what was done here.    He did a New Config and assigned the wrong drive as parity on a system that already had a failed drive.    What he COULD have done was simply replace the failed drive and let it rebuild instead of doing the new config ... and dual parity would have protected against a 2nd drive failure during that process.

 

But the New Config eliminated the possibility of doing that as soon as the data drive had been written to.    Even if dual parity was already implemented, and a New Config allowed you to "trust" either or both parity drives, it would not have helped here, as there was (a) already one failed drive; and (b) once the data drive had been written to, it was effectively a 2nd failed drive.

 

[Monkeyair]

Fortunately I have managed to mount the faulty drive for now.  It must be an intermittent fault.  So I only have 1 failed drive for the moment.

 

My parity now shows as valid, but when the array starts no rebuild option is available as the data disk that I used the wrong recovery program on is unmountable.

 

The option is given to me to create a file system and format the disk.  I want the disk to mount and rebuild the parity. 

 

I initially selected format but bottled it and restarted the system.  What is the correct thing to do? Have I compromised the data on my parity drive by letting the data disk format for a while?

[Monkeyair]

I now see that format was a BIG mistake!  :'(

[SSD]

If you are unsure if parity is valid based on the current disks in the array, you can always run a correcting parity check. UnRaid will update parity to be in sync with the drives. Even if parity is "way off", unRaid will slug through and at the end of the parity check parity will be correct.

 

The worst thing you can do with a disk you are trying to recover data from is write to that disk. Formatting is a form of writing, as is running a recovery program. It is best to ponder the options and only do a form that requires writing after exploring the non-destructive forms of recovery. And as you have learned, run the right recovery program for the right disk format.

 

Glad you were able to recover one of your data drives. Best of luck on the other. Sorry for the trouble.