Data Rebuild Mistake

[cpthook]

Hello Forum-  I'm hoping to get some assistance with recovering data from a disk that has gone bad.  Here is the story briefly:

 

I decided to upgrade drive 9 to a larger size, but during the Data Rebuild process Drive 7 started to return errors, however Drive 9 rebuilt successfully. So, I decided to replace/upgrade Drive 7 to a larger size but this time I was asked to Format.  Not sure why, as I did not select any changes to the Drive file system (reiserfs).  Well as you can guess, I selected the Format option which deleted all emulated data from Drive 7.  Now of course there is no data showing on Drive 7 during the rebuild process.  In an attempt to recover the data I tried the following:

 

1.  I tried to mount the old Drive using the Unassigned Devices plg which recognizes the Drive and the FS, but wont mount it

2.  I tried to recover in windows using this tool: http://www.diskinternals.com/linux-reader/ but it states that the "FS cannot be recognized"

 

I ran the reiserfsck --check /dev/sdo and attached the results

 

 

Thanks, in advance for any assistance provided!!!!

 

[trurl]

You need to run reiserfsck on the sdo1 partition, not the sdo device. Try again and post the results.

[JorgeB]

If there were read errors on disk7 during the rebuild of disk9 then disk9 will have some corrupted files, this in turn would make the parity invalid and the following disk7 rebuild unmountable.

 

If you have old disk9 you should use a file compare util with the new disk9, or copy data from the old disk and replace all files.

 

 

 

 

[cpthook]

You need to run reiserfsck on the sdo1 partition, not the sdo device. Try again and post the results.

 

New results posted per your instructions.

 

If there were read errors on disk7 during the rebuild of disk9 then disk9 will have some corrupted files, this in turn would make the parity invalid and the following disk7 rebuild unmountable.

 

Looks like I did a really good job of hosing up my server's files HUH!?   

 

If you have old disk9 you should use a file compare util with the new disk9, or copy data from the old disk and replace all files.

 

 

File compare and copy files from old disk sounds like a winner to me and I would do what you advise for disk7 if I could mount the old disk.  I tried both MC and my Krusader docker and could not see the old drive.  I still have it connected to my server VIA USB adapter, but "Unassigned Devices" wont mount the drive.  When placed back natively to the Array, unRAID says it's "un-mountable".  Could I be doing something better to reach the files on the old disk 7? 

 

Thanks guys for your support and advice, and please keep the ideas coming if possible.

 

EDIT

RE: Disk 9, I will follow your instructions as the old disk can still be mounted and recognized.   

[JorgeB]

For disk7 you should run reiserfsck on the old disk, but don't delete the rebuilt disk7 for now, if you can't recover from the old disk it may be possible to recover something from the new as a last resort, although if you don't have checksums of your files it will be difficult to know which ones are corrupt.

[cpthook]

For disk7 you should run reiserfsck on the old disk, but don't delete the rebuilt disk7 for now,

I posted the results from reiserfsck on the old disk 7 in my prior post.  So, Disk 7 rebuild is now complete and I will leave it in place per your advice.  unRAID Report VIA e-mail:

 

Event: unRAID Data rebuild: Subject: Notice [TOWER] - Data rebuild: finished (55288543 errors) 

 

This happens to be the exact same notice reported when disk 9 rebuild finished, which reinforces your comment about data on it being corrupt as well.  I assume running a Parity Check at this point will return the same bad news. 

 

if you can't recover from the old disk it may be possible to recover something from the new as a last resort, although if you don't have checksums of your files it will be difficult to know which ones are corrupt.

 

Ok I will go this route if we cant restore from the old disk.  I'm anxiously awaiting trurl to interpret the results of the reiserfsck results I posted and praying for good news! 

[JorgeB]

Ok I will go this route if we cant restore from the old disk.  I'm anxiously awaiting trurl to interpret the results of the reiserfsck results I posted and praying for good news!

 

You have to run reiserfsck with the --rebuild-tree option.

 

reiserfsck --rebuild-tree /dev/sdo1

[JorgeB]

This happens to be the exact same notice reported when disk 9 rebuild finished, which reinforces your comment about data on it being corrupt as well.  I assume running a Parity Check at this point will return the same bad news. 

 

This is a bug, a rebuild reports the same errors as the previous rebuild or parity check, unless there were more read errors the rebuild was done successfully, the problem is that because of the 55288543 on the previous rebuild there will be a lot of corrupt files.

[cpthook]

Ok I will go this route if we cant restore from the old disk.  I'm anxiously awaiting trurl to interpret the results of the reiserfsck results I posted and praying for good news!

 

You have to run reiserfsck with the --rebuild-tree option.

 

reiserfsck --rebuild-tree /dev/sdo1

 

Ok it's running..  Looks like it will take awhile, will report results after I return from work. 

[cpthook]

Looks like it aborted due to I/O error from a bad block.  Results attached :'(

[cpthook]

2nd try looks like it finished correctly.  Results posted, but not sure how to proceed from here.

[JorgeB]

Copy all the data you can from that disk to array, check for a lost+found folder, there could be some data there.

[cpthook]

Copy all the data you can from that disk to array, check for a lost+found folder, there could be some data there.

 

ok....  I figured that, but paranoia had me wait for confirmation and was amazed when the drive mounted with UD after so many times the mount option failed.

 

Fired up MC and copied all data from the old disk7 to the new disk7, however another problem still exists with drive9. The "Fix Common Problems" plug-in shows drive9 mounted as read-only, and my syslog shows reseirfs warnings/errors on Device md9.  The Syslog showing this information is attached.  So based upon your earlier conclusions and these errors, I'm thinking that I will have to fire up the New Config utility and rebuild my Parity drive.  I compared the files on the new Drive9 with the old Drive9 and they all seem to be in place.  Additionally, the USED/FREE percentage on the new drive9 is identical to the old.

 

Please advise on weather I'm incorrect or missing anything with my plan going forward.....

[JorgeB]

Disk9 rebuild had errors, so it's normal to have file system issues, besides there are almost certainly corrupt files, they look the same and occupy the same space but can be corrupt.

 

By using a file compare util I meant to use one that compares files on a byte level, but since you have the old disk I would format disk9 and copy all data from the old disk.

 

Simplest way to format is to stop the array, click on disk9, change filesystem to XFS, start array, disk 9 will appear as unmontable, click format.

 

You can leave it XFS, but if you want reiser just change fs back to reiser and reformat, then copy all data from old disk9.

 

PArity will remain valid.

[cpthook]

 

but since you have the old disk I would format disk9 and copy all data from the old disk.

 

Simplest way to format is to stop the array, click on disk9, change filesystem to XFS, start array, disk 9 will appear as unmontable, click format.

 

You can leave it XFS, but if you want reiser just change fs back to reiser and reformat, then copy all data from old disk9.

 

PArity will remain valid.

 

Thanks for all of your time and assistance.  I was able to salvage my data and get my server back up and running.  Issue marked Solved!