[SOLVED] Reiserfsck Rebuild Tree / Unformatted Drive Woes


Recommended Posts

Was running into problems with the mover script, and a drive appeared to be Read Only.  So through research looks like a reiserfsck --check was needed.  I did so and it suggested rebuild tree.  Ran that.  Then checked again.  Suggested rebuild tree still.  Ran one more time. Same error.

 

Will read-only check consistency of the filesystem on /dev/md1
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
###########
reiserfsck --check started at Sun Dec 30 23:54:59 2012
###########
Replaying journal: Done.
Reiserfs journal '/dev/md1' in blocks [18..8211]: 0 transactions replayed
Checking internal tree.. \block 126746674: The level of the node (2) is not correct, (4) expected
the problem in the internal node occured (126746674), whole subtree is skipped                       finished
Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs.
Bad nodes were found, Semantic pass skipped
1 found corruptions can be fixed only when running with --rebuild-tree
###########
reiserfsck finished at Sun Dec 30 23:57:04 2012
###########
root@Tower:~#

 

Now if I start the array, it shows the drive as unformatted.

 

So looking what I should try next  :-\

 

Attached is my syslog as well.

 

syslog-2012-12-31.zip

Link to comment

unformatted simply means "not able to be mounted as a reiserfs file system"

 

Once you fix the corruption, it should mount. 

 

DO NOT FORMAT THE DRIVE UNLESS YOU WANT TO ERASE ALL THE CONTENTS!!!!!!!

 

Just run

reiserfsck --rebuild-tree /dev/md1

If you are running a "long" test, make sure you disable spin-down, otherwise the test will abort when the disk spins down and never complete.

Link to comment

Here is my smart report.

 

Looks like I have a crap ton of relocated sectors, which I would assume this drive might be on its way out.  Might be why 2 --rebuild-tree runs aren't actually fixing the drive.

 

I have a drive that I just added to the array that is the same size and doesn't have much on it.  What steps should I take (assuming I remove whats on the drive) to use it to replace the failing drive.

smart.txt

Link to comment

Show the output with --rebuild-tree

 

Output is attached.

 

Note this is the third time to run --rebuild-tree.  I didn't export the command to a text file, and was unable to get all of the output through telnet :\  However I think from below you can see the significant part or where --rebuild-tree is doing something for better or worse.

 

Should be noted, that my drive is still showing as unformatted  :-[

rebuild-tree.txt

Link to comment

It appears as if the --rebuild-tree completed just fine.

 

Why did you run it three times?  Did it not complete the previous attempts?

 

You should not be able to run

reiserfsck --check

on the disk and have it show no errors.  Once you get to that point, just reboot the server, it should mount.

Link to comment

Should be noted, that my drive is still showing as unformatted  :-[

to unRAID, any disk it cannot mount, for any reason, is labeled as un-formatted.  That label will not go away until the disk is mounted. (and that can most easily be done by rebooting and have unRAID attempt to mount the disk again.)
Link to comment

It appears as if the --rebuild-tree completed just fine.

 

Why did you run it three times?  Did it not complete the previous attempts?

 

You should not be able to run

reiserfsck --check

on the disk and have it show no errors.  Once you get to that point, just reboot the server, it should mount.

 

Did it once, ran --check to see if it was good.  Wasn't good, so figured it maybe needed another pass to correct what was left.  Didn't work.  Then dgaschk wanted to know what it reported back, so ran it a third time. 

 

Check previous post of mine as to the error I get when I run check.. Still (even after 3 times) tells me to rebuild tree.

 

I haven't rebooted it after this last --rebuild-tree so I will do so now.  I have stop and restarted the array and it did not mount the drive.

 

Thanks for your help.

 

UPDATE:

Still didn't mount (unformatted) after reboot.

Link to comment

It appears as if the --rebuild-tree completed just fine.

 

Why did you run it three times?  Did it not complete the previous attempts?

 

You should not be able to run

reiserfsck --check

on the disk and have it show no errors.  Once you get to that point, just reboot the server, it should mount.

 

Did it once, ran --check to see if it was good.  Wasn't good, so figured it maybe needed another pass to correct what was left.  Didn't work.  Then dgaschk wanted to know what it reported back, so ran it a third time. 

 

Check previous post of mine as to the error I get when I run check.. Still (even after 3 times) tells me to rebuild tree.

 

I haven't rebooted it after this last --rebuild-tree so I will do so now.  I have stop and restarted the array and it did not mount the drive.

 

Thanks for your help.

 

UPDATE:

Still didn't mount (unformatted) after reboot.

I'd perform a memory test on your server.  If the --rebuild-tree keeps saying it was successful, and if the SMART report on the disk shows it is not experiencing un-readable sectors, then possibly something else is involved (memory, mb disk controller, etc)  Easiest to rule out is memory.

 

Reboot server, choose the memory test, let it run for several cycles.

 

Joe L.

Link to comment

Ouch, that's too bad you didn't see the bad sector issues before you went any further. You had a chance to simply replace the drive but you buggered the parity by messing with the failed drive.

 

At this point, I would get another drive to replace this failed drive and rebuild onto it. It will likely still be unformatted but reiserfsck might be able to fix a healthy drive.

 

Link to comment

I'd perform a memory test on your server.  If the --rebuild-tree keeps saying it was successful, and if the SMART report on the disk shows it is not experiencing un-readable sectors, then possibly something else is involved (memory, mb disk controller, etc)  Easiest to rule out is memory.

 

Reboot server, choose the memory test, let it run for several cycles.

 

Joe L.

 

Ran memory test while I was out last night, 7 Passes clear with no errors :\

 

I have a replacement drive that should arrive today.  Just need to get it cleared.

 

Joe do you recommend as lionelhutz pointed out to just  rebuild it and try to reiserfsck it again?  Definitely willing to try anything else though.

Link to comment

I'd perform a memory test on your server.  If the --rebuild-tree keeps saying it was successful, and if the SMART report on the disk shows it is not experiencing un-readable sectors, then possibly something else is involved (memory, mb disk controller, etc)  Easiest to rule out is memory.

 

Reboot server, choose the memory test, let it run for several cycles.

 

Joe L.

 

Ran memory test while I was out last night, 7 Passes clear with no errors :\

 

I have a replacement drive that should arrive today.  Just need to get it cleared.

 

Joe do you recommend as lionelhutz pointed out to just  rebuild it and try to reiserfsck it again?  Definitely willing to try anything else though.

Not likely to be memory then.

 

Just to be clear, if you run a --rebuild-tree, followed by a --check, the --check recommends another --rebuild tree?

 

Since the only disk involved in the --rebuild-tree is the one being rebuilt, you can give a try of re re-constructing onto a replacement disk... 

 

Be aware however that result will depend upon all the other disks and hardware in the server.  If you trust them, then go for it.

 

Joe L.

Link to comment

I'd perform a memory test on your server.  If the --rebuild-tree keeps saying it was successful, and if the SMART report on the disk shows it is not experiencing un-readable sectors, then possibly something else is involved (memory, mb disk controller, etc)  Easiest to rule out is memory.

 

Reboot server, choose the memory test, let it run for several cycles.

 

Joe L.

 

Ran memory test while I was out last night, 7 Passes clear with no errors :\

 

I have a replacement drive that should arrive today.  Just need to get it cleared.

 

Joe do you recommend as lionelhutz pointed out to just  rebuild it and try to reiserfsck it again?  Definitely willing to try anything else though.

Not likely to be memory then.

 

Just to be clear, if you run a --rebuild-tree, followed by a --check, the --check recommends another --rebuild tree?

 

Since the only disk involved in the --rebuild-tree is the one being rebuilt, you can give a try of re re-constructing onto a replacement disk... 

 

Be aware however that result will depend upon all the other disks and hardware in the server.  If you trust them, then go for it.

 

Joe L.

 

Correct.  Each --rebuild-tree has been followed with a check which asks for another --rebuild-tree.. Almost in the exact fashion.  I'll run checks on the rest of the drives just to be safe before proceeding.. 

 

It will be a a day or so since I will need to preclear, so if anyone does come up with an idea to try let me know.  Else I'll report back if I am able to reconstruct the disk...

 

Joe do I just need to bring array down, and replace disk in config with new disk and bring the array back up? (never actually replaced a disk before, but have read about quite a bit)

Link to comment

Update...

 

Replaced the drive, and ran the --rebuild-tree again and it appeared to fix it.  Currently running a parity check, but the drive came back as formatted so I am hoping we are good. 

 

Thanks all for the help!  Hopefully this will one day help someone else.

Link to comment

Update...

 

Replaced the drive, and ran the --rebuild-tree again and it appeared to fix it.  Currently running a parity check, but the drive came back as formatted so I am hoping we are good. 

 

Thanks all for the help!  Hopefully this will one day help someone else.

 

Run with --check again to confirm success.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.