Extremely Slow Disk Rebuild

[airic499]

Hello all,

 

I see a few topics that are similar, but none quite the same, so pardons if I have missed the solution somewhere.  I had a disk red ball on a solid box using version 5.0.2.  Spun everything down, replaced the problem drive with a pre-Cleared WD 3TB Green, spun back up everything normal.  Started format drive, as normal then the system moved on to the re-build.  The problem is the rebuild now shows anticipated completion time 47 days. 

 

I have rebooted a couple of times.  Checked all the drive connections, so running out of physical things to try.  The log is attached.  Far from good at reading logs, but I do see error handlers getting repeatedly called.  I see a couple of things that catch my attention, but no idea what to do about any of them?

 

Hoping for any help here!

 

Thanks

Tower_2_Log.txt

[dgaschk]

Disk 9 could have a bad or loose SATA cable.

 

Rebuilding a disk does not require a format. I hope that you did not format anything because this will cause the data disk to be formatted and thus empty. Rebuilding a disk reconstructs the entire partition; this includes the format.

[airic499]

Thanks,

 

Yes checked cables and seating first.  No problems there.  Disk was freshly pre-cleared, so no data to lose, but wouldn't rebuild without format.  No idea if that is significant?

[airic499]

Tried swapping with a second freshly pre-cleared drive and red balled after just a few minutes of rebuild.  Checked the original problem drive and it checks out fine on another computer.  Starting to think it must be a channel issue. 

 

I would like to at this point, just remove that drive slot for the time being.  But searching the forum that seems not to be possible?  Any suggestions?

[itimpi]

I would like to at this point, just remove that drive slot for the time being.  But searching the forum that seems not to be possible?  Any suggestions?

The only way you can currently reduce the number of drives is to use the new config utility to redefine your array and omit the drives you no longer want.

[airic499]

Thanks Itimipi,

 

I was looking at that option.  It does come with a pretty dire warning though about not to use if failed disk, which is what I have.  I should have plenty of room on existing disks, if using that option does in fact, I believe the term is, "trust existing parity" and rebuild from that?

[dgaschk]

If your removing all of the failed disks then New Config is applicable. Parity will have to be rebuilt after selecting New Config. Do NOT choose to trust existing parity. Try a new SATA cable first.

 

Thanks,

 

Yes checked cables and seating first.  No problems there.  Disk was freshly pre-cleared, so no data to lose, but wouldn't rebuild without format.  No idea if that is significant?

 

This sounds like your adding a new disk to an existing array rather than replacing an existing data disk.

[airic499]

No replacing red balled disk in existing array.  It is the re-build from parity that is taking forever.

[dgaschk]

Did the disk that is being replaced have data on it? This data should still be accessible via shares even if the physical disk has failed.

[airic499]

@dgaschk Yes, I can still get to the data via the share.  In fact I plugged the removed "bad" disk into my Linux box and it mounted and passes an extended SMART test.  So not worried about the data.  In point of fact, this is my backups server, so theoretically there is no good reason not to blow it all away and start over (other than I'm lazy and don't want to copy it all off and back  )

 

I am mostly concerned what is happening with the system overall, and that particular drive slot.  I did read one thread that sounded somewhat similar and what seems to have been the "fix" was a new flash drive.  Drive is only a bit over a year old, so that doesn't seem likely.  I took it out and ran a scandisk, no errors so don't think that is it. So looking for comfort level of getting to root cause of the problem, and then fixing that issue. 

 

Again, no expert reading these logs, but it looks to me like the system isn't accepting the "new" drive as having replaced the old drive and then failing with each attempt to restore. To me though if that was the case, then there should be no progress towards rebuilding, but it appears there is progress, just very slow.

 

itimpi's idea of the New command is tempting.  I would prefer that it not wipe whatever data is left on existing disks however if someone can let me know for certain there.  I do realize that anything from the missing disk that exists only in parity would be lost.  But files that exist currently in the existing file structure of the non-problem disks would be preserved correct?

 

Again, thanks for everyone's help!

 

[dgaschk]

Check the power connection. See here: http://lime-technology.com/forum/index.php?topic=4068.msg273535#msg273535

[airic499]

Yes, I am thinking there is something just messed up with the drive slot.  Unfortunately, it is one of those NORCO 3116 cases, so the power all comes from the same backplane.  So I am likely looking at replacing the backplane.  F