Drive-Failure/Replacement Data Rebuild Robustness

[magic144]

Hi, first time here on the forum and a brand new unRAID OS (and Hardware) owner!

 

My question is this, and I couldn't find the answer after a quick search of the forums so far:-

 

if a drive fails and I replace it and a Data Rebuild kicks off to re-construct thre replaced drive, what happens if something (e.g. a power cut) happens during that rebuild process and the server is rudely interrupted!?

 

I ask this because I'm currently running with 4*1.5TB Seagate drives (1 of which is obviously parity).

I had some initial data on the 1st data drive and I wanted to see what the proces would be like, so I shutdown the array, pulled out that one drive with the current data on it and restarted. I was comforted to see that the array still started, serving the data from my 'virtual' missing drive no probs.

 

When I reinserted the physical drive and wanted to perform a Data Rebuild to see what it would look like if I REPLACED the drive, I purposefully chose this option (rather than Restore) and the server duly obliged and set off on a Data Rebuild. This is due to take 8 hours! Therfore, the chances of something going wrong in such a time period seem high enough to prompt my question!!

 

This also prompts a follow-on question - if I have 15*1.5TB drives instead of just 4, what will that do the Data Rebuild time?

 

Thanks in advance for all of your feedback and advice :-)

 

m

[magic144]

Hmm, only just noticed the array is actually started/shared DURING the Data Rebuild - I assume I should avoid writing data during the rebuild (or the system will just cope anyway)??

 

Is it possible/advisable to Stop the array as a precaution during the rebuild (if, for example, you're doing the rebuild overnight and heading to bed) - and so does stopping the array (Stop button, NOT Cancel button) not stop the rebuild and allow it continue on the off-line array??

 

Thanks again,

 

m

[SSD]

Do not stop the rebuild!  The rebuild will restart on powerup, not resume.

 

unRAID is pretty robust in allowing you to access the array while rebuilds, parity builds, and other activities.  I'm a bit of a chicken myself, though.  If it is doing a heavy duty activity I tend to let it do its thing without trying to do much else in parallel.

 

Good luck!

[magic144]

Thanks bjp999!

 

Don't think I made myself clear, but you have answered 1 of my questions... if something goes wrong during a rebuild, e.g. a power cut, the rebuild process will start from scratch after a power-cycle :-)

 

Actually in my second post I was wondering if I could stop (i.e. take off-line) the array (not the rebuild) and allow the rebuild to continue (there is a STOP button to stop the array, and a CANCEL button to stop the rebuild - I meant the former)

 

Cheers,

m

[SSD]

If you stop the array the rebuild will stop.

[magic144]

OK then! Best to just leave it all alone to do its thing - thanks for the answer :-)

 

Any ideas on scalability time-wise? Looks now (I'm approaching the 70% complete mark) like it will take about 6 hours for these 4*1.5TB drives to rebuild 1. Anybody know how that would scale with additional array drives?

 

Cheers,

m

[Joe L.]

OK then! Best to just leave it all alone to do its thing - thanks for the answer :-)

 

Any ideas on scalability time-wise? Looks now (I'm approaching the 70% complete mark) like it will take about 6 hours for these 4*1.5TB drives to rebuild 1. Anybody know how that would scale with additional array drives?

 

Cheers,

m

The time it takes is entirely dependent on your hardware... if your drives are on PCIe bus, it will be much faster. If they all share a PCI bus, slower.

To rebuild a drive unRAID must read the equivalent block of data from all the other drives.  That is a lot of data to move through the PCI bus.

 

If some of your drives are smaller than the rest the speed will increase once the rebuild gets past their sizes.

 

Look here for examples of parity speed calculations (rebuilding a drive usually takes about the same time as a parity calculation) :

http://lime-technology.com/wiki/index.php?title=User_Benchmarks

 

Some friendly advice, get an inexpensive UPS for your server.  It will let it get through most power blips.

 

Joe L.

[SSD]

Parity check / drive rebuild speeds tend to stay pretty consistent as you add new drives if you are able to use PCIe controllers. With PCI controllers, these I/O intensive tasks will slow dramatically with each drive (after the first 2-3).

 

Note that for normal drive use, there is not a slowdown with the PCI controllers.

[magic144]

Thanks again!

 

I have bought the MD-1510/LI from Lime-Tech themselves.

This is supposed to have PCIe, so I guess that is the best option :-)

 

FYI the rebuild finished A-OK after ~6 hours I think (it was late, I was in bed!)

I seem to recall a figure of about 50+MB/s too.

 

Are the stats of the rebuild (speed/time) kept anywhere on the system after the fact?

 

Cheers,

m

[SSD]

Parity check times are logged in the syslog. I'd expect rebuild speed would as well. Follow the Troubleshooting link in my sig for instructions.

[magic144]

hmm, except I've power-cycled and I think my syslog is gone/overwritten...

 

oh well, at least I've got a ball-park figure now and I won't be too scared of it when one of my drives dies for real :-)

 

m