40+ TB of unRAID setup, want to add a new drive and avoid a long downtime?

[HiSoC8Y]

Ok

 

So last time I added a new disk to my array, I had to stop my array which caused a downtime for a long time until the drive was pre-cleared and then the array came online.

 

I want to avoid this long downtime.  Theoretically, I have a windows PC and have an external drive case that I can insert the new disk to, and want to pre-clear it, then add it to my unRAID array (which can take few minutes only), which will be fast.

 

so is this possible? if yes, how?

[itimpi]

Is this going to be an extra drive on the unRAID system, or is intended to replace an existing drive?

 

Your question makes it sound like an additional drive?  If so the easiest approach is to plug the drive into the unRAID a system and use the pre-clear script to clear it prior to adding it to the array.  That keeps the array downtime to a minute or so.

[HiSoC8Y]

Is this going to be an extra drive on the unRAID system, or is intended to replace an existing drive?

 

Your question makes it sound like an additional drive?  If so the easiest approach is to plug the drive into the unRAID a system and use the pre-clear script to clear it prior to adding it to the array.  That keeps the array downtime to a minute or so.

 

An additional drive.

 

But when it pre-clears, the whole array is down, and last time it took over 15 hours. Can't I just pre-clear it on a separate system, not the unRAID system?

[itimpi]

Is this going to be an extra drive on the unRAID system, or is intended to replace an existing drive?

 

Your question makes it sound like an additional drive?  If so the easiest approach is to plug the drive into the unRAID a system and use the pre-clear script to clear it prior to adding it to the array.  That keeps the array downtime to a minute or so.

 

An additional drive.

 

But when it pre-clears, the whole array is down, and last time it took over 15 hours. Can't I just pre-clear it on a separate system, not the unRAID system?

You could, but why not do it on the unRAID system using the script describe in this thread?  That is what most of us do if we have the free SATA port on the unRAID system.

 

Using the pre-clear script rather than letting unRAID do it means the pre-clear happens on the unRAID system with the array still running.  Only when the pre-clear completes successfully do you stop the array for the minute or so it takes to add it to the array.  This approach also has the advantage that as you are testing the drive 'in-situ' there is less chance of disturbing cables at the last moment when you add the drive.  A final point about the script is that it also carries out confidence checks on the drive - something the unRAID built-in pre-clear does not.

 

If you are going to use a telnet session (rather than the unRAID console)  to run the pre-clear script mentioned above then you might want to look into using 'screen' to control the pre-clear.  If you do not then closing the telnet window aborts the pre-clear and since on a large drive pre-clear can run for days this is a real danger.

 

At some point I think it is likely that unRAID v6 will include an equivalent to the above pre-clear script as standard, but it has not happened yet and the free-standing script is still the recommended way to do it.

[garycase]

You're correct that if you clear the drive within UnRAID it takes a long time, and the array is out of service for that time.

 

But if you PRE-Clear it, using the pre-clear script, it only takes a few minutes to add it to the array (actually it only takes a couple seconds -- but it then needs to be formatted to actually use, and that takes a few minutes).

 

You can pre-clear it on your UnRAID server without any impact to the array ... or you can use a different PC if that's more convenient.

 

[HiSoC8Y]

Is this going to be an extra drive on the unRAID system, or is intended to replace an existing drive?

 

Your question makes it sound like an additional drive?  If so the easiest approach is to plug the drive into the unRAID a system and use the pre-clear script to clear it prior to adding it to the array.  That keeps the array downtime to a minute or so.

 

An additional drive.

 

But when it pre-clears, the whole array is down, and last time it took over 15 hours. Can't I just pre-clear it on a separate system, not the unRAID system?

You could, but why not do it on the unRAID system using the script describe in this thread?  That is what most of us do if we have the free SATA port on the unRAID system.

 

Using the pre-clear script rather than letting unRAID do it means the pre-clear happens on the unRAID system with the array still running.  Only when the pre-clear completes successfully do you stop the array for the minute or so it takes to add it to the array.  This approach also has the advantage that as you are testing the drive 'in-situ' there is less chance of disturbing cables at the last moment when you add the drive.  A final point about the script is that it also carries out confidence checks on the drive - something the unRAID built-in pre-clear does not.

 

If you are going to use a telnet session (rather than the unRAID console)  to run the pre-clear script mentioned above then you might want to look into using 'screen' to control the pre-clear.  If you do not then closing the telnet window aborts the pre-clear and since on a large drive pre-clear can run for days this is a real danger.

 

At some point I think it is likely that unRAID v6 will include an equivalent to the above pre-clear script as standard, but it has not happened yet and the free-standing script is still the recommended way to do it.

Ok, i get it now. So you mean, insert the disk, but don't add it to the array, and start the array and use the system as usual, and on the side, you do the pre-clear on the unraid system itself, with the new disk which is not part of the array. am i correct?

[garycase]

Yes, that's correct.

 

Note that you have to be sure you've added the pre-clear script to your flash drive so you can use it.

 

Then you can either (a) start the pre-clear on the UnRAID system's console;  (b) start it via the Screen command from a client PC;  or ©  start it via Telnet.    But if you do © the Telnet session has to be active for the entire duration of the pre-clear ... closing the Telnet window will abort the pre-clear.    So you want to do either (a) or (b).    Doing it on the system console (a) is the best choice if you have a keyboard/monitor on your server.

 

[CHBMB]

I always use screen, lost count of the amount of preclears I've aborted due to accidentally closing down putty!

 

Easiest way to get screen is to install the nerdpack.

 

Use putty to SSH into the server.  Login and run screen, press space or enter, then run your preclear.  Can close down putty at anytime and will continue to preclear.  To check progress just open up putty, login and run screen -r 

[danioj]

I just glanced at a past post to get an indication of your build.

 

If you're reffering to the setup you have with the Supermicro X9SCM-IIF-O board you have IPMI. Why don't you use that method to run preclear rather than telent/ssh/screen. It's essentially like you're at the console itself - your browser uses Java (or similar) to attach itself to it and display it like you were sat in front of it.

 

If it disconnects you just log back into IPMI interface and attach yourself back to it. Easy peasy.

[trurl]

I always use screen, lost count of the amount of preclears I've aborted due to accidentally closing down putty!

 

Easiest way to get screen is to install the nerdpack.

 

Use putty to SSH into the server.  Login and run screen, press space or enter, then run your preclear.  Can close down putty at anytime and will continue to preclear.  To check progress just open up putty, login and run screen -r 

Not sure nerpack will work in v5.

 

Also, just to clarify the OPs terminology (yet again), the word preclear should only be used to refer to the method that uses the preclear script. If letting unRAID clear it instead (don't know why anyone would) that should be referred to as clear, no pre. The whole point of the pre is it gets cleared before you let unRAID try to use it.

 

Also, preclearing will test the disk and give you the test results. Much better than just trusting an unknown drive in your array.

[HiSoC8Y]

I always use screen, lost count of the amount of preclears I've aborted due to accidentally closing down putty!

 

Easiest way to get screen is to install the nerdpack.

 

Use putty to SSH into the server.  Login and run screen, press space or enter, then run your preclear.  Can close down putty at anytime and will continue to preclear.  To check progress just open up putty, login and run screen -r 

 

My unRAID is a VM on ESXi, so won't have a problem and wouldn't need any packs. thanks though!

[HiSoC8Y]

You're correct that if you clear the drive within UnRAID it takes a long time, and the array is out of service for that time.

 

But if you PRE-Clear it, using the pre-clear script, it only takes a few minutes to add it to the array (actually it only takes a couple seconds -- but it then needs to be formatted to actually use, and that takes a few minutes).

 

You can pre-clear it on your UnRAID server without any impact to the array ... or you can use a different PC if that's more convenient.

 

Just thought of this, my drives are all connected via RAID cards (of course no RAID), and if I follow your method described, will that impact any performance of the unRAID array? or no?

[garycase]

No, there's no impact on your server while you're running a pre-clear.

 

[HiSoC8Y]

thanks,

 

one last question before starting the preclear script. I added 2 drives, can i preclear both of them? if so, how?

[itimpi]

thanks,

 

one last question before starting the preclear script. I added 2 drives, can i preclear both of them? if so, how?

Yes.

 

Just run each preclear  from different console/telnet sessions.  Note if using the console you have up to 6 sessions available on ALT-1 to ALT-6.

[HiSoC8Y]

thanks,

 

one last question before starting the preclear script. I added 2 drives, can i preclear both of them? if so, how?

Yes.

 

Just run each preclear  from different console/telnet sessions.  Note if using the console you have up to 6 sessions available on ALT-1 to ALT-6.

 

tried that, didn't work. you sure it's Alt-1?

[itimpi]

thanks,

 

one last question before starting the preclear script. I added 2 drives, can i preclear both of them? if so, how?

Yes.

 

Just run each preclear  from different console/telnet sessions.  Note if using the console you have up to 6 sessions available on ALT-1 to ALT-6.

tried that, didn't work. you sure it's Alt-1?

Sorry - it is ALT-F1 to ALT-F6.    I do it frequently enough that I no longer think about it as my fingers know what to do

 

[HiSoC8Y]

One more thing,

 

While its preclwaring the disks, if I disconnect the network card and reconnect it, will it impact/stop the preclear?

 

I need to disconnect the network for some other reason.

[itimpi]

One more thing,

 

While its preclwaring the disks, if I disconnect the network card and reconnect it, will it impact/stop the preclear?

 

I need to disconnect the network for some other reason.

It should not unless you are running the preclear from a telnet session without using screen (as disconnecting the network terminates a telnet session).

[HiSoC8Y]

No, there's no impact on your server while you're running a pre-clear.

 

I started preclearing the disks, two disks, and it's COMPLETELY killing the performance of my array.  I can't even browse to my movies folder as it contains 3000+ folders, it times out even.

 

each preclear is showing a speed of 160 MB/s.

 

So can I now cancel one preclear, hoping that my array performance will improve? what are the consequences of cancelling the preclear?

 

Update 1

 

I stopped my array via the web, but when I click stop, it's taking time and the page is waiting for response.  It's been over 10 minutes now like this.

 

What can I do?

 

I also cancelled the preclear on my second disk, right now, only 1 preclear running.

 

Update 2

 

The reason I stopped the array because I noticed one of my drives (drive13) had a lot of reads, while i'm sure nothing was accessing the array.

 

Even the disk activity light is still on right now.

 

and, still the web GUI is waiting for response....it's stuck.

 

Seriously, help guys! thanks

[itimpi]

Stopping a preclear has no adverse affects other than that you have to start again from the beginning.

 

No idea why it should be killing your performance - it does not affect performance at all on my system.    The only think that I can think of is if the drive is connected to a disk controller that has limited throughput capability and other array drives are on the same controller.

[HiSoC8Y]

Stopping a preclear has no adverse affects other than that you have to start again from the beginning.

 

No idea why it should be killing your performance - it does not affect performance at all on my system.    The only think that I can think of is if the drive is connected to a disk controller that has limited throughput capability and other array drives are on the same controller.

 

that's what I mentioned before, All drives connected to a controller.....

 

now i'm stuck at what i described in my previous post (with the updates)....

 

I think I have no choice right now except to wait for the preclear to complete for the drive, and then maybe reboot the system forcefully.

 

no?

[garycase]

Clearly this has something to do with the fact that your UnRAID install is virtualized.  But as long as the drive being pre-cleared is on the controller that's passed through to the UnRAID VM it shouldn't matter.

 

Not sure what you can do at this point except just wait for it to finish.    Is there anything unique about the way this drive is connected vs. the way your other UnRAID drives are?  [e.g. a different controller]

 

[HiSoC8Y]

Clearly this has something to do with the fact that your UnRAID install is virtualized.  But as long as the drive being pre-cleared is on the controller that's passed through to the UnRAID VM it shouldn't matter.

 

Not sure what you can do at this point except just wait for it to finish.    Is there anything unique about the way this drive is connected vs. the way your other UnRAID drives are?  [e.g. a different controller]

 

all drives (the array drives) and the disk i'm preclearing connected to the same controller, which is passsthrued to the unRAID VM.

[garycase]

That's what I had assumed from your earlier comments.  Definitely strange behavior, as a pre-clear should NOT have any significant impact on the performance of your array (unless, as noted earlier, the drives are on a bandwidth-limited controller ... which is not the case here).