[SOLVED] Does Preclearing multiple drives simultaneously slow down preclear?


Recommended Posts

I recently pre-cleared four disks simultaneously that were attached to the same "half" of an AOC-SASLP-MV8. The first three completed within minutes of each other; the fourth took about an hour longer, so I think that fourth disk probably saturated the card. A more modern card (such as your AOC-SAS2LP-MV8) with more PCIe lanes can probably support more simultaneous pre-clears without saturating, so six should not be an unreasonable number to try. Not sure what you mean by "Can Preclear truly multi-task?" You can run multiple instances of the pre-clear script, certainly. It's the Linux kernel that schedules the pre-emptive multi-tasking.

Link to comment

Pre-clear doesn't multi-task at all.  You can, however, run multiple instances of it by opening new sessions within Linux.

 

As for whether or not running 6 at a time would slow things down => depends on the controller; the amount of memory you have; and your CPU.  In general I've never seen a problem running 3 or 4 at once ... but have never tried 6.

 

Personally, if I had 6 drives to clear, I'd do 3 at once; then repeat the process for the next 3.  But it won't hurt to try it ... just wait and minute or so after you start each one and see what the speeds look like.    If you start a pre-clear that seems to slow down the speeds, you've likely saturated your system's capability ... so don't start anymore.

 

 

Link to comment

Just for posterity's sake, I ran preclear on 7 disks simultaneously, and there were no issues.  And performance was quite snappy.

 

5 Drives were on the motherboard Sata III ports (ASUS M5A97 EVO) and two drives (WD1001FALS) are on a SIIG SC-SA0E12-S1 6gb/s 2 port card in a PCIE X1 Slot.  The DP SATA 6Gb/s 2S1P PCIe will add 2 SATA 6Gb/s & 1 PATA connectors to PCIe-equipped computers.  http://www.siig.com/download/search/?keyword=SC-SA0E12-S1

 

Here's a shot during the preclear.  I did not see any extended times due to preclearing multiple drives.  This screenshot was right after I started it, the mb/s actually increased during the process.  For instance, the ST1000DM000 made it up to 212 mb/s.

 

20151220-cjoc-95kb.jpg

Link to comment

Is there a way to calculate how many you could do at once based upon CPU and Memory?

 

A good way to estimate that would be to note the CPU and memory load (on the Dashboard) you are seeing when a particular # of pre-clears is underway.    Probably too late now, but if you had noted the loads with 3 underway; and then again after you started the next 3 (thus had 6 running); and finally with all 7, you would have had a pretty good indication of the load factor per pre-clear ... and could then make a very good educated guess as to just how many you could actually run at once.

 

I don't think pre-clears are very CPU intensive, so the primary factor is probably how much memory you have (and with 24GB you could probably run as many as you want within reason ... I've run 4 at a time on an Atom-based system with 4GB of RAM with no problem).    The CPU isn't likely to be an issue in most cases.

 

Link to comment

I'll have more drives to preclear eventually, so I'll give that a test.  I'm actually considering building and selling media servers.  Its very relaxing to do so (I spend most of my time building contracts, business plans and spreadsheets), so I like the 'hands on' aspect of putting the pieces together.  You mentioned an Atom processor in your system.  Is there a method for calculating the minimum configuration for an unRaid system with say 10GB of storage?  For the most cost effective, I think looking at 3TB drives would be best.  Thoughts?  Is there a place in the forums where such things are discussed specifically?

 

Thanks!

Link to comment

The configuration you need for a given amount of storage is easy to compute; but in addition to the raw capacity don't forget to also consider how many drives (& thus SATA ports) you need.

 

For example, if you wanted 10TB (clearly that's what you meant when you said 10GB), you could simply use 3 5TB drives -- and would only be using 3 SATA ports and have 3 installed drives.    To do this with 3TB drives, you'd need 5 drives (and would have 12TB) ... and would be using 5 SATA ports and drive slots.  If, for example, you were building that in a mini-ATX form factor that only held 6 drives, the 5TB drives would provide a lot more expandability.

 

As for computational resources => if it's strictly going to be a NAS, a low-power Atom-based setup works great [My trusty little D525-based Supermicro board is perfect for the server it's running];  on the other hand, if you're building a system to run Dockers and VM's, you'll want far more computational "horsepower."

 

You could start another thread if you want to isolate this discussion -- or just leave it here, and (eventually) add some more feedback r.e. the pre-clear results when you have a chance to monitor the actual impact on system resources of adding additional pre-clears.

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.