Is 40hrs to 60hrs parity check normal for 2TB to 3TB swap?

[dwoods99]

Is is normal to increase the parity check time from 40 hrs to taking 60 hrs

since in order to do a parity rebuild of a new Seagate 3TB drive (replacing Seagate 2TB)?

 

I'm guessing it makes sense sine 2/3 is the same as 40/60 trying to rebuild

20 x WD 2TB in the array.  I plan on upgrading drives to WD 3TB, but parity first.

 

One more question, at one time I had a problem with the 2TB Seagate needing a special

command or firmware upgrade to make sure it got set to the proper size, otherwise the

array complained about it being smaller than the WD drives I was trying to add.

Is this a problem with the 3TB Seagate drives?

[BRiT]

What is your hardware setup?

 

It seems excessive for 2TB parity checks to be 40 hours.

[dwoods99]

System Overview

  unRAID Ver:  unRAID Server Pro, Version 5.0-rc8a
Motherboard:  Intel - DG31PR - J3E1
    Processor:  Intel® Celeron® D CPU 440 @ 2.00GHz - 2 GHz
          Cache:  L1 = 32 kB   L2 = 512 kB   
       Memory:  2 GB - DIMM0 = 800 MHz   DIMM1 = 800 MHz     (total 4GB)
      Network:  1000Mb/s - Full Duplex

 

For my second question, the magic number for 2TB was 1953514552, and now the 3TB shows as 2930266532> I got these numbers from the Identification line on the Main screen for devices using SimpleFeatures plugin.

Can someone please confirm that's the number they get from a Seagate or WD 3TB drive?

 

 

[S80_UK]

    Processor:  Intel® Celeron® D CPU 440 @ 2.00GHz - 2 GHz

 

According to the Passmark benchmarks that's slower than an elderly Atom 330...

 

http://www.cpubenchmark.net/cpu.php?cpu=Intel+Celeron+440+%40+2.00GHz&id=652

 

http://www.cpubenchmark.net/cpu.php?cpu=Intel+Atom+330+%40+1.60GHz&id=604

 

Those speeds might simply reflect a lack of horse power, unless someone else can suggest otherwise.

[dgaschk]

How are the drives connected? What is the normal check time of the system? Attach a syslog.

[dwoods99]

Drives are attached via 4-ports on motherboard and 8-port Sata expansion card (Supermicro AOC-SAT2-MV8)

syslog.txt

[nars]

did you tested speed for all your hdd's? you can use something like:

for d in /dev/sd?; do echo $d:; dd if=$d of=/dev/null iflag=direct bs=10M count=20; done

(make sure all drives are spinning before doing it or run it twice else it may show lower values than real due to the time hdd's take to spinup...)

anyway you problem may be some bottleneck when reading them all at same time...

[S80_UK]

As I see it, you have four drives on the motherboard and eight on the AOC-SAT2-MV8.  That is a PCI hosted controller, and since your mother board only has 32 bit PCI it will be a major bottleneck.  A single drive can sometimes effectively saturate a standard PCI slot, and you have eight drives sharing the same bandwidth.  On that basis I think the speeds that you are seeing are reasonable.

 

Can you consider using a PCI-e controller instead (I assume that you use the on-board graphics on your motherboard)?  So long as your motherboard allows non-graphics cards in the PCI-e slot then something like the AOC-SASLP-MV8 would be a possible starting point.  A faster option could be the later AOC-SAS2LP-MV8, but I think the CPU could still be a limitation.  However, you should see a significant speed boost for relatively low cost (much cheaper than a controller and CPU, mobo and RAM upgrade). 

 

 

[garycase]

Your byte count is correct for a 3TB drive -- at least it's the same as my WD Red's

 

Your parity check times seem really long ==> my system with all WD 3TB Reds take a very consistent 8:05;  my other (older) system with a mix of 1TB, 1.5TB, and 2TB drives (mixing drives slows down parity checks) and running v4.7 takes just over 9 hours.

 

Are you talking about parity checks -- or is this the time to clear a drive and then do a "parity swap" operation?  [This will take a LOT longer than a parity check]

 

[garycase]

Just took a closer look at your system specs, and the reason for your very slow parity checks is clear.

 

You have an 8-port PCI-X SATA card that you've got plugged in to a PCI slot.    This is a HUGE bottleneck and is significantly slowing down your drive access speeds.    Probably doesn't make much difference when only one drive on it is being accessed -- but during a parity check when all 8 drives are being accessed that's a major bottleneck.    Even a 4-port PCI card will cause MAJOR slowdowns -- and you're using 8 !!

 

Your board has a PCIe x16 slot -- and I assume you're not using it since it also has onboard video.    You would get a VERY significant speed boost if you replaced your AOC-SAT2-MV8 with the PCIe x4  AOC-SASLP-MV8 [http://www.newegg.com/Product/Product.aspx?Item=N82E16816101358 ].    I'd expect your parity check speeds to drop to ~ 8 hrs with all 3TB drives;  or probably ~ 9-10 hrs with mixed 2 and 3 TB drives.

 

BTW, your parity check speed is almost certainly NOT due to your Celeron CPU ... parity checks are driven almost exclusively by the disk access speeds and require very little CPU "horsepower."    My newer system (with all WD 3TB Reds) is on a very low power Atom motherboard, and as I noted above, takes a very consistent 8:05 for a parity check.

[dwoods99]

is this the time to clear a drive and then do a "parity swap" operation?  [This will take a LOT longer than a parity check]

Yes, with the 2TB Seagate, parity checks and pre_clearing a new drive would take 30+ hours,

and yet I somehow thought that was normal :slap-me:

[dwoods99]

Can you consider using a PCI-e controller instead (I assume that you use the on-board graphics on your motherboard)?  So long as your motherboard allows non-graphics cards in the PCI-e slot then something like the AOC-SASLP-MV8 would be a possible starting point.  A faster option could be the later AOC-SAS2LP-MV8, but I think the CPU could still be a limitation.  However, you should see a significant speed boost for relatively low cost (much cheaper than a controller and CPU, mobo and RAM upgrade).

Thanks for helping out.

Is Supermicro still the best choice for quality/price?

I think I should stay away from Syba and Highpoint, not sure about Startech, LSI or Areca.

[dwoods99]

You have an 8-port PCI-X SATA card that you've got plugged in to a PCI slot.    This is a HUGE bottleneck and is significantly slowing down your drive access speeds.    Probably doesn't make much difference when only one drive on it is being accessed -- but during a parity check when all 8 drives are being accessed that's a major bottleneck.    Even a 4-port PCI card will cause MAJOR slowdowns -- and you're using 8 !!

 

The motherboard is a Gigabyte GA-965G-DS3

 

So I guess my confusion is the PCI Express graphics interface on the board (I believe) won't support the card and that's why it's plugged into a PCI slot. I didn't know better.

In that case, did I simply buy the wrong card for that motherboard, and what is the appropriate usage for the one I have... how can I use it properly?

 

EDIT: I am also considering changing this server to the newer one which is a Gigabyte GA-A75M-UD2H, so would the card work better on that one?

Although even with drives plugged in on this motherboard, pre_clearing takes over 30 hrs per cycle.

 

[JonathanM]

So I guess my confusion is the PCI Express graphics interface on the board (I believe) won't support the card and that's why it's plugged into a PCI slot. I didn't know better.

In that case, did I simply buy the wrong card for that motherboard, and what is the appropriate usage for the one I have... how can I use it properly?

 

EDIT: I am also considering changing this server to the newer one which is a Gigabyte GA-A75M-UD2H, so would the card work better on that one?

PCI-X is an old server interface, I doubt you would be able to find a current motherboard to fully use your HBA. Instead of shopping for a new motherboard, you would probably be better off getting a PCI-e HBA that will work in your current board. PCI-X is not compatible at all with PCI-Express. It will work as a standard PCI card, which is how you are using it now.

[dwoods99]

That helps to explain my mistake when I purchased it.

 

I've gone ahead and purchased the recommended PCIe x4  AOC-SASLP-MV8, so now I wait...

[Alex.vision]

Although even with drives plugged in on this motherboard, pre_clearing takes over 30 hrs per cycle.

 

I have my HDD running off of AOC-SAS Cards as well. And pre-clears still take around 24 hours. My parity check is about 8 hours though. That's with 14 WD 2TB Green Drives.  I just pre-cleared a 4TB drive and it took almost 40 hours per cycle.

[dwoods99]

Parity finished in less time than expected.

It seemed to speed up for the last 1TB -- which makes sense I guess since it's probably empty (for now).

 

[dwoods99]

The motherboard is a Gigabyte GA-965G-DS3

Oops, forgot I switched this for another. So in this case the motheboard is an Intel Desktop Board DQ965GF with the Celeron CPU -- it's old and used to have software raid before switching to unRaid (no regrets).

 

Thanks to all for the help.

[S80_UK]

Parity finished in less time than expected.

It seemed to speed up for the last 1TB -- which makes sense I guess since it's probably empty (for now).

It's nothing to do with whether there is data or not - parity checking reads the entire disk even for drives that have no files on them.  It is because some other smaller drives had finished being checked.  As the reading of any drive gets towards the inner cylinders it will slow down.  If you have mixed drive sizes (e.g. 2TB and 3TB) you will see a slow down at the end of the 2TB drives and then a partial speed up before slowing down again at the end of the 3TB drives.

 

See here...  http://lime-technology.com/forum/index.php?topic=25250.msg220939#msg220939