Greetings,

 

I've been reading about the increasing sizes of HDD's. There is a concern that raid setups aren't keeping up with HDD size increases -especially when considering the time it takes to parity check or rebuild a disk in a large disk array. My own setup has the max. 16xHDD's and takes approx 30 hours to run a parity check. I feel this is too long. 8xHDD's are on 2x PCI Promise TX4 expansion cards. They slow the whole process down considerably. Currently, I'm in the process of upgrading my parity to 6Tb, but I wonder if this is too big. Will my parity check and rebuild time reduce by cutting the number of drives to eight (all connected directly to Mobo). I'd like to think so. Is there some way to calculate how long a check should take?? Thx

It depends on your expectations. You seem to be concerned about parity check time, and using a 6TB drive. If none of your other drives is 6TB, the parity check will continue to run for the additional space of the 6TB. For example, if you currently have a mix of 4Tb and 2TB drives, swapping the 4TB parity for a 6TB parity will just lengthen the parity check. You should read this thread about using an 8TB drive for parity.

 

Start about here http://lime-technology.com/forum/index.php?topic=36749.msg353689#msg353689

and notice the mix of drives in the system, the parity check speed changes as different drives are involved in the check.

Greetings,

 

I've been reading about the increasing sizes of HDD's. There is a concern that raid setups aren't keeping up with HDD size increases -especially when considering the time it takes to parity check or rebuild a disk in a large disk array. My own setup has the max. 16xHDD's and takes approx 30 hours to run a parity check. I feel this is too long. 8xHDD's are on 2x PCI Promise TX4 expansion cards. They slow the whole process down considerably. Currently, I'm in the process of upgrading my parity to 6Tb, but I wonder if this is too big. Will my parity check and rebuild time reduce by cutting the number of drives to eight (all connected directly to Mobo). I'd like to think so. Is there some way to calculate how long a check should take?? Thx

 

I believe those Promise cards are on the PCI bus - so you are definitely bus limited with 8 drives!

 

I'd suggest that you consider a system update so that all of the drives are running on PCIe or MB ports. You 6TB parity checks will likely be faster than your current 4TB parity checks if you do.

 

I believe those Promise cards are on the PCI bus - so you are definitely bus limited with 8 drives!

 

I'd suggest that you consider a system update so that all of the drives are running on PCIe or MB ports. You 6TB parity checks will likely be faster than your current 4TB parity checks if you do.

 

Thx. That's my current intention. To run a system without any expansion cards at all. I have 8xSATA ports on the Mobo. If all 7xdata HDD's were 6tb, then that would double my current space. I doubt I will need that much space anytime soon

I have 8xSATA ports on the Mobo. If all 7xdata HDD's were 6tb, then that would double my current space. I doubt I will need that much space anytime soon

 

Quite true => and if they're all on motherboard ports, your parity checks would be much faster than what you're seeing now.

 

I'm not sure of your MB, but a small investment in a PCIe controller may be helpful.

I'm not sure of your MB ...

 

Clearly shown in his signature as an Asus P5B-E 

 

... but a small investment in a PCIe controller may be helpful.

 

With only x1 slots available, it's probably not worth the bother.  Yes, it would improve the bandwidth somewhat, but it would still be a bottleneck if 4 drives were connected to the controller.  [The board does have an x16 slot, but there's no onboard video capability, so I'm sure that's in use for a video card.]

 

flambot's plan to simply move off of the add-in cards altogether is a far better idea.

 

 

I'm not sure of your MB ...

 

Clearly shown in his signature as an Asus P5B-E 

 

... but a small investment in a PCIe controller may be helpful.

 

With only x1 slots available, it's probably not worth the bother.  Yes, it would improve the bandwidth somewhat, but it would still be a bottleneck if 4 drives were connected to the controller.  [The board does have an x16 slot, but there's no onboard video capability, so I'm sure that's in use for a video card.]

 

flambot's plan to simply move off of the add-in cards altogether is a far better idea.

 

How about a cheap PCI video card and use the x16 slot for an HBA controller?

... How about a cheap PCI video card and use the x16 slot for an HBA controller?

 

Not worth the bother.  His goal is to (a) get rid of all the small, low areal density disks, and in the process (b) only use the motherboard ports.

 

Buying both a video card and an HBA controller is spending money that can better be put towards the new disks 

 

... or (eventually) towards a new motherboard/CPU/memory combo

 

For now, buying the new disks will provide the speed and capacity gains he's looking for, with NO additional costs for supporting hardware.

 

 

 

Thank you for all the great input here. Lots of good stuff to mull over.

 

Let's just hope the new 6Tb drive is faulty (see other thread) and the replacement is recognised by my setup. Not sure what to do if it's not - or why it isn't recognised.

 

 

Thank you for all the great input here. Lots of good stuff to mull over.

 

Let's just hope the new 6Tb drive is faulty (see other thread) and the replacement is recognised by my setup. Not sure what to do if it's not - or why it isn't recognised.

 

Every reason to believe they would.

 

Not worth the bother. ...

 

His choice. I don't see going this route as being a bad option:

- Cheap PCI Video Card - $12.99

- Areca ARC-1430 12 port Controller = $69.99 OR

- Used Adaptec 1430SA

 

My longest lasting assets in my servers are cases, drive cages and controllers.

 

If he put out a feeler for the 4 drive Adaptec 1430sa in the FSBO forum, I'd bet a lot of them would be available cheap, but the Areca is bigger and more versatile (RAID0 parity, for example), and is a super buy for 12 ports!

 

 

IF he needs more ports those are good options ... but the simple fact is he doesn't.

 

He's already bought a 6TB parity drive ... and after that's installed and the current 4TB replaces his 1TB drive on a motherboard port, he can copy the data from his 4 750GB drives to it, and then change his setup so he only has 2 drives on each of the TX4's.  This will very significantly mitigate his bandwidth restrictions, and result in MUCH better parity check speeds.

 

One more 6TB drive and he'll be down to only one drive on one TX4 ... with all the rest on motherboard ports.

 

Simply no need for any additional hardware.

 

Agree 1430SA's are likely available -- they're a great little card.  I've got 2 of them on my "spares" shelf (along with several other 2-8 port cards) 

I would suggest bigger newer drives over the course of a few months as well.

 

The motherboard is PCI 2.2 and the promise TX4 supports the 66MHZ bus yielding a max bandwidth of 266MB/s.

However,  it's still not enough with two cards each having 4 drives. Everything comes to a standstill with PCI until the devices and CPU finish communication. The Promise TX4 has a FIFO buffer, but that only goes so far.

 

The best bang for the buck would be the HGST 7200 RPM 6TB drive for parity as it's very fast.

Then wait on the lower priced Seagate 6TB's when they have a sale, sometimes you can score them at ~$200ish.

 

I have both.

From the SMART data a full drive scan is from

(hgst)

Extended self-test routine     

recommended polling time:      785 minutes.

to

(seagate)

Extended self-test routine     

recommended polling time:      632 minutes.

 

This yields a sweep from approximately 10 hours to 12 hours.

 

In my experience on a modern high speed motherboard with 8 drives, (4) 6TB, (2) 3TB, (2) 4TB, I see about 12 hours.

I don't think you'll see much faster then that.

However the chances of approaching that are pretty high if drives are spread around with various controllers and bus paths.

 

First order is more modern high density drives to see if parity speed is acceptable.

After that moving video to PCI and a x8 LSI, Supermicro, Areca controller in that x16 slot could provide a boost.

The best bang for the buck would be the HGST 7200 RPM 6TB drive for parity as it's very fast.

 

I'm in New Zealand. A HGST Ultrastar He6 HUS726060ALA640 64MB 6TB (the only 6tb HGST on the pricespy list) is currenlt $800.32NZ

Roughly twice the price of a 6tb WD Red.

 

WeeboTech...I would be extremely stoked if my parity checks were only 12 hours!! 

The WD Red 6TB isn't as fast as the 7200rpm Hitachi, but it's still a VERY fast drive with its 1.2TB platters.  You'll get very good parity check speeds once you've moved to all 1TB platter drives => and even before that you'll notice a BIG improvement once you've eliminated the 750GB drives and changed from 4 drives per TX4 to 2 drives per TX4 (that interface change will make a BIG difference).

 

If you had all 6TB Reds I'd expect a parity check in the 13-14 hour range.  With a 6TB parity, 4TB parity, and some older 2TB drives with lower density platters you'll probably be closer to 20 hours, depending on the exact platter density on the 2TB drives (as I noted in your other post).  Once you get rid of the lower density units, a mix of 6TB and 4TB drives will likely take ~ 15 hours.

 

 

 

+1 for the The WD Red. I am a big fan of these drives. I am not sure how much of a comparison this is (given you are talking about 6TB drives) but if it helps at all:

 

Last checked on Fri Mar 13 00:28:32 2015 EST (eleven days ago), finding 0 errors.

Duration: 7 hours, 54 minutes, 8 seconds. Average speed: 105.5 MB/sec

 

Parity Disk (3TB) and 4 data disks (4x3TB)

 

Given what I have experienced with these WD 3TB Red's (when funds permit) I will be migrating to the WD 6TB Red's.

The best bang for the buck would be the HGST 7200 RPM 6TB drive for parity as it's very fast.

 

I'm in New Zealand. A HGST Ultrastar He6 HUS726060ALA640 64MB 6TB (the only 6tb HGST on the pricespy list) is currenlt $800.32NZ

Roughly twice the price of a 6tb WD Red.

 

I was thinking more of the HGST Deskstars which are in the $300 range here in the US.

HGST Deskstar NAS H3IKNAS600012872SN 6TB 7200 RPM SATA 6.0Gb/s 3.5"

 

The Helium filled drives are very expensive right now and I wouldn't expect them to be cost effective unless under some very specific requirements.

 

WeeboTech...I would be extremely stoked if my parity checks were only 12 hours!! 

It's feasible once you start swapping out low density drives to higher density drives and get rid of the PCI based controllers.

However from the Estimated polling time, I do not see that an array wth 6tb drives will ever get under 10 hours.

Which is why I posted those two times. It provides a good median estimate .

... I do not see that an array wth 6tb drives will ever get under 10 hours

 

At least not until somebody builds an array with 6TB SSDs 

... or even spinners with 2TB or 3TB / platter areal density.

 

I agree, however, that with CURRENT 6TB drives 10-12 hours is likely the "floor" for parity check times.

 

FYI, just to give you some comparables...

 

I'm running a P5B with a PCI video card and an SAP2LP in the x16 slot.  I have 5 drives in the array, 6TB parity, 1 6TB data and 3x3TB data.  My parity check takes around 26 hours...

 

It was faster when I ran parity on the motherboard but that results in unstable operation for my motherboard/sata controller combo under unRAID6.  Things are very stable with everything on the expansion card, but a bit slower.

 

I attribute the slower speeds to the old PCIex 1.1 motherboard design.  6TB parity checks are never going to be fast, but they should be faster on newer motherboards.

I have a server with mostly 4 TB drives and several 6Tb drives, a parity check takes about 17-18 hours (95Mb/sec average).

 

I could even live with 18 hours - beats my current setup by almost 10 hours (and that'll go even further south with the intro of a 6Tb parity)  :'(

 

Would be nice to be back to a small array, but I have too much media. 

FYI, just to give you some comparables...

 

I'm running a P5B with a PCI video card and an SAP2LP in the x16 slot.  I have 5 drives in the array, 6TB parity, 1 6TB data and 3x3TB data.  My parity check takes around 26 hours...

 

It was faster when I ran parity on the motherboard but that results in unstable operation for my motherboard/sata controller combo under unRAID6.  Things are very stable with everything on the expansion card, but a bit slower.

 

I attribute the slower speeds to the old PCIex 1.1 motherboard design.  6TB parity checks are never going to be fast, but they should be faster on newer motherboards.

 

 

How are the drives distributed among controllers? does the SAS2LP have only the 6TB drives?

FWIW, I'm using an X10SL7-F, all data drives are on the 8 port LSI controller.

http://www.supermicro.com/products/motherboard/Xeon/C220/X10SL7-F.cfm

 

I use the other 6 SATA ports for SSD's and removable drives for pre-clearing.

When I had the 6tb drives on an N54L HP Microserver, the parity check speeds were within 10-12 hours as well.

 

WeeboTech's 10-12 hrs are with 7200 rpm 6TB drives.  With WD Reds the times will be a bit longer because of the lower rotational speed (both units have 1.2TB platters, so the areal densities are the same).

 

I'd expect about 14 hours or so with all 6TB WD Reds (They're about 20% slower than a 7200 rpm unit).  If you mix in some 3 or 4TB units that'll likely add 2-3 hours, due to the lower areal density (1TB/platter vs. the 1.2TB of the 6TB units)  and the slowdown when the smaller units reach their inner cylinders.    This is consistent with Wimpie's 17-18 hour experience.

 

Note that as long as you have drives with even lower areal density in the mix, the times will be even longer.  Basically, you'll notice two nice performance "bumps" => one when you reduce the bandwidth restrictions of the TX4's (when you go from 4 drives per TX4 to 2 drives per TX4 it'll help a LOT ... and when you later get down to just one drive on a TX4 that'll completely eliminate that as a bottleneck);  and a second one when you eliminate all of your lower areal density disks.

 

 

 

 

 

WeeboTech's 10-12 hrs are with 7200 rpm 6TB drives.  With WD Reds the times will be a bit longer because of the lower rotational speed (both units have 1.2TB platters, so the areal densities are the same).

 

 

2 7200's, 2 5900's. 3TB's are 7200, the 4TB's are 5900.