Hello all,

I have 192tb in my array spread out across 15 disks. As the array has grown, parity checks obviously have started to take longer and longer - as of right now, a parity check takes just a hair under 48 hours even. I run a parity check as a scheduled check monthly, which is I believe the default expectation.

My concern is - in a 720 hour month, I now have a parity check running basically 7% of the time. Is this dangerous to the longevity of my drives - and specifically to the parity drive, getting the shit hammered out of it for 7% of the time? Should I be doing parity checks less frequently (since 2011 I have literally never had a parity check fail).

Not sure what the best practice is here, or if I am worrying about nothing.

Unraid 7.2.3, all drives are XFS (if it matters).

Thank you!

Edited June 16Jun 16 by Ouze

Parity checks should not take longer just because you add additional drives. The time should be dependent on the size of the largest drive (parity). What is the size of your largest drive?

In terms of wear a parity check should be one of the least stressful operations on a drive as it works serially through the drive so minimal head movements.

My parity is a 24tb WD Red Pro, WDC_WD241KFGX. I guess I keep getting bigger and bigger drives so it gave me the erroneous impression that the pool size was a factor.

Thank you for the rapid response!

Edited June 16Jun 16 by Ouze

For reference, I have 2 28TB Seagate Ironwolf NAS drives. Pairty Check finishes a little under 35 hours [1 day 10 Hours 58 Minutes and 14 Seconds].

I use the Parity Tuner plugin with settings to split it across 4 intervals so it only runs overnight when no one is using the array.

One more important point about arrays with very large size parity drive(s). If you have many different size data drives in the array, the speed of the parity check is control by the read speed of the slowest drive in the array. As an example, in my Backup Server, I have a 3TB drive and its read speed is about 90MB/s on the very inner tracks of the drive. (Everyone should realize that all HD's have their slowest read speeds on those inner tracks.) At that point, the potential read speeds of the 12TB parity drive and the other 12TB data drives is about 180MB/s!

As a point of reference, the parity check times for my two servers (See specs in signature below) are:

Media Server 22 hours, 29 minutes, 24 seconds. Average speed: 148.2 MB/s

Backup Server for Data and Media 21 hours, 32 minutes, 42 seconds. Average speed: 154.7 MB/s

I have now fixed on only using 12TB drives in my servers. This allows me to have only one size of 'spare drive' to have on hand for expansion and/or replacement

@Ouze , your average data drive size is approximately 12TB (192TB/15drives) . Your potential data storage with this array as presently designed and configured is 360TB. You should have asked yourself if you really can see the need for that much storage in the next five years before deciding on the first 24TB drive purchase. I know that there is a tendency to want the latest and biggest in all of us. But there are also disadvantages to doing so without looking at the much larger picture. I can also understand that designing and configuring a brand new server can result a completely different configuration that one would choose when re-configuring an old server. (In the past, When I was re-configuring my servers, I was was doubling the size of the parity drive. That simplistic approach no longer makes sense as it results in potential array sizes that I can never fill and parity check times that I find simply unacceptable.

Edited June 17Jun 17 by Frank1940

Check your drives' datasheets for workload ratings.

As mentioned parity checks themselves are easy on the drives, the fact you say it takes 48h means either you have HBA bottlenecks or that the array is doing other stuff at the same time, if the latter than that'll be more "wear" than the check itself.

Also some people increase intervals once the system is "trusted", I do one every 2 months but some people do 4-6.

31 minutes ago, Frank1940 said:

One more important point about arrays with very large size parity drive(s). If you have many different size data drives in the array, the speed of the parity check is control by the read speed of the slowest drive in the array. As an example, in my Backup Server, I have a 3TB drive and its read speed is about 90MB/s on the very inner tracks of the drive. (Everyone should realize that all HD's have their slowest read speeds on those inner tracks.) At that point, the potential read speeds of the 12TB parity drive and the other 12TB data drives is about 180MB/s!

As a point of reference, the parity check times for my two servers (See specs in signature below) are:

Media Server 22 hours, 29 minutes, 24 seconds. Average speed: 148.2 MB/s

Backup Server for Data and Media 21 hours, 32 minutes, 42 seconds. Average speed: 154.7 MB/s

I have now fixed on only using 12TB drives in my servers. This allows me to have only one size of 'spare drive' to have on hand for expansion and/or replacement

@Ouze , your average data drive size is approximately 12TB (192TB/15drives) . Your potential data storage with this array as presently designed and configured is 360TB. You should have asked yourself if you really can see the need for that much storage in the next five years before deciding on the first 24TB drive purchase. I know that there is a tendency to want the latest and biggest in all of us.

This is the first time I think I've seen someone make an argument for not getting too much capacity. In this case, there was a big sale coupled with a paypal pay-in-4 incentive in November that wound up with me getting 3x 24TB red pros for $1400 total. Those drives are now selling for $1289 each, so yeah, I feel pretty good about that choice and I will definitely fill them.

I think your point makes more sense when the prices of hard drives would come down over time, instead of inexplicable getting more and more expensive as they now seem to do - what stupid times we live in.

I think I have my answer - I am going to change the parity to every 2 months instead of monthly.

Thank you, all!

2 hours ago, Ouze said:

This is the first time I think I've seen someone make an argument for not getting too much capacity. In this case, there was a big sale coupled with a paypal pay-in-4 incentive in November that wound up with me getting 3x 24TB red pros for $1400 total. Those drives are now selling for $1289 each, so yeah, I feel pretty good about that choice and I will definitely fill them.

In a former life, I was an Senior Electrical Engineer and I was involved in the design of 'systems'. Much of that task involved making decisions about size, capability, reliability, and cost as well as usability. Believe me when I say that not one design solution ever meets all of the requirements that were originally set. The end result usually involves some compromises on the part of everyone. (I remember that in my class in ROTC, an Army reserve major, who had gone ashore at Normandy, said, "You will never have enough men, equipment, time or money to do the job. But you will still have to do it somehow!") You now have a running system that now has some fixed design constraints. You will now need a plan to manage that system. So be working on that plan and get one that will work for you.

The AI build-out has made a total mess of the Electronic component supply system. I don't know how long the current situation can exist. I suspect that it might well be a bubble with people, who have not really looked at the operating cost plus sunk cost for those data centers, throwing money at the concept to be the first one to have one running. (Unless, they are going to provide the AI services at a loss, those costs have to priced into every query made...)

If as a result of getting larger drives you now have quite a bit of spare capacity it might be worth considering removing some of the smaller drives as they tend to be both the most power hungry and also the lower performance dtives.

36 minutes ago, itimpi said:

If as a result of getting larger drives you now have quite a bit of spare capacity it might be worth considering removing some of the smaller drives as they tend to be both the most power hungry and also the lower performance dtives.

This is another option that can be employed BUT you should do an analysis of how many free space you lose and with decent estimate of how much storage space you will be in the future. A time window of five years is probably enough.

Another option to be explored is removing some smaller drives by transferring their contents to some of older larger drives to fill them up closer to their capacity. That way if you do need to swap a new capacity drive for more storage room, you lose less space by replacing drive. (If you replace an almost full 8TB drive with a 24TB drive, you only gain 16TB in the process. If that drive was a 4TB drive, you would gain 20TB on the swap. Remember that the cost of the 24TB drive is a constant in analysis.)

7 hours ago, Frank1940 said:

all HD's have their slowest read speeds on those inner tracks

Constant RPM, shorter tracks, less data per revolution.

3 hours ago, itimpi said:

If as a result of getting larger drives you now have quite a bit of spare capacity it might be worth considering removing some of the smaller drives as they tend to be both the most power hungry and also the lower performance dtives.

And as I always say, each additional drive is an additional point of failure.