How big a drive is too big?

[Squazz]

Hi

I have a bunch of WD Green 3Tb drives. And I'm beginning to run out of space (both physical in the computer case, and on the shares)

 

I'm considering buying new Seagate Exos X16 16 or 14 Tb drives. But, are they gonna be too big?

I'm considering the parity checks. Right now it takes me about 8 hours for my 3Tb drives. If it's gonna tke me 5,5 times as much time with 16Tb drives, we are talking 40+ hours of parity checking. That's almost two days where I will see degraded performance on my shares.

Right now I do parity checks every month. That doesn't seam feasable with a 2 days parity check.

 

What is the experience from other people?

Are 16Tb drives too big?

How often do you do parity checks with so large drives?

How long does it take to take to do parity checks of drives of this size?

My main fear is data degradation through bitrot or the like.

Edited May 25, 2022 by Squazz

[itimpi]

The normal recommendation is to go for as big a drive as you can afford.  Large drives tend to perform better, consume less power and be less points of potential failure.   Bear in mind, though, that you have to upgrade the parity drive(s) first before you can have large data drives.

 

You can use the Parity Check Tuning plugin to minimise the effect on daily use of parity checks if you have large drives.

[DMills]

For me, it kind of depends on the system the drives are on. I know on my previous unRAID "server", I had a number of 8TB drives and it took upwards of 24 hours to finish the monthly parity check, mainly due to the machine being overtaxed by me with lots of containers running, and using the machine during that time. Now on my new server with 16TB drives, and lots more horsepower, I don't even notice the check is running until I go onto the server and see it's either running or finished. The parity check now takes around 26 hours or so, so there is no direct correlation from one size to the next, there are lots of other factors. I checked the prices and found the 16TB drives to be the best $/TB so went with them, parity check times be damned!

[Squazz]

15 minutes ago, itimpi said:

The normal recommendation is to go for as big a drive as you can afford.  Large drives tend to perform better, consume less power and be less points of potential failure.

I guess I'll go with the 16Tb drives then

 

15 minutes ago, itimpi said:

Bear in mind, though, that you have to upgrade the parity drive(s) first before you can have large data drives.

I'm considering removing one of my two parity drives, and then just go with a single 16Tb parity drive

 

17 minutes ago, itimpi said:

You can use the Parity Check Tuning plugin to minimise the effect on daily use of parity checks if you have large drives.

Sounds interresting, I'll give it a look

 

12 minutes ago, DMills said:

For me, it kind of depends on the system the drives are on. I know on my previous unRAID "server", I had a number of 8TB drives and it took upwards of 24 hours to finish the monthly parity check, mainly due to the machine being overtaxed by me with lots of containers running, and using the machine during that time. Now on my new server with 16TB drives, and lots more horsepower, I don't even notice the check is running until I go onto the server and see it's either running or finished. The parity check now takes around 26 hours or so, so there is no direct correlation from one size to the next, there are lots of other factors. I checked the prices and found the 16TB drives to be the best $/TB so went with them, parity check times be damned!

Interresting that you only see 26 hours for a parity check on the 16Tb drives.

 

If one doesn't feel the impact of the parity check, a monthly check might be OK.

 

@DMills could the reason you don't see an impact be that you have a big cache? Personally I don't have that big a cache as I'm only running SSDs as my cache

 

[DMills]

2 hours ago, Squazz said:

@DMills could the reason you don't see an impact be that you have a big cache? Personally I don't have that big a cache as I'm only running SSDs as my cache

 

 

It's possible I guess, although the cache drives are just regular HDDs so they're as slow as the rest of the array, parity included. I just think my old quad core couldn't handle the load as well as this one can. I'm still going to try out the tuning plugin though

[ConnerVT]

No one ever came home after buying a new television and has said "I should have bought the smaller one."  😄

 

Buy as much storage as you anticipate you will use now, and in the future.  Replacing all of those small drives with a few large ones will reduce your power consumption (and noise, and heat...) as well as reduce the number of points of failure.

 

The Parity Tuning plug in works well.  You can schedule parity checking to run during non-prime time hours if you do feel any impact.

[Hoopster]

19 minutes ago, ConnerVT said:

No one ever came home after buying a new television and has said "I should have bought the smaller one."

You never met my wife.  😀

 

Other than that small point, the rest of your post is spot on.  I am a firm believer that fewer larger drives is better than many smaller drives.  I have consolidated twice.  First it was 3TB to 8TB drives and now I am starting to move from 8TB to 14TB and 16TB drives.

[Squazz]

2 hours ago, ConnerVT said:

No one ever came home after buying a new television and has said "I should have bought the smaller one."  😄

 

Buy as much storage as you anticipate you will use now, and in the future.  Replacing all of those small drives with a few large ones will reduce your power consumption (and noise, and heat...) as well as reduce the number of points of failure.

 

2 hours ago, Hoopster said:

I am a firm believer that fewer larger drives is better than many smaller drives.  I have consolidated twice.  First it was 3TB to 8TB drives and now I am starting to move from 8TB to 14TB and 16TB drives.

 

I feel there's beginning to be a consensus.

I'm propably gonna end up with buying 2x Seagate Exos X16 ST16000NM001G 16TB drives, that's where I get most space for the money.

 

And it also seems that parity checks is not that big of a problem with drives in the sizes of above 10Tb

Edited May 25, 2022 by Squazz

[John_M]

Unraid 6.10 has built-in support for splitting the scheduled parity check into multiple shorter periods. See Settings -> Scheduler

 

[itimpi]

Just now, John_M said:

Unraid 6.10 has built-in support for splitting the scheduled parity check into multiple shorter periods. See Settings -> Scheduler

 

True, but it is not as flexible as the Parity Check Tuning plugin (which also has more capabilities than the built-in feature).

[Frank1940]

I am going to be a bit of a contrarian here.  It appears that you have 18TB of storage space on your current server.  How long did it take you to fill it?

 

To replace that first disk with one of 16TB, you will have purchased three 16TB drives at that point and you will add a minimum of 13TB of additional storage to your server.  That is all well and good.  Now, lets assume that you have a second drive fail.  (Don't know the age of your drives and, of course, no one can predict if and when a drive will go belly-up, but it will happen!)  You will then be adding another 13TB-to-14TB of storage.  You could easily end up  with a server that is only a third filled.  

 

Hard drives in Unraid servers seem to last 5 to 8 years.   Do some analysis of so that by the time you get to the point with your current up-sized drives are starting to fail, they aren't half empty. 

 

It might make sense to go with 16TB or it might make sense to with 8 or 10 TB drives. 

[ConnerVT]

55 minutes ago, Squazz said:

And it also seems that parity checks is not that big of a problem with drives in the sizes of above 10Tb

 

The time to run the parity check scales pretty linearly with the size of the parity drive (as it is the largest).  My system is similar in processing power to yours (if your siggy is correct_ - 1st Gen Ryzen 1700 to my 1500X gives you 2 more cores and the same everywhere else.  With my 8TB drives and the system mostly idle (while running about a dozen dockers) a parity check runs about 14.5 hours.  I've streamed Plex movies during a parity check (both Direct Play at home and hardware transcoded away) and had no issues (stuttering/pauses/etc).  One month when I was doing this, the parity check time increased to about 16 hours.

 

So it comes down to what activity is happening on the server when running a parity check, and does it have any meaningful impact to users.  That a parity check may take longer, or a file transfer may take a little longer isn't worth worrying over.  If video playback suffers, or other noticeably annoying behavior is observer, do some planning as to when you can schedule your parity checks for the least intrusive times.  Even if it takes 12 days @ 2 hr/day really doesn't matter, as long as the maintenance task is done.

[Squazz]

23 minutes ago, Frank1940 said:

I am going to be a bit of a contrarian here.  It appears that you have 18TB of storage space on your current server.  How long did it take you to fill it?

 

... Do some analysis of so that by the time you get to the point with your current up-sized drives are starting to fail, they aren't half empty. 

 

It might make sense to go with 16TB or it might make sense to with 8 or 10 TB drives. 

 

That's a fair point! Which is also what is making me consider if I should just go with 8Tb 7E8 drives instead.

 

15 minutes ago, ConnerVT said:

 

The time to run the parity check scales pretty linearly with the size of the parity drive (as it is the largest).  My system is similar in processing power to yours (if your siggy is correct)

 

...

 

Even if it takes 12 days @ 2 hr/day really doesn't matter, as long as the maintenance task is done.

 

My signature is updated now  

 

Good to hear about your experience, and a fair point about the 2hr/day is also viable, as long as it is done.

 

[Frank1940]

One more point--  I would get CMR Drives rather than SMR drives.  They have the potential for much better write performance. 

 

7200RPM drives also increase both full-array read and write performance BUT you have to be planning ahead to get the full benefit.  All the drives have to be 7200 RPM.  One slow drive will be a bottleneck.  However, that bottleneck will only last until that slow drive is out in the mix.  (A 5400RPM 2TB will only slow things down until the array operation moves past that 2TB point.)

Edited May 25, 2022 by Frank1940

[ConnerVT]

4 hours ago, Squazz said:

My signature is updated now  

 

Just read through your build thread.  I touched on it in an earlier post, but you'll appreciate the reduction in power.  My HGST drives are a bit power hungry.  I keep 3 spinning all of the time (parity + 2 with video media) plus a NVMe (cache + appdata) and a SSD (usually idle - my transcode cache drive, until I kill it, it is ancient).  The screenshot below is when sitting idle in this state.  56-64W.  When all drive are spun up, and CPU busy, mid 90W.  I have to max the system out to get it to pull 120W.

[Squazz]

On 5/26/2022 at 3:05 AM, ConnerVT said:

 

Just read through your build thread.  I touched on it in an earlier post, but you'll appreciate the reduction in power. 

Buying new harddrives just for the reduction in power, seems to not really be worth it. At least here in Denmark

We are talking a payback time of at least a decade, when looking at powerdraw exclusively

 

It's a nice bonus. But not something I'd buy new drive for

 

[ConnerVT]

I agree that it isn't a major factor in making a change.  And let's hope, given the world's current economic issues, it stays that way.

 

But for others that may read this thread, the reduction in power usage also bring less heat, noise and cabling.  Perhaps a smaller sized power supply and/or case, or the elimination of a HBA or two.  Just things to consider when spec'ing out a new build.

[Be-Art]

On 5/25/2022 at 11:21 PM, Frank1940 said:

One more point--  I would get CMR Drives rather than SMR drives.  They have the potential for much better write performance. 

 

7200RPM drives also increase both full-array read and write performance BUT you have to be planning ahead to get the full benefit.  All the drives have to be 7200 RPM.  One slow drive will be a bottleneck.  However, that bottleneck will only last until that slow drive is out in the mix.  (A 5400RPM 2TB will only slow things down until the array operation moves past that 2TB point.)

Seeking some clarity and sanity check on possible issues with mixing rpm and cache capability drives. This is my current config after adding 2 x 7200rpm drives to my array.

 

1 x SSD Crucial/Micron CT240BX500SSD1
2 x SSD Crucial/Micron CT500MX500SSD1
1 x NVME SKHynix_HFS256GD9TNI-L2B0B

3 x WD Red 4TB WDC WD40EFRX-68N32N0 5400rpm (parity 1, disk 4, disk 6)
1 x WD Red 10TB WDC WD40EZRZ-00GXCB0 5400rpm (parity 2)
2 x Seagate Iron Wolf 2TB ST2000VN004-2E4164 5900rpm (disk 1 & 2)
1 x WD Enterprise 4TB WDC WD4000FYYZ-01UL1B1 7200rpm (disk 5)
1 x HDD TOSHIBA P300 2TB HDWD120 7200rpm (disk 3)
1 x WD Blue WDC WD40EZRZ-00GXCB0 4TB 5400rpm (disk

New Additions
2 x HDD Toshiba X300 4TB 7200rpm (disks 7 & 9)
1 x SSD Crucial_CT525MX300SSD1 525GB

 

All SSD's are outside the array.

 

I have concern, as in the past i have used drobo (i know its a raid 5), but i had a huge corruption before coming to unraid this was likely (in opinion of the data recovery expert i paid a stack of cash too in order to retrieve the drobo data) mixing wd blue with toshiba x300 and two other drives. One failed and was unable to rebuild.

 

Is the bottleneck concept applicable to parity processes between 5400, 5900 & 7400rpm's.  My current thoughts are i have been running unraid for 2+ years, its been running ok with 1 x 7200rpm (toshiba P300) and 2 x ironwolf NAs at 5900rpm rating.

[JorgeB]

It's not a problem to mix different speed disks in the array, parity check/rebuilds will be limited by the speed of the slowest disk, other than that there are no issues.

[Be-Art]

1 hour ago, JorgeB said:

It's not a problem to mix different speed disks in the array, parity check/rebuilds will be limited by the speed of the slowest disk, other than that there are no issues.

Thank you for the feedback.