Data drive larger than Parity

[Leseratte10]

I have not yet set up my UnRAID machine (I'm just in the process of buying all the parts to build a new machine), so I apologize if this is already mentioned somewhere in the UnRAID UI, but I haven't found it being discussed on the forum anywhere. 

 

Right now, the parity drive(s) for UnRAID need to be the largest drives in the array, and if you try to add a new data drive that's larger than your parity drive, UnRAID won't let you do that. I do understand why that's needed - on a data drive that's larger than the parity drive, the storage at the "end" of the data disk wouldn't be protected by parity. 

 

But lets assume I have a setup with one 8TB parity drive and two 8TB data drives each, and the data drives are at capacity and I need additional storage NOW, as fast as possible. I run to the store and grab a new 12TB drive (or whatever size is currently cheapest in terms of $/TB), and then I would either need to throw away my parity and re-build it on the new 12TB drive (giving me three 8TB data drives and one 12TB parity drive), or I would need to migrate the parity data to the new drive. 

 

Both solutions are not ideal - the first one means my array is entirely unprotected until the parity is rebuilt and if a drive dies, I'm screwed. And the second solution means it's going to take a long time to copy / migrate the parity data, probably even with a long-ish downtime, and I cannot use the additional storage immediately - I would need to wait until the parity move is over.

 

However, just by looking at the documentation about how UnRAID works, there should be a theoretical third solution, which I haven't found anything about online. I believe UnRAID currently doesn't support that, so that's why I'm writing this post. 

 

Assuming I have an array with 2x 8TB data and 1x 8TB parity - what if it was possible to add the new 12TB drive as an additional data drive, but have UnRAID limit all writes to that drive to the first 8TB? Create an xfs partition that's just 8TB and ignore the rest.

 

That'd mean my new 12TB data drive would act like it was just an 8TB data drive, with the last 4TB unused / unavailable. But it would also mean I'd get an additional 8TB of usable storage *right now* without having to mess with parity. And then in the future I have all the time in the world to get yet another new 12TB drive, migrate the existing 8TB parity drive to a new 12TB parity drive, and at that point unRAID would recognize "Hey, the parity drive is now larger, that means I can now unlock/use the last 4TB on that data drive by resizing the file system". 

 

Basically, it would mean that you could *add* data drives to your array which are larger than your parity drive, but you'd only be able to use the same amount of space that your parity drive has. 

 

Am I missing something that makes this not work as I described? Or is this just not implemented in UnRAID for whatever reason? I'm not saying I absolutely need this feature, but it would be nice to be able to expand an array with a larger data disk, even if it means I cannot use 100% of it (until I get a larger parity drive).

 

[trurl]

Why would you want to waste space? Why pay for 12TB when you only want to use 8?

 

Just rebuild parity onto the 12 then add the old parity as data and you will be able to use 12TB data disks later since parity will be 12

[trurl]

52 minutes ago, Leseratte10 said:

grab a new 12TB drive (or whatever size is currently cheapest in terms of $/TB), and then I would either need to throw away my parity and re-build it on the new 12TB drive (giving me three 8TB data drives and one 12TB parity drive)

Note that you haven't "thrown away" anything because the former parity disk becomes a data disk. This is, of course, exactly what I said above and is the simplest and usual way to proceed.

 

54 minutes ago, Leseratte10 said:

second solution means it's going to take a long time to copy / migrate the parity data, probably even with a long-ish downtime

There is procedure known as "parity swap", and is normally used to replace a failed data disk with original parity while upsizing parity to a new disk. There is no simple way to copy parity yourself, but this procedure is built-in. The array is offline while parity is copied, of course, since it can't be allowed to change until copy is complete. I have even used parity swap to upsize parity and upsize a data disk by replacing it with old parity.

 

 

[Leseratte10]

The goal of the way I suggested would be *quickly* adding more storage to an UnRAID array when needed, without losing parity. 

 

The first method (which I described as "throw away parity" would mean adding the old parity drive as data drive, rebuilding a new parity onto the new 12TB drive. Which means my array will be unprotected while that happens. 

The second method ("parity swap") would require the array to be offline until the whole 8TB are copied, which will be quite a while.

 

Yes, the method I suggested would result in you wasting space, but only until you manage to swap parity to a new 12TB disk at a later time, when a downtime is convenient. It was a suggestion for a method to add more storage to an UnRAID array without downtime and without losing parity. Yes, it would waste 4TB in this scenario, but I still feel it would be good to have this as an optional feature if needed - if I need a bunch of more storage immediately and don't want to wait for a parity swap.

[JonathanM]

21 minutes ago, Leseratte10 said:

The goal of the way I suggested would be *quickly* adding more storage to an UnRAID array when needed, without losing parity.

Proper planning eliminates hasty decisions, as trying to do things "quickly" often results in problems.

 

You should always try to keep a minimum free space across the parity array equal to your largest data drive, when you get below that it's time to source another drive and expand the array.

 

You should never be in a hurry to do anything with computers, as proper planning is required to keep your data safe.

[ChatNoir]

I agree with JonathanM's view and there are things you say that makes me think you do not understand how Unraid Parity works

 

3 hours ago, Leseratte10 said:

Create an xfs partition that's just 8TB and ignore the rest.

Parity is unaware of file systems (maybe also of partitions ?). It only consider raw bit status on disks sectors.

 

Also, that part seems sketchy to me.

2 hours ago, Leseratte10 said:

"Hey, the parity drive is now larger, that means I can now unlock/use the last 4TB on that data drive by resizing the file system". 

 

Is this even possible without formatting the whole drive (and thus losing the data on the drive) ?

If it was, the whole parity would become obsolete and would have to be rebuilt, and then :

3 hours ago, Leseratte10 said:

my array is entirely unprotected until the parity is rebuilt and if a drive dies, I'm screwed

[trurl]

Computers are faster than ever but our data is larger than ever.

 

Patience is a virtue.

[Leseratte10]

14 minutes ago, ChatNoir said:

Parity is unaware of file systems (maybe also of partitions ?). It only consider raw bit status on disks sectors.

 

That is true, but doing that (creating a smaller FS) makes sure that data is only written to the first 8TB of the disk (which would be parity-protected), doesn't it? 

 

16 minutes ago, ChatNoir said:

Is this even possible without formatting the whole drive (and thus losing the data on the drive) ?

If it was, the whole parity would become obsolete and would have to be rebuilt, and then :

Yes, it is typically possible to resize a filesystem. Why would that invalidate the parity? It's just normal writes to the disk ...

 

35 minutes ago, JonathanM said:

You should always try to keep a minimum free space across the parity array equal to your largest data drive, when you get below that it's time to source another drive and expand the array.

 

I definitely agree with that. My suggestion was just about a solution when you've ran into the issue already, didn't plan ahead, and need more storage immediately. But looking at the other answers, it's probably not as simple as I imagined it to be.

[MrAndyBurns]

So ... what did you do??

 

I have a 3tb WD Red parity and 2x older 3tb standard disks ... I need some more space and have an 8tb WD Green drive I would like to add to the array while saving up for a new 8tb 'NAS' spec drive (Ironwolf or Red).

 

My initial hope is add the 8tb standard drive to the array and get extra 3tb storage due to the size of my parity (3tb parity with 8tb (3tb usable)+3tb+3tb=9tb space) THEN in a few months, add new 8tb NAS drive as parity drive which then allows me to use all 8tb of the standard drive in the array (8tb parity with 8tb+3tb+3tb and retire oldest drive)

 

ps

I say NAS drive for the parity working on the principle that the parity does a lot more read/write so do not want to use an older standard 8tb shucked drive.

[JorgeB]

5 minutes ago, MrAndyBurns said:

My initial hope is add the 8tb standard drive to the array and get extra 3tb storage due to the size of my parity

You can't do that, you can use the new 8TB for parity, then add old parity to the array.