If one has three drives in Unraid:

A) 250GB

B) 500GB

C) 1TB (parity)

 

My understanding is that only 500GB of the parity drive is utilized in the above example. What if drives A and B are joined in a JBOD-like manner, thus utilizing 750GB of the parity drive?

 

This could further reduce unneeded drive spinup of the two data drives and make it more practical to join many small drives together with a large parity drive. I'm sure I'm not the only one with piles of older and smaller drives. ( http://www.hardocp.com/news/2009/03/21/50_bmg_api_vs_18_hard_drives/ )

While an interesting idea at first glance, it makes the situation of drive expansion (replace a smaller drive with a larger drive) insanely complex. It also requires large fundamental changes in the low level device drivers to track drive offsets and perform sector translations on every single write operations and in a failed drive simulation every single read operation.

 

It also doesn't reduce drive spin-up in typical happy-day usage scenarios. It only does so in the situations of parity calculation or parity checks and failed drive being simulated.

You can essentially do that yourself, by putting 2 drives into a HW RAID, and then select that RAID set as your parity drive.

Speaking Frankly, I don't see the need to join two smaller drives to a large one for data.

 

The benefit of unRAID has always been, 1 spindle = 1 filesystem.

 

The user share filesystem presents a "merged" view of multiple spindles so applications accessing files over the network never need to know about the hardware layout.

So in the example presented. 750GB of dataspace is readily available. The largest single file is limited to 500GB.

 

I suppose you could do the jBOD joining with LVM and create a larger volume, but then you loose some of management ease.

 

There are SATA hardware addon's that allow you to do all sorts of cool BIG/SPAN/RAID0/RAID1 merging of multiple volumes.

My thought here is that with the current Unraid, in order to write to Disk A, Unraid must also read from Disk B in order to compute parity.

 

With Disk A and B in quasi-JBOD, writing to Disk A does not require spinning up disk B.

Your thought is wrong.

 

unRAID only ever needs to read/write to the 1 Data disk and the Parity disk.

 

In a 20 data drive array, if you're writing to disk 1, unRAID will only ever need to read from data disk 1 and parity disks and then only ever needs to write to data disk 1 and parity disks. Data disks 2 through 20 will not be used.

 

The only time the other disks come into the mix is during the initial parity calculation, during any parity checks, and during failed drive conditions. This should occur less than 1% of the time in a normal array.

 

 

Those are good points Barzija.

 

However, once again it makes the situation of drive expansion (replace a smaller drive with a larger drive) insanely complex. It also requires large fundamental changes in the low level device drivers to track drive offsets and perform sector translations on every single write operations and in a failed drive simulation every single read operation.

 

Let's say you have a 2tb parity disk, and two 1tb data disks.

If one of the 1tb data disks figures in the parity calculations with a 1tb offset,

then you can have two data disks fail at the same time, and still be able to reconstruct.

 

If you have a large array with a 2tb parity disk, and a whole bunch of data disks of various sizes,

there could be a smart way to pick groups of disks whose combined sizes are less or equal the parity drive.

Assign the disks of such a group the appropriate offsets, to spread them along the whole parity disk.

Now two or more disks can fail at the same time, and if you're lucky enough that they were assigned to

non-overlapping regions of the parity disk, then all those disks can be reconstructed from parity without problem.

 

 

Cool idea, Good explanation. I missed this one.

At what point, do you start overlapping matching drive sector parity instead if concatenating ?

I.E. when you exceed the maximum spindle size of parity or (2TB as of now).

 

If I had 3 1tb drives, how would it be handled?

 

Is it worth all the effort in programming?

However, once again it makes the situation of drive expansion (replace a smaller drive with a larger drive) insanely complex.

For drive expansion, it wouldn't really matter much if the old smaller drive used to start at offset 0 or at offset 750gb.

 

It does. When you're replacing with a larger drive, do you also offset the other drives too? There's a ripple effect to consider. If you don't, you could lose the possible benefits yet remain with all the software complexities. If you do, during the parity rebuild/recalculate the array is vulnerable to another drive failure.

 

For instance, if you were to replace Data Drive B from a 500gb drive with a 750gb drive in this contrived situation.

 

Pre Condition

Parity Drive: 2000gb, no offsets

Data Drive A: 750gb, offset 0

Data Drive B: 500gb, offset 750gb

Data Drive C: 250gb, offset 1250gb

Data Drive D: 250gb, offset 1500gb

Data Drive E: 2000gb, offset 0

 

Ideal Post Condition:

Parity Drive: 2000gb, no offsets

Data Drive A: 750gb, offset 0

Data Drive B: 750gb, offset 750gb

Data Drive C: 250gb, offset 1500gb

Data Drive D: 250gb, offset 1750gb

Data Drive E: 2000gb, offset 0

 

Other Possible Post Conditions:

Parity Drive: 2000gb, no offsets

Data Drive A: 750gb, offset 0

Data Drive B: 750gb, offset 750gb

Data Drive C: 250gb, offset 1250gb

Data Drive D: 250gb, offset 1500gb

Data Drive E: 2000gb, offset 0

 

Then every once in a while, if you wish, you can decide to completely reset the configuration, assign new disk offsets, and recalculate parity.

 

 

Or, simply require that adding a new drive always requires new parity computation (since a new JBOD is being created).

 

This idea putting data drives in JBOS is somewhat derived from the proposed/planned feature of having the parity drive span two drives.

Or, simply require that adding a new drive always requires new parity computation (since a new JBOD is being created).

 

That's a bad idea from a data integrity perspective. The reason why new discs are cleared is so parity recalculations are not required. The reason parity recalculations are not desired is whenever it's being done your data is vulnerable until the process completes. Only then is your data protected.

Or, simply require that adding a new drive always requires new parity computation (since a new JBOD is being created).

 

That's a bad idea from a data integrity perspective. The reason why new discs are cleared is so parity recalculations are not required. The reason parity recalculations are not desired is whenever it's being done your data is vulnerable until the process completes. Only then is your data protected.

Also, your idea provides NO benefit.  You gain no additional storage space over having those exact same disks assigned as data disks in the array.

 

Each of the JBOD disks would need their own file system, exactly the same as if they were all set as disk2 disk3, disk4, etc.  If you can physically put the disks in the array, then you can have them all assigned as disks in the array.  It does not matter how big or small.  at times I've had an 8 Gig drive in my array.

 

Joe L.

Also, your idea provides NO benefit.

 

Barzija noted the benefit that one could lose all the data drives in the JBOD data set (in the original example, both drive A and drive B) and still restore from the one parity drive (drive C).

 

I see what you are saying about rebuilding parity every time a drive is added, though.

 

Also, is there something that makes this idea fundamentally different from the concatenated parity drive that is on the Laundry List?

Also, your idea provides NO benefit.

 

Barzija noted the benefit that one could lose all the data drives in the JBOD data set (in the original example, both drive A and drive B) and still restore from the one parity drive (drive C).

 

I see what you are saying about rebuilding parity every time a drive is added, though.

 

Also, is there something that makes this idea fundamentally different from the concatenated parity drive that is on the Laundry List?

Not really, other than the concatenated parity disk does not need a separate file system per disk...(because the parity disk has no file-system)

 

But, it has the same "flaw" in implementation.  You don't really gain storage space.  Its only advantage is that you might be able to fit a larger file on a larger single data disk, because the file-system might be bigger.