I've just been having an argument with ChatGPT.

 

I had a thought about the possibility of using two smaller disks in place of a single large disk for cases where people have a single parity setup. For example, 2x8TB drives instead of a single 16TB drive. My idea is, When doing a scheduled parity check, instead of UNRAID scanning the entire system from 0 to 16TB (say) serially, it could be reading from 0 to 8TB for the first 8TB drive, while simultaneously reading from 8TB to 16TB for the second 8TB drive. This should significantly increase performance, nearly doubling, I would think.

 

Even if strict requirements were put in place to be able to enable this feature, such as being the same model drive, or speed, and only when in a single-parity system, etc., I'm having trouble seeing any significant reasons why this wouldn't be possible.

 

ChatGPT insisted it would increase parity computation complexity amongst other things, but it seemed to be thinking that the parity calculation itself would be split across the drives.

 

Having an option to have parity setup in this way would allow people to use older, smaller disks they may have laying around instead of buying increasingly larger, more expensive disks just to use them for parity as the array grows. Personally, I'd just love to have my parity checks done in less than 1 day instead of 2.

Parity involves all drives simultaneously already.

 

It would not increase performance as all disks are read serially from start to end during a parity sync/check.

It seems I haven't made myself clear.

 

When downloading a file, you can download it one byte at a time from start to finish, or you can download it twice as fast by downloading from the start AND from the middle. This is how applications like IDM work, except usually with more than two simultaneous streams.

 

I'm not talking about reading parity from start to finish filling up one parity disk before spilling over onto a second one, I'm talking about reading parity info from the beginning AND the middle, writing both simultaneously to both parity disks. This way, the longest this process will take is the time to read the first 8TB, as the second 8TB was read at the same time.

This could be kind of interesting, but it would be a very niche setup, if you wanted to add for example a 16TB data disk, parity check would be super slow due to all the seeks, you can use the future multiple array option to achieve something similar, just create two smaller arrays.

Downloading and disk reading are not comparable. 

 

You can't read the beginning and the middle of a disk simultaneously. There isn't the internal hardware to allow that to happen. It can read the beginning, then seek to read the middle, repeat,etc.

 

Seeks are by far the slowest thing the disk hardware does. So it would make it much slower. 

3 hours ago, trurl said:

You can't read the beginning and the middle of a disk simultaneously.

I don't think that's what OP is asking about.

Conceptually - someone has 16TB of space. 

If that space is comprised of a single 16TB disk, OP argues that should take 2x as long because the process has to go from 1 to 16 on a single disk.

If that space is comprised of 2x 8TB disks, then OP wants to run a parity check against both disks at the same time. In theory, simultaneously running the parity check against 1-8 (disk 1) and 9 - 16 (disk 2), in parallel.

Colloquially, starting both at the beginning and in the middle of your data at the same time. 

I've no horse in this race, just trying to make sure I understand OP correctly.

Edited November 16, 20241 yr by whipdancer

4 hours ago, whipdancer said:

Colloquially, starting both at the beginning and in the middle of your data at the same time. 

I've no horse in this race, just trying to make sure I understand OP correctly.

That is what I assumed.   The problem is that it would dramatically slow down the check as the disk heads continually move back and forth between the two positions. 

9 hours ago, whipdancer said:

simultaneously running the parity check against 1-8 (disk 1) and 9 - 16 (disk 2)

12 hours ago, trurl said:

You can't read the beginning and the middle of a disk simultaneously.

I was talking about the parity disk.

 

Parity would have to be 2 independent disks synced to independent data disks. In other words, multiple arrays.

Just use smaller disks for the entire array.

You can't split a big disk into two partitions and read from both without a performance hit.

On 11/16/2024 at 9:04 PM, trurl said:

Parity would have to be 2 independent disks synced to independent data disks. In other words, multiple arrays.

I didn't think it would work, but I didn't know why. This clears it up in my head, at least.