...and I'm wondering what will happen once the 4TB mark is reached.

 

I'll have an answer later tonight or sometime tomorrow from real-life experience, so that's why I'm bringing this question up in the lounge. It's just idle conversational curiosity while I wait.

 

Once 4TB is reach, all that's really left to do is "zero out" (I don't know if zero bits are actually used) the last half of each parity drive, right?

Should I expect the parity rebuilding to race through this process of dealing with the last 4TB? Or will Unraid poke along as if it were still doing all of the work needed to build meaningful parity?

What I'm up to at the moment is the experiment of taking what had been an all-SSD array (yes, I'm aware that support for all-SSD is itself considered only experimental) and replacing the parity drives with HDDs, figuring they have to deal with the highest level of write activity. Also, spin up delays and head seeking delays when writing data are of little importance to me. Fast reading without worry about mechanical delays (especially head contention) for video streaming is my primary concern.

Since the HDDs are so much cheaper than SSDs I figured I might as well make my parity drives 8TB now so that I can easily add larger SSDs, or replace existing 4TB drives with larger SSDs, in the future.

I'm also curious, if for some reason my array crashed while building parity, but did so somewhere beyond the 4TB point, if Unraid would be smart enough to know it had all of the parity it needs to emulate and rebuild any of my 4TB data drives, or if Unraid won't consider the parity drives to be at all valid and useful until the very last byte of their 8TB capacity has been filled.

The Parity build process just continues normally until it reaches the end of the larger parity drive.

Nothing special is done to take into account the actual data drive size.

 

There was a good rationale, its just that I cannot remember it at the moment.  

Parity build will continue as normal writing the correct parity to the whole disk, i.e. zeros. Check does the same, it checks they are zeros. That is so that whenever you add a new drive that's larger and now uses that parity space it's already prepared for it.

Edited February 8, 20251 yr by Kilrah

25 minutes ago, ChatNoir said:

The Parity build process just continues normally until it reaches the end of the larger parity drive...

I wasn't expecting it to give up. I'm wondering if the rebuild process will do the last half a lot faster, since there's nothing to read from other drives and nothing to compute.

 

As the rebuild is at 49% right now, I should know fairly soon. The data drives are slightly larger than half the size of the parity drives, so it will be just a little past 50% when there's nothing left to read from the data drives anymore.

Edited February 8, 20251 yr by RasterEyes

It looks like the answer is yes, the last 4TB will go a lot faster than the first 4.

 

Oddly enough, the speed picked up quite a bit a little before the very end of the data drives was reached, as if the last dregs of real data could somehow be read and processed faster.

When progress hit 51.2% all reading from the data drives stopped. The parity drives are now being zeroed out at a fairly steady 180-190 MB/sec.

 

Estimated remaining time is a little over five and a half hours -- I think that's a fairly good, reliable estimate because the value isn't wildly bouncing around as it often does.

 

The first 4TB of parity took over one full day to be completed. That perhaps could be really be more like a little under 24 hours if you subtract a 2-3 hours when I paused the rebuild (I'm not sure if the paused time gets subtracted from the elapsed time or not).

But either way, yes, when there's no real work to do reading and computer parity, it looks like the parity building process can run at steady, fast pace, about as quickly as the drives will accept data.

49 minutes ago, RasterEyes said:

since there's nothing to read from other drives and nothing to compute.

Parity is not computationally intensive, it's all 100% limited by drive speed (unless you have controller bottlenecks too).

 

The 8TB drives are just both much faster than the 4TB ones and only half their way in on the actual platters while the 4TBs were at the inner end where they're slowest. It'll gradually slow down again now, but yeah you get a "jump" after the end of a disk.

52 minutes ago, Kilrah said:

The 8TB drives are just both much faster than the 4TB ones and only half their way in on the actual platters while the 4TBs were at the inner end where they're slowest. It'll gradually slow down again now, but yeah you get a "jump" after the end of a disk.

Except in this case the 4TB data drives are SSDs, only the parity drives are now HDDs. (Using HDDs for parity, instead of those also being SSDs, is what I've been experimenting with.)

OK, SSDs in array isn't recommeded so wouldn't have expected that.

4 hours ago, Kilrah said:

OK, SSDs in array isn't recommeded so wouldn't have expected that.

Hence the "(yes, I'm aware that support for all-SSD is itself considered only experimental)" in my OP. 😁

On 2/8/2025 at 3:42 AM, RasterEyes said:

I wasn't expecting it to give up. I'm wondering if the rebuild process will do the last half a lot faster, since there's nothing to read from other drives and nothing to compute.

 

Two things to consider:

 

Access time (read/write time) of the slowest drive being accessed during Parity Check/Rebuild is typically the gating factor as to how fast it is running at any one time.

Spinning HDD access times are always the fastest at the start of the Parity Check/rebuild, and slowest at the end.

 

If all you had are spinning rust drives, you probably would have seen an increase in speed once the 4TB drives were finished.  Since they are SSD, there will be no speed increase, just slowly getting slower (or maybe not - I don't know if Unraid is smart enough to recognize that it just needs to write zeros for the rest of the task).

One other thing to add...  If you're writing to the array while the parity check is running then everything will slow down.

 

Here's a bit of a comparison for you.  Notice that my last check was quite slow because a massive sync happened to be running when the scheduled parity check kicked in.