Is parity-check/-rebuild single threaded?

[hawihoney]

From my own experience it looks as if parity-check/-rebuild runs single threaded? Is this true?

 

[JorgeB]

1 hour ago, hawihoney said:

if parity-check/-rebuild runs single threaded? Is this true?

Yes.

[hawihoney]

Thanks.

 

Seems that single-thread performance is more important then on Unraid parity-check/-rebuild than number of CPUs, cores.

 

[Pixel5]

How many disks are in your array that the CPU is pegged at 100% in this scenario?

 

im running a fairly small array with 4 disks and also have a cpu with high single clock performance so i never even noticed that it puts any load on the CPU, at which number of disks could this become the limiting factors instead of the drive speed itself?

[itimpi]

On most cases parity-check is input/output limited by the speeds of the disks rather than limitations of the CPU.

[jpporter]

I just spent the last few months updating my Unraid system with an updated Supermicro case with 12GB SAS expanders, 12 more drive bays, bought new 2x 12gb HBAs, updated the CPU to lower power, lower speed (but with 4 more cores)...  Did all the testing, using old crappy drives, assuming they were the reason parity was slow.  Went ahead and moved over the real drives... the end result was 1/3 the speed of parity check, from ~24 hours to 3+ days...  Today I figured out that the parity check is single threaded.  Just bought some overkill CPUs that are twice the clockrate and 70% more power usage.  I'll try to reply once I get the CPUs replaced.

 

In short, it would be nice if the parity check/build was multithreaded.

[John_M]

On 2/22/2021 at 10:45 PM, jpporter said:

In short, it would be nice if the parity check/build was multithreaded.

 

Why? How would that speed things up?

[trurl]

Parity is completely I/O bound so CPU cycles aren't an issue except for very old CPUs on dual parity.

 

If you installed port multipliers that probably explains your slowdown.

[Jan Hoker]

On 3/3/2021 at 10:27 PM, trurl said:

Parity is completely I/O bound so CPU cycles aren't an issue except for very old CPUs on dual parity.

 

If you installed port multipliers that probably explains your slowdown.

I cannot agree with you. I have similar experience as jpporter. I have Ryzen 3700X thats not slow CPU, 2x1TB Samsung 980 PRO NVMe as chace drives and 3x12TB WD Gold HDDs one of them parity drive. Every time when parity check starts one thread is pegged to 100% and all my work must stop, shares are usually not accesible during that time and services like Plex or HOOBS are not working or at very limited speed/laggy. That includes shares which are only in cache drives(game libraries), because CPU is struggling with that kind of load. It would be awesome if this proces would be multitreaded, not because speeding up, i dont care how long it takes, but because I could access data. It's maybe issue just for Ryzen CPUs caused by architecture and one 100% pegged thread can cause this trouble. But I'm really unsatisfied by performance during parity check.

[primeval_god]

I would suggest taking a closer look at you system performance with something like Netdata, the unRAID dashboard is not particularly detailed about system performance. I would suspect you will find very high iowait and that your pegged core is mostly busy waiting for IO. Why that would bog down your entire system and what to do about it I unfortunately dont known.