Slow Parity Checks

[thezyth]

I'll try to provide as much info as I can because I'm kind of lost as to what the issue could be. I've been getting super slow parity checks dropping from the usual 80-120mb/s to roughly 20mb/s. I did upgrade my single 8tb parity drive to two parity drives 14tb.

 

System Specs:

 

The ASR 72405 controller is set to HBA Mode

CPU scaling governor is set to ON-Demand

Unradi Version: 6.10.0-rc5 

 

Below are some Diskspeed Test and the diagnostic. Please let me know if you have any suggestions I can try. I don't have another controller that I can use to test if it is still somehow the controller. 

 

Tests were run with no docker containers running or any VM's. The diagnostic was pulled while the parity check was running.

 

Diskspeed Test:

 

Controller Benchmark:

 

The cache_downloads SSD is dying but I don't think that should cause any issues with parity checks. 

 

Please let em know if you have any suggestions that I can try to fix this!

 

 

 

 

 

 

zeus-diagnostics-20220504-1359.zip

[JorgeB]

10 hours ago, thezyth said:

CPU scaling governor is set to ON-Demand

See if it get faster if you set it to performance, or slower if set to powersave.

 

Parity sync is single threaded, and with a parity2 it uses more CPU, still with just 8 disks I would think it should be faster than that with that CPU.

[thezyth]

 

11 hours ago, JorgeB said:

See if it get faster if you set it to performance, or slower if set to powersave.

 

Parity sync is single threaded, and with a parity2 it uses more CPU, still with just 8 disks I would think it should be faster than that with that CPU.

Okay as soon as I changed it form On-Demand to Performance it boosted up to 100mb/s. Will report back and let you know if it stays stable at that speed. Attached another diagnostic after setting it to performance.

zeus-diagnostics-20220505-1214.zip

[thezyth]

12 hours ago, JorgeB said:

See if it get faster if you set it to performance, or slower if set to powersave.

 

Parity sync is single threaded, and with a parity2 it uses more CPU, still with just 8 disks I would think it should be faster than that with that CPU.

 

Unforunately that did not last...its back at 20mb/s

[Squid]

Another thing to bear in mind is that with 6.10, parity checks are automatically throttled if access to the array is detected (this is a good thing)

[thezyth]

Some more info I can provide. The parity checks seem to speed up to 100mb/s and drops down to anywhere between 10-25mb/s. The speeds seem to increase and just drop down again. I have also attached another diagnostic while the speeds were at 2.9 mb/s. Hopefully this can be of use.

 

The speeds also seem to be slow even when the server is not being accessed. 

 

 

 

zeus-diagnostics-20220505-1649.zip

[JorgeB]

8 hours ago, thezyth said:

The parity checks seem to speed up to 100mb/s and drops down to anywhere between 10-25mb/s.

That basically rules out any CPU issues, disks seem to be performing well so my main suspect for me would be the controller, not a bandwidth issue but some driver/compatibility issue, that controller is kind of an unknown since there aren't many used with Unraid.

 

 

[thezyth]

11 hours ago, JorgeB said:

That basically rules out any CPU issues, disks seem to be performing well so my main suspect for me would be the controller, not a bandwidth issue but some driver/compatibility issue, that controller is kind of an unknown since there aren't many used with Unraid.

 

 

After doing some more research it seems that the controller might be overheating. Going to repaste the controller and maybe zapstrap a 40mm fan on top of it. 

 

[flyize]

On 5/6/2022 at 4:08 PM, thezyth said:

After doing some more research it seems that the controller might be overheating. Going to repaste the controller and maybe zapstrap a 40mm fan on top of it. 

 

Apologies for bumping an old thread, but how would one see controller temps?

[JonathanM]

On 11/28/2022 at 3:49 PM, flyize said:

Apologies for bumping an old thread, but how would one see controller temps?

[flyize]

Well that's less exciting than I was hoping for.