Upgraded parity drive - Parity check very slow

[flixxx]

Hello,

 

I recently upgraded my parity from a 4TB to 20TB hard drive. 

Precleared it, did the parity write.

 

Now, I want to ensure it's correct before i install any new hard drives. I am running a parity check (without write correction) and it's running extremely slow - 18MB/s, 12 days to finish.

With my 4 TB, it would take a little over a day.

 

I've attached the diagnostics while parity check is running.

kenny-diagnostics-20231121-0707.zip

Edited November 21, 2023 by flixxx

[itimpi]

According to the diagnostics the check may only have been running a couple of minutes at the point they were taken?    Is this actually the case?   If so new diagnostics after the check has been running for longer might show something.

[flixxx]

Thanks, yes i stopped it because i was running a short self test and was taking very long and i thought it might have something to do with it. I restarted it now and will keep it, still showing at about 17-20MB/s

[flixxx]

Nevermind, found the issue. I had the syslog server running and pointing to the unraid while i was troubleshooting other things a couple of days ago and forgot to turn it off. 

The moment i turned it off, the speed jumped and it will complete at a reasonable time.

 

It dropped back down again - will continue to monitor and post diags in a few hours

 

Edited November 21, 2023 by flixxx

[flixxx]

Attached is an updated diagnostics 24 hours after parity has started.

 

I have a scheduler to check parity every month and for years it wouldn't take more than 1.5 days. 

I upgraded to a 20TB and this is the behaviour now - hoping to get to the bottom of it.

kenny-diagnostics-20231122-0703.zip

[JorgeB]

Nothing obvious in the diags, you can run the DiskSpeed docker to see if all disks and controllers are performing normally.

[flixxx]

I've narrowed down the problem to the docker Codeproject AI. When i turn it off, it jumps back to about 140MB/s. With it on it crawls to about 20 MB/s

Total CPU load with it on goes to a max of 20%, docker stats commands shows it peak at 2%.

 

I just tried pinning cpu on that particular docker to all except 0/1 and it seems to get a bit better to 95 MB/s.

Still gonna play around with it - i've had this docker running for at least a year with no issue and the config hasn't changed.