Parity Check and Disk Pre Clear Running Slow

[mason.dale]

Hi,

 

    I have recently realized my unraid was massively out of date. I upgraded from 6.7 to 6.11.5.  At first it was quite slow, but as it did if first start up and restart, everything sped up to normal. For some reason the initial drive mounting on start up, Parity Check and disk pre clearing take an awful long time.  Originally parity check was taking about 1000 days, since checking the file systems of each disk, it has come down to  about 60 days, which is still quite a long time.  There are two parity drives @ 10TB. Pre clear on a new 6tb has been running for about 24hrs and it is only at 1%.  Have attached system logs.  If anyone has any ideas, they would be much appreciated. 

 

Thanks!

 

timothy-server-diagnostics-20230416-1520.zip

Edited April 16, 2023 by mason.dale
wrong file uploaded

[JorgeB]

Please post the diagnostics during a parity check.

[mason.dale]

Spoiler

Lol clearly its my first day.... see attached correct file.

 

Thanks!

 

timothy-server-diagnostics-20230416-1520.zip

[JorgeB]

There's something writing to multiple disks, looks like the mover is still running, cancel the check and wait for any other array activity to stop.

[mason.dale]

Okay cool will do, thank you for the help.

 

Does that also explain the 1mb/s pre clear speed?

 

Thanks!

[JorgeB]

That should be unrelated, it it's still that slow try a different disk.

[mason.dale]

Hi, some more developments on this.  I was wondering if it was a new install thing so I let it sit for a few days just to see if that did anything, it didn't.  I decided to start up the server in Safe mode with nothing running.  First i did a file system check and repair on every drive including the cache drives.  Same thing was happening Parity checks would average out at about 2 MB/s and pre-clear was about the same, it would also take about 30 minutes to mount all of the disks to the array.  At this point I was pretty stumped so i decided to restore the whole installation to 6.7.2.  Works completely fine now.  Parity is now averaging at around 90 MB/s and Pre Clear is also faster.  I am not sure of the changes between 6.7.2 to 6.11.5 but i am sure its alot.  I'm not very experienced with Linux/Unraid, but could it be a driver incapability with my HBA cards? I know they are older, how easy is it to check driver versions and such?  I have attached two diagnostics.  One for 6.11.5 in safe mode with parity check running for about an hour, and one in 6.7.2 started up normally with parity check running.  If anyone has any ideas they would be much appreciated as I would like to update to the newest OS version. 

 

Thank you in advanced.

 

timothy-server-diagnostics-20230425-2109 - 6.7.2.zip timothy-server-diagnostics-20230425-1249- 6.11.5.zip

[JorgeB]

Those controllers have not been recommended for a long time, but they should still perform normally with v6.11, post the diags during a parity check with v6.11.5

[mason.dale]

Hi,

 

     It's attached above. It's parity check running in 6.11.5 for about an hour in safe mode.

 

Thanks.

[JorgeB]

There appears to be something reading from multiple disks, do it again in maintenance mode and if still slow post new diags.

[mason.dale]

Hi,

 

    Attached.  This is the server in version 6.11.5  running in safe mode and the array mounted in maintenance mode running a parity check.  Parity check is still only running at 1MB/S. Let me know if you see anything. 

 

Thanks!

timothy-server-diagnostics-20230428-1626 - 6.11.5.zip

[itimpi]

The syslog in those diagnostics show continual resets happening on ata15 which is

Apr 28 14:59:20 Timothy-SERVER kernel: ata15.00: ATA-11: Samsung SSD 860 PRO 256GB, RVM01B6Q, max UDMA/133

Whether this is the cause of the slow speed I am not sure but it should definitely be rectified as the messages about this are flooding the syslog making it impossible to see if anything else is being reported.

[mason.dale]

Hello,

 

   Apologies,  disk disabled in BIOS (its remote server).  Weirdly that disk has no issues in 6.7.5, its only in 6.11.5 that it starts flagging up errors, not sure if that means anything.  Diagnostics attached.  Again, Server started in safe mode and array mounted in maintenance mode. Parity check still only averaging out at 1 - 2 MB/s, and normal array disk mounting takes about 20-25 minutes.  Let me know if you see anything. 

 

Thanks for all the help.

 

 

timothy-server-diagnostics-20230430-1703.zip

[JorgeB]

Nothing obvious, run the diskspeed docker to confirm all disks are performing normally.

[mason.dale]

Hi,

 

    Ran it see attached screen shots, there is an error but not knowing the docker, I'm not sure what it all means.

 

Thanks!

[JorgeB]

If the test is not running you should post in the docker support thread.

[mason.dale]

Hi,

 

   Apologies, attached the results.  All drives are around 1.5 - 2MB/s. Not sure why, could it be a driver thing since on 6.7.5 it is significantly faster?

 

Thanks!

 

Edited May 1, 2023 by mason.dale
Added One

[mason.dale]

Hi,

 

    Just for reference here is Disk Speed running on 7.5.2.

Thanks!

[JorgeB]

Yeah, that looks really bad, don't remember seeing a similar issue, but those controllers are being used by less users with every release, I still have some let me see if I can test with one.

[JorgeB]

It's not just a driver problem, you are using a SASLP correct? There are other models that look similar on lspci and use the same driver.

 

This is with a SASLP and v6.11.5, the max speed is still being limited by the controller, but it's not far from the controller max usable bandwidth with older Unraid releases, around 640MB/s