My parity check is stuck, and seems to be eating huge resources.

[atsmusz]

I had a cache drive that was going bad so I removed it and swapped it out for a much larger one following an article I found on how to do this. I brought the server back up and due to some other issues decided to reload all my dockers.  I put the Docker appdata on cache-only since my cache drive was now 2TB (used to be 256GB) and then when my weekly parity check started it got to 47% and just hung there for days and a number of my CPU cores are constantly running at 100%.  It now seems that there are SMART errors with my parity drive which is quite new 4TB.

 

New Diags. 

tower-diagnostics-20210923-0827.zip

tower-diagnostics-20210923-1019.zip

Edited September 23, 2021 by atsmusz
New Daigs 10-19am US PDT

[trurl]

Did you ever get diagnostics? If not reboot and try again.

 

10 hours ago, atsmusz said:

cache drive that was going bad so I removed it and swapped it out

You must always double check connections when mucking about inside. Bad connections are much more common than bad disks.

 

10 hours ago, atsmusz said:

SMART errors with my parity drive

Which SMART attributes exactly? UDMA CRC errors are bad connections.

[atsmusz]

Yeah thnx.  will re-check connections. Good idea.  When I woke up diags had still not finished.  Will attempt re-start and try again, but seems I have to do ugly power-off as command line re-start/shutdown not working either.

 

Diags now uploaded.  System fine after re-boot ....... probably until parity check kicks in.

Edited September 23, 2021 by atsmusz
new info

[atsmusz]

Parity Drive "short" SMART test finished w no errors.  Parity check now running and at 4.8%.  Got some warnings about Parity drive but can't find the text now.  The parity-check usually seems to hang around 40%.

Edited September 23, 2021 by atsmusz

[trurl]

post new diagnostics if you want us to see how things are going.

[itimpi]

32 minutes ago, atsmusz said:

Parity Drive "short" SMART test finished w no errors.  Parity check now running and at 4.8%.  Got some warnings about Parity drive but can't find the text now.  The parity-check usually seems to hang around 40%.

It is not at all unusual for a drive to pass the short test and fail the long one so if possible do that as well.

[atsmusz]

Yeah, I figured that was the case.  Running the long test now but it seems to be stuck at 10% for a long time.

 

That drive is less then 6mos old.  Gonna be irritated w WD if it's already failing.  😞

 

SMART self-test "full" now at 20% so still moving.  

Edited September 23, 2021 by atsmusz

[trurl]

Long test takes several hours and doesn't really give much in the way of updates as it goes. And you may need to disable spindown on the disk to get it to complete.

[atsmusz]

SMART test still running ...... now at 90%

 

Do see this yellow as shown in screen-shot but no idea what it means.

[trurl]

Pending sectors are not good, it means they can't be reliably read and will be reallocated next time they are written. Disks have extra sectors for reallocation. Since there aren't many the disk can still be used but you need to get those reallocated. Since this is parity and contains none of your data, no danger in rebuild, which should get those reallocated.

[atsmusz]

2 hours ago, trurl said:

Pending sectors are not good, it means they can't be reliably read and will be reallocated next time they are written. Disks have extra sectors for reallocation. Since there aren't many the disk can still be used but you need to get those reallocated. Since this is parity and contains none of your data, no danger in rebuild, which should get those reallocated.

When you say rebuild, is this something that will happen or you mean I need to do something specific to initialize the rebuild process? Apologize for these probably noob questions but while I have many years in IT related stuff, I have not had too much experience dealing with HDD hardware issues.

 

It's likely I can get this drive replaced from WD and even have the replacement drive cross-shipped, in the past I have always just replaced drives that had any issues.

[atsmusz]

This is my partiy check progress so far.  In past checks, it seems to "hang" at around 46 or 47% and then says it will take over 31 days instead of the 2 days and change showing now.

[trurl]

How long does parity check usually take for you? I would expect 4TB parity check to be done in less than 12 hours.

[atsmusz]

Parity drive is in external enclosure on USB3 w USBA connector.

Edited September 24, 2021 by atsmusz

[trurl]

USB NOT recommended for array or pool disks. USB does not provide the permanent, reliable connections required.

[atsmusz]

Yeah ...... that makes sense ...... 

 

My case only has one more slot for a drive internally and it already runs too hot ......( case not designed for "server" use )

 

I'm going to see if I can figure out another way to put the drive inside the case but just in the main compartment and get it back on SATA cable.

[atsmusz]

Looks like it was HDD hardware issue.  Many errors (over 900) once I got the drive on SATA instead of USB.

 

Going to RMA it and replace.

 

Thanks @trurl and anybody else that assisted. Appreciate your patience and support!