Huge number of errors when adding parity disk?

[Fencer]

Whew... Finally got all my files copied over to my array and added the parity disk to it this morning. 

I checked it after about 13 hours and it showed 92 million (!) errors and the temperature reading for disk 5 has mysteriously disappeared (see attached pic). 

 

The number of errors hasn't increased in several hours and it seems to be chugging along but this is highly concerning. Disks 6 and 7 are attached to one of the (formerly) approved Marvell controllers and I would have thought if something was going to error out that it would be one of those. No SMART issues are reported on any of the disks as of the time I started the parity build. 

 

Thoughts on this? Should I rerun the parity, restart the system? Kind of lost here at the moment.

Edited May 3, 2022 by Fencer
forgot to add picture

[JorgeB]

Please post the diagnostics.

[Fencer]

5 minutes ago, JorgeB said:

Please post the diagnostics.

OK, Parity run still has a few hours to go but this is the zip with the diags that I just ran. If there's a particular one I should be posting or a different format please let me know

powertower-diagnostics-20220503-0006.zip

[JorgeB]

Disk 5 dropped offline, could a controller issue since SASLP are not recommended, could be other problem, start by canceling the sync and power cycling the server to see if the disk comes back online, then post new diags.

[Fencer]

16 hours ago, JorgeB said:

Disk 5 dropped offline, could a controller issue since SASLP are not recommended, could be other problem, start by canceling the sync and power cycling the server to see if the disk comes back online, then post new diags.

It didn't come back up and all 91 million errors came up off the drive. I'll open it up shortly and see which disk it is and what controller it's connected to. I thought 6-7 were connected to the other controller but I'll double check that and the cables. Attached is the diags after the power cycle: 

powertower-diagnostics-20220503-1715.zip

[Fencer]

Let's see, updates and info:  Turns out Disk 5 is in fact on the Marvell controller and going over my drive notes from before, it and one other drive had reallocated sector counts. There was some kind of event that happened when they were in my old server and 3 of the 7 drives popped up with errors at the same moment. (reallocated sector counts in varying amounts and one had another one. The 3rd drive with the multiple error values didn't get reused but the two with the reallocated sector errors I did reuse after running a surface scan and multiple extended spart checks on and kept an eye on.  The reallocated sector count for Drive 5 has held steady at 48 for over 6 months and remains so and the other has held steady at 72 for the same amount of time. 

 

Per instructions in the Storage Management manual, I stopped the array, unmounted Disk 5, restarted the array, let it see the missing disk and then stopped it, re-added disk 5 and it's now going through a rebuild. So far no errors at about 10% rebuild so hopefully it will rebuild successfully and be good for the moment. It's going to take a month or two to scrape together a few nickels and buy another controller card and new drives to eventually replace disks 5 and 6. 

 

Am I on the right track here? One thing I had read about that controller was that it would possibly need to be reflashed to use but it came right up and didn't have raid enabled. I believe you are correct in the controller being the reason it dropped off. I need to learn more about this stuff going forward...

[JorgeB]

1 hour ago, Fencer said:

One thing I had read about that controller was that it would possibly need to be reflashed

Those controllers are not recommend for a long time, they have several known issues including dropping disks without a reason, I would recommend replacing it with an LSI HBA or another recommended controller.

[Fencer]

14 hours ago, JorgeB said:

Those controllers are not recommend for a long time, they have several known issues including dropping disks without a reason, I would recommend replacing it with an LSI HBA or another recommended controller.

That's the plan. When I was building this I was unfortunately on an older compatibility list and the one I got was the only particular model of the Marvells  still showing compatible at that time. Missed the blurb at the top of the page so my bad. I've gotten a bit better at navigating the forum and the guides since then. Rebuild should be done by the time I get home and it'll be time to add the cache drive and hope the thing doesn't cough up furballs until I can get a compatible replacement one.