Parity check slow and lots of errors

[kbrutlag]

Hello -   I run a parity check monthly, but this last one was taking a very long time to finish.  When I checked, it had been running for a little over 23 hrs and had over 500K errors.  I've since tried it a few more times and each one starts off with good speed, then drops way down.  All the the drives are not reporting any trouble and I've even tried stopping Docker thinking that may have something to do with it since I recently set it up.  I recently moved to new hardware (second time in 8 years), and nothing stood out - no errors, etc.  I did add a 10 GB network card, and the driver is throwing errors about the SPF module not being supported.  I asked our Linux administrator about it and his guess was that it wasn't configured to support generic modules, but that should not affect a parity check.

 

I appreciate any insight, and have attached a diagnostic log.

 

Thanks

Ken  

unraid-1-diagnostics-20180819-1348.zip

[trurl]

You rebooted before getting the diagnostics so we can't see anything that was happening before that. Get another check and get us the diagnostics while the problem is exhibiting.

[kbrutlag]

Thanks trurl - started a new check and it ran at good speed for a little over 2 hours then dropped.  I've added a new diagnostic data file.

unraid-1-diagnostics-20180820-1533.zip

[trurl]

Looks like it is disconnecting on multiple ports. Controller issue?

[John_M]

2 hours ago, trurl said:

Controller issue?

 

It certainly is, and I bet the reason is this:

03:00.0 RAID bus controller [0104]: Marvell Technology Group Ltd. 88SE9485 SAS/SATA 6Gb/s controller [1b4b:9485] (rev c3)
	Subsystem: Marvell Technology Group Ltd. 88SE9485 SAS/SATA 6Gb/s controller [1b4b:9480]
	Kernel driver in use: mvsas
	Kernel modules: mvsas

The mvsas driver is broken under unRAID 6 (maybe it was the move to 64-bit) and Marvell HBAs are generally problematic. The symptoms are that disks randomly get disconnected. It's often worse if you have IOMMU enabled but the workround of disabling IOMMU doesn't seem to work any more. You really need to replace it with a card based on an LSI controller. This issue crops up time after time and I've been affected by it myself. I replaced my Marvell controllers with Dell Perc H310s bought second hand on ebay (quality server pulls) and cross-flashed to LSI/Avago/Broadcom IT mode.

[kbrutlag]

Thanks everyone - I have a new LSI based controller on order.  I'll let you know how things go in a few days.