Multiple Drive Failures but SMART says no errors

[Zangief]

Hello wonderful UNRAID community peeps :-)

 

My eternal gratitude is yours for solving this frustrating situation.

 

My Unraid system experiences drive failures every few months. (image 1 - Error Messages)

 

My diagnostics files are attached (both before any reboots whilst the drives are in error and after a cold restart).

 

My hardware is:

Model: Custom

M/B: Supermicro - X11SSL-CF

CPU: Intel® Xeon® CPU E3-1240 v5 @ 3.50GHz

HVM: Enabled

IOMMU: Enabled

Cache: 128 kB, 1024 kB, 8192 kB

Memory: 64 GB (max. installable capacity 64 GB)

Network: bond0: fault-tolerance (active-backup), mtu 1500 
 eth0: 1000 Mb/s, full duplex, mtu 1500 
 eth1: 1000 Mb/s, full duplex, mtu 1500

Kernel: Linux 4.9.30-unRAID x86_64

OpenSSL: 1.0.2k

 

SMART status reports indicate no errors, and indeed when I have put the disks back in the array they seem to continue functioning for months at a time before it happens again....

 

Has anyone experienced something similar? Could it be some issue at the hardware (RAID/SCSI controller level)?

 

Thank you in advance

Zangy

 

 

 

foxserver-diagnostics-after-cold-start.zip

foxserver-diagnostics-before-cold-start.zip

[JorgeB]

Looks like a connection issue, though the LSI driver is not very helpful with the error messages.

 

I see you're using some sort of enclosure with an expander, do you have any way of connecting 4 disks to the free onboard SATA ports? Then you could check if the problem stayed with the disks still on the enclosure or not.

[Zangief]

Thanks for the response...

 

I have a rack mounted server that looks like this... (24 hot swap bay enclosure)

 

How it works internally I am not sure, but someone else suggested maybe I should invest in a separate hard disk controller? 

[JorgeB]

It may be the controller, the enclosure itself (like the builtin expander), power supply, etc, you'll need to start swapping some parts, if you had a way of connecting 4 disks to the onboard SATA it would be the fastest way to see if the issue is related to the HBA or the enclosure/expander.

[Zangief]

15 minutes ago, johnnie.black said:

It may be the controller, the enclosure itself (like the builtin expander), power supply, etc, you'll need to start swapping some parts, if you had a way of connecting 4 disks to the onboard SATA it would be the fastest way to see if the issue is related to the HBA or the enclosure/expander.

 

This issue does not occur frequently though (i.e. it may not be an issue for several months if not longer)... So connecting to an onboard SATA may not immediately prove or disprove anything. But I hear what you are saying, there is some sort of issue between the motherboard and the hard drives.