UDMA CRC issues

[unsainted]

I have a 3U supermicro system running 2X X5650 Xeons. I have 17 disks total and on at least 3 of them I am getting UDMA_CRC errors pretty frequently. Drives are running from 2X LSI 9211-81 in IT Mode through a Supermicro direct attach backplane. I have already replaced the cables from the HBA to the backplane and things have improved. I have been fighting this for a few weeks now with drives dropping out of the array etc.. I was finally able to get all drives added back into the array after the cable swap, but I am still seeing 3 drives producing UDMA_CRC errors. I have a new backplane on order and plan to switch that out. I am running a 920-SQ power supply (supermicro)

 

My question is, can the HBA be causing this, and if so, wouldnt it affect all drives, or do you think this is something in the backplane, I have swapped cables twice now, first set cheap from amazon caused more problems than i had originally, so ordered some better quality ones.

 

In the screenshot, it shows errors on 2 drives, but smart test says theyre ok.

 

Any advice on where to look next? Is replacing the backplane a valid idea?

 

mars-diagnostics-20180530-0838.zip

 

Edited May 30, 2018 by unsainted
added diagnostics

[JorgeB]

3 minutes ago, unsainted said:

In the screenshot, it shows errors on 2 drives, but smart test says theyre ok

Those are read errors, possibly related to connections issues, but parity sync will have some corruption.

 

4 minutes ago, unsainted said:

My question is, can the HBA be causing this, and if so, wouldnt it affect all drives, or do you think this is something in the backplane,

I would say the backplane is more likely the culprit, you could swap cables and disks on the affected slots and see if the CRC errors stay with the slots.

[unsainted]

2 minutes ago, johnnie.black said:

Those are read errors, possibly related to connections issues, but parity sync will have some corruption.

 

I would say the backplane is more likely the culprit, you could swap cables and disks on the affected slots and see if the CRC errors stay with the slots.

I will definitely try that, its about 1/2 way through parity sync so once that is done, I can try your suggestion.

[unsainted]

Do you think the read errors are related to the parity sync?

[JorgeB]

Related as they happened during the sync, but not caused by it, like I said parity won't be 100% valid with those errors, diagnostics might give some more clues.

[unsainted]

I have attached diagnostics to this post. Appreciate any advice you have and thanks for taking the time to respond.

[JorgeB]

SMART for both disks looks fine, and both have a very high number of CRC errors, so more likely connection related.

 

You can let the sync finish but you'll need to do a correcting parity check after to fix those sync errors.

[unsainted]

1 minute ago, johnnie.black said:

SMART for both disks looks fine, and both have a very high number of CRC errors, so more likely connection related.

 

You can let the sync finish but you'll need to do a correcting parity check after to fix those sync errors.

thanks! was looking through syslog and wow seems i have a few other issues too.

EDAC MC0: 369 CE error on CPU#0Channel#1_DIMM#0

 

bad ram stick(s)?

 

I have 96GB in the server but only 80 is being reported..

 

[JorgeB]

bad ram stick(s)?

Possibly, check the board's SEL (system event log), might have more info.