Problem solved!

Problem has been solved by @Faceman. Thank you so much. You can scroll down further to see my original post.

I'm going to put these in Italic so you know that this is the latest update about what happened if you want a TL:DR. 

My friend and I bought this LSI 9207-8i HBA card (Based on the SAS 2308 chip) from a reputable source so we were convinced it's not a fake LSI card. I also had a spare set of SAS->SATA connectors on standby and after swapping those, the UDMA CRC Errors still persisted. 

Therefore, my post below was where I was. To start with, my card were bought as follows:

Adapter: LSI 9207-8i (SAS 2308)
Firmware: 20.00.00.00-IT
BIOS: 7.39.00.00
Chip Revision: (0x05) 

I eliminated the following:

• Heat Issue: I had a Arctic P12 Max blowing at both cards at 50% speed and both cards are cool to touch.
• Cable Issue: I had a spare set from another online provider and they still gave the same problems.
• Badly Seated Ports: I always did a light tension test and ensured that all the ports were secure. 
• Card: I bought it from a place where most folks bought theirs from as well so while plausible, is more improbable than the above. 

Faceman made an educated guess that the issue was the firmware (and after doing some research, it seems this was a common culprit) so if this is the firmware your card came in, this is the problem causing those errors to pop out.

Scroll down and you'll see that i find out that I found a post on reddit that shows someone who copied two files from two downloads into their Unraid USB Stick and executed the commands. See the solution for Faceman's very detailed instructions on how to do this within Unraid. 

!!!! ENSURE YOUR ARRAY HAS BEEN STOPPED BEFORE DOING SO !!!!

After doing the flash and a reboot later, 

mpt2sas_cm0: LSISAS2308: FWVersion(20.00.07.00), ChipRevision(0x05), BiosVersion(07.39.00.00)

And as you will see in the last post, i started a parity check with no more UDMA CRC Check Errors. 

Thank you so much once again @Faceman for the help!!! I have learnt a lot and I hope this post will help others who encounter this problem!

________________________________________________________

                                 ORIGINAL POST

_________________________________________________________
So I moved my system from an AMD 3950X to a Threadripper 2990WX with 128GB of RAM and shifted from using SATA ports to a LSI card. Booted up Unraid and made sure the drives were in the correct order and was amazed at how easy it was to migrate to another system.

I went on with my way and my guys just finshed a Satisfactory game session (hosted on the same server) when I saw my browser window flashing orange at the side, I was being thrown a lot of Notifications from the Dashboard about my drives having SMART problems and upon investigation, my drives were throwing out CRC errors because it was doing a parity check (It's 1 October where I Live now and its set to do once every month on the 1st)

I immediately stopped the parity check because it was going into the thousands. I have attached the diagnostics file. 

From what I see, most people say that it might just go up a few times then stop but mine just keeps going up.

I looked into the SMART Error logs and this was what it said (The latest error):

Error 2754 occurred at disk power-on lifetime: 47506 hours (1979 days + 10 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  84 41 e0 00 00 00 00  Error: ICRC, ABRT at LBA = 0x00000000 = 0

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 30 a8 6f 08 40 00      09:24:39.031  READ FPDMA QUEUED
  60 00 38 a8 73 08 40 00      09:24:39.022  READ FPDMA QUEUED
  60 00 28 a8 6b 08 40 00      09:24:39.020  READ FPDMA QUEUED
  60 00 20 a8 67 08 40 00      09:24:39.020  READ FPDMA QUEUED
  60 00 18 a8 63 08 40 00      09:24:39.020  READ FPDMA QUEUED

Then I went to the Main page and to my horror, I saw errors on Disk 2 and 3:


I am at wit's end on what to do. Please do provide some guidance on how I can help to resolve this! Thanks for taking time off to read this!

Edit: I plugged everything out and plugged all the cables back in again. Still the same problem.

diagnostics.zip

Edited October 1, 20241 yr by rescend
For Clarity and updated information.

was the LSI card's firmware up to date and it's definitely not overheating?

 

I have a Arctic P12 MAX 120mm fan blowing directly at the card directly since I read that both the LSI Cards and 10GbE NICs run HOT so they get a dedicated fan blowing at it.

As for the firmware, is this the correct place to download them?

As an update, this is what the BIOS of the LSI card says on boot:

 

Avago Technologies MPT SAS2 BIOS
MPT2BIOS-7.39.00.00 (2014.09.18)
Copyright 2000-2014 Avago Technologies, All Rights Reserved

I'm going to get into the boot screen so that I can see what Firmware I'm on.

I'm using MPT Firmware Revision 20.00.00.00-IT @Faceman is that considered new or old?

Edited October 1, 20241 yr by rescend
Updating as I go.

What model is the card?

There are issues with 20.00.00.00, usually fixed in 20.00.07.00

 

 

 

by the way you dont need to reboot to check firmware, you can pull it out of the terminal with

dmesg|grep -i lsi

 

23 minutes ago, Faceman said:

What model is the card?

There are issues with 20.00.00.00, usually fixed in 20.00.07.00

 

 

 

I'm sorry for sounding really dumb, how do I find out the model of the card?

Edit: Sorry, I think the model is SAS9207-8i?
 

Edited October 1, 20241 yr by rescend
Added Adapter Name?

did you buy it from somewhere or were you given it? if its got firmware 20 on it then it's probably in the 9200 series, but it's good the have the right variant (9211-8i, 9207-8i, etc etc) for a firmware flash

 

there is a way to find out, but the only way I know is with LSIutil which does require some poking around in the terminal to get it to run.

 

 

 

Yeah, its the 9207-8i if thats the model number you're looking for @Faceman

Can I just use this guy's way of updating the firmware? Except I won't be using Linux, I'll be using FreeDOS (so I have to use the provided sas2flsh.exe instead of using the Linux sas2flash) but otherwise its the same command I take it?

 

Edited October 1, 20241 yr by rescend

since its an IT to IT upgrade you should be able to use the linux executable and do it from within unraid (with the array offline of course), no need to deal with DOS or EFI.

 

 

7 minutes ago, Faceman said:

since its an IT to IT upgrade you should be able to use the linux executable and do it from within unraid (with the array offline of course), no need to deal with DOS or EFI.

 

 

@Faceman If you could show me (or point me in the right direction) on how to do this, I'd love to learn! I've been using Unraid and I'd like to learn more on how to actually work the console a little to do stuff. Thank you most kindly in advance.

well, it's pretty much as per that reddit post.

Get the executable sas2flash from here

 

get the firmware file 9207-8.bin from here (its in /Firmware/HBA_9207_8i_IT/ )

 

put them onto your boot USB, then boot up and either log in to the terminal on an attached monitor, or open the terminal from the webUI.

The unraid terminal is always root so be very careful to double check commands as you can do a lot of damage with it.

 

change directory to boot, which is the unraid usb

cd /boot 

here you can type 'ls' for a file listing to make sure the files are there if you want to check

copy the files sas2flash and 9207-8.bin from the current directory to /tmp (just a place to stick things)

cp sas2flash 9207-8.bin /tmp

 

now we go to the /tmp directory

cd /tmp

 

make sas2flash executable

chmod +x sas2flash

 

now we can use sas2flash to flash the firmware

./sas2flash -c 0 -o -f 9207-8.bin

 

That should do it, after a reboot you should be good to go assuming that known firmware bug was your issue, and i think it was.

 

I will try this out now! Thanks a lot, keep you updated on whether it works!

while you're tinkering, reseat all your sata cables and the sas connectors too. just in case.

 

4 minutes ago, Faceman said:

while you're tinkering, reseat all your sata cables and the sas connectors too. just in case.

 

Yeah, standard sanity check. I'll do just that.

Just finished flashing, into the hands of fate with a reboot!

Update: 

There are no problems so far! It previously couldn't get past 0.1% without throwing thousands of UDMA CRC Check errors! 

Thank you for everything so far @Faceman I will be updating the top of this post for people to refer to in the future.

Edited October 1, 20241 yr by rescend
Success!

just keep in mind that CRC errors already logged will stay on the disks Smart data, but you just want to see too many new ones.

One every now and then isnt too uncommon, nor is it a sign of a major issue, thats usually just a tired old cable or something simple like that.

 

 

14 minutes ago, Faceman said:

just keep in mind that CRC errors already logged will stay on the disks Smart data, but you just want to see too many new ones.

One every now and then isnt too uncommon, nor is it a sign of a major issue, thats usually just a tired old cable or something simple like that.

 

Noted, I'll keep an eye on it until then. 

Just wanted to thank @Faceman for the super-clear guidance in the solution. Came across this exact issue installing my first HBA today, this thread made for a relevant and easy solve 🙏