Intel Corporation 82599 10 Gigabit TN Network Connection - ixgbe 0000:04:00.1: Warning firmware error detected FWSM: 0x00000000

[Dal]

I added an Intel NIC to my Unraid-server, but now the cache disk (a NVMe M2 disk) is gone, and the logged is filled with messages like this:
 

ixgbe 0000:04:00.1: Warning firmware error detected FWSM: 0x00000000

 

ethtool eth2 and ethtool eth1 says:

 

root@Unraid:~# ethtool -i eth1
driver: ixgbe
version: 5.19.17-Unraid
firmware-version: 0x800001a5
expansion-rom-version: 
bus-info: 0000:04:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

 

I know there are similar posts about this, but it's unclear if it ever were fixed or not.

Is it possible to fix, or should I ditch the NIC?

 

I'm running Unraid v6.11.5

 

Thanks

[JorgeB]

Look for a firmware update for the NIC.

[Dal]

12 minutes ago, JorgeB said:

Look for a firmware update for the NIC.

.... sure.

But where? All I seem to find are drivers, not firmware

[JorgeB]

Intel support page is where it should be, if one exists.

[notdrunkjustsloppy]

Sorry for the trouble, I don't currently have much too offer in the way of solutions, will think on it. Writing to ask, TN as in 'Nooga?

YopuThouight about it, you should try earlier in-kernel drivers if possible with unraid, surely it is. This is obviously my first post. New around here. That, or an updated one, seeing as you're on a newer version from 5.19 (obviously) try these https://downloadmirror.intel.com/732258/ixgbe-5.15.2.tar.gz circa june'22

[Dal]

38 minutes ago, notdrunkjustsloppy said:

Sorry for the trouble, I don't currently have much too offer in the way of solutions, will think on it. Writing to ask, TN as in 'Nooga?

YopuThouight about it, you should try earlier in-kernel drivers if possible with unraid, surely it is. This is obviously my first post. New around here. That, or an updated one, seeing as you're on a newer version from 5.19 (obviously) try these https://downloadmirror.intel.com/732258/ixgbe-5.15.2.tar.gz circa june'22

 

Thank you for a proper answer. Can I use those drivers directly in Unraid, or must I put the card inside a machine running linux? I have put it inside a Windows Server machine now, looking for drivers for that one as well

[notdrunkjustsloppy]

Sorry bud, I just saw ythat you had replied. So, aafter some looking

just came across this -

Linux kernel

Linux 5.15.30-Unraid

oot: md/unraid: Upgraded to version 2.9.22 (revert setting min sector size to 4096)

oot: Added Intel ixgbe: version 5.14.6 so I am going to assume they likely still are on that but it'd be easy enough to check

 

from 6.10.rc4 changelog. That is dated March 21st, 2022.

I can't find where they added a later release in, could have missed it.

 

I think I / we may be barking up the wrong tree here though. Seems like this may be a PCIe lane issue. So, you added the network card, and then your m.2 slot went out? I don't think this is actually a firmware issue, per se. but a lane/motherboard/cpu issue.

 

Look at your error output, it is reporting for 0000:04:00.1  Now look at your eth tool -i references, they do not report the same  info. I'm not a linux wiz, but I find this interesting and it's where I would start looking, which brought me to my own results (haven't had a chance to change defaults yet) - I'm guessing you just happened to grab that port as a copy and paste?

```

root@Tower:~# ethtool -i eth2 <-------
driver: igb
version: 5.19.17-Unraid
firmware-version: 1.63, 0x80000ae6
expansion-rom-version:
bus-info: 0000:04:00.1 <-------------
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
root@Tower:~# ethtool -i eth1 <-------------
driver: igb
version: 5.19.17-Unraid
firmware-version: 1.63, 0x80000ae6
expansion-rom-version:
bus-info: 0000:04:00.0 <-----------
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
root@Tower:~#```

 

Do you have a diagnostics zip by chance?

What does ' lspci | grep X5 ' (Or grep i) say? 

 

Here are my results as an example:

04:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
04:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
04:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

 

So, a few questions, is it only reporting on 0000:04:00.*? Or is that just one you happened to grab?

Havew you tried / are you able to try taking out the NIC, and putting it in a different slot?

Do you have your motherboard manuual, or can you name your motherboard so we can check the lane table?

Have you checked here: https://bugzilla.kernel.org/show_bug.cgi?id=207555 ?

Is your card of Dell origin? Who's name is slapped up there other than Intel? Sun Microsystems? HP? etc.

And also, take a look here, as this seems like it is interfering with your PCIe lanes, as the m.2 and the GPU and the, anything pcie all use lane resources. Perhaps you simply do not have enough lanes (a) or (b) need to properly bifurcate them (google brings up a pretty good explanation, but this can be and be done in the bios but is typically done automatically, although can be done manually on some boards)  which is why I say may not be the problem that it seems to be. :

 

"Is it possible to fix, or should I ditch the NIC?" No, no, no, no , no, no, no, no --- We don't give up that easily, my friend! Every once in a while if you bash something so hard with a hammer, it can turn out to uncover some issues that don't actually pertain to the damage caused from the hammer, just keep at it! if you don't have an extra pcie slot, but have on board graphics, try taking out your graphics card and, as a good measure in regards to bifurcation, put the card in that slot. Boot, see what happens. Any error messages?

[Dal]

Damn, you're thorough

Makes me think you read logs for a living.

 

And I think you're right; Tested some more an the m2 disk does not even show up in BIOS when the NIC is attached.

So yes, this is probably NOT a driver / firmware problem.

 

I seem to vaguely remember that some lanes were disabled if all SATA ports where enabled or something like that.

Going to read the motherboard manual now.

 

Thanks so far.

[Dal]

I think the answer was in the Motherboard manual

Some highlights:

 

2x PCIe 3.0 x16 slots (supports x16/x4 mode)*
4x PCIe 3.0 x1 slots**
* The PCI_E4 slot will be unavailable when an M.2 SSD module has been 
   installed in the M.2_2 slot.

 

And according to this picture,

 

 

That is exactly where I have the NIC (PCI_E4) and the M2 disk (M2_2)

 

So in theory, if I move the M2 disk to the M2_1 slot, everything should work fine.

Trying that tomorrow. To be continued

 

Thanks for helping out, and I guess the moral is: If all else fails, RTFM

[Dal]

Hello again.

As suspected, when movin the M2 disk to the other slot, the NIC seems to work also

But the log still fills up with these messages:

 

Unraid kernel: ixgbe 0000:04:00.1: Warning firmware error detected FWSM: 0x00000000

 

So unless it's fixable or the logs supressable, it's still a no go really.

 

Thanks

[Dal]

Can I download and install drivers myself in this OS?

I downloaded the latest (5.18.6)  from Intel to tmp diretory using wget, but don't really know where to go from there

[itimpi]

1 hour ago, Dal said:

Can I download and install drivers myself in this OS?

I downloaded the latest (5.18.6)  from Intel to tmp diretory using wget, but don't really know where to go from there

No.   Drivers are not easily installed by end-users under Unraid.

[JorgeB]

Still looks to me more like a firmware problem, I would contact Intel support and ask if a firmware update is available for that NIC.

[Dal]

Yeah, Intel is impossible to reach
And I think this card came from a Dell server. And I found new firmware on the Dell pages, but all the packages are not compatible with Unraid it seems

There are packages for Redhat, SUSE and Ubuntu, and all gives me the same error:
 

This Update Package is not compatible with your system configuration.

 

Must try to move the card to another server I guess

[Dal]

Tried in a Windows 2022 server, but no go there either.

Error:

Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Server Adapter   64.04(40.04   151C 00:002 Update not
    X520-T2                                                      available


Tool execution completed with the following status: Device not found.
Press any key to exit.

 

The card seems to work fine, though. So I don't understand what is wrong.

Time to give up

[Original_Vecna]

On 12/12/2022 at 8:56 AM, Dal said:

Tried in a Windows 2022 server, but no go there either.

Error:

Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Server Adapter   64.04(40.04   151C 00:002 Update not
    X520-T2                                                      available


Tool execution completed with the following status: Device not found.
Press any key to exit.

 

The card seems to work fine, though. So I don't understand what is wrong.

Time to give up

We're you ever able to figure this out? Having the same issue.