10GB NIC showing in devices, not on networking page

[mihcox]

I have installed 2 10gb nics, both in different unraid servers, to try and directly connect them. My older box shows the new nics in the network panel, but the precision3450 does something different. It show the devices in system devices:

But they do not show up on the network page as expected:

 

Working computer same views:

 

[mihcox]

Bumping this. I noticed that the server with the issue does not have a network-rules.cfg file generated while the server that is working does.

 

Any suggestions on how to fix this?

[mihcox]

attaching diagnosticsprecision3450-diagnostics-20240118-2301.zip

[JorgeB]

Both NICs are failing to initialize, so the driver is not loaded, but don't see why:

 

Jan 18 22:59:50 Precision3450 kernel: ixgbe: probe of 0000:01:00.0 failed with error -5
Jan 18 22:59:50 Precision3450 kernel: ixgbe: probe of 0000:01:00.1 failed with error -5

 

[ADvorsky]

I had a similar issue including the vanishing network-rules.cfg after each restart...
(btw, for this everybody replying to a previous thread of mine insists i have something installed that removes it, which is actually not the case. i still think that its unraid itself removing the file due to reasons during startup after reading it.)
The machine has 2 NICs, the onboard (which whyever cannot be disabled in the bios) and the 10GBit Nic

Here is my network-rules.cfg:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="e4:1d:9c:92:ce:44", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="22:48:5c:05:16:45", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

It is merely to make sure the 10GBit is eth0 because Unraid has some special handling for eth0 as we all know...

As well, I made sure its there before restarting the server (user script just writes it to /boot/config each hour )

hope this helps you as well

Edited January 19 by ADvorsky

[JorgeB]

In this case I suspect network-rules.cfg is not being created because the other NICs are not loading a driver, so basically Unraid only sees one NIC.

[mihcox]

7 hours ago, JorgeB said:

Both NICs are failing to initialize, so the driver is not loaded, but don't see why:

 

Jan 18 22:59:50 Precision3450 kernel: ixgbe: probe of 0000:01:00.0 failed with error -5
Jan 18 22:59:50 Precision3450 kernel: ixgbe: probe of 0000:01:00.1 failed with error -5

 

Any suggestion on how to get them to initialize? I have the same 10gb nic working in another unraid server without issue

[ADvorsky]

Try my network-rules.cfg with your mac addresses... If they dont come up, its something different...

Worth a try though.

[JackieWu]

7 minutes ago, mihcox said:

Any suggestion on how to get them to initialize? I have the same 10gb nic working in another unraid server without issue

 

Maybe switch 10G NIC PCIE slot to another. 

[mihcox]

18 hours ago, ADvorsky said:

I had a similar issue including the vanishing network-rules.cfg after each restart...
(btw, for this everybody replying to a previous thread of mine insists i have something installed that removes it, which is actually not the case. i still think that its unraid itself removing the file due to reasons during startup after reading it.)
The machine has 2 NICs, the onboard (which whyever cannot be disabled in the bios) and the 10GBit Nic

Here is my network-rules.cfg:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="e4:1d:9c:92:ce:44", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="22:48:5c:05:16:45", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

It is merely to make sure the 10GBit is eth0 because Unraid has some special handling for eth0 as we all know...

As well, I made sure its there before restarting the server (user script just writes it to /boot/config each hour )

hope this helps you as well

This did not help, after formatting it like the below, it was deleted after rebooting.

 

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="b0:4f:13:11:06:70", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="90:e2:ba:ea:71:18", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="90:e2:ba:ea:71:19", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

[mihcox]

18 hours ago, JorgeB said:

In this case I suspect network-rules.cfg is not being created because the other NICs are not loading a driver, so basically Unraid only sees one NIC.

yes, thats probably why the one i was creating was getting deleted.

 

13 hours ago, JackieWu said:

 

Maybe switch 10G NIC PCIE slot to another. 

 Tried in both slots, x16 and x4 with the same result. Seen in the bios, and in devices, but not on the network tab.

21 hours ago, JorgeB said:

Both NICs are failing to initialize, so the driver is not loaded, but don't see why:

 

Jan 18 22:59:50 Precision3450 kernel: ixgbe: probe of 0000:01:00.0 failed with error -5
Jan 18 22:59:50 Precision3450 kernel: ixgbe: probe of 0000:01:00.1 failed with error -5

This is unfortunately not very helpful

[mihcox]

Just installed windows on the same machine, and had no issues adding the drivers for the x520. It seems like it isnt a hardware issue, but is unraid specific as it works with windows.

[JorgeB]

Could be a kernel in-tree driver problem, you can try booting a different Linux distro, ideally with a similar/same kernel.

[ADvorsky]

9 hours ago, mihcox said:

This did not help, after formatting it like the below, it was deleted after rebooting.

Yes, it vanishes on my system as well after each reboot, but the nics are present as configured in the file. Without the file present during boot the 10gb nic is not available.

That's why i have a copy of that file to recreating it after each reboot on array start

[mihcox]

8 hours ago, JorgeB said:

Could be a kernel in-tree driver problem, you can try booting a different Linux distro, ideally with a similar/same kernel.

Same issue in ubuntu 20.04.3, what does that indicate?

 

4 hours ago, ADvorsky said:

Yes, it vanishes on my system as well after each reboot, but the nics are present as configured in the file. Without the file present during boot the 10gb nic is not available.

That's why i have a copy of that file to recreating it after each reboot on array start

I tried that with mine, but when the server restarts it deletes the file. How are you generating it?

 

[mihcox]

ubuntu 20.04.3

[JorgeB]

14 hours ago, mihcox said:

Same issue in ubuntu 20.04.3, what does that indicate?

It does confirm it's not an Unraid problem, could be an issue with the Linux in-tree driver, or a problem that the Windows driver ignores and Linux one doesn't, either way I'm afraid I don't have a good solution, other then maybe trying a different NIC if possible.

[mihcox]

10 hours ago, JorgeB said:

It does confirm it's not an Unraid problem, could be an issue with the Linux in-tree driver, or a problem that the Windows driver ignores and Linux one doesn't, either way I'm afraid I don't have a good solution, other then maybe trying a different NIC if possible.

Yes, so the difference is in the machines where it is working, i have an I217-LM intel NIC as the main connection, but in the ones where its failing its an I219-LM NIC. This means it is some sort of kernel/driver issue i guess?

 

I had purchased an x520-DA2 because it was seemed to work easily. Is there another suggestion for cheap 10GB NICs, with 1 and 2 ports?

[JorgeB]

Personally I like the Mellanox Connect-X NICs, Connect-X 2 and 3 models can be found pretty cheap on eBay, and they usually work great with Unraid.

[mihcox]

Thank you, ill look into purchasing a few of these