eth0 does't exist error on integrated Intel 82579LM with 6.9.2

[quattro]

I've been running 6.9.2 for a while with no issues.  Recently after a reboot, I started getting 169.254 IP addresses with my DHCP configuration.  I tried setting up a static IP and that didn't help, there still was no network connectivity.

 

I cleaned up my network.cfg to a minimal list of settings and began getting "eth0 doesn't exist" errors.

 

I created a clean usb to boot from, and got the same error, so seem to have ruled out any usb issues (it's plugged into a usb2 port and named unraid).

 

I then found the tg3 thread.  I created the blank tg3.conf on the clean usb and still no network.

 

I then went into my bios and disabled VT-d, put in the original licensed USB and it booted with an IP address from DHCP and was up and running.  I just rebooted and now I'm getting the "eth0 doesn't exist" errors again.  I have tried enabling and disabling VT-d and placing a blank tg3.conf on the USB with no change.

 

Thanks in advanced for any assistance getting to the bottom of this pesky unraid issue.

[quattro]

to clarify, it's an integrated intel 82579LM on a Dell motherboard.

 

Also, when I put in a Windows 10 SSD, the system boots up fine live on the network.  

Edited July 3, 2022 by quattro

[trurl]

attach diagnostics to your NEXT post in this thread

[quattro]

Thanks for the quick response, greatly appreciated.  I've attached my diagnostics log summary.

tower-diagnostics-20220703-1424.zip

[trurl]

There is no ethtool result in your diagnostics. Are you sure you don't have a cable, port, switch, router issue?

[quattro]

I've swapped cables, booted into windows with network with the exact configuration, and tried both dhcp and setting a static IP. 

I really don't think there is a network issue, but I'm certainly open to the possibility I've missed something.

 

Is there anything else that would prevent an ethtool result?

[quattro]

I logged in and typed ethtool eth0 at the console and got this:

 

netlink error: no device matches name (offset 24)

[quattro]

ifconfig shows loopback and wg0 for wireguard

[JorgeB]

Jul  3 14:23:37 Tower kernel: e1000e: probe of 0000:00:19.0 failed with error -3

 

NIC is failing to initialize, try updating to v6.10.3 to see if there's any difference.

[quattro]

Thank you very much for taking the time to look at my diagnostics and logs.  I greatly appreciate it, given how frustrating the last month or so has been with unraid.

 

Any idea why would it suddenly fail to initialize after working for months?  And why would it suddenly work again after disabling virtualization in the bios.  And then fail again after a simple reboot? 

 

I had already created the blank tg3.conf file on the usb using windows.  But I found a thread mentioning creating the file at the unraid console and suddenly it boots again.  All I did was enter this command at the unraid console after logging in:

 

touch /boot/config/modprobe.d/tg3.conf

 

To be honest, I'm afraid to do anything now that it's running with no explanation of why it has been down most of the time for nearly a month.

Why would 6.9.2 be doing this?  Isn't the tg3 thread all about issues in 6.10?

 

is the Intel 82579LM one of the NICs with corruption issues.  Around the time the network issue began I was getting weird parity errors.

 

 

 

Edited July 5, 2022 by quattro

[JorgeB]

33 minutes ago, quattro said:

Any idea why would it suddenly fail to initialize after working for months?  And why would it suddenly work again after disabling virtualization in the bios.  And then fail again after a simple reboot? 

That suggests a NIC problem.

 

33 minutes ago, quattro said:

I had already created the blank tg3.conf

You don't need to do that for v6.10.3, also your NIC wasn't affected even when you needed to use that.

 

34 minutes ago, quattro said:

is the Intel 82579LM one of the NICs with corruption issues. 

No, and those are fixed in v6.10.3, there could be some issue with your hardware though.

[quattro]

46 minutes ago, JorgeB said:

That suggests a NIC problem.

 

You don't need to do that for v6.10.3, also your NIC wasn't affected even when you needed to use that.

 

No, and those are fixed in v6.10.3, there could be some issue with your hardware though.

 

I thoroughly ruled out all physical NIC issues, it's perfectly fine in windows.   It's been working for 24 hours so far after the touch command.

 

I'm running 6.9.2 and I'm not upgrading to 6.10 until the NIC issue is clearly documented.

 

I booted about 15 times and got eth0 does not exist every time.  I then logged into unraid and ran the touch command and booted with a successful DHCP IP assignment.

 

 

Edited July 5, 2022 by quattro

[JorgeB]

28 minutes ago, quattro said:

I'm running 6.9.2 and I'm not upgrading to 6.10 until the NIC issue is clearly documented.

It's up to you, but the data corruption issue is not NIC related, and even when we suspected it was it was with Broadcom NICs, not Intel.

[quattro]

46 minutes ago, JorgeB said:

It's up to you, but the data corruption issue is not NIC related, and even when we suspected it was it was with Broadcom NICs, not Intel.

I was referring more to NICs being disabled, then the corruption issue that triggered the code change.

 

Another question, if I have a NIC problem(I'm assuming you meant hardware), are you thinking there is something in 6.10 that might fix it?

 

If I don't have a hardware issue with my NIC, what do you think happened?

 

Edited July 5, 2022 by quattro

[JorgeB]

3 minutes ago, quattro said:

I was referring more to NICs being disabled

Again, it only affects Broadcom NIC, and it's resolved on v6.10.3, no NICs are disabled there.

 

4 minutes ago, quattro said:

Another question, if I have a NIC problem(I'm assuming you meant hardware), are you thinking there is something in 6.10 that might fix it?

If it's really hardware it won't fix it, but since the driver is likely newer it might help if it's not.

[quattro]

4 minutes ago, JorgeB said:

Again, it only affects Broadcom NIC, and it's resolved on v6.10.3, no NICs are disabled there.

 

If it's really hardware it won't fix it, but since the driver is likely newer it might help if it's not.

 

Cool, I appreciate your time. It never really felt like a driver issue, but who knows.

 

I finally got the gust to reboot and still have network.  So I'm still standing by my assessment that it's not hardware, but I've been troubleshooting for far too many years to be foolish enough to think that a hardware issue in this scenario is impossible.

 

I'll have time for more testing in a few days, and I'll update here in case it helps anyone else.

[quattro]

So I got the guts to reboot today, no problem.  The blank tg3.conf seems to have fixed the issue that has been plaguing me for weeks. 

 

I'm feeling gutsy, so will attempt an update to 6.10.3 😮

Edited July 9, 2022 by quattro

[quattro]

On 7/5/2022 at 2:13 AM, JorgeB said:

Again, it only affects Broadcom NIC, and it's resolved on v6.10.3, no NICs are disabled there.

 

Sure seems to have been the culprit with my Intel NIC.  NOTHING else would fix it after weeks of trying.  

[JorgeB]

8 hours ago, quattro said:

The blank tg3.conf seems to have fixed the issue that has been plaguing me for weeks. 

As already mentioned that won't affect an Intel NIC, it can't help or hurt your issue.

[quattro]

10 hours ago, JorgeB said:

As already mentioned that won't affect an Intel NIC, it can't help or hurt your issue.

So, if I go into the bios and enable Vt-d, it won't disable my NIC?

 

What if you're wrong?

Edited July 10, 2022 by quattro

[jmztaylor]

1 hour ago, quattro said:

So, if I go into the bios and enable Vt-d, it won't disable my NIC?

 

What if you're wrong?

 

He is one of the few going around helping most of the members here on his free time.  I think he knows what he is talking about.  But lets entertain.  tg3 is a driver.  Which has been said in this thread multiple times wouldn't affect you no matter what.

From your diags.
 

00:19.0 Ethernet controller [0200]: Intel Corporation 82579LM Gigabit Network Connection (Lewisville) [8086:1502] (rev 04)
	DeviceName:  Onboard LAN
	Subsystem: Dell 82579LM Gigabit Network Connection (Lewisville) [1028:052c]
	Kernel modules: e1000e

 

The driver being loaded for your NIC is e1000e not tg3.  And nowhere is the tg3 driver being loaded for you.  So it was shear coincidence that adding the blank tg3 worked for you.  Also, 6.9.2 wasn't affected by that bug anyways.  So again.  tg3 is a moot point. 

[quattro]

Except that it's not a coincidence.  I can reproduce it every time.  If I enable virtualization I get the eth0 doesn't exist error.  If I disable virtualization it doesn't work until I create the blank tg3.conf.   I've done it 6 times just know.  Multiple reboots after disabling Vt-d won't fix the eth0 does not exist error.  The second I touch the tg3.conf it works the next boot.

 

I'm fully aware of JorgeB's efforts, you can refer to my post thanking him for taking the time to look at my issue.  My forum account is new, but this is not my first day at the rodeo.  I'm also aware of the driver used.  However the blank tg3.conf was a workaround specifically for the "eth0 doesn't exist" issue that was caused by Unraid disabling the NIC on those system that were documented.  I'm aware my config hasn't been tested or documented, but that doesn't mean it's impossible that it's affected by the Vt-d issue.

 

I'm primarily posting in detail to help others who might come across the same problem as it completely disables unraid.   The Dell 7010/9010 platform is insanely popular for projects like unraid that are designed to utilize old hardware.  The posts that others made helped me find a fix for my system, which was completely disabled due to this bug.

 

Unraid is disabling my NIC, I'm certainlly curious as to why it's happening.  Remeber, win10 boots fine all day long with Vt-d enabled.  

 

 

[JorgeB]

11 hours ago, quattro said:

If I disable virtualization it doesn't work until I create the blank tg3.conf. 

I don't see how that is possible, let me explain what creating a blank tg3.conf does:

 

-with Unraid v6.10.2, and only with that release, if VT-d is enabled and there is a NIC that used the tg3 driver that driver won't be loaded Unraid unless a blank tg3.conf file exists.

-with any other Unraid release, including v6.10.3 that code doesn't exist, i.e., it won't make any difference having that file or not, even if you have a NIC that uses that driver.

 

If your NIC isn't loading with VT-d enable it's a different problem, please post diags after booting witn VT-d enabled and VT-d disabled.

 

 

[quattro]

On 7/11/2022 at 1:51 AM, JorgeB said:

I don't see how that is possible, let me explain what creating a blank tg3.conf does:

 

-with Unraid v6.10.2, and only with that release, if VT-d is enabled and there is a NIC that used the tg3 driver that driver won't be loaded Unraid unless a blank tg3.conf file exists.

-with any other Unraid release, including v6.10.3 that code doesn't exist, i.e., it won't make any difference having that file or not, even if you have a NIC that uses that driver.

 

If your NIC isn't loading with VT-d enable it's a different problem, please post diags after booting witn VT-d enabled and VT-d disabled.

 

 

Thanks for continuing to help with this issue.  I'm sure you can imagine it's been frustrating, so my apologies if I at all seemed unappreciative.

 

I always understood it was a different driver, but thought perhaps the workaround might not be driver specific since it's just a blank file (even though it's the name of the driver config file).  I guess I kinda figured the blank tg3.conf was just turning off iommu passthrough or something similar.  Thanks for clarifying exactly how the fix behaves, even though it's been kinda obvious that's what it was doing, it was the only thing I tried that worked.    Imagine getting to a thread with some fixes posted and those suddenly fix your issue that has kept a system down for weeks.  Prior to that, I tried EVERYTHING under the sun to troubleshoot it.

 

Like I said, clearly those changes are(seem to be) affecting my system, and it's too regular to give up and conclude it's just a coincidence that my machine boots when I make those changes.  

 

I will troubleshoot some more this weekend and get some diagnostics.   Thanks again for your assistance. 

 

 

Edited July 12, 2022 by quattro

[SenorPantalones]

I have the same onboard NIC and I have a similar problem although maybe it's different enough to justify another thread. Every time I upgrade to 6.10.x (I've tried all 3 releases) I get the "eth0 not found" message and the "e1000e: probe of 0000:00:19.0 failed with error -3" message in the syslog. Reverting to 6.9.2 fixes it. Messing with virtualization or the tg3 file doesn't have a lasting effect. I have had a handful of times it suddenly worked only to break again on a simple reboot (without making any more changes). The only way I've been able to fix it consistently/permanently has been to downgrade to 6.9.2.