I've been fighting this for a couple of days now, thinking it was a bad fiber cable. I just put in a ConnectX-2, then left the ethernet in, booted up, and it was in bond. I then unplugged the ethernet and lost WebUI access as well as all my Docker containers. The weird thing is that Unraid Connect shows it’s online and is able to poll Docker containers running, disk usage, and everything. This happens on reboots too—Unraid Connect shows it's offline, then when it boots, it goes online.

I went into flash config, deleted the network config, and rebooted with the ethernet unplugged. It gets an IP address according to my DHCP logs. My firewall is OPNsense, and that interface shows only 2 errors, but the receive and transmit packet count goes up without the error count rising. The interfaces are in a bridge for my LAN VLAN (I even undid all VLANs to rule that out). I then replaced the fiber cable because, when I looked at the cable, I only saw one side light up (I know, bad idea—it was a quarter of a second, I'm fine). The new fiber cable is from a reputable brand (Tripp Lite). This is my first fiber setup, so I am out of ideas.

I also set the listening interface to eth1, which is the ConnectX-2. I also reassigned eth0 to the card (all of this with ethernet plugged in). I can't get it running. I have attached my diagnostic logs. I am pretty sure these were made when the ethernet was also plugged in but can also do it from the console with only the ConnectX-2 if needed. Let me know, as I am tearing my hair out.

tower-diagnostics-20240515-2038.zip

Edited May 18, 20242 yr by Lucid_Enemy
accidently left my localAI request text in there lol

2 hours ago, Lucid_Enemy said:

The weird thing is that Unraid Connect shows it’s online

Does still on-line even 2.5g port plug-out ? Your switch is manage 10G switch ?

 

To simplify troubleshoot, disable bonding, put ConnecX-2 be eth0, disconnect eth1 ( also don't setting eth0,eth1 in same subnet), disable array auto-start until you solve the problem.

 

Also temporary setting "use SSL/TLS"  to "no".

 

Edited May 18, 20242 yr by Vr2Io

It still shows online when the 2.5G port was unplugged (this reset not anymore, but that's probably because bridging was disabled). The 10G port is straight out of my firewall's bridge as I don't have a 10G switch yet (ix1 bridged to igc1, and LAN is the bridge - showing a picture of that).

I got as far as disabling bonding/bridging, put ConnectX-2 as eth0 with the 2.5G port unplugged, and the system came up (enabled EFI because I wanted to see if PCI bifurcation could help, but I can't enable it apparently). So, I wasn't able to set SSL to off and auto-start, but I figure that's just to speed things up for this diag, as I also tried with a ping. I did just reset network.cfg after running diagnostics from the shell. I'm also going to try to disable autotweaks TCP stuff as I saw a bunch of it doing things on boot. However, when I did a lot of the diag the other day, I did boot it into safe mode with no plugins to rule that out, and that didn't help.

tower-diagnostics-20240517-2342.zip

 

2 hours ago, Lucid_Enemy said:

The 10G port is straight out of my firewall's bridge as I don't have a 10G switch yet

 

igc means 2.5G and ix1 is 10G ? But why ix1 was linkdown ?

 

Does mlx4 00:02:c9:56:8c:1e show in router ?

 

# PCI device 0x15b3:0x6750 (mlx4_core)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:02:c9:56:8c:1e", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

 

If you can access Unraid locally and make test on 10g have pass some network test, i.e. ping other client and router, then problem likely opensense relate.

Edited May 18, 20242 yr by Vr2Io

I am an idiot thank you for helping me narrow it down to the firewall. I learned today FreeBSD handles the bridge members thru the packet filter rather then the bridging interface went into tunable and changed net.link.bridge.pfil_member to 0 (off) and net.link.bridge.pfil_bridge (on) which changes the member iterfaces of the packet filter to the full on packet interface.