August 28, 20241 yr Hi, I am having this issue where once in a while the network to my Unraid just drops and stops working until I unplug and replug the ethernet cable. No restart needed, just a simple unplug replug of the Ethernet cable fixes it. I have an instance of Uptime Kuma running on a friend's server off-site monitoring various of my services, and it's giving me a pretty reliable picture of when/how the server is going down This seems to have started ever since I got Ubiquiti's UDM SE. I did not have that problem for the past year using my ISP's router. Details/oddities I have noticed: - I'm on the latest stable version of Unraid. - My server is connected to a Ubiquiti UDM SE through an SFP to Ethernet module. There doesn't seem to be any logs or any evidence of this issue on the router's side. - My NIC is: Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05) - I do NOT have the custom driver package from the Unraid Apps Store, never had issues without it. - Seems to be unrelated to DHCP renewals, the renewal happened at 4:30am, way before the server went down. - The server is still working, physical access to the console still works - The server still shows up as connected on my UDM's UI. It only sees a disconnect once I unplug/replug to fix the outage. - My docker is set to ipvlan. this isn't a macvlan issue. None of my containers use custom networks anyways. All on the default bridge or host mode. - I am using a bonded + bridged setup even if I only have one ethernet port. I just have it like this because it's the default in Unraid. - The logs do not show ANYTHING about the NIC going down or anything of that sort. The problem is completely "silent". Here is a snippet of the system logs around the time the server went down (it went down at exactly 6:00am). As you can see nothing happened Aug 28 05:42:18 Endurance kernel: mdcmd (53): set md_write_method 0 Aug 28 05:42:18 Endurance kernel: Aug 28 06:00:01 Endurance root: Starting Mover Aug 28 06:00:01 Endurance root: ionice -c 2 -n 7 nice -n 5 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start 0 0 0 "/mnt/user/cache-only/mover-ignore/global.txt" '' '' '' '' '' '' '' 0 Aug 28 06:01:23 Endurance emhttpd: spinning down /dev/sdf Aug 28 07:00:01 Endurance root: Starting Mover - Everything web related (meaning my plex, all my websites hosted on my reverse-proxy, the Unraid's web-ui, etc.) goes down, both for LAN and for WAN. But for some reason my Minecraft server doesn't go down, it still works. (could it be differences in UDP vs TCP? I'm really at a loss for ideas here lol) - While the problem is still present, SSH does NOT work, but I see logs of my computer trying to reach the server but failing authentication. So it would seem there's still some communication happening, but only one-way? The letsencrypt/certificate error you see is probably from me TRYING TO reach the Unraid WebUI (which doesn't load) roughly at the same time as my SSH attempt Aug 28 08:00:01 Endurance root: Starting Mover Aug 28 08:00:01 Endurance root: ionice -c 2 -n 7 nice -n 5 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start 0 0 0 "/mnt/user/cache-only/mover-ignore/global.txt" '' '' '' '' '' '' '' 0 Aug 28 08:58:15 Endurance emhttpd: spinning down /dev/sde Aug 28 09:00:01 Endurance root: Starting Mover Aug 28 09:00:01 Endurance root: ionice -c 2 -n 7 nice -n 5 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start 0 0 0 "/mnt/user/cache-only/mover-ignore/global.txt" '' '' '' '' '' '' '' 0 Aug 28 09:05:53 Endurance sshd-session[15967]: Connection from 192.168.2.10 port 64652 on 192.168.2.200 port 22 rdomain "" Aug 28 09:05:57 Endurance nginx: 2024/08/28 09:05:57 [error] 14180#14180: OCSP responder sent invalid "Content-Type" header: "text/html" while requesting certificate status, responder: r10.o.lencr.org, peer: 184.150.39.43:80, certificate: "/boot/config/ssl/certs/certificate_bundle.pem" Aug 28 09:06:03 Endurance sshd-session[15967]: Connection reset by 192.168.2.10 port 64652 [preauth] Aug 28 09:06:03 Endurance sshd[9379]: srclimit_penalise: ipv4: new 192.168.2.10/32 deferred penalty of 1 seconds for penalty: connections without attempting authentication - Here are the logs of when I unplug/replug the ethernet cable and the outage gets fixed. Note that this "Link is down" never ever showed up when the outage started. This is just from me physically unplugging and replugging the cable. Aug 28 09:06:14 Endurance kernel: r8169 0000:06:00.0 eth0: Link is Down Aug 28 09:06:14 Endurance kernel: bond0: (slave eth0): link status definitely down, disabling slave Aug 28 09:06:14 Endurance kernel: device eth0 left promiscuous mode Aug 28 09:06:14 Endurance kernel: bond0: now running without any active interface! Aug 28 09:06:14 Endurance kernel: br0: port 1(bond0) entered disabled state Aug 28 09:06:23 Endurance kernel: r8169 0000:06:00.0 eth0: Link is Up - 2.5Gbps/Full - flow control off Aug 28 09:06:23 Endurance kernel: bond0: (slave eth0): link status definitely up, 2500 Mbps full duplex Aug 28 09:06:23 Endurance kernel: bond0: (slave eth0): making interface the new active one Aug 28 09:06:23 Endurance kernel: device eth0 entered promiscuous mode Aug 28 09:06:23 Endurance kernel: bond0: active interface up! Aug 28 09:06:23 Endurance kernel: br0: port 1(bond0) entered blocking state Aug 28 09:06:23 Endurance kernel: br0: port 1(bond0) entered forwarding state I am at a loss for ideas on how to fix this. Let me know if anyone else experienced this.
August 28, 20241 yr 1 hour ago, hotdogdirt said: through an SFP to Ethernet module. This is common problem, at ethernet side was 2.5g, does UDMSE sfp+ side in 2.5g or 10g ? FYR, best if both end was 2.5g. Edited August 28, 20241 yr by Vr2Io
August 28, 20241 yr Author Just now, Vr2Io said: This is common problem, at ethernet side was 2.5g, does sfp+ side in 2.5g or 10g. FYR, best was both in 2.5g. The SFP port on the UDM technically only supports 1G and 10G, so those are the only options I have on the router. but from my understanding, this wasn't really a problem since the SFP module itself could negociate 1/2.5/5/10. And I know that Unraid negotiates successfully in 2.5g root@Endurance:~# ethtool eth0 Settings for eth0: Supported ports: [ TP MII ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full 2500baseT/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: Yes Supported FEC modes: Not reported Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full 2500baseT/Full Advertised pause frame use: Symmetric Receive-only Advertised auto-negotiation: Yes Advertised FEC modes: Not reported Link partner advertised link modes: 10baseT/Full 100baseT/Full 1000baseT/Full 10000baseT/Full 2500baseT/Full 5000baseT/Full Link partner advertised pause frame use: No Link partner advertised auto-negotiation: Yes Link partner advertised FEC modes: Not reported Speed: 2500Mb/s Duplex: Full Auto-negotiation: on master-slave cfg: preferred slave master-slave status: slave Port: Twisted Pair PHYAD: 0 Transceiver: external MDI-X: Unknown Supports Wake-on: pumbg Wake-on: d Link detected: yes
August 28, 20241 yr 8 minutes ago, hotdogdirt said: The SFP port on the UDM technically only supports 1G and 10G, so those are the only options I have on the router. but from my understanding, this wasn't really a problem since the SFP module itself could negociate 1/2.5/5/10. And I know that Unraid negotiates successfully in 2.5g I am not sure UDMSE support 2.5g or not, but my us-16-xg SFP+ support 1, 2.5, 10g only. I confirm auto negotiation usually not a good setting for those converter. Pls note you also claim you have unstable issue. Edited August 28, 20241 yr by Vr2Io
August 28, 20241 yr Author 9 minutes ago, Vr2Io said: I am not sure UDMSE support 2.5g or not, but my us-16-xg SFP+ support 1, 2.5, 10g only. I confirm auto negotiation usually not a good setting for those converter. Pls note you also claim you have unstable issue. I'll plug in my Unraid server to a real 1G Ethernet port on my router for now and see if the issue happens again. It's just very hard to troubleshoot since it only shows up every couple of days/every week.
August 28, 20241 yr 5 minutes ago, hotdogdirt said: I'll plug in my Unraid server to a real 1G Ethernet port on my router That's fine too.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.