Hi,
I am having this issue where once in a while the network to my Unraid just drops and stops working until I unplug and replug the ethernet cable. No restart needed, just a simple unplug replug of the Ethernet cable fixes it.
I have an instance of Uptime Kuma running on a friend's server off-site monitoring various of my services, and it's giving me a pretty reliable picture of when/how the server is going down
This seems to have started ever since I got Ubiquiti's UDM SE. I did not have that problem for the past year using my ISP's router.
Details/oddities I have noticed:
- I'm on the latest stable version of Unraid.
- My server is connected to a Ubiquiti UDM SE through an SFP to Ethernet module. There doesn't seem to be any logs or any evidence of this issue on the router's side.
- My NIC is: Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
- I do NOT have the custom driver package from the Unraid Apps Store, never had issues without it.
- Seems to be unrelated to DHCP renewals, the renewal happened at 4:30am, way before the server went down.
- The server is still working, physical access to the console still works
- The server still shows up as connected on my UDM's UI. It only sees a disconnect once I unplug/replug to fix the outage.
- My docker is set to ipvlan. this isn't a macvlan issue. None of my containers use custom networks anyways. All on the default bridge or host mode.
- I am using a bonded + bridged setup even if I only have one ethernet port. I just have it like this because it's the default in Unraid.
- The logs do not show ANYTHING about the NIC going down or anything of that sort. The problem is completely "silent". Here is a snippet of the system logs around the time the server went down (it went down at exactly 6:00am). As you can see nothing happened
Aug 28 05:42:18 Endurance kernel: mdcmd (53): set md_write_method 0
Aug 28 05:42:18 Endurance kernel:
Aug 28 06:00:01 Endurance root: Starting Mover
Aug 28 06:00:01 Endurance root: ionice -c 2 -n 7 nice -n 5 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start 0 0 0 "/mnt/user/cache-only/mover-ignore/global.txt" '' '' '' '' '' '' '' 0
Aug 28 06:01:23 Endurance emhttpd: spinning down /dev/sdf
Aug 28 07:00:01 Endurance root: Starting Mover
- Everything web related (meaning my plex, all my websites hosted on my reverse-proxy, the Unraid's web-ui, etc.) goes down, both for LAN and for WAN. But for some reason my Minecraft server doesn't go down, it still works. (could it be differences in UDP vs TCP? I'm really at a loss for ideas here lol)
- While the problem is still present, SSH does NOT work, but I see logs of my computer trying to reach the server but failing authentication. So it would seem there's still some communication happening, but only one-way? The letsencrypt/certificate error you see is probably from me TRYING TO reach the Unraid WebUI (which doesn't load) roughly at the same time as my SSH attempt
Aug 28 08:00:01 Endurance root: Starting Mover
Aug 28 08:00:01 Endurance root: ionice -c 2 -n 7 nice -n 5 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start 0 0 0 "/mnt/user/cache-only/mover-ignore/global.txt" '' '' '' '' '' '' '' 0
Aug 28 08:58:15 Endurance emhttpd: spinning down /dev/sde
Aug 28 09:00:01 Endurance root: Starting Mover
Aug 28 09:00:01 Endurance root: ionice -c 2 -n 7 nice -n 5 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start 0 0 0 "/mnt/user/cache-only/mover-ignore/global.txt" '' '' '' '' '' '' '' 0
Aug 28 09:05:53 Endurance sshd-session[15967]: Connection from 192.168.2.10 port 64652 on 192.168.2.200 port 22 rdomain ""
Aug 28 09:05:57 Endurance nginx: 2024/08/28 09:05:57 [error] 14180#14180: OCSP responder sent invalid "Content-Type" header: "text/html" while requesting certificate status, responder: r10.o.lencr.org, peer: 184.150.39.43:80, certificate: "/boot/config/ssl/certs/certificate_bundle.pem"
Aug 28 09:06:03 Endurance sshd-session[15967]: Connection reset by 192.168.2.10 port 64652 [preauth]
Aug 28 09:06:03 Endurance sshd[9379]: srclimit_penalise: ipv4: new 192.168.2.10/32 deferred penalty of 1 seconds for penalty: connections without attempting authentication
- Here are the logs of when I unplug/replug the ethernet cable and the outage gets fixed. Note that this "Link is down" never ever showed up when the outage started. This is just from me physically unplugging and replugging the cable.
Aug 28 09:06:14 Endurance kernel: r8169 0000:06:00.0 eth0: Link is Down
Aug 28 09:06:14 Endurance kernel: bond0: (slave eth0): link status definitely down, disabling slave
Aug 28 09:06:14 Endurance kernel: device eth0 left promiscuous mode
Aug 28 09:06:14 Endurance kernel: bond0: now running without any active interface!
Aug 28 09:06:14 Endurance kernel: br0: port 1(bond0) entered disabled state
Aug 28 09:06:23 Endurance kernel: r8169 0000:06:00.0 eth0: Link is Up - 2.5Gbps/Full - flow control off
Aug 28 09:06:23 Endurance kernel: bond0: (slave eth0): link status definitely up, 2500 Mbps full duplex
Aug 28 09:06:23 Endurance kernel: bond0: (slave eth0): making interface the new active one
Aug 28 09:06:23 Endurance kernel: device eth0 entered promiscuous mode
Aug 28 09:06:23 Endurance kernel: bond0: active interface up!
Aug 28 09:06:23 Endurance kernel: br0: port 1(bond0) entered blocking state
Aug 28 09:06:23 Endurance kernel: br0: port 1(bond0) entered forwarding state
I am at a loss for ideas on how to fix this. Let me know if anyone else experienced this.