Hello,

 

I wanted to see if anybody would be able to help me understand why my server keeps crashing. Today has been the most amount of times it's crashed in a single day (4 times) and I'm not sure why it's doing it and I don't know where to begin to troubleshoot. This doesn't appear to be specific to 6.12.13 as it was happening before and had calmed down a bit over the past few months however it's progressively gotten worse in the past 2 weeks.

 

Any help on where to start would be greatly appreciated.

turn on syslog and supply diagnostics after the next crash.

Maybe we can find something then...

 

The issue happened today at 18:17. Looks like it might be an issue associated with IPv6?

syslog-previous syslog

I've said "diagnostics" not just syslog...

Sorry, here is the diagnostic zip. Hopefully this helps.

 

I also disabled IPv6 on the server for the time being in case that was the problem.

salisbury-htpc-diagnostics-20241018-2352.zip

2 minutes ago, Papi_Sosa said:

I also disabled IPv6 on the server for the time being in case that was the problem.

Unwise, since it was your only Internet connection as far as I could see from the syslog.

Problem seems that your router announces a V6 configuration (somewhere in Canada), but your PC is unable to configure that address to the Intel 2,5G card.

Maybe a driver issue, the card also complains that the slot she is in does not support a specific feature (dunno if this needed or essential)

Maybe its just a loose / faulty cable.

I will check the diagnostics now...

 

The server primarily uses IPv4. I don't have it statically configured in the server as I do all of my IP reservations from the DHCP server itself. IPv6 was just an addition from the ISP that I allowed my router to propagate to clients on my networks and isn't really required for anything. If it's the cause of the problem then I'll leave it disabled.

You dont have real IPv4, you are behind a CGNAT gateway and just get a fake address.

This maybe usable to reach outside internet, but in no way can be used to get into the server from the internet. If you do not need this, it maybe ok.

The V6 is your real Internet connection.

 

IPv6 to the server itself was disabled but the IPv6 addressing to the ISP router and my router are still active. The ISP router is in bridged mode so my router is seeing the public IP directly.

hmm... what happened at 22:09 yesterday? Did you manually stop the server and (re)configured the LAN settings, or was it a crash?

 

 

That was when I disabled the Docker Manager and VM Manager so I could change the network protocol from "IPv4 + IPv6" to "IPv4 only".

 

I just checked the switch port that the server is connected to and I see some Tx and Rx errors. The errors don't appear to be incrementing so I'll keep an eye on it.

Too bad, the current logs do not show anything that points to an error.

Let the box run untouched until it happens. Then reboot and supply new diagnostics.

But my educated guess currently still would be "cabling problem". What kind/brand of switch is the box connected to and what kind of router do you have???

(Its strange to see address leases for V6 given out for only 1 hr. That looks like a mobile connection...)

 

The router is a Ubiquiti UDM-SE and the server is connected to a USW-8-Pro-PoE that is acting as my core switch.

 

IPv6 address leasing for an hour sounds strange. It's configured for 24 hours in the LAN configuration for the subnet.

I've checked all the logs and settings but cannot find anything wrong.

The only "problem" caught is the unreachablilty of the router (checked only on V6,so by turning it off, you just surpress the messages not the culprit).

Seems that some devices (LAN Card, some USB Light Stuff (is this a computer or a lightshow in a disco??? 🙂 ) ) dropping off frequently, beeing reset and working again for some minutes. Unlikely that this is a software problem, looks more like the hardware on the mobo is failing now and then.

Can be anything from bad capicitor, to bad power supply to...

I am afraid, by changing some settings you will not be able to fix this 😞

But you could try to reset the BIOS to defaults (no overclocking and such evil things) and pray.

 

I’m not sure what the Mystic Light device is (Could be an RGB controller in my keyboard), I do have a KVM that I use to move my mouse and keyboard between my main computer and the server for my VM so sometimes that spits out errors when I click the switch. There is a BIOS update available so I’ll try updating that tonight.

11 minutes ago, Papi_Sosa said:

Could be an RGB controller in my keyboard

Unlikely. the keyboard and mouse lightshows are detected seperately. Must be something on the motherboard I would say.

 

So I just want to post an update to this.

 

I think this was closely related to the network issue I had seen earlier that clued to the IPv6 spam in the log. As mentioned before, disabling IPv6 did not resolve the issue and just eliminated the noise in the logs.

 

I looked into my networking equipment a bit and found out that the switchport that my server was plugged into was actually faulty and generating a bunch of errors. I swapped out the switch and the server has been stable now for the last 3 weeks with no errors being reported on the port. I'll keep the post open a little longer just to clear my suspicions but so far it's looking good.

Edited November 26, 20241 yr by Papi_Sosa