System locked out twice in last couple of weeks

[baboolian]

Hi, my system has been running smooth for a year until it complete froze up twice in the last two weeks. I have it hooked up to GUI mode as backup in case of networking issues, and GUI was locked out as well as ssh and http access.

 

I've read other users having stability issues due to macvlan recently, I've been using this since the start and never had an issue until recently so i'm not sure if that's it.

 

I've attached the two diagnostics logs, which i took immediately after hard rebooting.

 

Any help would be greatly appreciated!

 

 

unraid-diagnostics-20240127-1517.zip unraid-diagnostics-20240116-2140.zip

[itimpi]

The syslog in the diagnostics is the RAM copy and only shows what happened since the reboot.   It could be worth enabling the syslog server to get a log that survives a reboot so we can see what happened prior to the reboot. The mirror to flash option is the easiest to set up, but if you are worried about excessive wear on the flash drive you can put your server’s address into the Remote Server field.

[baboolian]

I'll set that up now and provide more useful logs next time it happens. Thanks for the advice!

[baboolian]

Hello, unfortunately I'm back.

 

Everything was running smoothly since my last post, until i got a system lockup last week. It happened around the same time that my download cache got full, and i figured it may have been related. Just a couple days later on an empty cache I got another system lockout. 

 

I enabled syslog server following your advice. My log file is extremely long, with 99% of it being composed of the following lines repeated multiple times per minute:

Jun  7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered blocking state

Jun  7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered forwarding state

Jun  7 18:48:11 Unraid kernel: veth248cea2: renamed from eth0

Jun  7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered disabled state

Jun  7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered disabled state

Jun  7 18:48:11 Unraid kernel: device veth8a464e8 left promiscuous mode

Jun  7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered disabled state

Jun  7 18:49:11 Unraid kernel: docker0: port 20(veth3e437da) entered blocking state

Jun  7 18:49:11 Unraid kernel: docker0: port 20(veth3e437da) entered disabled state

Jun  7 18:49:11 Unraid kernel: device veth3e437da entered promiscuous mode

Jun  7 18:49:11 Unraid kernel: eth0: renamed from vethf9f02e6

Jun  7 18:49:11 Unraid kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth3e437da: link becomes ready

 

The only content i see in my syslog file (/boot/logs/syslog-previous) aside from the above being repeated, is two errors which I've attached.

 

I am running macvlan, which I'm aware is known to cause issues for many folks. I see it mentioned in my error logs, but it's not clear to me what they mean. If we can conclude with evidence that that is the culprit of my issues, I'll bite the bullet and switch to ipvlan and work my way through drawback of it. I've got over a dozen docker containers setup and running smoothly on this unraid server for well over a year, which I'd like to avoid messing with unless absolutely necessary. 

 

Happy to add a diagnostics dump if that would be helpful, but not a lot has changed since the last time I posted one. I'm currently on version 6.12.6, will update to 6.12.10 after my parity check finishes (usually takes ~30 hours). 

 

Thanks for all the help!

 

syslog-errors.txt

[JorgeB]

Jun  7 04:08:41 Unraid kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Jun  7 04:08:41 Unraid kernel: ? _raw_spin_unlock+0x14/0x29
Jun  7 04:08:41 Unraid kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

Macvlan call traces will usually end up crashing the server, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), then reboot.