baboolian Posted January 27 Share Posted January 27 Hi, my system has been running smooth for a year until it complete froze up twice in the last two weeks. I have it hooked up to GUI mode as backup in case of networking issues, and GUI was locked out as well as ssh and http access. I've read other users having stability issues due to macvlan recently, I've been using this since the start and never had an issue until recently so i'm not sure if that's it. I've attached the two diagnostics logs, which i took immediately after hard rebooting. Any help would be greatly appreciated! unraid-diagnostics-20240127-1517.zip unraid-diagnostics-20240116-2140.zip Quote Link to comment
itimpi Posted January 28 Share Posted January 28 The syslog in the diagnostics is the RAM copy and only shows what happened since the reboot. It could be worth enabling the syslog server to get a log that survives a reboot so we can see what happened prior to the reboot. The mirror to flash option is the easiest to set up, but if you are worried about excessive wear on the flash drive you can put your server’s address into the Remote Server field. Quote Link to comment
baboolian Posted January 28 Author Share Posted January 28 I'll set that up now and provide more useful logs next time it happens. Thanks for the advice! Quote Link to comment
baboolian Posted June 9 Author Share Posted June 9 Hello, unfortunately I'm back. Everything was running smoothly since my last post, until i got a system lockup last week. It happened around the same time that my download cache got full, and i figured it may have been related. Just a couple days later on an empty cache I got another system lockout. I enabled syslog server following your advice. My log file is extremely long, with 99% of it being composed of the following lines repeated multiple times per minute: Jun 7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered blocking state Jun 7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered forwarding state Jun 7 18:48:11 Unraid kernel: veth248cea2: renamed from eth0 Jun 7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered disabled state Jun 7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered disabled state Jun 7 18:48:11 Unraid kernel: device veth8a464e8 left promiscuous mode Jun 7 18:48:11 Unraid kernel: docker0: port 20(veth8a464e8) entered disabled state Jun 7 18:49:11 Unraid kernel: docker0: port 20(veth3e437da) entered blocking state Jun 7 18:49:11 Unraid kernel: docker0: port 20(veth3e437da) entered disabled state Jun 7 18:49:11 Unraid kernel: device veth3e437da entered promiscuous mode Jun 7 18:49:11 Unraid kernel: eth0: renamed from vethf9f02e6 Jun 7 18:49:11 Unraid kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth3e437da: link becomes ready The only content i see in my syslog file (/boot/logs/syslog-previous) aside from the above being repeated, is two errors which I've attached. I am running macvlan, which I'm aware is known to cause issues for many folks. I see it mentioned in my error logs, but it's not clear to me what they mean. If we can conclude with evidence that that is the culprit of my issues, I'll bite the bullet and switch to ipvlan and work my way through drawback of it. I've got over a dozen docker containers setup and running smoothly on this unraid server for well over a year, which I'd like to avoid messing with unless absolutely necessary. Happy to add a diagnostics dump if that would be helpful, but not a lot has changed since the last time I posted one. I'm currently on version 6.12.6, will update to 6.12.10 after my parity check finishes (usually takes ~30 hours). Thanks for all the help! syslog-errors.txt Quote Link to comment
JorgeB Posted June 9 Share Posted June 9 Jun 7 04:08:41 Unraid kernel: macvlan_broadcast+0x10a/0x150 [macvlan] Jun 7 04:08:41 Unraid kernel: ? _raw_spin_unlock+0x14/0x29 Jun 7 04:08:41 Unraid kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan] Macvlan call traces will usually end up crashing the server, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), then reboot. Quote Link to comment
baboolian Posted September 7 Author Share Posted September 7 It's been consistently crashing about once per month since my initial post. I finally switch to IPVLAN and to my surprise, it's been crashing more frequently. Anecdotally about once per week, and just now it crashed less than an hour after hard rebooting from a prior crash during a parity check. I've attached the complete remote syslog between the two crashes. The only notable thing I see is `dnsmasq-dhcp[24984]: DHCP, IP range 192.168.122.2 -- 192.168.122.254`, i've recently upgraded to ubiquiti gear and changed my network ip range to 10.0.*.* Any ideas? I know this isn't a lot of information to go off of, how can I provide more data? Thanks! unraid.logs.txt Quote Link to comment
Solution JorgeB Posted September 7 Solution Share Posted September 7 That syslog only covers a few minutes up time, it does show filesystem corruption in your pool, check filesystem there, if issues persist update to latest stable since the macvlan is fixed there, then enable the syslog server and post that after a crash Quote Link to comment
baboolian Posted September 10 Author Share Posted September 10 Thanks! I do have remote syslog enabled, it was short because that's how long it was up before crashing again. I ran xfs_repair and fixed the corruptions, which were on the system cache drive. Already on the latest Unraid version, and moved back to macvlan. Hopefully that was it! Is there a way to get notifications of corruptions like this in the future, without having to dig through syslogs following a hang? Thanks again! Quote Link to comment
JorgeB Posted September 10 Share Posted September 10 46 minutes ago, baboolian said: Is there a way to get notifications of corruptions like this in the future, without having to dig through syslogs following a hang? Nope Quote Link to comment
baboolian Posted September 17 Author Share Posted September 17 So far so good! Seems that the system drive cache corruption was the culprit (hopefully I'm not jynxing it). Thanks for the help!! 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.