Jump to content

Unstable server after UPS failure, 6.12.3


Go to solution Solved by JorgeB,

Recommended Posts

My 6.12.3 UnRaid install has been stable for many years.  Have added drives and a couple years ago, new motherboard and almost all other hardware.  Still been rock solid.  My UPS failed a couple weeks ago and since then I have fought with keeping the server up over 24 hours.  Sometimes I might get 36 hours, Lately seems only 8 to 12 hours.  Seems that my cache pool with my appdata goes read-only at some point after the reboot, if the server is still responsive by the time I notice it.  This morning, the dockers were unhappy as the appdata was read-only, but the server was responsive, so I could get the diagnostics downloaded.

 

Attempted fixes:

- Replaced both m.2 drives in the appdata cache as that seemed to be the issue in the beginning, but I think that was my ignorance.

-- Through that, had to completely format and start from scratch with the FS on that RAID.

- Have VMs disabled hoping there was an issue there.

- The server used to boot from power off with zero issues, but seems that now, it will not boot unless I unplug the thumbdrive, let it go in to the BIOS, plug in the thumbdrive, then hit ctrl-alt-del, then it goes into UnRaid.

- Replaced thumbdrive thinking that could be causing the above.  Same issue (maybe motherboard is having issues?)

- Have downloaded MemTest and that had a successful run with no errors.

- Remote Syslog does indicate some "kernel warning" and "kernel error" messages, but they don't mean anything to me.

-- I have seen these happen as I have been watching, and the server does not always stop working (might not be the exact error though).

 

Diagnostics attached.  I hate to. keep swapping hardware unless I know it will help.  Having a stable UnRaid server for years certainly does not mean I can troubleshoot it at this level.

 

Any support would be appreciated.

spearsmedia00-diagnostics-20240123-0635.zip

Link to comment
  • Solution
Jan 22 21:20:05 SpearsMedia00 kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Jan 22 21:20:05 SpearsMedia00 kernel: ? _raw_spin_unlock+0x14/0x29
Jan 22 21:20:05 SpearsMedia00 kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

Macvlan call traces will usually end up crashing the server, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), then reboot.

Link to comment

I had seen a "discussion" about macvlan, but had not implemented that as I was unsure of the level of effort.  I have some dockers that are running on the server's IP and some that I have assigned static IPs to.  Do I need to make any physical network changes first?  I thought I had read that I needed to add another physical network adapter to support this.

 

Thank you for the quick response...

Link to comment
23 hours ago, JorgeB said:
Jan 22 21:20:05 SpearsMedia00 kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Jan 22 21:20:05 SpearsMedia00 kernel: ? _raw_spin_unlock+0x14/0x29
Jan 22 21:20:05 SpearsMedia00 kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

Macvlan call traces will usually end up crashing the server, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), then reboot.

Almost 24 hours in to the IPVLAN change and all seems very stable.  Also, no kernel.critical errors.  A few kernel.warning that say "kernel: x86/split lock detection", but from my research, that looks to be related to a Windows VM that is running, but not seeming to be causing stability issues.

 

Thank you again.  After another 24'ish hours, I think we can call this fixed, but will monitor.  Just wanted to post an update.

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...