Unraid freezes every few days

[0rca]

Hi all,

 

I have been using Unraid for two years now and have always been able to fix any issues by searching the forum or the web. I love the software, the versatility as well as the stability. But the latter has declined drastically since a few weeks. I have been wrecking my brain and tried many things, but can't pinpoint the problem and need your help.

 

First things first: I'll attach the diagnostics file and the (redirected) syslog. The latter is quite big, since it was captured from the moment I tried to get to the bottom of things end of May. So those of you who can read syslogs (I cannot, I am a bit of a Linux noob tbh, more of a Windows pro) will see  my various tries at "fixing" my issues. I only edited out the mover file operations.
 

It all started either with the latest Unraid update, or with me trying to better separate my networks. I have only Unifi devices and set up 3 vlans, one untagged, one for all media purposes (vlan2 10.2.0.0/24) and one for IoT devices (vlan3 10.3.0.0/24). My Unraid box (awesome Qnap TS-873A-8G, can only recommend) has one 10GbE NIC and the two onboard 1GbE NICs. My goal was to run all file and streaming traffic through the 10GbE (10.0.0.40), most dockers on vlan2 and all VMs on vlan3. "All" meaning just one Home Assistant OS VM in the end.

I hope the diagnostic shows my current settings - what I assumed to be correct after hours and hours of reading and changing network settings.

Now to the issue itself: Every few days the machine "freezes", meaning it stops responding (no Web, no SSH, no Ping). The only way to get it back is to reset it (using a long press on the power button of the NAS). Naturally it starts a parity check each time (which takes more than two days) and after that I am looking at the logs to see what happened. Since the problem is relatively new and the hardware has not changed, I am still assuming this to be a software issue, so I have not yet started to swap out parts (insofar this is possible with this NAS), but naturally will do so, if your verdict points to hardware.

Since I changed the vlans, I tried looking into the network side of things and switched to ipvlan and back to macvlan, but turned bridging and applied some other recommendations I found on the forum. But the Unraid kept freezing up. After some more changes, eliminating one Windows VM and reassigning all dockers to br2 instead of Bridge I thought it was stable, but the it froze again. Believing the HASS VM might be the culprit I deactivated VMs fo the time being, but yesterday it froze again. Except for deactivating Dockers too or swapping out hardware I don't know what else to try and am at my wits end.

Can anyone be so kind and have a look at the attached logs and point me in the right direction?

executor-diagnostics-20240626-1239.zip syslog-10.0.0.40.zip

[JorgeB]

Jun 24 18:21:35 Executor kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Jun 24 18:21:35 Executor kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

Macvlan call traces will usually end up crashing the server, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), then reboot.

[0rca]

Hi Jorge,

 

thanks for the lightning fast answer. I assumed this was fixed with 6.12.4 and followed these instructions: https://docs.unraid.net/unraid-os/release-notes/6.12.4/#fix-for-macvlan-call-traces ?!?

 

[JorgeB]

Yep

[0rca]

Yep? Sorry I don't get it. It was fixed, and following the instructions completely should prevent those crashes, shouldn't they? If so, why are they still happening for me? Did I implement anything wrong? As mentioned, I have spent a lot of time reading about this and adapting all of my network-related settings, but naturally I might have misunderstood or mixed up some things, so again, any pointers are most welcome.

Edited June 26 by 0rca

[JorgeB]

7 minutes ago, 0rca said:

Yep? Sorry I don't get it

You need to either use ipvlan, or disabled bridging, you have bridging enabled with macvlan, that can be a problem, and in this case it is, since there are call traces.

[0rca]

Afaict and if I interpreted everything correctly I do have disabled bridging as recommended. Bridging is disabled for eth0 and eth1 and on eth2 I am only using bridging (br2) for most of my dockers, the rest are using eth0. The instructions and recommendations regarding this are not very clear when it comes to multiple NICs, so it was my understanding this should work. The help tells me "Bridging is a feature which creates a virtual bridge and allows VMs and Docker containers to communicate directly with the physical Ethernet port" so I am assuming I need that bridge/br2 in order to connect those dockers to the physical network.

And since I am using Unifi for all things network and Unifi and ipvlan don't work well together, I am trying not to use ipvlan.

Please correct me, if I got anything of this wrong.

[JorgeB]

17 minutes ago, 0rca said:

bridging (br2) for most of my dockers

This is the main source of the problem problem, using bridging with dockers and macvlan.

 

18 minutes ago, 0rca said:

so I am assuming I need that bridge/br2 in order to connect those dockers to the physical network.

You don't need that, bridging is mostly needed for VMs.

[0rca]

So then most topics about the issue, the documentation and especially the inline-help are rather misleading, I think. A simple "Don't ever use macvlan and bridging ony any network interface together at the same time" should be written in big red letters everywhere, or am I wrong?

 

Be that as it may, I will switch all dockers to eth2 and disable bridging alltogether and report back.

[0rca]

Jorge, may I ask you one more thing? I have assigned static IPs to each interface (eth0 - 10.0.0.40, eth1 - 10.2.0.40, eth2 - 10.3.0.40), each in a different vlan (as described above).  Why is Unraid publishing additional MAC-adresses for each interface, each having the same IP as eth0. unifi is complaining that I have several devices with the same IP on the network.

Why is it doing that and how can I prevent it from doing so?

[JorgeB]

Sorry can't help with that, most network issues are out of my wheel house.

[0rca]

Ok, thanks for your help so far anyway.

 

Anybody else who can shed some light on this? I have read several other threads mentioning this behaviour but have yet to find a definitive answer why this is happeing, let alone a way to fix this. Or maybe I have to open a new thread for this.....

[JorgeB]

39 minutes ago, 0rca said:

Or maybe I have to open a new thread for this...

Probably best, to have a relevant title.