Hey folks, ever since I have set up my unRAID, my server kept crashing on average once per day (it's been ~3 weeks) since I set this up, presumably because of kernel panic.

During this crash, both unRAID GUI is not accessible and also the terminal interface is not showing up when I plug my server to monitor (no video input). For me the fix has always been forcing electrical power off and turning it on manually again, and I'm starting to get worried about my drive being corrupted during this process since this happens so often now.

I've finally got the time to set up a server syslog to be able to debug this, so I'm attaching both the diagnostics and also the server kernel here. 

 

At first I thought it's probably because I'm using macvlan, so I tried using ipvlan now and it seems like the issue is even worse with ipvlan since the server crashed within a few hours. When I was using macvlan it was better, usually it's pretty stable for 1-2 day before it crashed.

P.S great job on unRAID, I'm a software engineer by day, and the setup is super easy and the Docker container is really seamless. I would love it if you guys can advise me on how to debug and solve this issue

fujitower-diagnostics-20240130-1346.zip server_kernel.log

Jan 27 19:30:56 FujiTower kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Jan 27 19:30:56 FujiTower kernel: ? _raw_spin_unlock+0x14/0x29
Jan 27 19:30:56 FujiTower kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

Macvlan call traces will usually end up crashing the server, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), then reboot.
 

1 hour ago, Albertus Angga Raharja said:

Hey folks, ever since I have set up my unRAID, my server kept crashing on average once per day (it's been ~3 weeks) since I set this up,

Is this a new install? For some time now that a new install should default to ipvlan, did you change that?

@JorgeB thanks for the quick response here! 🫡

If you inspect the server log again on Jan 30, I think that's where it's showing my latest crash even with `ipvlan` - do you perhaps know why it's still crashing with `ipvlan` ?
I just changed the network type to `ipvlan` today and it crashed after 4 hours, let me see if it will crash again in the next few days, if yes my backup plan is probably not to assign any of the Docker container on custom networks and see if I can get by just by using bridge network for all containers, and maybe host mode on a few of those.

 

55 minutes ago, JorgeB said:

Is this a new install? For some time now that a new install should default to ipvlan, did you change that?

It's been a month, I think! And Yes I think I changed it to macvlan for some reason 😅

1 hour ago, Albertus Angga Raharja said:

I just changed the network type to `ipvlan` today and it crashed after 4 hours

Did you reboot after changing?

@JorgeB I did not, only turned off Docker enabled settings and turned it on again. Due I need to reboot before the custom network types' changes apply here?

You don't need to reboot to apply the change, but the time it ran with macvlan may still be enough to crash the server after the change and before a reboot.

Got it, When I switched it from macvlan to ipvlan, it was just right after the crash and so macvlan probably runs for a minute or two before I disabled the docker to switch it to ipvlan.

 

So far it has not crashed again, but will keep this thread updated in 2-3 days, hopefully this will be the fix ✌️

@JorgeB my server crashed again today and seems like somehow `macvlan` is still linkedin into the module, do you perhaps know why?

Before this crash, it crashed around 2 days ago because of a failing USB drive, so I restored a backup from the unRAID connect website, would that be a potential root cause here? If so, can you advise what would the fix be here?

syslog.log

macvlan call traces should only happen with macvlan is use, post the output of:

docker network ls

 

 

6 hours ago, JorgeB said:

macvlan call traces should only happen with macvlan is use, post the output of:

docker network ls

 

you are right, `docker network ls` was showing `br0` driver as `macvlan` - I modified the docker settings and rebooted and it's now properly `ipvlan`, hopefully this is the proper fix now. Thank you so much!