Server crashing after 6.12 update

[DarkMain]

Yesterday I updated to 6.12.5 from 6.11.

Everything seemed to go fine, but when I got home from work the server had crashed.
I reset it and it was running fine... I watched a movie from Plex and then went to bed, then today when I woke up it had crashed again.

 

There has been no major changes to the system. Hardware is the same and plugins were updated before the OS update but that's it.

 

I have attached the diagnostics to this post.
Syslog server was not enabled during the first 2 crashes but its on now (however it doesn't seem to be writing anything to the 

Local syslog folder).

tower-diagnostics-20231203-1532.zip

[JorgeB]

Enable the syslog server and post that after a crash.

 

[DarkMain]

K, here's the syslog.

Looks like it might be something to do with unassigned drives and a btrfs file system?

 

Note: These drives were all working perfectly fine before the update.

syslog-127.0.0.1.log

[JorgeB]

Docker image is corrupt, delete and recreate.

[DarkMain]

3 minutes ago, JorgeB said:

Docker image is corrupt, delete and recreate.

 

Just out of curiosity, what in the log gave you that answer?

And for my own piece of mind... are all following messages ok?
Dec  4 20:51:00 Tower kernel: BTRFS error (device loop2: state EA): parent transid verify failed on logical 335855616 mirror 2 wanted 5033596 found 5032512

[JorgeB]

11 minutes ago, DarkMain said:

(device loop2

 

This is the docker image, and those btrfs messages mean it's corrupt.

[DarkMain]

So I rebuilt the docker image last night and when I got home from work today the server had crashed again.

 

I've attached the new syslog.

syslog-127.0.0.1.log

[JorgeB]

Dec  5 17:41:37 Tower kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Dec  5 17:41:37 Tower kernel: ? _raw_spin_unlock+0x14/0x29
Dec  5 17:41:37 Tower kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

Macvlan call traces will usually end up crashing the server, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), then reboot.

[DarkMain]

I did see that in the patch notes, but a couple of things stopped me from changing it.

1 - Unless the setting was changed when I updated to 6.12, its been macvlan for ages and never been a problem. I figured (perhaps incorrectly), its not caused a problem in the past so why change it.

2 - The "help" says "The ipvlan type is best when connection to the physical network is not needed.". Maybe in interpreting that incorrectly but my containers are a combination of host / bridge and one is br0 and has its own IP address.
I was worried that by changing it to ipvlan I might break something so I just left it.

I'll give it a shot though and see how it goes.

[JorgeB]

1 hour ago, DarkMain said:

Unless the setting was changed when I updated to 6.12, its been macvlan for ages and never been a problem.

It's fairly typical for users that never had issues before having them after an upgrade, and not just with 6.12, it's been a thing for a long time.

 

1 hour ago, DarkMain said:

2 - The "help" says "The ipvlan type is best when connection to the physical network is not needed.". Maybe in interpreting that incorrectly but my containers are a combination of host / bridge and one is br0 and has its own IP address.
I was worried that by changing it to ipvlan I might break something so I just left it.

They should still work, but if you want to keep macvaln you can (with 6.12.4+), but need to disable bridging for eth0, see the release notes.

 

[DarkMain]

5 days and there have been no more crashes.
Looks like the Macvlan to ipvlan was the fix.

Cheers.