System crash, nothing odd showing in the logs

[aglyons]

My system, lately, has started to get erratic with it's stability. It crashed hard just this morning and checking the logs showed nothing out of the ordinary. This however was displayed on the monitor I have connected.

 

The biggest problem is I have, aside from losing a ton of data I'd rather not lose, is a PiHole Docker container runs on here. When the system goes down that kills the internet for the whole house (= angry wife & kid).

 

Can anyone offer any advice?

 

SyncMaster SA650 </TASK>

Modules linked in: maculan xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE nf_c onntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag _ipv6 nf_defrag_ipv4 br_netfilter xfs md_mod iptable_filter ip6_tables iptable_ filter ip_tables x_tables atlantic bmx2 intel_powerclamp coretemp crc32c_intel i ntel_cstate ipmi_ssif i2c_core intel_uncore megaraid_sas input_leds led_class ip mi_si umi acpi_power_meter button [last unloaded: atlantic] --[ end trace 9e2cc8776e9cb2ed ]---

RIP: 0010:nf_nat_setup_info+0x146/0x773 [nf_nat]

Code: 15 c0 6a 00 00 89 c0 48 8d 04 c2 48 8b 10 48 85 d2 74 07 48 81 ea 98 00 00 00 45 31 c0 48 85 d2 of 84 6b ff ff ff 8a 44 24 4e <38> 42 46 74 19 48 8b 92 98

00 00 00 48 85 d2 0f 84 52 ff ff ff 48

RSP: 0018:ffffc90000274790 EFLAGS: 00010202

RAX: ffff888177ac0206 RBX: ffff888c85f 183c0 RCX: 1cd53ef870e390c7

RDX: 002900002b26ff68 RSI: 5b01b43358f245ef RDI: ffffc90000274768

RBP: ffffc90000274860 RO8: 0000000000000000 R09: b7ff817916811ca9 R10: 8e54dbffc0bc8b40 R11: cca2307651abe750 R12: 0000000000000000

R13: ffffc90000274874 R14: 0000000000000000 R15: ffffc900002747b8

FS: 000014b46c781840(0000) GS: ffff8897dfc00000(0000) kn1GS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CRO: 0000000080050033

CRZ: 0000557b78fa6298 CR3: 00000007be032000 CR4: 00000000000006e0 Kernel panic not syncing: Fatal exception in interrupt

Kernel Offset: disabled

---[ end Kernel panic not syncing: Fatal exception in interrupt ]---

 

[JorgeB]

See if this applies to you, if yes, upgrading to v6.10 and switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), or see below for more info.:

 

https://forums.unraid.net/topic/70529-650-call-traces-when-assigning-ip-address-to-docker-containers/

 

See also here:

https://forums.unraid.net/bug-reports/stable-releases/690691-kernel-panic-due-to-netfilter-nf_nat_setup_info-docker-static-ip-macvlan-r1356/

[aglyons]

So I realized I had to shutdown the Docker service to change this setting. Once down I could change it but then, Docker won't start.

 

There is a line in the console;

 

Aug 30 11:30:53 KNOXX root: '/mnt/user/system/docker/docker.img' is in-use, cannot mount

 

I'm trying a warm boot to see if that clears it up.

 

 

UPDATE: So the warm boot solved the problem of Docker not starting. I have it set to IPVLAN. I'll watch and see how this runs.

 

Thanks for your assistance!

Edited August 30, 2022 by aglyons
Situation Update

[JorgeB]

Rebooting should fix that.

[aglyons]

So it appears that this may have caused another issue.

 

I started to notice instability with some containers on br0 networks and now bridge networks as well. One of my containers is trying to connect to an external SFTP server and it's timing out failing to maintain a constant connection.

 

I noticed in my router, Ubiquiti UDMProSE that the MAC address of the server shows the various IP's of the containers flipping from one IP to another.

 

Could this be the result of IPVLAN instead of the MACVLAN?

 

I do have two other NIC's in the server that I could assign one dedicated to the PiHole. I'd have to figure out how to create the custom network in Docker and attach a specific eth to it. Any advice there?

 

UPDATE: So I have confirmed that when Docker is set to IPVLAN, it causes issues, at least with Ubiquiti routers, where outbound connections are timing out and failing. I changed back to MACVLAN and the problems disappeared. Set it to IPVLAN and the problems came back again.

 

I am not experienced enough to troubleshoot further on this but I am willing to try suggestions and to perform testing.

Edited August 31, 2022 by aglyons

[JorgeB]

Sorry, I don't use either, probably best to do another post with a title about that.