zarklin Posted November 9, 2021 Share Posted November 9, 2021 Please forgive me if I've overlooked something. Only 15 days into my trial key and very new to server side. After the previous crash I ran memtest for around 10 hours and did not show any errors. Durning this crash I was able to access the unifi docker (didn't think to try Emby) so not sure if it was a full crash. I could not access web or run a diagnostics so that was ran post crash. Additional Syslog is from the syslog server share i was still able to access this time before restarting. I'm assuming I misconfigured something as the system ran well for the first week and only in the last few days has been crashing every 24-30 hours it seems. Kennel Panic and RIP: 0010:kmem_cache_alloc+0x8d/0x130 is what I've been able to see from the IPMI preview and log. Not sure if the 504 timeouts when trying to access the web portal are related. Wondering if it may be a problem with my cache drive. I unfotunatley made a lot of changes in the days before this started. Installed a cache drive, moved appdata, setup pihole which I disabled after the previous crash and repointed the router dns to a public one to see if that was the problem. Thank you in advance. Details below. No VMs. Installed dockers : Binhex Krusader - was not started Emby - was running Unifi - was running PiHole - wasn not started Plugins : CA Auto Update Applications, CA Bacup / Restore Appdata, CA Cleanup Appdata, Community Applications, Dynamix SSD TRIM, Fix Common Problems, Preclear Disks, Unassigned Devicves, Unassigned Devices Plus, VM Backup System info : M/B: Supermicro X9DRD-7LN4F Version REV:1.02 BIOS: American Megatrends Inc. Version 3.3. Dated: 08/23/2018 CPU: Intel® Xeon® CPU E5-2620 0 @ 2.00GHz HVM: Enabled IOMMU: Enabled Cache: 384 KiB, 1536 KiB, 15 MB, 384 KiB, 1536 KiB, 15 MB Memory: 32 GiB DDR3 Multi-bit ECC (max. installable capacity 1024 GiB) Network: bond0: fault-tolerance (active-backup), mtu 1500 eth0: 1000 Mbps, full duplex, mtu 1500 eth1: interface down eth2: interface down eth3: interface down Kernel: Linux 5.10.28-Unraid x86_64 OpenSSL: 1.1.1j moria-diagnostics-20211108-2352.zip syslog-10.214.152.15.log Quote Link to comment
ChatNoir Posted November 9, 2021 Share Posted November 9, 2021 1 hour ago, zarklin said: Installed dockers : Binhex Krusader - was not started Emby - was running Unifi - was running PiHole - wasn not started Nov 6 11:45:48 Moria kernel: ------------[ cut here ]------------ Nov 6 11:45:48 Moria kernel: WARNING: CPU: 2 PID: 26556 at net/netfilter/nf_conntrack_core.c:1120 __nf_conntrack_confirm+0x9b/0x1e6 [nf_conntrack] Nov 6 11:45:48 Moria kernel: Modules linked in: xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle nf_tables vhost_net tun vhost vhost_iotlb tap xt_nat xt_tcpudp macvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter xfs md_mod ip6table_filter ip6_tables iptable_filter ip_tables x_tables bonding igb i2c_algo_bit sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd glue_helper rapl intel_cstate mpt3sas ahci i2c_i801 intel_uncore i2c_smbus input_leds i2c_core libahci raid_class led_class acpi_ipmi scsi_transport_sas wmi ipmi_si button acpi_pad [last unloaded: i2c_algo_bit] Nov 6 11:45:48 Moria kernel: CPU: 2 PID: 26556 Comm: kworker/2:2 Not tainted 5.10.28-Unraid #1 Nov 6 11:45:48 Moria kernel: Hardware name: Supermicro X9DRD-7LN4F(-JBOD)/X9DRD-EF/X9DRD-7LN4F, BIOS 3.3 08/23/2018 Nov 6 11:45:48 Moria kernel: Workqueue: events macvlan_process_broadcast [macvlan] Nov 6 11:45:48 Moria kernel: RIP: 0010:__nf_conntrack_confirm+0x9b/0x1e6 [nf_conntrack] Nov 6 11:45:48 Moria kernel: Code: e8 dc f8 ff ff 44 89 fa 89 c6 41 89 c4 48 c1 eb 20 89 df 41 89 de e8 36 f6 ff ff 84 c0 75 bb 48 8b 85 80 00 00 00 a8 08 74 18 <0f> 0b 89 df 44 89 e6 31 db e8 6d f3 ff ff e8 35 f5 ff ff e9 22 01 Nov 6 11:45:48 Moria kernel: RSP: 0018:ffffc90003380dd8 EFLAGS: 00010202 Nov 6 11:45:48 Moria kernel: RAX: 0000000000000188 RBX: 0000000000003a60 RCX: 000000002605bc03 Nov 6 11:45:48 Moria kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffa0123f48 Nov 6 11:45:48 Moria kernel: RBP: ffff888164e8c500 R08: 00000000d6b7bd8e R09: 0000000000000000 Nov 6 11:45:48 Moria kernel: R10: 0000000000000098 R11: ffff88811b862800 R12: 000000000000f352 Nov 6 11:45:48 Moria kernel: R13: ffffffff8210b440 R14: 0000000000003a60 R15: 0000000000000000 Nov 6 11:45:48 Moria kernel: FS: 0000000000000000(0000) GS:ffff88846fa80000(0000) knlGS:0000000000000000 Nov 6 11:45:48 Moria kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 6 11:45:48 Moria kernel: CR2: 0000145982658080 CR3: 000000000200a006 CR4: 00000000000606e0 Nov 6 11:45:48 Moria kernel: Call Trace: Nov 6 11:45:48 Moria kernel: <IRQ> Nov 6 11:45:48 Moria kernel: nf_conntrack_confirm+0x2f/0x36 [nf_conntrack] Nov 6 11:45:48 Moria kernel: nf_hook_slow+0x39/0x8e Nov 6 11:45:48 Moria kernel: nf_hook.constprop.0+0xb1/0xd8 Nov 6 11:45:48 Moria kernel: ? ip_protocol_deliver_rcu+0xfe/0xfe Nov 6 11:45:48 Moria kernel: ip_local_deliver+0x49/0x75 Nov 6 11:45:48 Moria kernel: __netif_receive_skb_one_core+0x74/0x95 Nov 6 11:45:48 Moria kernel: process_backlog+0xa3/0x13b Nov 6 11:45:48 Moria kernel: net_rx_action+0xf4/0x29d Nov 6 11:45:48 Moria kernel: __do_softirq+0xc4/0x1c2 Nov 6 11:45:48 Moria kernel: asm_call_irq_on_stack+0x12/0x20 Nov 6 11:45:48 Moria kernel: </IRQ> Nov 6 11:45:48 Moria kernel: do_softirq_own_stack+0x2c/0x39 Nov 6 11:45:48 Moria kernel: do_softirq+0x3a/0x44 Nov 6 11:45:48 Moria kernel: netif_rx_ni+0x1c/0x22 Nov 6 11:45:48 Moria kernel: macvlan_broadcast+0x10e/0x13c [macvlan] Nov 6 11:45:48 Moria kernel: macvlan_process_broadcast+0xf8/0x143 [macvlan] Nov 6 11:45:48 Moria kernel: process_one_work+0x13c/0x1d5 Nov 6 11:45:48 Moria kernel: worker_thread+0x18b/0x22f Nov 6 11:45:48 Moria kernel: ? process_scheduled_works+0x27/0x27 Nov 6 11:45:48 Moria kernel: kthread+0xe5/0xea Nov 6 11:45:48 Moria kernel: ? __kthread_bind_mask+0x57/0x57 Nov 6 11:45:48 Moria kernel: ret_from_fork+0x22/0x30 Nov 6 11:45:48 Moria kernel: ---[ end trace 1f1b68cca18c7de8 ]--- Do you have custom IP address on some of your dockers ? Quote Link to comment
JorgeB Posted November 9, 2021 Share Posted November 9, 2021 Macvlan call traces are usually the result of having dockers with a custom IP address, upgrading to v6.10 and switching to ipvlan might fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enable, top right)), or see below for more info. https://forums.unraid.net/topic/70529-650-call-traces-when-assigning-ip-address-to-docker-containers/ See also here: https://forums.unraid.net/bug-reports/stable-releases/690691-kernel-panic-due-to-netfilter-nf_nat_setup_info-docker-static-ip-macvlan-r1356/ Quote Link to comment
zarklin Posted November 10, 2021 Author Share Posted November 10, 2021 Thanks for the info. I do have individual IPs for the dockers. I'll take a look through the link and see if I can get the Vlan setup. My home network is flat right now and the server is hanging off a dumb switch so may need to do some rearanaging Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.