Gbcue Posted February 5, 2021 Share Posted February 5, 2021 (edited) Last time, it took 20+ days, this time, 4-5-ish days. Can't access shares or the web interface. I have to hard reset via IPMI and then parity check runs, 0 errors. The dockers I have running are Plex, Locast2Plex, Glances, and binhex-krusader. No VMs. I wasn't even home when the server went weird. I think it might have something to do with Locast2Plex (host with its own IP) but stopping it off for a long time didn't do anything. Anyone have any insight? Mobo: Supermicro X9DRH-7TF CPU: 2x Intel Xeon E5-2630 v2 RAM: 128gb DDR3 ECC Connected to power via UPS. The two interfaces are bonded via 802.3ad to a switch that supports it. 6 array disks (WD Reds) xfs, 2 parity (WD Reds), 2 cache (WD Black) btrfs. Edited February 5, 2021 by Gbcue Quote Link to comment
JorgeB Posted February 5, 2021 Share Posted February 5, 2021 You can try this and then post that log, as well the complete diagnostics. Quote Link to comment
Gbcue Posted February 6, 2021 Author Share Posted February 6, 2021 (edited) 23 hours ago, JorgeB said: You can try this and then post that log, as well the complete diagnostics. I enabled syslog, here's an entire snip of the kernel panic. But the server is not crashed or hung up. Nothing prompted me to grab this log, I was just curious what it said at this hour. I also see the BTRFS critical errors, but no SMART errors on any drives. syslog-10.0.10.x-5.txt serverone-diagnostics-20210206-0039.zip Edited February 6, 2021 by Gbcue Quote Link to comment
JorgeB Posted February 6, 2021 Share Posted February 6, 2021 Btrfs errors are likely the result of the previous crashes, I don't know how critical that particular error is, but it might be a good idea to re-format the pool just in case. Your main issue is likely related to the macvlan call traces, these are usually the result of having dockers with a custom IP address, and may end up crashing Unraid, see below for more info. Quote Link to comment
Gbcue Posted February 17, 2021 Author Share Posted February 17, 2021 Thanks, it seemed like setting a VLAN with custom IPs solved my issue. Can you point me to more info on reformatting my cache pool? Quote Link to comment
JorgeB Posted February 17, 2021 Share Posted February 17, 2021 With the array stopped use wipefs on both cache devices, first the partition then the device (all data there will be lost): wipefs -a /dev/sdX1 then wipefs -a /dev/sdX Start array and there will be an option to format cache. Quote Link to comment
Gbcue Posted February 18, 2021 Author Share Posted February 18, 2021 Thanks, I got my cache to a single drive since I didn't need a pool. I moved my dockers to their own VLAN and they all have custom IPs. But I just got this error in the syslog: Feb 17 21:47:56 ServerOne kernel: WARNING: CPU: 0 PID: 151 at net/netfilter/nf_conntrack_core.c:945 __nf_conntrack_confirm+0xa0/0x69e Feb 17 21:47:56 ServerOne kernel: Modules linked in: ccp xt_nat macvlan ipt_MASQUERADE iptable_filter iptable_nat nf_nat_ipv4 nf_nat ip_tables xfs md_mod ipmi_devintf ixgbe(O) sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp ipmi_ssif kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd mpt3sas glue_helper isci libsas intel_cstate ahci i2c_i801 intel_uncore i2c_core libahci raid_class intel_rapl_perf scsi_transport_sas pcc_cpufreq wmi ipmi_si button [last unloaded: ixgbe] Feb 17 21:47:56 ServerOne kernel: CPU: 0 PID: 151 Comm: kworker/0:2 Tainted: G O 4.19.107-Unraid #1 Feb 17 21:47:56 ServerOne kernel: Hardware name: Supermicro X9DRH-7TF/7F/iTF/iF/X9DRH-7TF/7F/iTF/iF, BIOS 3.3 07/13/2018 Feb 17 21:47:56 ServerOne kernel: Workqueue: events macvlan_process_broadcast [macvlan] Feb 17 21:47:56 ServerOne kernel: RIP: 0010:__nf_conntrack_confirm+0xa0/0x69e Feb 17 21:47:56 ServerOne kernel: Code: 04 e8 56 fb ff ff 44 89 f2 44 89 ff 89 c6 41 89 c4 e8 7f f9 ff ff 48 8b 4c 24 08 84 c0 75 af 48 8b 85 80 00 00 00 a8 08 74 26 <0f> 0b 44 89 e6 44 89 ff 45 31 f6 e8 95 f1 ff ff be 00 02 00 00 48 Feb 17 21:47:56 ServerOne kernel: RSP: 0018:ffff88903f803d58 EFLAGS: 00010202 Feb 17 21:47:56 ServerOne kernel: RAX: 0000000000000188 RBX: ffff88903b55ef00 RCX: ffff888fd5eb1318 Feb 17 21:47:56 ServerOne kernel: RDX: 0000000000000001 RSI: 0000000000000017 RDI: ffffffff81e0865c Feb 17 21:47:56 ServerOne kernel: RBP: ffff888fd5eb12c0 R08: 00000000e1f538c4 R09: ffffffff81c8aa80 Feb 17 21:47:56 ServerOne kernel: R10: 0000000000000158 R11: ffffffff81e91080 R12: 0000000000000c17 Feb 17 21:47:56 ServerOne kernel: R13: ffffffff81e91080 R14: 0000000000000000 R15: 000000000000c80d Feb 17 21:47:56 ServerOne kernel: FS: 0000000000000000(0000) GS:ffff88903f800000(0000) knlGS:0000000000000000 Feb 17 21:47:56 ServerOne kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Feb 17 21:47:56 ServerOne kernel: CR2: 00001516f289a718 CR3: 0000000001e0a004 CR4: 00000000001606f0 Feb 17 21:47:56 ServerOne kernel: Call Trace: Feb 17 21:47:56 ServerOne kernel: <IRQ> Feb 17 21:47:56 ServerOne kernel: ipv4_confirm+0xaf/0xb9 Feb 17 21:47:56 ServerOne kernel: nf_hook_slow+0x3a/0x90 Feb 17 21:47:56 ServerOne kernel: ip_local_deliver+0xad/0xdc Feb 17 21:47:56 ServerOne kernel: ? ip_sublist_rcv_finish+0x54/0x54 Feb 17 21:47:56 ServerOne kernel: ip_sabotage_in+0x38/0x3e Feb 17 21:47:56 ServerOne kernel: nf_hook_slow+0x3a/0x90 Feb 17 21:47:56 ServerOne kernel: ip_rcv+0x8e/0xbe Feb 17 21:47:56 ServerOne kernel: ? ip_rcv_finish_core.isra.0+0x2e1/0x2e1 Feb 17 21:47:56 ServerOne kernel: __netif_receive_skb_one_core+0x53/0x6f Feb 17 21:47:56 ServerOne kernel: process_backlog+0x77/0x10e Feb 17 21:47:56 ServerOne kernel: net_rx_action+0x107/0x26c Feb 17 21:47:56 ServerOne kernel: __do_softirq+0xc9/0x1d7 Feb 17 21:47:56 ServerOne kernel: do_softirq_own_stack+0x2a/0x40 Feb 17 21:47:56 ServerOne kernel: </IRQ> Feb 17 21:47:56 ServerOne kernel: do_softirq+0x4d/0x5a Feb 17 21:47:56 ServerOne kernel: netif_rx_ni+0x1c/0x22 Feb 17 21:47:56 ServerOne kernel: macvlan_broadcast+0x111/0x156 [macvlan] Feb 17 21:47:56 ServerOne kernel: ? __switch_to_asm+0x41/0x70 Feb 17 21:47:56 ServerOne kernel: macvlan_process_broadcast+0xea/0x128 [macvlan] Feb 17 21:47:56 ServerOne kernel: process_one_work+0x16e/0x24f Feb 17 21:47:56 ServerOne kernel: worker_thread+0x1e2/0x2b8 Feb 17 21:47:56 ServerOne kernel: ? rescuer_thread+0x2a7/0x2a7 Feb 17 21:47:56 ServerOne kernel: kthread+0x10c/0x114 Feb 17 21:47:56 ServerOne kernel: ? kthread_park+0x89/0x89 Feb 17 21:47:56 ServerOne kernel: ret_from_fork+0x35/0x40 Feb 17 21:47:56 ServerOne kernel: ---[ end trace e6dff886c5bb9d83 ]--- But no server hangs, dockers are still working as normal. Quote Link to comment
JorgeB Posted February 19, 2021 Share Posted February 19, 2021 Macvlan call traces are usually the result of having dockers with a custom IP address, more info below. https://forums.unraid.net/topic/70529-650-call-traces-when-assigning-ip-address-to-docker-containers/ Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.