Unraid Computer quit responding

[brbjr]

Twice in the last month my unraid server quit responding and i had to power cycle it. How can i troubleshoot the issue? Attached is the screenshot of the screen after the issue.

[JorgeB]

You should start by posting the diagnostics, there are some known issues with some hardware, e.g. Ryzen, besides that see this.

[brbjr]

9 minutes ago, johnnie.black said:

You should start by posting the diagnostics, there are some known issues with some hardware, e.g. Ryzen, besides that see this.

Thanks. I enabled remote sys logging. Now we wait until it happens again?

[JorgeB]

Yep, then post the syslog, it might catch something.

[brbjr]

Looking at the log i think i captured an error and the system must have rebooted because i did not notice it was down. Please let me know if you see the issue. Maybe i need a new flash drive. 

 

<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: WARNING: CPU: 0 PID: 0 at net/netfilter/nf_conntrack_core.c:945 __nf_conntrack_confirm+0xa0/0x69e
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: Modules linked in: veth dm_mod dax xt_CHECKSUM ipt_REJECT ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap macvlan xt_nat ipt_MASQUERADE iptable_filter iptable_nat nf_nat_ipv4 nf_nat ip_tables ext4 mbcache jbd2 xfs md_mod bonding mlx4_en edac_mce_amd kvm_amd ccp kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc mpt3sas aesni_intel aes_x86_64 crypto_simd cryptd k10temp raid_class mxm_wmi fam15h_power cdc_acm scsi_transport_sas pcc_cpufreq mlx4_core glue_helper ahci pata_atiixp i2c_piix4 libahci i2c_core wmi button acpi_cpufreq
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.19.98-Unraid #1
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./990FXA-UD5, BIOS FC 06/24/2015
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RIP: 0010:__nf_conntrack_confirm+0xa0/0x69e
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: Code: 04 e8 56 fb ff ff 44 89 f2 44 89 ff 89 c6 41 89 c4 e8 7f f9 ff ff 48 8b 4c 24 08 84 c0 75 af 48 8b 85 80 00 00 00 a8 08 74 26 <0f> 0b 44 89 e6 44 89 ff 45 31 f6 e8 94 f1 ff ff be 00 02 00 00 48
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RSP: 0018:ffff88881ea03868 EFLAGS: 00010202
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RAX: 0000000000000188 RBX: ffff8886e810bb00 RCX: ffff88810a48d098
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RDX: 0000000000000001 RSI: 00000000000001c5 RDI: ffffffff81e091b4
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RBP: ffff88810a48d040 R08: 0000000040323eb3 R09: ffff8887e22984c0
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: R10: ffff88871848c040 R11: ffffffff81e91140 R12: 000000000000f5c5
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: R13: ffffffff81e91140 R14: 0000000000000000 R15: 00000000000022ed
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: FS:  0000000000000000(0000) GS:ffff88881ea00000(0000) knlGS:0000000000000000
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: CR2: ffff8009da896c08 CR3: 00000007e2060000 CR4: 00000000000406f0
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: Call Trace:
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: <IRQ>
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ipv4_confirm+0xaf/0xb9
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: nf_hook_slow+0x3a/0x90
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ip_local_deliver+0xad/0xdc
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? ip_sublist_rcv_finish+0x54/0x54
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ip_sabotage_in+0x38/0x3e
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: nf_hook_slow+0x3a/0x90
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ip_rcv+0x8e/0xbe
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? ip_rcv_finish_core.isra.0+0x2e1/0x2e1
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: __netif_receive_skb_one_core+0x53/0x6f
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: netif_receive_skb_internal+0x9f/0xba
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: br_pass_frame_up+0x128/0x14a
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? br_port_flags_change+0x29/0x29
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: br_handle_frame_finish+0x342/0x383
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? br_pass_frame_up+0x14a/0x14a
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: br_nf_hook_thresh+0xa3/0xc3
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? br_pass_frame_up+0x14a/0x14a
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: br_nf_pre_routing_finish+0x24a/0x271
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? br_pass_frame_up+0x14a/0x14a
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? br_handle_local_finish+0xe/0xe
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? nf_nat_ipv4_in+0x1e/0x62 [nf_nat_ipv4]
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? br_handle_local_finish+0xe/0xe
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: br_nf_pre_routing+0x31c/0x343
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? br_nf_forward_ip+0x362/0x362
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: nf_hook_slow+0x3a/0x90
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: br_handle_frame+0x27e/0x2bd
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? br_pass_frame_up+0x14a/0x14a
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: __netif_receive_skb_core+0x464/0x76e
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? udp_gro_receive+0x4b/0x136
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: __netif_receive_skb_one_core+0x35/0x6f
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: netif_receive_skb_internal+0x9f/0xba
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: napi_gro_frags+0x15a/0x196
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: mlx4_en_process_rx_cq+0x7ee/0x952 [mlx4_en]
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? mlx4_en_rx_irq+0x23/0x3e [mlx4_en]
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ? mlx4_eq_int+0xb30/0xb5c [mlx4_core]
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: mlx4_en_poll_rx_cq+0x68/0xc9 [mlx4_en]
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: net_rx_action+0x107/0x26c
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: __do_softirq+0xc9/0x1d7
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: irq_exit+0x5e/0x9d
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: do_IRQ+0xb2/0xd0
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: common_interrupt+0xf/0xf
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: </IRQ>
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RIP: 0010:native_safe_halt+0xe/0xf
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: Code: 60 02 df f0 83 44 24 fc 00 48 8b 00 a8 08 74 0b 65 81 25 4d b2 98 7e ff ff ff 7f c3 e9 07 00 00 00 0f 00 2d d2 cd 57 00 fb f4 <c3> e9 07 00 00 00 0f 00 2d c3 cd 57 00 f4 c3 53 e8 a5 15 a1 ff 65
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RSP: 0018:ffffffff81e03e28 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffdc
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RAX: 0000000080000000 RBX: 0000000000000001 RCX: 000000000000001f
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RDX: ffff88881ea00000 RSI: ffffffff81e5e2a0 RDI: ffff88881a538064
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: RBP: ffff88881a538064 R08: 000050a8d9595152 R09: 0000000000000021
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: R10: 0000000000000d14 R11: 071c71c71c71c71c R12: 0000000000000001
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: R13: ffff88881a538000 R14: 0000000000000000 R15: ffffffff81e5e318
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: acpi_safe_halt+0x15/0x1f
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: acpi_idle_enter+0x1db/0x21a
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: cpuidle_enter_state+0xa1/0x141
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: do_idle+0x17e/0x1fc
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: cpu_startup_entry+0x6a/0x6c
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: start_kernel+0x44e/0x46c
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: secondary_startup_64+0xa4/0xb0
<4>1 2020-03-15T16:53:15-05:00 Storage1 kernel - - - kernel: ---[ end trace ebaa749755b3fbee ]---
 

[JorgeB]

Please post the complete syslog.

[brbjr]

Ok. My system is down now. I attached the complete syslog

messages.txt

[JorgeB]

Macvlan call traces are usually related to having dockers with custom IP addresses, more info here, in the end there was a general protection fault, possibly unrelated and might be a hardware issue, but see if getting rid of the call traces helps.