Kernel Panic issue


Recommended Posts

Hi,

Got a kernel panic about an hour ago. Can you help me figure out what caused this?

Attached diagnostics, picture of the error and logfile right before crash. Thanks
 

Apr 21 07:00:01 Tower Docker Auto Update: Community Applications Docker Autoupdate running
Apr 21 07:00:01 Tower Docker Auto Update: Checking for available updates
Apr 21 07:00:34 Tower Docker Auto Update: Found update for Jackett.  Not set to autoupdate
Apr 21 07:00:34 Tower Docker Auto Update: No updates will be installed
Apr 21 07:17:54 Tower kernel: mdcmd (291): spindown 0
Apr 21 07:22:02 Tower kernel: rcu: INFO: rcu_sched self-detected stall on CPU
Apr 21 07:22:02 Tower kernel: rcu: #0115-....: (59999 ticks this GP) idle=88e/1/0x4000000000000002 softirq=111016460/111016460 fqs=14913
Apr 21 07:22:02 Tower kernel: rcu: #011 (t=60000 jiffies g=89357665 q=39319)
Apr 21 07:22:02 Tower kernel: NMI backtrace for cpu 5
Apr 21 07:22:02 Tower kernel: CPU: 5 PID: 2865 Comm: worker Tainted: G        W  O      4.19.107-Unraid #1
Apr 21 07:22:02 Tower kernel: Hardware name: System manufacturer System Product Name/Z170 PRO GAMING, BIOS 3805 05/16/2018
Apr 21 07:22:02 Tower kernel: Call Trace:
Apr 21 07:22:02 Tower kernel: <IRQ>
Apr 21 07:22:02 Tower kernel: dump_stack+0x67/0x83
Apr 21 07:22:02 Tower kernel: nmi_cpu_backtrace+0x71/0x83
Apr 21 07:22:02 Tower kernel: ? lapic_can_unplug_cpu+0x97/0x97
Apr 21 07:22:02 Tower kernel: nmi_trigger_cpumask_backtrace+0x57/0xd4
Apr 21 07:22:02 Tower kernel: rcu_dump_cpu_stacks+0x8b/0xb4
Apr 21 07:22:02 Tower kernel: rcu_check_callbacks+0x296/0x5a0
Apr 21 07:22:02 Tower kernel: update_process_times+0x24/0x47
Apr 21 07:22:02 Tower kernel: tick_sched_timer+0x36/0x64
Apr 21 07:22:02 Tower kernel: __hrtimer_run_queues+0xb7/0x10b
Apr 21 07:22:02 Tower kernel: ? tick_sched_handle.isra.0+0x2f/0x2f
Apr 21 07:22:02 Tower kernel: hrtimer_interrupt+0xf4/0x20e
Apr 21 07:22:02 Tower kernel: smp_apic_timer_interrupt+0x7b/0x93
Apr 21 07:22:02 Tower kernel: apic_timer_interrupt+0xf/0x20
Apr 21 07:22:02 Tower kernel: </IRQ>
Apr 21 07:22:02 Tower kernel: RIP: 0010:__radix_tree_create+0x101/0x167
Apr 21 07:22:02 Tower kernel: Code: 74 65 48 83 c8 01 48 85 db 8b 4c 24 0c 49 89 07 74 18 fe 43 02 eb 13 83 e0 03 48 ff c8 74 0b 4d 85 e4 74 36 49 89 1c 24 eb 30 <48> 8b 5c 24 10 48 8d 74 24 10 4c 89 f2 89 4c 24 0c 48 83 e3 fe 48
Apr 21 07:22:02 Tower kernel: RSP: 0018:ffffc90010a53870 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
Apr 21 07:22:02 Tower kernel: RAX: 0000000000000000 RBX: ffff888508fca480 RCX: 000000000000000c
Apr 21 07:22:02 Tower kernel: RDX: ffff888508fcb6c1 RSI: ffffc90010a53880 RDI: ffff888508fca480
Apr 21 07:22:02 Tower kernel: RBP: ffffc90010a538f0 R08: 000000000000001d R09: 000000001677db78
Apr 21 07:22:02 Tower kernel: R10: 0000000000000000 R11: ffff8886004b5130 R12: ffffc90010a538e8
Apr 21 07:22:02 Tower kernel: R13: ffff8886004b5130 R14: 000000001677db78 R15: ffff888508fca590
Apr 21 07:22:02 Tower kernel: __radix_tree_insert+0x43/0x1c4
Apr 21 07:22:02 Tower kernel: ? __radix_tree_lookup+0x69/0xa2
Apr 21 07:22:02 Tower kernel: alloc_extent_buffer+0x22a/0x325
Apr 21 07:22:02 Tower kernel: read_tree_block+0x1e/0x5f
Apr 21 07:22:02 Tower kernel: read_block_for_search.isra.0+0x2ef/0x345
Apr 21 07:22:02 Tower kernel: btrfs_search_slot+0x751/0x84a
Apr 21 07:22:02 Tower kernel: btrfs_del_csums+0xc4/0x384
Apr 21 07:22:02 Tower kernel: __btrfs_free_extent+0x5e2/0x90f
Apr 21 07:22:02 Tower kernel: __btrfs_run_delayed_refs+0xa77/0xbf4
Apr 21 07:22:02 Tower kernel: btrfs_run_delayed_refs+0x5d/0x16d
Apr 21 07:22:02 Tower kernel: ? btrfs_release_path+0x5c/0x71
Apr 21 07:22:02 Tower kernel: btrfs_start_dirty_block_groups+0x32a/0x41d
Apr 21 07:22:02 Tower kernel: btrfs_commit_transaction+0xc5/0x79c
Apr 21 07:22:02 Tower kernel: ? dput+0x26/0x10e
Apr 21 07:22:02 Tower kernel: ? btrfs_log_dentry_safe+0x52/0x65
Apr 21 07:22:02 Tower kernel: btrfs_sync_file+0x2af/0x2f7
Apr 21 07:22:02 Tower kernel: do_fsync+0x2a/0x44
Apr 21 07:22:02 Tower kernel: __x64_sys_fdatasync+0xe/0x11
Apr 21 07:22:02 Tower kernel: do_syscall_64+0x57/0xf2
Apr 21 07:22:02 Tower kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Apr 21 07:22:02 Tower kernel: RIP: 0033:0x1539d3d3822b
Apr 21 07:22:02 Tower kernel: Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 f3 2c f8 ff 8b 7c 24 0c 41 89 c0 b8 4b 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 31 2d f8 ff 8b 44
Apr 21 07:22:02 Tower kernel: RSP: 002b:000015393b6fd740 EFLAGS: 00000293 ORIG_RAX: 000000000000004b
Apr 21 07:22:02 Tower kernel: RAX: ffffffffffffffda RBX: 00001539d206e7b0 RCX: 00001539d3d3822b
Apr 21 07:22:02 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000b
Apr 21 07:22:02 Tower kernel: RBP: 00001539d20860f0 R08: 0000000000000000 R09: 00000000ffffffff
Apr 21 07:22:02 Tower kernel: R10: 000015393b6fd730 R11: 0000000000000293 R12: 0000561790dc4a18
Apr 21 07:22:02 Tower kernel: R13: 00001539d2086158 R14: 000015394e191a80 R15: 00001539d20860e0
Apr 21 07:25:02 Tower kernel: rcu: INFO: rcu_sched self-detected stall on CPU
Apr 21 07:25:02 Tower kernel: rcu: #0115-....: (240002 ticks this GP) idle=88e/1/0x4000000000000002 softirq=111016460/111016460 fqs=59513
Apr 21 07:25:02 Tower kernel: rcu: #011 (t=240003 jiffies g=89357665 q=159225)
Apr 21 07:25:02 Tower kernel: NMI backtrace for cpu 5
Apr 21 07:25:02 Tower kernel: CPU: 5 PID: 2865 Comm: worker Tainted: G        W  O      4.19.107-Unraid #1
Apr 21 07:25:02 Tower kernel: Hardware name: System manufacturer System Product Name/Z170 PRO GAMING, BIOS 3805 05/16/2018
Apr 21 07:25:02 Tower kernel: Call Trace:
Apr 21 07:25:02 Tower kernel: <IRQ>
Apr 21 07:25:02 Tower kernel: dump_stack+0x67/0x83
Apr 21 07:25:02 Tower kernel: nmi_cpu_backtrace+0x71/0x83
Apr 21 07:25:02 Tower kernel: ? lapic_can_unplug_cpu+0x97/0x97
Apr 21 07:25:02 Tower kernel: nmi_trigger_cpumask_backtrace+0x57/0xd4
Apr 21 07:25:02 Tower kernel: rcu_dump_cpu_stacks+0x8b/0xb4
Apr 21 07:25:02 Tower kernel: rcu_check_callbacks+0x296/0x5a0
Apr 21 07:25:02 Tower kernel: update_process_times+0x24/0x47
Apr 21 07:25:02 Tower kernel: tick_sched_timer+0x36/0x64
Apr 21 07:25:02 Tower kernel: __hrtimer_run_queues+0xb7/0x10b
Apr 21 07:25:02 Tower kernel: ? tick_sched_handle.isra.0+0x2f/0x2f
Apr 21 07:25:02 Tower kernel: hrtimer_interrupt+0xf4/0x20e
Apr 21 07:25:02 Tower kernel: smp_apic_timer_interrupt+0x7b/0x93
Apr 21 07:25:02 Tower kernel: apic_timer_interrupt+0xf/0x20
Apr 21 07:25:02 Tower kernel: </IRQ>
Apr 21 07:25:02 Tower kernel: RIP: 0010:__radix_tree_preload+0x7f/0x82
Apr 21 07:25:02 Tower kernel: Code: 8b 50 08 48 89 56 08 ff 00 48 89 70 08 eb c3 48 8b 3d 8b fd c3 00 e8 57 07 ac ff eb b5 31 c0 eb 05 b8 f4 ff ff ff 5b 5d 41 5c <41> 5d c3 0f ba e7 15 72 02 0f 0b be 15 00 00 00 e9 6c ff ff ff be
Apr 21 07:25:02 Tower kernel: RSP: 0018:ffffc90010a53928 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
Apr 21 07:25:02 Tower kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
Apr 21 07:25:02 Tower kernel: RDX: ffffffffffffffff RSI: 0000000000000015 RDI: 0000000000600040
Apr 21 07:25:02 Tower kernel: RBP: 0000000000000000 R08: ffff888650bd41e8 R09: 0000000000000000
Apr 21 07:25:02 Tower kernel: R10: 0000000000000000 R11: ffff8886004b5130 R12: ffff888132e3b928
Apr 21 07:25:02 Tower kernel: R13: 000000000001f2d0 R14: 000001677db78000 R15: ffff8886004b512c
Apr 21 07:25:02 Tower kernel: alloc_extent_buffer+0x203/0x325
Apr 21 07:25:02 Tower kernel: read_tree_block+0x1e/0x5f
Apr 21 07:25:02 Tower kernel: read_block_for_search.isra.0+0x2ef/0x345
Apr 21 07:25:02 Tower kernel: btrfs_search_slot+0x751/0x84a
Apr 21 07:25:02 Tower kernel: btrfs_del_csums+0xc4/0x384
Apr 21 07:25:02 Tower kernel: __btrfs_free_extent+0x5e2/0x90f
Apr 21 07:25:02 Tower kernel: __btrfs_run_delayed_refs+0xa77/0xbf4
Apr 21 07:25:02 Tower kernel: btrfs_run_delayed_refs+0x5d/0x16d
Apr 21 07:25:02 Tower kernel: ? btrfs_release_path+0x5c/0x71
Apr 21 07:25:02 Tower kernel: btrfs_start_dirty_block_groups+0x32a/0x41d
Apr 21 07:25:02 Tower kernel: btrfs_commit_transaction+0xc5/0x79c
Apr 21 07:25:02 Tower kernel: ? dput+0x26/0x10e
Apr 21 07:25:02 Tower kernel: ? btrfs_log_dentry_safe+0x52/0x65
Apr 21 07:25:02 Tower kernel: btrfs_sync_file+0x2af/0x2f7
Apr 21 07:25:02 Tower kernel: do_fsync+0x2a/0x44
Apr 21 07:25:02 Tower kernel: __x64_sys_fdatasync+0xe/0x11
Apr 21 07:25:02 Tower kernel: do_syscall_64+0x57/0xf2
Apr 21 07:25:02 Tower kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Apr 21 07:25:02 Tower kernel: RIP: 0033:0x1539d3d3822b
Apr 21 07:25:02 Tower kernel: Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 f3 2c f8 ff 8b 7c 24 0c 41 89 c0 b8 4b 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 31 2d f8 ff 8b 44
Apr 21 07:25:02 Tower kernel: RSP: 002b:000015393b6fd740 EFLAGS: 00000293 ORIG_RAX: 000000000000004b
Apr 21 07:25:02 Tower kernel: RAX: ffffffffffffffda RBX: 00001539d206e7b0 RCX: 00001539d3d3822b
Apr 21 07:25:02 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000b
Apr 21 07:25:02 Tower kernel: RBP: 00001539d20860f0 R08: 0000000000000000 R09: 00000000ffffffff
Apr 21 07:25:02 Tower kernel: R10: 000015393b6fd730 R11: 0000000000000293 R12: 0000561790dc4a18
Apr 21 07:25:02 Tower kernel: R13: 00001539d2086158 R14: 000015394e191a80 R15: 00001539d20860e0
Apr 21 07:28:02 Tower kernel: rcu: INFO: rcu_sched self-detected stall on CPU
Apr 21 07:28:02 Tower kernel: rcu: #0115-....: (420005 ticks this GP) idle=88e/1/0x4000000000000002 softirq=111016460/111016460 fqs=104120
Apr 21 07:28:02 Tower kernel: rcu: #011 (t=420006 jiffies g=89357665 q=289644)
Apr 21 07:28:02 Tower kernel: NMI backtrace for cpu 5
Apr 21 07:28:02 Tower kernel: CPU: 5 PID: 2865 Comm: worker Tainted: G        W  O      4.19.107-Unraid #1
Apr 21 07:28:02 Tower kernel: Hardware name: System manufacturer System Product Name/Z170 PRO GAMING, BIOS 3805 05/16/2018
Apr 21 07:28:02 Tower kernel: Call Trace:
Apr 21 07:28:02 Tower kernel: <IRQ>
Apr 21 07:28:02 Tower kernel: dump_stack+0x67/0x83
Apr 21 07:28:02 Tower kernel: nmi_cpu_backtrace+0x71/0x83
Apr 21 07:28:02 Tower kernel: ? lapic_can_unplug_cpu+0x97/0x97
Apr 21 07:28:02 Tower kernel: nmi_trigger_cpumask_backtrace+0x57/0xd4
Apr 21 07:28:02 Tower kernel: rcu_dump_cpu_stacks+0x8b/0xb4
Apr 21 07:28:02 Tower kernel: rcu_check_callbacks+0x296/0x5a0
Apr 21 07:28:02 Tower kernel: update_process_times+0x24/0x47
Apr 21 07:28:02 Tower kernel: tick_sched_timer+0x36/0x64
Apr 21 07:28:02 Tower kernel: __hrtimer_run_queues+0xb7/0x10b
Apr 21 07:28:02 Tower kernel: ? tick_sched_handle.isra.0+0x2f/0x2f
Apr 21 07:28:02 Tower kernel: hrtimer_interrupt+0xf4/0x20e
Apr 21 07:28:02 Tower kernel: smp_apic_timer_interrupt+0x7b/0x93
Apr 21 07:28:02 Tower kernel: apic_timer_interrupt+0xf/0x20
Apr 21 07:28:02 Tower kernel: </IRQ>
Apr 21 07:28:02 Tower kernel: RIP: 0010:__radix_tree_lookup+0x42/0xa2
Apr 21 07:28:02 Tower kernel: Code: 8b 04 25 28 00 00 00 48 89 44 24 10 31 c0 48 8d 54 24 08 48 89 e6 4c 89 df 49 89 e8 e8 27 eb ff ff 48 39 5c 24 08 72 44 31 ff <48> 8b 04 24 48 89 c2 83 e2 03 48 ff ca 75 21 48 83 f8 01 74 cf 48
Apr 21 07:28:02 Tower kernel: RSP: 0018:ffffc90010a538f0 EFLAGS: 00000212 ORIG_RAX: ffffffffffffff13
Apr 21 07:28:02 Tower kernel: RAX: 000000000000002d RBX: 000000001677db78 RCX: ffff8880458f8dd8
Apr 21 07:28:02 Tower kernel: RDX: ffff888650bd4001 RSI: ffffc90010a538f0 RDI: ffff8880458f8db0
Apr 21 07:28:02 Tower kernel: RBP: ffff8886004b5138 R08: ffff8880458f8f40 R09: 0000000000000000
Apr 21 07:28:02 Tower kernel: R10: 0000000000000000 R11: ffff8886004b5130 R12: ffff888132e3b928
Apr 21 07:28:02 Tower kernel: R13: ffff8886004b4000 R14: 000001677db78000 R15: ffff8886004b512c
Apr 21 07:28:02 Tower kernel: find_extent_buffer+0x13/0x64
Apr 21 07:28:02 Tower kernel: alloc_extent_buffer+0x246/0x325
Apr 21 07:28:02 Tower kernel: read_tree_block+0x1e/0x5f
Apr 21 07:28:02 Tower kernel: read_block_for_search.isra.0+0x2ef/0x345
Apr 21 07:28:02 Tower kernel: btrfs_search_slot+0x751/0x84a
Apr 21 07:28:02 Tower kernel: btrfs_del_csums+0xc4/0x384
Apr 21 07:28:02 Tower kernel: __btrfs_free_extent+0x5e2/0x90f
Apr 21 07:28:02 Tower kernel: __btrfs_run_delayed_refs+0xa77/0xbf4
Apr 21 07:28:02 Tower kernel: btrfs_run_delayed_refs+0x5d/0x16d
Apr 21 07:28:02 Tower kernel: ? btrfs_release_path+0x5c/0x71
Apr 21 07:28:02 Tower kernel: btrfs_start_dirty_block_groups+0x32a/0x41d
Apr 21 07:28:02 Tower kernel: btrfs_commit_transaction+0xc5/0x79c
Apr 21 07:28:02 Tower kernel: ? dput+0x26/0x10e
Apr 21 07:28:02 Tower kernel: ? btrfs_log_dentry_safe+0x52/0x65
Apr 21 07:28:02 Tower kernel: btrfs_sync_file+0x2af/0x2f7
Apr 21 07:28:02 Tower kernel: do_fsync+0x2a/0x44
Apr 21 07:28:02 Tower kernel: __x64_sys_fdatasync+0xe/0x11
Apr 21 07:28:02 Tower kernel: do_syscall_64+0x57/0xf2
Apr 21 07:28:02 Tower kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Apr 21 07:28:02 Tower kernel: RIP: 0033:0x1539d3d3822b
Apr 21 07:28:02 Tower kernel: Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 f3 2c f8 ff 8b 7c 24 0c 41 89 c0 b8 4b 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 31 2d f8 ff 8b 44
Apr 21 07:28:02 Tower kernel: RSP: 002b:000015393b6fd740 EFLAGS: 00000293 ORIG_RAX: 000000000000004b
Apr 21 07:28:02 Tower kernel: RAX: ffffffffffffffda RBX: 00001539d206e7b0 RCX: 00001539d3d3822b
Apr 21 07:28:02 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000b
Apr 21 07:28:02 Tower kernel: RBP: 00001539d20860f0 R08: 0000000000000000 R09: 00000000ffffffff
Apr 21 07:28:02 Tower kernel: R10: 000015393b6fd730 R11: 0000000000000293 R12: 0000561790dc4a18
Apr 21 07:28:02 Tower kernel: R13: 00001539d2086158 R14: 000015394e191a80 R15: 00001539d20860e0

tower-diagnostics-20200421-0815.zip

IMG_20200421_080325.jpg

Link to comment

New crash a few minutes ago and now it's crashing at startup. Any ideas?

 

Apr 21 22:42:15 Tower kernel: WARNING: CPU: 5 PID: 71 at net/netfilter/nf_conntrack_core.c:945 __nf_conntrack_confirm+0xa0/0x69e
Apr 21 22:42:15 Tower kernel: Modules linked in: veth nfsv3 nfs arc4 ecb md4 xt_CHECKSUM ipt_REJECT sha512_ssse3 sha512_generic cmac cifs ccm ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap macvlan xt_nat ipt_MASQUERADE iptable_filter iptable_nat nf_nat_ipv4 nf_nat ip_tables xfs nfsd lockd grace sunrpc md_mod ipmi_devintf nct6775 hwmon_vid e1000e igb(O) mxm_wmi wmi_bmof x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd glue_helper intel_cstate intel_uncore intel_rapl_perf i2c_i801 i2c_core mpt3sas ahci libahci pcc_cpufreq cdc_acm raid_class scsi_transport_sas wmi thermal fan video backlight button acpi_pad [last unloaded: e1000e]
Apr 21 22:42:15 Tower kernel: CPU: 5 PID: 71 Comm: kworker/5:1 Tainted: G O 4.19.107-Unraid #1
Apr 21 22:42:15 Tower kernel: Hardware name: System manufacturer System Product Name/Z170 PRO GAMING, BIOS 3805 05/16/2018
Apr 21 22:42:15 Tower kernel: Workqueue: events macvlan_process_broadcast [macvlan]
Apr 21 22:42:15 Tower kernel: RIP: 0010:__nf_conntrack_confirm+0xa0/0x69e
Apr 21 22:42:15 Tower kernel: Code: 04 e8 56 fb ff ff 44 89 f2 44 89 ff 89 c6 41 89 c4 e8 7f f9 ff ff 48 8b 4c 24 08 84 c0 75 af 48 8b 85 80 00 00 00 a8 08 74 26 <0f> 0b 44 89 e6 44 89 ff 45 31 f6 e8 95 f1 ff ff be 00 02 00 00 48
Apr 21 22:42:15 Tower kernel: RSP: 0018:ffff888816b43d90 EFLAGS: 00010202
Apr 21 22:42:15 Tower kernel: RAX: 0000000000000188 RBX: ffff8880a2d0f600 RCX: ffff8882592be198
Apr 21 22:42:15 Tower kernel: RDX: 0000000000000001 RSI: 00000000000001fc RDI: ffffffff81e08df0
Apr 21 22:42:15 Tower kernel: RBP: ffff8882592be140 R08: 00000000696b6111 R09: ffff8886fbf3ada0
Apr 21 22:42:15 Tower kernel: R10: 0000000000000000 R11: ffff8887b8b49400 R12: 00000000000035fc
Apr 21 22:42:15 Tower kernel: R13: ffffffff81e91080 R14: 0000000000000000 R15: 000000000000644f
Apr 21 22:42:15 Tower kernel: FS: 0000000000000000(0000) GS:ffff888816b40000(0000) knlGS:0000000000000000
Apr 21 22:42:15 Tower kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 21 22:42:15 Tower kernel: CR2: 000014a80a17d000 CR3: 0000000001e0a005 CR4: 00000000003626e0
Apr 21 22:42:15 Tower kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 21 22:42:15 Tower kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Apr 21 22:42:15 Tower kernel: Call Trace:
Apr 21 22:42:15 Tower kernel: <IRQ>
Apr 21 22:42:15 Tower kernel: ipv4_confirm+0xaf/0xb9
Apr 21 22:42:15 Tower kernel: nf_hook_slow+0x3a/0x90
Apr 21 22:42:15 Tower kernel: ip_local_deliver+0xad/0xdc
Apr 21 22:42:15 Tower kernel: ? ip_sublist_rcv_finish+0x54/0x54
Apr 21 22:42:15 Tower kernel: ip_rcv+0xa0/0xbe
Apr 21 22:42:15 Tower kernel: ? ip_rcv_finish_core.isra.0+0x2e1/0x2e1
Apr 21 22:42:15 Tower kernel: __netif_receive_skb_one_core+0x53/0x6f
Apr 21 22:42:15 Tower kernel: process_backlog+0x77/0x10e
Apr 21 22:42:15 Tower kernel: net_rx_action+0x107/0x26c
Apr 21 22:42:15 Tower kernel: __do_softirq+0xc9/0x1d7
Apr 21 22:42:15 Tower kernel: do_softirq_own_stack+0x2a/0x40
Apr 21 22:42:15 Tower kernel: </IRQ>
Apr 21 22:42:15 Tower kernel: do_softirq+0x4d/0x5a
Apr 21 22:42:15 Tower kernel: netif_rx_ni+0x1c/0x22
Apr 21 22:42:15 Tower kernel: macvlan_broadcast+0x111/0x156 [macvlan]
Apr 21 22:42:15 Tower kernel: macvlan_process_broadcast+0xea/0x128 [macvlan]
Apr 21 22:42:15 Tower kernel: process_one_work+0x16e/0x24f
Apr 21 22:42:15 Tower kernel: worker_thread+0x1e2/0x2b8
Apr 21 22:42:15 Tower kernel: ? rescuer_thread+0x2a7/0x2a7
Apr 21 22:42:15 Tower kernel: kthread+0x10c/0x114
Apr 21 22:42:15 Tower kernel: ? kthread_park+0x89/0x89
Apr 21 22:42:15 Tower kernel: ret_from_fork+0x1f/0x40
Apr 21 22:42:15 Tower kernel: ---[ end trace fa968b3a6dcc1b00 ]---
Apr 21 22:42:19 Tower kernel: ------------[ cut here ]------------
Apr 21 22:42:19 Tower kernel: kernel BUG at fs/btrfs/extent-tree.c:6862!
Apr 21 22:42:19 Tower kernel: invalid opcode: 0000 [#1] SMP NOPTI
Apr 21 22:42:19 Tower kernel: CPU: 1 PID: 1510 Comm: kworker/u16:8 Tainted: G W O 4.19.107-Unraid #1
Apr 21 22:42:19 Tower kernel: Hardware name: System manufacturer System Product Name/Z170 PRO GAMING, BIOS 3805 05/16/2018
Apr 21 22:42:19 Tower kernel: Workqueue: btrfs-extent-refs btrfs_extent_refs_helper
Apr 21 22:42:19 Tower kernel: RIP: 0010:__btrfs_free_extent+0x558/0x90f
Apr 21 22:42:19 Tower kernel: Code: ba c8 1a 00 00 e9 36 fd ff ff 48 81 3c 24 ff 00 00 00 76 18 48 8b 74 24 48 48 89 ef e8 a8 79 ff ff 3b 84 24 b8 00 00 00 74 02 <0f> 0b 48 83 7c 24 48 00 8b 45 40 74 0c 39 44 24 08 0f 84 0c ff ff
Apr 21 22:42:19 Tower kernel: RSP: 0018:ffffc9000357bc58 EFLAGS: 00010206
Apr 21 22:42:19 Tower kernel: RAX: 0000000058000001 RBX: 0000000000000001 RCX: ffff888663719000
Apr 21 22:42:19 Tower kernel: RDX: 0000000000003000 RSI: 0000000000000003 RDI: ffff88811449ef50
Apr 21 22:42:19 Tower kernel: RBP: ffff88806f711e70 R08: ffffc9000357bbd8 R09: ffffc9000357bbe0
Apr 21 22:42:19 Tower kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Apr 21 22:42:19 Tower kernel: R13: ffff8887d5ac3a90 R14: ffff88811449ef50 R15: 00000190de43d000
Apr 21 22:42:19 Tower kernel: FS: 0000000000000000(0000) GS:ffff888816a40000(0000) knlGS:0000000000000000
Apr 21 22:42:19 Tower kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 21 22:42:19 Tower kernel: CR2: 000014d881476020 CR3: 0000000001e0a001 CR4: 00000000003626e0
Apr 21 22:42:19 Tower kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 21 22:42:19 Tower kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Apr 21 22:42:19 Tower kernel: Call Trace:
Apr 21 22:42:19 Tower kernel: __btrfs_run_delayed_refs+0xa77/0xbf4
Apr 21 22:42:19 Tower kernel: ? start_transaction+0x178/0x333
Apr 21 22:42:19 Tower kernel: btrfs_run_delayed_refs+0x5d/0x16d
Apr 21 22:42:19 Tower kernel: delayed_ref_async_start+0x4c/0x84
Apr 21 22:42:19 Tower kernel: normal_work_helper+0xd0/0x1c7
Apr 21 22:42:19 Tower kernel: process_one_work+0x16e/0x24f
Apr 21 22:42:19 Tower kernel: worker_thread+0x1e2/0x2b8
Apr 21 22:42:19 Tower kernel: ? rescuer_thread+0x2a7/0x2a7
Apr 21 22:42:19 Tower kernel: kthread+0x10c/0x114
Apr 21 22:42:19 Tower kernel: ? kthread_park+0x89/0x89
Apr 21 22:42:19 Tower kernel: ret_from_fork+0x1f/0x40
Apr 21 22:42:19 Tower kernel: Modules linked in: veth nfsv3 nfs arc4 ecb md4 xt_CHECKSUM ipt_REJECT sha512_ssse3 sha512_generic cmac cifs ccm ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap macvlan xt_nat ipt_MASQUERADE iptable_filter iptable_nat nf_nat_ipv4 nf_nat ip_tables xfs nfsd lockd grace sunrpc md_mod ipmi_devintf nct6775 hwmon_vid e1000e igb(O) mxm_wmi wmi_bmof x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd glue_helper intel_cstate intel_uncore intel_rapl_perf i2c_i801 i2c_core mpt3sas ahci libahci pcc_cpufreq cdc_acm raid_class scsi_transport_sas wmi thermal fan video backlight button acpi_pad [last unloaded: e1000e]
Apr 21 22:42:19 Tower kernel: ---[ end trace fa968b3a6dcc1b01 ]---
Apr 21 22:42:19 Tower kernel: RIP: 0010:__btrfs_free_extent+0x558/0x90f
Apr 21 22:42:19 Tower kernel: Code: ba c8 1a 00 00 e9 36 fd ff ff 48 81 3c 24 ff 00 00 00 76 18 48 8b 74 24 48 48 89 ef e8 a8 79 ff ff 3b 84 24 b8 00 00 00 74 02 <0f> 0b 48 83 7c 24 48 00 8b 45 40 74 0c 39 44 24 08 0f 84 0c ff ff
Apr 21 22:42:19 Tower kernel: RSP: 0018:ffffc9000357bc58 EFLAGS: 00010206
Apr 21 22:42:19 Tower kernel: RAX: 0000000058000001 RBX: 0000000000000001 RCX: ffff888663719000
Apr 21 22:42:19 Tower kernel: RDX: 0000000000003000 RSI: 0000000000000003 RDI: ffff88811449ef50
Apr 21 22:42:19 Tower kernel: RBP: ffff88806f711e70 R08: ffffc9000357bbd8 R09: ffffc9000357bbe0
Apr 21 22:42:19 Tower kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Apr 21 22:42:19 Tower kernel: R13: ffff8887d5ac3a90 R14: ffff88811449ef50 R15: 00000190de43d000
Apr 21 22:42:19 Tower kernel: FS: 0000000000000000(0000) GS:ffff888816a40000(0000) knlGS:0000000000000000
Apr 21 22:42:19 Tower kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 21 22:42:19 Tower kernel: CR2: 000014d881476020 CR3: 0000000001e0a001 CR4: 00000000003626e0
Apr 21 22:42:19 Tower kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 21 22:42:19 Tower kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

 

IMG_20200421_222745.jpg

Link to comment

So i just ran a "Check Filesystem Status" in maint mode and got this. 
I had the same error on my old cache (4 SSDs in RAID10) and replaced all the disks and rebuilt it. What's happening?
 

[1/7] checking root items
[2/7] checking extents
ref mismatch on [1721715888128 4096] extent item 72057080919784639, found 1
incorrect local backref count on 1721715904512 root 5 owner 405599 offset 325214208 found 1 wanted 1476395009 back 0x8be1630
backpointer mismatch on [1721715904512 4096]
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
Opening filesystem to check...
Checking filesystem on /dev/sdb1
UUID: 226c5fe0-1cd9-4939-babd-09067f2aa381
found 880229638144 bytes used, error(s) found
total csum bytes: 288978308
total tree bytes: 839352320
total fs tree bytes: 423247872
total extent tree bytes: 98713600
btree space waste bytes: 101541917
file data blocks allocated: 20352307535872
 referenced 858553622528

 

Link to comment
2 hours ago, Daggr said:

What's happening?

There's corruption on the cache filesystem, possibly from all the crashing.

 

2 hours ago, Daggr said:

Apr 21 22:42:15 Tower kernel: macvlan_broadcast+0x111/0x156 [macvlan] Apr 21 22:42:15 Tower kernel: macvlan_process_broadcast+0xea/0x128 [macvlan]

Macvlan call traces are usually caused by dockers with a custom IP address, more info here.

Link to comment
7 hours ago, johnnie.black said:

There's corruption on the cache filesystem, possibly from all the crashing.

 

Macvlan call traces are usually caused by dockers with a custom IP address, more info here.

Thank you for answering! 

I have had custom IP addresses for a long time. But i'll check it out. Thanks.

Link to comment
  • 1 month later...

Dont know if it made any difference. Server still freeze from time to time. This time three times in a row. Two first times nothing at all in the logs. Third time btrfs again. I'm giving up Unraid.

 

Jun 4 06:04:52 Tower kernel: ------------[ cut here ]------------
Jun 4 06:04:52 Tower kernel: kernel BUG at fs/btrfs/extent-tree.c:6862!
Jun 4 06:04:52 Tower kernel: invalid opcode: 0000 [#1] SMP NOPTI
Jun 4 06:04:52 Tower kernel: CPU: 5 PID: 9691 Comm: worker Tainted: G O 4.19.107-Unraid #1
Jun 4 06:04:52 Tower kernel: Hardware name: System manufacturer System Product Name/Z170 PRO GAMING, BIOS 3805 05/16/2018
Jun 4 06:04:52 Tower kernel: RIP: 0010:__btrfs_free_extent+0x558/0x90f
Jun 4 06:04:52 Tower kernel: Code: ba c8 1a 00 00 e9 36 fd ff ff 48 81 3c 24 ff 00 00 00 76 18 48 8b 74 24 48 48 89 ef e8 a8 79 ff ff 3b 84 24 b8 00 00 00 74 02 <0f> 0b 48 83 7c 24 48 00 8b 45 40 74 0c 39 44 24 08 0f 84 0c ff ff
Jun 4 06:04:52 Tower kernel: RSP: 0018:ffffc90003c0fc20 EFLAGS: 00010206
Jun 4 06:04:52 Tower kernel: RAX: 0000000040000001 RBX: 0000000000000001 RCX: ffff888062cc5000
Jun 4 06:04:52 Tower kernel: RDX: 0000000000002000 RSI: 0000000000000002 RDI: ffff88805aaa34c8
Jun 4 06:04:52 Tower kernel: RBP: ffff88805e74c8c0 R08: ffffc90003c0fba0 R09: ffffc90003c0fba8
Jun 4 06:04:52 Tower kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Jun 4 06:04:52 Tower kernel: R13: ffff8887ccd172d8 R14: ffff88805aaa34c8 R15: 00000190ea53a000
Jun 4 06:04:52 Tower kernel: FS: 00001514028fd700(0000) GS:ffff888816b40000(0000) knlGS:0000000000000000
Jun 4 06:04:52 Tower kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 4 06:04:52 Tower kernel: CR2: 0000155369e8b020 CR3: 000000079d8e2005 CR4: 00000000003626e0
Jun 4 06:04:52 Tower kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jun 4 06:04:52 Tower kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Jun 4 06:04:52 Tower kernel: Call Trace:
Jun 4 06:04:52 Tower kernel: __btrfs_run_delayed_refs+0xa77/0xbf4
Jun 4 06:04:52 Tower kernel: btrfs_run_delayed_refs+0x5d/0x16d
Jun 4 06:04:52 Tower kernel: btrfs_commit_transaction+0x54/0x79c
Jun 4 06:04:52 Tower kernel: ? dput+0x26/0x10e
Jun 4 06:04:52 Tower kernel: ? btrfs_log_dentry_safe+0x52/0x65
Jun 4 06:04:52 Tower kernel: btrfs_sync_file+0x2af/0x2f7
Jun 4 06:04:52 Tower kernel: do_fsync+0x2a/0x44
Jun 4 06:04:52 Tower kernel: __x64_sys_fdatasync+0xe/0x11
Jun 4 06:04:52 Tower kernel: do_syscall_64+0x57/0xf2
Jun 4 06:04:52 Tower kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jun 4 06:04:52 Tower kernel: RIP: 0033:0x15140488522b
Jun 4 06:04:52 Tower kernel: Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 f3 2c f8 ff 8b 7c 24 0c 41 89 c0 b8 4b 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 0c e8 31 2d f8 ff 8b 44
Jun 4 06:04:52 Tower kernel: RSP: 002b:00001514028fc740 EFLAGS: 00000293 ORIG_RAX: 000000000000004b
Jun 4 06:04:52 Tower kernel: RAX: ffffffffffffffda RBX: 0000151402c6e7b0 RCX: 000015140488522b
Jun 4 06:04:52 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000b
Jun 4 06:04:52 Tower kernel: RBP: 0000151402c860f0 R08: 0000000000000000 R09: 00000000ffffffff
Jun 4 06:04:52 Tower kernel: R10: 00001514028fc730 R11: 0000000000000293 R12: 000055edd9de5a18
Jun 4 06:04:52 Tower kernel: R13: 0000151402c86158 R14: 0000151402c6ef90 R15: 0000151402c860e0
Jun 4 06:04:52 Tower kernel: Modules linked in: veth nfsv3 nfs arc4 ecb md4 sha512_ssse3 sha512_generic cmac cifs ccm macvlan xt_CHECKSUM ipt_REJECT ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap xt_nat ipt_MASQUERADE iptable_filter iptable_nat nf_nat_ipv4 nf_nat ip_tables xfs nfsd lockd grace sunrpc md_mod ipmi_devintf nct6775 hwmon_vid e1000e igb(O) wmi_bmof mxm_wmi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd glue_helper intel_cstate intel_uncore intel_rapl_perf i2c_i801 i2c_core mpt3sas cdc_acm ahci libahci raid_class scsi_transport_sas wmi video pcc_cpufreq backlight button fan thermal acpi_pad [last unloaded: e1000e]
Jun 4 06:04:52 Tower kernel: ---[ end trace ccf65c4091aee82e ]---
Jun 4 06:04:52 Tower kernel: RIP: 0010:__btrfs_free_extent+0x558/0x90f
Jun 4 06:04:52 Tower kernel: Code: ba c8 1a 00 00 e9 36 fd ff ff 48 81 3c 24 ff 00 00 00 76 18 48 8b 74 24 48 48 89 ef e8 a8 79 ff ff 3b 84 24 b8 00 00 00 74 02 <0f> 0b 48 83 7c 24 48 00 8b 45 40 74 0c 39 44 24 08 0f 84 0c ff ff
Jun 4 06:04:52 Tower kernel: RSP: 0018:ffffc90003c0fc20 EFLAGS: 00010206
Jun 4 06:04:52 Tower kernel: RAX: 0000000040000001 RBX: 0000000000000001 RCX: ffff888062cc5000
Jun 4 06:04:52 Tower kernel: RDX: 0000000000002000 RSI: 0000000000000002 RDI: ffff88805aaa34c8
Jun 4 06:04:52 Tower kernel: RBP: ffff88805e74c8c0 R08: ffffc90003c0fba0 R09: ffffc90003c0fba8
Jun 4 06:04:52 Tower kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Jun 4 06:04:52 Tower kernel: R13: ffff8887ccd172d8 R14: ffff88805aaa34c8 R15: 00000190ea53a000
Jun 4 06:04:52 Tower kernel: FS: 00001514028fd700(0000) GS:ffff888816b40000(0000) knlGS:0000000000000000
Jun 4 06:04:52 Tower kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 4 06:04:52 Tower kernel: CR2: 0000155369e8b020 CR3: 000000079d8e2005 CR4: 00000000003626e0
Jun 4 06:04:52 Tower kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jun 4 06:04:52 Tower kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

 

Edited by Daggr
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.