Ok Thanks, here are the errors I was able to capture, the entire system would hard lock so I could not get the logs or anything. I did catch it mid-crash once and was able to extract this from the log before it died completely, this is actually what led me to suspect tartube was the issue due to the next image:
ErrorWarningSystemArrayLogin
Jan 16 07:29:58 NAS kernel: asm_call_irq_on_stack+0x12/0x20
Jan 16 07:29:58 NAS kernel: </IRQ>
Jan 16 07:29:58 NAS kernel: sysvec_apic_timer_interrupt+0x71/0x95
Jan 16 07:29:58 NAS kernel: asm_sysvec_apic_timer_interrupt+0x12/0x20
Jan 16 07:29:58 NAS kernel: RIP: 0010:xas_descend+0x1e/0x49
Jan 16 07:29:58 NAS kernel: Code: 08 48 d3 e6 48 01 f0 48 89 47 08 c3 0f b6 0e 49 89 f9 48 8b 47 08 48 d3 e8 48 89 c1 83 e1 3f 89 c8 48 83 c0 04 4c 8b 44 c6 08 <48> 89 77 18 4c 89 c7 e8 77 ff ff ff 84 c0 74 13 49 c1 e8 02 44 89
Jan 16 07:29:58 NAS kernel: RSP: 0018:ffffc90034a67a38 EFLAGS: 00000216
Jan 16 07:29:58 NAS kernel: RAX: 0000000000000021 RBX: 0000000000000007 RCX: 000000000000001d
Jan 16 07:29:58 NAS kernel: RDX: 0000000000000001 RSI: ffff88a28bc6c490 RDI: ffffc90034a67a48
Jan 16 07:29:58 NAS kernel: RBP: ffff88923c7c3358 R08: ffff88821d8eb688 R09: ffffc90034a67a48
Jan 16 07:29:58 NAS kernel: R10: ffffc90034a67a48 R11: 000000000000000e R12: ffff88821d8eb688
Jan 16 07:29:58 NAS kernel: R13: ffff88923c7c34c8 R14: 000000000001c05d R15: ffff88a64cb1ea68
Jan 16 07:29:58 NAS kernel: ? xas_descend+0x2a/0x49
Jan 16 07:29:58 NAS kernel: xas_load+0x2d/0x39
Jan 16 07:29:58 NAS kernel: find_get_entry+0x57/0xba
Jan 16 07:29:58 NAS kernel: pagecache_get_page+0x20/0x13e
Jan 16 07:29:58 NAS kernel: prepare_pages+0x73/0x133
Jan 16 07:29:58 NAS kernel: btrfs_buffered_write+0x285/0x5a2
Jan 16 07:29:58 NAS kernel: ? update_load_avg+0x39/0x2c4
Jan 16 07:29:58 NAS kernel: btrfs_file_write_iter+0x3cc/0x4b1
Jan 16 07:29:58 NAS kernel: do_iter_readv_writev+0x97/0xd7
Jan 16 07:29:58 NAS kernel: do_iter_write+0x7f/0xc0
Jan 16 07:29:58 NAS kernel: iter_file_splice_write+0x215/0x313
Jan 16 07:29:58 NAS kernel: direct_splice_actor+0x2e/0x2f
Jan 16 07:29:58 NAS kernel: splice_direct_to_actor+0x125/0x1cd
Jan 16 07:29:58 NAS kernel: ? generic_file_splice_read+0x15e/0x15e
Jan 16 07:29:58 NAS kernel: do_splice_direct+0x94/0xbd
Jan 16 07:29:58 NAS kernel: do_sendfile+0x185/0x24f
Jan 16 07:29:58 NAS kernel: __do_sys_sendfile64+0x61/0xa7
Jan 16 07:29:58 NAS kernel: do_syscall_64+0x5d/0x6a
Jan 16 07:29:58 NAS kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 16 07:29:58 NAS kernel: RIP: 0033:0x7efe3948e68e
Jan 16 07:29:58 NAS kernel: Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 bd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d2 a7 0c 00 f7 d8 64 89 01 48
Jan 16 07:29:58 NAS kernel: RSP: 002b:00007ffe62eb1b68 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
Jan 16 07:29:58 NAS kernel: RAX: ffffffffffffffda RBX: 00007ffe62eb1c20 RCX: 00007efe3948e68e
Jan 16 07:29:58 NAS kernel: RDX: 00007ffe62eb1c40 RSI: 0000000000000005 RDI: 0000000000000007
Jan 16 07:29:58 NAS kernel: RBP: 00007ffe62f31fb0 R08: 0000000000000000 R09: 000000000008871d
Jan 16 07:29:58 NAS kernel: R10: 0000000000080000 R11: 0000000000000246 R12: 00007ffe62eb1c10
Jan 16 07:29:58 NAS kernel: R13: 00007ffe62eb1c40 R14: 00007ffe62eb1bd8 R15: 00007ffe62eb1bd0
Jan 16 07:32:58 NAS kernel: rcu: INFO: rcu_sched self-detected stall on CPU
Jan 16 07:32:58 NAS kernel: rcu: 36-....: (420005 ticks this GP) idle=692/1/0x4000000000000000 softirq=29122084/29122084 fqs=104638
Jan 16 07:32:58 NAS kernel: (t=420006 jiffies g=214476629 q=4370531)
Jan 16 07:32:58 NAS kernel: NMI backtrace for cpu 36
Jan 16 07:32:58 NAS kernel: CPU: 36 PID: 4502 Comm: file.so Tainted: G O 5.10.28-Unraid #1
Jan 16 07:32:58 NAS kernel: Hardware name: Supermicro X9DAX/X9DAX, BIOS 3.2a 06/30/2015
Jan 16 07:32:58 NAS kernel: Call Trace:
Jan 16 07:32:58 NAS kernel: <IRQ>
Jan 16 07:32:58 NAS kernel: dump_stack+0x6b/0x83
Jan 16 07:32:58 NAS kernel: ? lapic_can_unplug_cpu+0x8e/0x8e
Jan 16 07:32:58 NAS kernel: nmi_cpu_backtrace+0x7d/0x8f
Jan 16 07:32:58 NAS kernel: nmi_trigger_cpumask_backtrace+0x56/0xd3
Jan 16 07:32:58 NAS kernel: rcu_dump_cpu_stacks+0x9f/0xc6
Jan 16 07:32:58 NAS kernel: rcu_sched_clock_irq+0x1ec/0x543
Jan 16 07:32:58 NAS kernel: ? _raw_spin_unlock_irqrestore+0xd/0xe
Jan 16 07:32:58 NAS kernel: update_process_times+0x50/0x6e
Jan 16 07:32:58 NAS kernel: tick_sched_timer+0x36/0x64
Jan 16 07:32:58 NAS kernel: __hrtimer_run_queues+0xb7/0x10b
Jan 16 07:32:58 NAS kernel: ? tick_sched_do_timer+0x39/0x39
Jan 16 07:32:58 NAS kernel: hrtimer_interrupt+0x8d/0x15b
Jan 16 07:32:58 NAS kernel: __sysvec_apic_timer_interrupt+0x5d/0x68
Jan 16 07:32:58 NAS kernel: asm_call_irq_on_stack+0x12/0x20
Jan 16 07:32:58 NAS kernel: </IRQ>
Jan 16 07:32:58 NAS kernel: sysvec_apic_timer_interrupt+0x71/0x95
Jan 16 07:32:58 NAS kernel: asm_sysvec_apic_timer_interrupt+0x12/0x20
Jan 16 07:32:58 NAS kernel: RIP: 0010:__page_cache_add_speculative.constprop.0+0x17/0x1f
Jan 16 07:32:58 NAS kernel: Code: 00 00 48 89 ef e8 7a fd ff ff 48 89 ef 5d e9 87 fa ff ff 8b 57 34 85 d2 74 10 8d 4a 01 89 d0 f0 0f b1 4f 34 74 04 89 c2 eb ec <31> c0 85 d2 0f 95 c0 c3 41 54 65 48 8b 04 25 c0 7b 01 00 55 89 fd
Jan 16 07:32:58 NAS kernel: RSP: 0018:ffffc90034a67a40 EFLAGS: 00000246
Jan 16 07:32:58 NAS kernel: RAX: ffff88821d8eb688 RBX: 0000000000000007 RCX: 000000000000001d
Jan 16 07:32:58 NAS kernel: RDX: 0000000000000000 RSI: ffff88a28bc6c490 RDI: ffff88821d8eb688
Jan 16 07:32:58 NAS kernel: RBP: ffff88923c7c3358 R08: ffff88821d8eb688 R09: ffffc90034a67a48
Jan 16 07:32:58 NAS kernel: R10: ffffc90034a67a48 R11: 000000000000000e R12: ffff88821d8eb688
Jan 16 07:32:58 NAS kernel: R13: ffff88923c7c34c8 R14: 000000000001c05d R15: ffff88a64cb1ea68
Jan 16 07:32:58 NAS kernel: find_get_entry+0x7b/0xba
Jan 16 07:32:58 NAS kernel: pagecache_get_page+0x20/0x13e
Jan 16 07:32:58 NAS kernel: prepare_pages+0x73/0x133
Jan 16 07:32:58 NAS kernel: btrfs_buffered_write+0x285/0x5a2
Jan 16 07:32:58 NAS kernel: ? update_load_avg+0x39/0x2c4
Jan 16 07:32:58 NAS kernel: btrfs_file_write_iter+0x3cc/0x4b1
Jan 16 07:32:58 NAS kernel: do_iter_readv_writev+0x97/0xd7
Jan 16 07:32:58 NAS kernel: do_iter_write+0x7f/0xc0
Jan 16 07:32:58 NAS kernel: iter_file_splice_write+0x215/0x313
Jan 16 07:32:58 NAS kernel: direct_splice_actor+0x2e/0x2f
Jan 16 07:32:58 NAS kernel: splice_direct_to_actor+0x125/0x1cd
Jan 16 07:32:58 NAS kernel: ? generic_file_splice_read+0x15e/0x15e
Jan 16 07:32:58 NAS kernel: do_splice_direct+0x94/0xbd
Jan 16 07:32:58 NAS kernel: do_sendfile+0x185/0x24f
Jan 16 07:32:58 NAS kernel: __do_sys_sendfile64+0x61/0xa7
Jan 16 07:32:58 NAS kernel: do_syscall_64+0x5d/0x6a
Jan 16 07:32:58 NAS kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 16 07:32:58 NAS kernel: RIP: 0033:0x7efe3948e68e
Jan 16 07:32:58 NAS kernel: Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 bd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d2 a7 0c 00 f7 d8 64 89 01 48
Jan 16 07:32:58 NAS kernel: RSP: 002b:00007ffe62eb1b68 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
Jan 16 07:32:58 NAS kernel: RAX: ffffffffffffffda RBX: 00007ffe62eb1c20 RCX: 00007efe3948e68e
Jan 16 07:32:58 NAS kernel: RDX: 00007ffe62eb1c40 RSI: 0000000000000005 RDI: 0000000000000007
Jan 16 07:32:58 NAS kernel: RBP: 00007ffe62f31fb0 R08: 0000000000000000 R09: 000000000008871d
Jan 16 07:32:58 NAS kernel: R10: 0000000000080000 R11: 0000000000000246 R12: 00007ffe62eb1c10
Jan 16 07:32:58 NAS kernel: R13: 00007ffe62eb1c40 R14: 00007ffe62eb1bd8 R15: 00007ffe62eb1bd0
Jan 16 07:33:04 NAS nginx: 2022/01/16 07:33:04 [error] 38735#38735: *1873274 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.1.20, server: , request: "GET /favicon.ico HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "nas.local", referrer: "https://nas.local/logging.htm"
Jan 16 07:35:58 NAS kernel: rcu: INFO: rcu_sched self-detected stall on CPU
Jan 16 07:35:58 NAS kernel: rcu: 36-....: (600008 ticks this GP) idle=692/1/0x4000000000000000 softirq=29122084/29122084 fqs=149437
Jan 16 07:35:58 NAS kernel: (t=600009 jiffies g=214476629 q=6281381)
Jan 16 07:35:58 NAS kernel: NMI backtrace for cpu 36
Jan 16 07:35:58 NAS kernel: CPU: 36 PID: 4502 Comm: file.so Tainted: G O 5.10.28-Unraid #1
Jan 16 07:35:58 NAS kernel: Hardware name: Supermicro X9DAX/X9DAX, BIOS 3.2a 06/30/2015
Jan 16 07:35:58 NAS kernel: Call Trace:
Jan 16 07:35:58 NAS kernel: <IRQ>
Jan 16 07:35:58 NAS kernel: dump_stack+0x6b/0x83
Jan 16 07:35:58 NAS kernel: ? lapic_can_unplug_cpu+0x8e/0x8e
Jan 16 07:35:58 NAS kernel: nmi_cpu_backtrace+0x7d/0x8f
Jan 16 07:35:58 NAS kernel: nmi_trigger_cpumask_backtrace+0x56/0xd3
Jan 16 07:35:58 NAS kernel: rcu_dump_cpu_stacks+0x9f/0xc6
Jan 16 07:35:58 NAS kernel: rcu_sched_clock_irq+0x1ec/0x543
Jan 16 07:35:58 NAS kernel: ? _raw_spin_unlock_irqrestore+0xd/0xe
Jan 16 07:35:58 NAS kernel: update_process_times+0x50/0x6e
Jan 16 07:35:58 NAS kernel: tick_sched_timer+0x36/0x64
Jan 16 07:35:58 NAS kernel: __hrtimer_run_queues+0xb7/0x10b
Jan 16 07:35:58 NAS kernel: ? tick_sched_do_timer+0x39/0x39
Jan 16 07:35:58 NAS kernel: hrtimer_interrupt+0x8d/0x15b
Jan 16 07:35:58 NAS kernel: __sysvec_apic_timer_interrupt+0x5d/0x68
Jan 16 07:35:58 NAS kernel: asm_call_irq_on_stack+0x12/0x20
Jan 16 07:35:58 NAS kernel: </IRQ>
Jan 16 07:35:58 NAS kernel: sysvec_apic_timer_interrupt+0x71/0x95
Jan 16 07:35:58 NAS kernel: asm_sysvec_apic_timer_interrupt+0x12/0x20
Jan 16 07:35:58 NAS kernel: RIP: 0010:find_get_entry+0x62/0xba
Jan 16 07:35:58 NAS kernel: Code: 20 00 00 00 00 48 c7 44 24 28 00 00 00 00 48 c7 44 24 18 03 00 00 00 48 89 e7 e8 8d c0 26 00 48 3d 06 04 00 00 49 89 c4 74 e4 <48> 3d 02 04 00 00 74 dc 48 85 c0 74 27 a8 01 75 23 48 89 c7 e8 b8
Jan 16 07:35:58 NAS kernel: RSP: 0018:ffffc90034a67a48 EFLAGS: 00000286
Jan 16 07:35:58 NAS kernel: RAX: ffff88821d8eb688 RBX: 0000000000000007 RCX: 000000000000001d
Jan 16 07:35:58 NAS kernel: RDX: 0000000000000000 RSI: ffff88a28bc6c490 RDI: ffff88821d8eb688
Jan 16 07:35:58 NAS kernel: RBP: ffff88923c7c3358 R08: ffff88821d8eb688 R09: ffffc90034a67a48
Jan 16 07:35:58 NAS kernel: R10: ffffc90034a67a48 R11: 000000000000000e R12: ffff88821d8eb688
Jan 16 07:35:58 NAS kernel: R13: ffff88923c7c34c8 R14: 000000000001c05d R15: ffff88a64cb1ea68
Jan 16 07:35:58 NAS kernel: ? find_get_entry+0x57/0xba
Jan 16 07:35:58 NAS kernel: pagecache_get_page+0x20/0x13e
Jan 16 07:35:58 NAS kernel: prepare_pages+0x73/0x133
Jan 16 07:35:58 NAS kernel: btrfs_buffered_write+0x285/0x5a2
Jan 16 07:35:58 NAS kernel: ? update_load_avg+0x39/0x2c4
Jan 16 07:35:58 NAS kernel: btrfs_file_write_iter+0x3cc/0x4b1
Jan 16 07:35:58 NAS kernel: do_iter_readv_writev+0x97/0xd7
Jan 16 07:35:58 NAS kernel: do_iter_write+0x7f/0xc0
Jan 16 07:35:58 NAS kernel: iter_file_splice_write+0x215/0x313
Jan 16 07:35:58 NAS kernel: direct_splice_actor+0x2e/0x2f
Jan 16 07:35:58 NAS kernel: splice_direct_to_actor+0x125/0x1cd
Jan 16 07:35:58 NAS kernel: ? generic_file_splice_read+0x15e/0x15e
Jan 16 07:35:58 NAS kernel: do_splice_direct+0x94/0xbd
Jan 16 07:35:58 NAS kernel: do_sendfile+0x185/0x24f
Jan 16 07:35:58 NAS kernel: __do_sys_sendfile64+0x61/0xa7
Jan 16 07:35:58 NAS kernel: do_syscall_64+0x5d/0x6a
Jan 16 07:35:58 NAS kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 16 07:35:58 NAS kernel: RIP: 0033:0x7efe3948e68e
Jan 16 07:35:58 NAS kernel: Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 bd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d2 a7 0c 00 f7 d8 64 89 01 48
Jan 16 07:35:58 NAS kernel: RSP: 002b:00007ffe62eb1b68 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
Jan 16 07:35:58 NAS kernel: RAX: ffffffffffffffda RBX: 00007ffe62eb1c20 RCX: 00007efe3948e68e
Jan 16 07:35:58 NAS kernel: RDX: 00007ffe62eb1c40 RSI: 0000000000000005 RDI: 0000000000000007
Jan 16 07:35:58 NAS kernel: RBP: 00007ffe62f31fb0 R08: 0000000000000000 R09: 000000000008871d
Jan 16 07:35:58 NAS kernel: R10: 0000000000080000 R11: 0000000000000246 R12: 00007ffe62eb1c10
Jan 16 07:35:58 NAS kernel: R13: 00007ffe62eb1c40 R14: 00007ffe62eb1bd8 R15: 00007ffe62eb1bd0
Jan 16 07:37:14 NAS nginx: 2022/01/16 07:37:14 [error] 38735#38735: *1874493 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.1.20, server: , request: "GET /Dashboard HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "nas.local", referrer: "https://nas.local/script"
Jan 16 07:38:58 NAS kernel: rcu: INFO: rcu_sched self-detected stall on CPU
Jan 16 07:38:58 NAS kernel: rcu: 36-....: (780011 ticks this GP) idle=692/1/0x4000000000000000 softirq=29122084/29122084 fqs=194300
Jan 16 07:38:58 NAS kernel: (t=780012 jiffies g=214476629 q=8153649)
Jan 16 07:38:58 NAS kernel: NMI backtrace for cpu 36
Jan 16 07:38:58 NAS kernel: CPU: 36 PID: 4502 Comm: file.so Tainted: G O 5.10.28-Unraid #1
Jan 16 07:38:58 NAS kernel: Hardware name: Supermicro X9DAX/X9DAX, BIOS 3.2a 06/30/2015
Jan 16 07:38:58 NAS kernel: Call Trace:
Jan 16 07:38:58 NAS kernel: <IRQ>
Jan 16 07:38:58 NAS kernel: dump_stack+0x6b/0x83
Jan 16 07:38:58 NAS kernel: ? lapic_can_unplug_cpu+0x8e/0x8e
Jan 16 07:38:58 NAS kernel: nmi_cpu_backtrace+0x7d/0x8f
Jan 16 07:38:58 NAS kernel: nmi_trigger_cpumask_backtrace+0x56/0xd3
Jan 16 07:38:58 NAS kernel: rcu_dump_cpu_stacks+0x9f/0xc6
Jan 16 07:38:58 NAS kernel: rcu_sched_clock_irq+0x1ec/0x543
Jan 16 07:38:58 NAS kernel: ? _raw_spin_unlock_irqrestore+0xd/0xe
Jan 16 07:38:58 NAS kernel: update_process_times+0x50/0x6e
Jan 16 07:38:58 NAS kernel: tick_sched_timer+0x36/0x64
Jan 16 07:38:58 NAS kernel: __hrtimer_run_queues+0xb7/0x10b
Jan 16 07:38:58 NAS kernel: ? tick_sched_do_timer+0x39/0x39
Jan 16 07:38:58 NAS kernel: hrtimer_interrupt+0x8d/0x15b
Jan 16 07:38:58 NAS kernel: __sysvec_apic_timer_interrupt+0x5d/0x68
Jan 16 07:38:58 NAS kernel: asm_call_irq_on_stack+0x12/0x20
Jan 16 07:38:58 NAS kernel: </IRQ>
Jan 16 07:38:58 NAS kernel: sysvec_apic_timer_interrupt+0x71/0x95
Jan 16 07:38:58 NAS kernel: asm_sysvec_apic_timer_interrupt+0x12/0x20
Jan 16 07:38:58 NAS kernel: RIP: 0010:xa_is_sibling+0x6/0x1a
Jan 16 07:38:58 NAS kernel: Code: 10 c3 c6 07 00 0f 1f 40 00 c3 48 89 f8 83 e0 03 48 83 f8 02 0f 94 c0 48 81 ff 00 10 00 00 0f 97 c2 21 d0 c3 48 89 f8 83 e0 03 <48> 83 f8 02 0f 94 c0 48 81 ff fd 00 00 00 0f 96 c2 21 d0 c3 40 f6
Jan 16 07:38:58 NAS kernel: RSP: 0018:ffffc90034a67a30 EFLAGS: 00000202
Jan 16 07:38:58 NAS kernel: RAX: 0000000000000002 RBX: 0000000000000007 RCX: 000000000000001c
Jan 16 07:38:58 NAS kernel: RDX: 0000000000000001 RSI: ffff88a3f19fc490 RDI: ffff88a28bc6e482
Jan 16 07:38:58 NAS kernel: RBP: ffff88923c7c3358 R08: ffff88a28bc6e482 R09: ffffc90034a67a48
Jan 16 07:38:58 NAS kernel: R10: ffffc90034a67a48 R11: 000000000000000e R12: ffff88821d8eb688
Jan 16 07:38:58 NAS kernel: R13: ffff88923c7c34c8 R14: 000000000001c05d R15: ffff88a64cb1ea68
Jan 16 07:38:58 NAS kernel: xas_descend+0x2a/0x49
Jan 16 07:38:58 NAS kernel: xas_load+0x2d/0x39
Jan 16 07:38:58 NAS kernel: find_get_entry+0x57/0xba
Jan 16 07:38:58 NAS kernel: pagecache_get_page+0x20/0x13e
Jan 16 07:38:58 NAS kernel: prepare_pages+0x73/0x133
Jan 16 07:38:58 NAS kernel: btrfs_buffered_write+0x285/0x5a2
Jan 16 07:38:58 NAS kernel: ? update_load_avg+0x39/0x2c4
Jan 16 07:38:58 NAS kernel: btrfs_file_write_iter+0x3cc/0x4b1
Jan 16 07:38:58 NAS kernel: do_iter_readv_writev+0x97/0xd7
Jan 16 07:38:58 NAS kernel: do_iter_write+0x7f/0xc0
Jan 16 07:38:58 NAS kernel: iter_file_splice_write+0x215/0x313
Jan 16 07:38:58 NAS kernel: direct_splice_actor+0x2e/0x2f
Jan 16 07:38:58 NAS kernel: splice_direct_to_actor+0x125/0x1cd
Jan 16 07:38:58 NAS kernel: ? generic_file_splice_read+0x15e/0x15e
Jan 16 07:38:58 NAS kernel: do_splice_direct+0x94/0xbd
Jan 16 07:38:58 NAS kernel: do_sendfile+0x185/0x24f
Jan 16 07:38:58 NAS kernel: __do_sys_sendfile64+0x61/0xa7
Jan 16 07:38:58 NAS kernel: do_syscall_64+0x5d/0x6a
Jan 16 07:38:58 NAS kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jan 16 07:38:58 NAS kernel: RIP: 0033:0x7efe3948e68e
Jan 16 07:38:58 NAS kernel: Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 bd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d2 a7 0c 00 f7 d8 64 89 01 48
Jan 16 07:38:58 NAS kernel: RSP: 002b:00007ffe62eb1b68 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
Jan 16 07:38:58 NAS kernel: RAX: ffffffffffffffda RBX: 00007ffe62eb1c20 RCX: 00007efe3948e68e
Jan 16 07:38:58 NAS kernel: RDX: 00007ffe62eb1c40 RSI: 0000000000000005 RDI: 0000000000000007
Jan 16 07:38:58 NAS kernel: RBP: 00007ffe62f31fb0 R08: 0000000000000000 R09: 000000000008871d
Jan 16 07:38:58 NAS kernel: R10: 0000000000080000 R11: 0000000000000246 R12: 00007ffe62eb1c10
Jan 16 07:38:58 NAS kernel: R13: 00007ffe62eb1c40 R14: 00007ffe62eb1bd8 R15: 00007ffe62eb1bd0
Jan 16 07:39:09 NAS shutdown[8721]: shutting down for system halt
Jan 16 07:39:09 NAS init: Switching to runlevel: 0
Jan 16 07:39:09 NAS init: Trying to re-exec init
Jan 16 07:39:10 NAS root: Starting Run at Shutdown script
Jan 16 07:39:10 NAS root: Stopping Dockers
Jan 16 07:40:14 NAS kernel: rcu: INFO: rcu_sched detected expedited stalls on CPUs/tasks: { 36-... } 63968 jiffies s: 2909 root: 0x4/.
Jan 16 07:40:14 NAS kernel: rcu: blocking rcu_node structures: l=1:28-39:0x100/.
Jan 16 07:40:14 NAS kernel: Task dump for CPU 36:
Jan 16 07:40:14 NAS kernel: task:file.so state:R running task stack: 0 pid: 4502 ppid: 11242 flags:0x0000000e
Jan 16 07:40:14 NAS kernel: Call Trace:
Jan 16 07:40:14 NAS kernel: ? do_iter_readv_writev+0x97/0xd7
Jan 16 07:40:14 NAS kernel: ? do_iter_write+0x7f/0xc0
Jan 16 07:40:14 NAS kernel: ? iter_file_splice_write+0x215/0x313
Jan 16 07:40:14 NAS kernel: ? direct_splice_actor+0x2e/0x2f
Jan 16 07:40:14 NAS kernel: ? splice_direct_to_actor+0x125/0x1cd
Jan 16 07:40:14 NAS kernel: ? generic_file_splice_read+0x15e/0x15e
Jan 16 07:40:14 NAS kernel: ? do_splice_direct+0x94/0xbd
Jan 16 07:40:14 NAS kernel: ? do_sendfile+0x185/0x24f
Jan 16 07:40:14 NAS kernel: ? __do_sys_sendfile64+0x61/0xa7
Jan 16 07:40:14 NAS kernel: ? do_syscall_64+0x5d/0x6a
Jan 16 07:40:14 NAS kernel: ? entry_SYSCALL_64_after_hwframe+0x44/0xa9
The rest are just screen shots of the IPMI screen after the crash, they are all pretty simular just with slight variations of the errors. Most of them mention kernel panic and sync errors.
I have more if you think they would help.