I am having regular crashes. I was on 6.12 but I have downgraded to 6.11.5 which did not work. 

 

In Netdata, I can see that CPU and Memory utilization both climb and sustain right before each crash. Per netdata, it is NOT from any dockers utilization.  In order to minimize variables, I have also disabled all dockers except for netdata and plex. (All the arr's, downloaders, etc turned off)

Once the problem starts, if I happen to notice it in time, the Webui remains responsive, and sometimes even whatever dockers are on. But then eventually it crashes.  If I catch it early enough, I can do a reboot from the gui. If I don't catch it early enough, I have to power cycle the NAS. 

 

The "system" seems to be spawning processes and taking up CPU before the crash, but I don't know what the actual process is. 

 

Currently the box is only running on 4gb of RAM, which is tight, because I removed a 16gb chip to diagnose possible hardware issues. It did not work. 

 

The crashes DO appear to be time based, prefering to crash right around midnight, or 4 am. 4 am is the mover time, which I tried disabling, and it did not make a difference.  To my knowledge nothing is scheduled for midnight.  It could also NOT be o'clock based, but uptime based, as when it crashes I reboot it in the morning, and so it could be something happening X hours after boot. 

 

 

 

Something is spawning a lot of processes

 

 

Notably plex and netdata had low cpu utilization during the window. 

tower-diagnostics-20231127-0854.zip

The interesting part of the syslog

 

Nov 26 23:11:46 Tower nginx: 2023/11/26 23:11:46 [error] 2653#2653: *150636 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 192.168.50.205, server: 192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net, request: "GET /login/plugins/dynamix.vm.manager/include/VMMachines.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net:444"
Nov 26 23:29:05 Tower nginx: 2023/11/26 23:29:05 [error] 2653#2653: *160807 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 192.168.50.205, server: 192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net, request: "GET /login/plugins/dynamix.docker.manager/include/DockerContainers.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net:444"
Nov 26 23:29:05 Tower nginx: 2023/11/26 23:29:05 [error] 2653#2653: *160807 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 192.168.50.205, server: 192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net, request: "GET /login/plugins/dynamix.vm.manager/include/VMMachines.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net:444"
Nov 26 23:29:05 Tower nginx: 2023/11/26 23:29:05 [error] 2653#2653: *160807 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 192.168.50.205, server: 192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net, request: "GET /login/webGui/include/ShareList.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net:444"
Nov 26 23:29:05 Tower nginx: 2023/11/26 23:29:05 [error] 2653#2653: *160807 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 192.168.50.205, server: 192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net, request: "GET /login/plugins/dynamix.apcupsd/include/UPSstatus.php?all=false HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192-168-50-106.e47e8f1e894e2bec274a8bf7ba93b1d2b891ade5.myunraid.net:444"
Nov 27 00:23:01 Tower kernel: general protection fault, probably for non-canonical address 0xffff0181001cef00: 0000 [#1] PREEMPT SMP NOPTI
Nov 27 00:23:01 Tower kernel: CPU: 3 PID: 7109 Comm: monitor Tainted: G     U            5.19.17-Unraid #2
Nov 27 00:23:01 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:23:01 Tower kernel: RIP: 0010:folio_memcg_lock+0x2e/0x88
Nov 27 00:23:01 Tower kernel: Code: 00 41 55 41 54 49 89 fc 55 53 e8 93 0d eb ff e8 a5 b3 ff ff 84 c0 75 61 49 8b 7c 24 38 e8 51 cc ff ff 48 85 c0 48 89 c3 74 4f <8b> 80 00 0f 00 00 85 c0 7e 45 4c 8d ab 40 04 00 00 4c 89 ef e8 9a
Nov 27 00:23:01 Tower kernel: RSP: 0018:ffffc90002bd3c70 EFLAGS: 00010286
Nov 27 00:23:01 Tower kernel: RAX: ffff0181001ce000 RBX: ffff0181001ce000 RCX: ffffea0005d628c0
Nov 27 00:23:01 Tower kernel: RDX: ffff8880050a1d80 RSI: ffff888012606f00 RDI: ffff0181001ce000
Nov 27 00:23:01 Tower kernel: RBP: ffff888012606f00 R08: ffff88817c1b0c80 R09: ffff88817c1b0c90
Nov 27 00:23:01 Tower kernel: R10: ffff88804867b460 R11: 0000000000000000 R12: ffffea0005d628c0
Nov 27 00:23:01 Tower kernel: R13: 00000001758a3025 R14: 0000000000000000 R15: ffff888012606f00
Nov 27 00:23:01 Tower kernel: FS:  0000000000000000(0000) GS:ffff88817c180000(0000) knlGS:0000000000000000
Nov 27 00:23:01 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 27 00:23:01 Tower kernel: CR2: 0000148a476821e0 CR3: 000000011f40c000 CR4: 0000000000350ee0
Nov 27 00:23:01 Tower kernel: Call Trace:
Nov 27 00:23:01 Tower kernel: <TASK>
Nov 27 00:23:01 Tower kernel: page_remove_rmap+0x1b/0x239
Nov 27 00:23:01 Tower kernel: unmap_page_range+0x451/0x66e
Nov 27 00:23:01 Tower kernel: unmap_vmas+0x87/0xbb
Nov 27 00:23:01 Tower kernel: exit_mmap+0xdc/0x15a
Nov 27 00:23:01 Tower kernel: __mmput+0x43/0xdb
Nov 27 00:23:01 Tower kernel: do_exit+0x30b/0x8e5
Nov 27 00:23:01 Tower kernel: do_group_exit+0x8f/0x8f
Nov 27 00:23:01 Tower kernel: __x64_sys_exit_group+0x14/0x14
Nov 27 00:23:01 Tower kernel: do_syscall_64+0x68/0x81
Nov 27 00:23:01 Tower kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd
Nov 27 00:23:01 Tower kernel: RIP: 0033:0x148a4b3ec7c9
Nov 27 00:23:01 Tower kernel: Code: Unable to access opcode bytes at RIP 0x148a4b3ec79f.
Nov 27 00:23:01 Tower kernel: RSP: 002b:00007ffc8930ba98 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
Nov 27 00:23:01 Tower kernel: RAX: ffffffffffffffda RBX: 0000148a4b4f49e0 RCX: 0000148a4b3ec7c9
Nov 27 00:23:01 Tower kernel: RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
Nov 27 00:23:01 Tower kernel: RBP: 0000000000000000 R08: ffffffffffffff78 R09: 0000148a4b4ffac0
Nov 27 00:23:01 Tower kernel: R10: 0000000000000003 R11: 0000000000000246 R12: 0000148a4b4f49e0
Nov 27 00:23:01 Tower kernel: R13: 0000000000000000 R14: 0000148a4b4fa2c8 R15: 0000148a4b4fa2e0
Nov 27 00:23:01 Tower kernel: </TASK>
Nov 27 00:23:01 Tower kernel: Modules linked in: wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha veth xt_nat xt_tcpudp xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge xfs md_mod efivarfs ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp stp llc r8169 realtek i915 x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel wmi_bmof aesni_intel crypto_simd cryptd intel_cstate ahci nvme i2c_i801 drm i2c_smbus nvme_core libahci intel_gtt agpgart i2c_core processor_thermal_device_pci_legacy processor_thermal_device processor_thermal_rfim processor_thermal_mbox syscopyarea int340x_thermal_zone sysfillrect sysimgblt intel_soc_dts_iosf input_leds
Nov 27 00:23:01 Tower kernel: led_class iosf_mbi fb_sys_fops fan thermal wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_pad button unix [last unloaded: realtek]
Nov 27 00:23:01 Tower kernel: ---[ end trace 0000000000000000 ]---
Nov 27 00:23:01 Tower kernel: RIP: 0010:folio_memcg_lock+0x2e/0x88
Nov 27 00:23:01 Tower kernel: Code: 00 41 55 41 54 49 89 fc 55 53 e8 93 0d eb ff e8 a5 b3 ff ff 84 c0 75 61 49 8b 7c 24 38 e8 51 cc ff ff 48 85 c0 48 89 c3 74 4f <8b> 80 00 0f 00 00 85 c0 7e 45 4c 8d ab 40 04 00 00 4c 89 ef e8 9a
Nov 27 00:23:01 Tower kernel: RSP: 0018:ffffc90002bd3c70 EFLAGS: 00010286
Nov 27 00:23:01 Tower kernel: RAX: ffff0181001ce000 RBX: ffff0181001ce000 RCX: ffffea0005d628c0
Nov 27 00:23:01 Tower kernel: RDX: ffff8880050a1d80 RSI: ffff888012606f00 RDI: ffff0181001ce000
Nov 27 00:23:01 Tower kernel: RBP: ffff888012606f00 R08: ffff88817c1b0c80 R09: ffff88817c1b0c90
Nov 27 00:23:01 Tower kernel: R10: ffff88804867b460 R11: 0000000000000000 R12: ffffea0005d628c0
Nov 27 00:23:01 Tower kernel: R13: 00000001758a3025 R14: 0000000000000000 R15: ffff888012606f00
Nov 27 00:23:01 Tower kernel: FS:  0000000000000000(0000) GS:ffff88817c180000(0000) knlGS:0000000000000000
Nov 27 00:23:01 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 27 00:23:01 Tower kernel: CR2: 0000148a476821e0 CR3: 000000011f40c000 CR4: 0000000000350ee0
Nov 27 00:23:01 Tower kernel: note: monitor[7109] exited with preempt_count 1
Nov 27 00:23:01 Tower kernel: Fixing recursive fault but reboot is needed!
Nov 27 00:23:01 Tower kernel: BUG: using smp_processor_id() in preemptible [00000000] code: monitor/7109
Nov 27 00:23:01 Tower kernel: caller is __schedule+0x2f/0x5f6
Nov 27 00:23:01 Tower kernel: CPU: 3 PID: 7109 Comm: monitor Tainted: G     UD           5.19.17-Unraid #2
Nov 27 00:23:01 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:23:01 Tower kernel: Call Trace:
Nov 27 00:23:01 Tower kernel: <TASK>
Nov 27 00:23:01 Tower kernel: dump_stack_lvl+0x44/0x5c
Nov 27 00:23:01 Tower kernel: check_preemption_disabled+0xcb/0xe0
Nov 27 00:23:01 Tower kernel: __schedule+0x2f/0x5f6
Nov 27 00:23:01 Tower kernel: ? _printk+0x58/0x73
Nov 27 00:23:01 Tower kernel: do_task_dead+0x3f/0x41
Nov 27 00:23:01 Tower kernel: make_task_dead+0xb2/0xba
Nov 27 00:23:01 Tower kernel: rewind_stack_and_make_dead+0x17/0x17
Nov 27 00:23:01 Tower kernel: RIP: 0033:0x148a4b3ec7c9
Nov 27 00:23:01 Tower kernel: Code: Unable to access opcode bytes at RIP 0x148a4b3ec79f.
Nov 27 00:23:01 Tower kernel: RSP: 002b:00007ffc8930ba98 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
Nov 27 00:23:01 Tower kernel: RAX: ffffffffffffffda RBX: 0000148a4b4f49e0 RCX: 0000148a4b3ec7c9
Nov 27 00:23:01 Tower kernel: RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
Nov 27 00:23:01 Tower kernel: RBP: 0000000000000000 R08: ffffffffffffff78 R09: 0000148a4b4ffac0
Nov 27 00:23:01 Tower kernel: R10: 0000000000000003 R11: 0000000000000246 R12: 0000148a4b4f49e0
Nov 27 00:23:01 Tower kernel: R13: 0000000000000000 R14: 0000148a4b4fa2c8 R15: 0000148a4b4fa2e0
Nov 27 00:23:01 Tower kernel: </TASK>
Nov 27 00:23:01 Tower kernel: BUG: scheduling while atomic: monitor/7109/0x00000000
Nov 27 00:23:01 Tower kernel: Modules linked in: wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha veth xt_nat xt_tcpudp xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge xfs md_mod efivarfs ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp stp llc r8169 realtek i915 x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel wmi_bmof aesni_intel crypto_simd cryptd intel_cstate ahci nvme i2c_i801 drm i2c_smbus nvme_core libahci intel_gtt agpgart i2c_core processor_thermal_device_pci_legacy processor_thermal_device processor_thermal_rfim processor_thermal_mbox syscopyarea int340x_thermal_zone sysfillrect sysimgblt intel_soc_dts_iosf input_leds
Nov 27 00:23:01 Tower kernel: led_class iosf_mbi fb_sys_fops fan thermal wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_pad button unix [last unloaded: realtek]
Nov 27 00:23:01 Tower kernel: Preemption disabled at:
Nov 27 00:23:01 Tower kernel: [<0000000000000000>] 0x0
Nov 27 00:23:01 Tower kernel: CPU: 3 PID: 7109 Comm: monitor Tainted: G     UD           5.19.17-Unraid #2
Nov 27 00:23:01 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:23:01 Tower kernel: Call Trace:
Nov 27 00:23:01 Tower kernel: <TASK>
Nov 27 00:23:01 Tower kernel: dump_stack_lvl+0x44/0x5c
Nov 27 00:23:01 Tower kernel: __schedule_bug+0xa3/0xb6
Nov 27 00:23:01 Tower kernel: __schedule+0x59/0x5f6
Nov 27 00:23:01 Tower kernel: ? _printk+0x58/0x73
Nov 27 00:23:01 Tower kernel: do_task_dead+0x3f/0x41
Nov 27 00:23:01 Tower kernel: make_task_dead+0xb2/0xba
Nov 27 00:23:01 Tower kernel: rewind_stack_and_make_dead+0x17/0x17
Nov 27 00:23:01 Tower kernel: RIP: 0033:0x148a4b3ec7c9
Nov 27 00:23:01 Tower kernel: Code: Unable to access opcode bytes at RIP 0x148a4b3ec79f.
Nov 27 00:23:01 Tower kernel: RSP: 002b:00007ffc8930ba98 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
Nov 27 00:23:01 Tower kernel: RAX: ffffffffffffffda RBX: 0000148a4b4f49e0 RCX: 0000148a4b3ec7c9
Nov 27 00:23:01 Tower kernel: RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
Nov 27 00:23:01 Tower kernel: RBP: 0000000000000000 R08: ffffffffffffff78 R09: 0000148a4b4ffac0
Nov 27 00:23:01 Tower kernel: R10: 0000000000000003 R11: 0000000000000246 R12: 0000148a4b4f49e0
Nov 27 00:23:01 Tower kernel: R13: 0000000000000000 R14: 0000148a4b4fa2c8 R15: 0000148a4b4fa2e0
Nov 27 00:23:01 Tower kernel: </TASK>
Nov 27 00:23:01 Tower kernel: ------------[ cut here ]------------
Nov 27 00:23:01 Tower kernel: Voluntary context switch within RCU read-side critical section!
Nov 27 00:23:01 Tower kernel: WARNING: CPU: 3 PID: 7109 at kernel/rcu/tree_plugin.h:318 rcu_note_context_switch+0x6a/0x3b9
Nov 27 00:23:01 Tower kernel: Modules linked in: wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha veth xt_nat xt_tcpudp xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge xfs md_mod efivarfs ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp stp llc r8169 realtek i915 x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel wmi_bmof aesni_intel crypto_simd cryptd intel_cstate ahci nvme i2c_i801 drm i2c_smbus nvme_core libahci intel_gtt agpgart i2c_core processor_thermal_device_pci_legacy processor_thermal_device processor_thermal_rfim processor_thermal_mbox syscopyarea int340x_thermal_zone sysfillrect sysimgblt intel_soc_dts_iosf input_leds
Nov 27 00:23:01 Tower kernel: led_class iosf_mbi fb_sys_fops fan thermal wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_pad button unix [last unloaded: realtek]
Nov 27 00:23:01 Tower kernel: CPU: 3 PID: 7109 Comm: monitor Tainted: G     UD W         5.19.17-Unraid #2
Nov 27 00:23:01 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:23:01 Tower kernel: RIP: 0010:rcu_note_context_switch+0x6a/0x3b9
Nov 27 00:23:01 Tower kernel: Code: 75 2a 41 8b 84 24 0c 03 00 00 85 c0 7e 1e 80 3d cb c5 2a 01 00 75 15 48 c7 c7 35 97 0c 82 c6 05 bb c5 2a 01 01 e8 0e 7f 74 00 <0f> 0b 65 48 8b 04 25 c0 bb 01 00 8b 80 0c 03 00 00 85 c0 0f 8e bc
Nov 27 00:23:01 Tower kernel: RSP: 0018:ffffc90002bd3e90 EFLAGS: 00010086
Nov 27 00:23:01 Tower kernel: RAX: 0000000000000000 RBX: ffff8880050a1d80 RCX: 0000000000000027
Nov 27 00:23:01 Tower kernel: RDX: 0000000000000002 RSI: ffffffff820d7be1 RDI: 00000000ffffffff
Nov 27 00:23:01 Tower kernel: RBP: ffff88817c1ace80 R08: 0000000000000000 R09: ffffc90002bd3e28
Nov 27 00:23:01 Tower kernel: general protection fault, probably for non-canonical address 0xffff0181001cef00: 0000 [#2] PREEMPT SMP NOPTI
Nov 27 00:23:01 Tower kernel: R10: 0000000000000001 R11: 0000000000000001 R12: ffff8880050a1d80
Nov 27 00:23:01 Tower kernel: CPU: 0 PID: 7128 Comm: startBackground Tainted: G     UD W         5.19.17-Unraid #2
Nov 27 00:23:01 Tower kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
Nov 27 00:23:01 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:23:01 Tower kernel: FS:  0000000000000000(0000) GS:ffff88817c180000(0000) knlGS:0000000000000000
Nov 27 00:23:01 Tower kernel: RIP: 0010:folio_memcg_lock+0x2e/0x88
Nov 27 00:23:01 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 27 00:23:01 Tower kernel: Code: 00 41 55 41 54 49 89 fc 55 53 e8 93 0d eb ff e8 a5 b3 ff ff 84 c0 75 61 49 8b 7c 24 38 e8 51 cc ff ff 48 85 c0 48 89 c3 74 4f <8b> 80 00 0f 00 00 85 c0 7e 45 4c 8d ab 40 04 00 00 4c 89 ef e8 9a
Nov 27 00:23:01 Tower kernel: CR2: 0000148a476821e0 CR3: 000000011f40c000 CR4: 0000000000350ee0
Nov 27 00:23:01 Tower kernel: RSP: 0018:ffffc90002c53c70 EFLAGS: 00010286
Nov 27 00:23:01 Tower kernel: Call Trace:
Nov 27 00:23:01 Tower kernel: 
Nov 27 00:23:01 Tower kernel: <TASK>
Nov 27 00:23:01 Tower kernel: RAX: ffff0181001ce000 RBX: ffff0181001ce000 RCX: ffffea0005d628c0
Nov 27 00:23:01 Tower kernel: __schedule+0xa9/0x5f6
Nov 27 00:23:01 Tower kernel: RDX: ffff888176933b00 RSI: ffff888010fef180 RDI: ffff0181001ce000
Nov 27 00:23:01 Tower kernel: ? _printk+0x58/0x73
Nov 27 00:23:01 Tower kernel: RBP: ffff888010fef180 R08: ffff88817c030c80 R09: ffff88817c030c90
Nov 27 00:23:01 Tower kernel: do_task_dead+0x3f/0x41
Nov 27 00:23:01 Tower kernel: R10: ffff88804867bf50 R11: 0000000000000000 R12: ffffea0005d628c0
Nov 27 00:23:01 Tower kernel: make_task_dead+0xb2/0xba
Nov 27 00:23:01 Tower kernel: R13: 00000001758a3025 R14: 0000000000000000 R15: ffff888010fef180
Nov 27 00:23:01 Tower kernel: rewind_stack_and_make_dead+0x17/0x17
Nov 27 00:23:01 Tower kernel: FS:  0000000000000000(0000) GS:ffff88817c000000(0000) knlGS:0000000000000000
Nov 27 00:23:01 Tower kernel: RIP: 0033:0x148a4b3ec7c9
Nov 27 00:23:01 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 27 00:23:01 Tower kernel: Code: Unable to access opcode bytes at RIP 0x148a4b3ec79f.
Nov 27 00:23:01 Tower kernel: CR2: 00001459b43211e0 CR3: 000000011b2a2000 CR4: 0000000000350ef0
Nov 27 00:23:01 Tower kernel: RSP: 002b:00007ffc8930ba98 EFLAGS: 00000246
Nov 27 00:23:01 Tower kernel: Call Trace:
Nov 27 00:23:01 Tower kernel: ORIG_RAX: 00000000000000e7
Nov 27 00:23:01 Tower kernel: <TASK>
Nov 27 00:23:01 Tower kernel: RAX: ffffffffffffffda RBX: 0000148a4b4f49e0 RCX: 0000148a4b3ec7c9
Nov 27 00:23:01 Tower kernel: page_remove_rmap+0x1b/0x239
Nov 27 00:23:01 Tower kernel: RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
Nov 27 00:23:01 Tower kernel: unmap_page_range+0x451/0x66e
Nov 27 00:23:01 Tower kernel: RBP: 0000000000000000 R08: ffffffffffffff78 R09: 0000148a4b4ffac0
Nov 27 00:23:01 Tower kernel: unmap_vmas+0x87/0xbb
Nov 27 00:23:01 Tower kernel: R10: 0000000000000003 R11: 0000000000000246 R12: 0000148a4b4f49e0
Nov 27 00:23:01 Tower kernel: exit_mmap+0xdc/0x15a
Nov 27 00:23:01 Tower kernel: R13: 0000000000000000 R14: 0000148a4b4fa2c8 R15: 0000148a4b4fa2e0
Nov 27 00:23:01 Tower kernel: __mmput+0x43/0xdb
Nov 27 00:23:01 Tower kernel: </TASK>
Nov 27 00:23:01 Tower kernel: do_exit+0x30b/0x8e5
Nov 27 00:23:01 Tower kernel: ---[ end trace 0000000000000000 ]---
Nov 27 00:23:01 Tower kernel: do_group_exit+0x8f/0x8f
Nov 27 00:23:01 Tower kernel: __x64_sys_exit_group+0x14/0x14
Nov 27 00:23:01 Tower kernel: do_syscall_64+0x68/0x81
Nov 27 00:23:01 Tower kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd
Nov 27 00:23:01 Tower kernel: RIP: 0033:0x1459b808a7c9
Nov 27 00:23:01 Tower kernel: Code: Unable to access opcode bytes at RIP 0x1459b808a79f.
Nov 27 00:23:01 Tower kernel: RSP: 002b:00007ffdb5e3db28 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
Nov 27 00:23:01 Tower kernel: RAX: ffffffffffffffda RBX: 00001459b81929e0 RCX: 00001459b808a7c9
Nov 27 00:23:01 Tower kernel: RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
Nov 27 00:23:01 Tower kernel: RBP: 0000000000000000 R08: ffffffffffffff78 R09: 00001459b819dac0
Nov 27 00:23:01 Tower kernel: R10: 0000000000000003 R11: 0000000000000246 R12: 00001459b81929e0
Nov 27 00:23:01 Tower kernel: R13: 0000000000000000 R14: 00001459b81982c8 R15: 00001459b81982e0
Nov 27 00:23:01 Tower kernel: </TASK>
Nov 27 00:23:01 Tower kernel: Modules linked in: wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha veth xt_nat xt_tcpudp xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge xfs md_mod efivarfs ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp stp llc r8169 realtek i915 x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel wmi_bmof aesni_intel crypto_simd cryptd intel_cstate ahci nvme i2c_i801 drm i2c_smbus nvme_core libahci intel_gtt agpgart i2c_core processor_thermal_device_pci_legacy processor_thermal_device processor_thermal_rfim processor_thermal_mbox syscopyarea int340x_thermal_zone sysfillrect sysimgblt intel_soc_dts_iosf input_leds
Nov 27 00:23:01 Tower kernel: led_class iosf_mbi fb_sys_fops fan thermal wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_pad button unix [last unloaded: realtek]
Nov 27 00:23:01 Tower kernel: ---[ end trace 0000000000000000 ]---
Nov 27 00:23:01 Tower kernel: RIP: 0010:folio_memcg_lock+0x2e/0x88
Nov 27 00:23:01 Tower kernel: Code: 00 41 55 41 54 49 89 fc 55 53 e8 93 0d eb ff e8 a5 b3 ff ff 84 c0 75 61 49 8b 7c 24 38 e8 51 cc ff ff 48 85 c0 48 89 c3 74 4f <8b> 80 00 0f 00 00 85 c0 7e 45 4c 8d ab 40 04 00 00 4c 89 ef e8 9a
Nov 27 00:23:01 Tower kernel: RSP: 0018:ffffc90002bd3c70 EFLAGS: 00010286
Nov 27 00:23:01 Tower kernel: RAX: ffff0181001ce000 RBX: ffff0181001ce000 RCX: ffffea0005d628c0
Nov 27 00:23:01 Tower kernel: RDX: ffff8880050a1d80 RSI: ffff888012606f00 RDI: ffff0181001ce000
Nov 27 00:23:01 Tower kernel: RBP: ffff888012606f00 R08: ffff88817c1b0c80 R09: ffff88817c1b0c90
Nov 27 00:23:01 Tower kernel: R10: ffff88804867b460 R11: 0000000000000000 R12: ffffea0005d628c0
Nov 27 00:23:01 Tower kernel: R13: 00000001758a3025 R14: 0000000000000000 R15: ffff888012606f00
Nov 27 00:23:01 Tower kernel: FS:  0000000000000000(0000) GS:ffff88817c000000(0000) knlGS:0000000000000000
Nov 27 00:23:01 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 27 00:23:01 Tower kernel: CR2: 00001459b43211e0 CR3: 000000011b2a2000 CR4: 0000000000350ef0
Nov 27 00:23:01 Tower kernel: note: startBackground[7128] exited with preempt_count 1
Nov 27 00:23:01 Tower kernel: Fixing recursive fault but reboot is needed!
Nov 27 00:23:01 Tower kernel: BUG: using smp_processor_id() in preemptible [00000000] code: startBackground/7128
Nov 27 00:23:01 Tower kernel: caller is __schedule+0x2f/0x5f6
Nov 27 00:23:01 Tower kernel: CPU: 0 PID: 7128 Comm: startBackground Tainted: G     UD W         5.19.17-Unraid #2
Nov 27 00:23:01 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:23:01 Tower kernel: Call Trace:
Nov 27 00:23:01 Tower kernel: <TASK>
Nov 27 00:23:01 Tower kernel: dump_stack_lvl+0x44/0x5c
Nov 27 00:23:01 Tower kernel: check_preemption_disabled+0xcb/0xe0
Nov 27 00:23:01 Tower kernel: __schedule+0x2f/0x5f6
Nov 27 00:23:01 Tower kernel: ? _printk+0x58/0x73
Nov 27 00:23:01 Tower kernel: do_task_dead+0x3f/0x41
Nov 27 00:23:01 Tower kernel: make_task_dead+0xb2/0xba
Nov 27 00:23:01 Tower kernel: rewind_stack_and_make_dead+0x17/0x17
Nov 27 00:23:01 Tower kernel: RIP: 0033:0x1459b808a7c9
Nov 27 00:23:01 Tower kernel: Code: Unable to access opcode bytes at RIP 0x1459b808a79f.
Nov 27 00:23:01 Tower kernel: RSP: 002b:00007ffdb5e3db28 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
Nov 27 00:23:01 Tower kernel: RAX: ffffffffffffffda RBX: 00001459b81929e0 RCX: 00001459b808a7c9
Nov 27 00:23:01 Tower kernel: RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
Nov 27 00:23:01 Tower kernel: RBP: 0000000000000000 R08: ffffffffffffff78 R09: 00001459b819dac0
Nov 27 00:23:01 Tower kernel: R10: 0000000000000003 R11: 0000000000000246 R12: 00001459b81929e0
Nov 27 00:23:01 Tower kernel: R13: 0000000000000000 R14: 00001459b81982c8 R15: 00001459b81982e0
Nov 27 00:23:01 Tower kernel: </TASK>
Nov 27 00:23:01 Tower kernel: BUG: scheduling while atomic: startBackground/7128/0x00000000
Nov 27 00:23:01 Tower kernel: Modules linked in: wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha veth xt_nat xt_tcpudp xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge xfs md_mod efivarfs ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp stp llc r8169 realtek i915 x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel wmi_bmof aesni_intel crypto_simd cryptd intel_cstate ahci nvme i2c_i801 drm i2c_smbus nvme_core libahci intel_gtt agpgart i2c_core processor_thermal_device_pci_legacy processor_thermal_device processor_thermal_rfim processor_thermal_mbox syscopyarea int340x_thermal_zone sysfillrect sysimgblt intel_soc_dts_iosf input_leds
Nov 27 00:23:01 Tower kernel: led_class iosf_mbi fb_sys_fops fan thermal wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_pad button unix [last unloaded: realtek]
Nov 27 00:23:01 Tower kernel: Preemption disabled at:
Nov 27 00:23:01 Tower kernel: [<0000000000000000>] 0x0
Nov 27 00:23:01 Tower kernel: CPU: 0 PID: 7128 Comm: startBackground Tainted: G     UD W         5.19.17-Unraid #2
Nov 27 00:23:01 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:23:01 Tower kernel: Call Trace:
Nov 27 00:23:01 Tower kernel: <TASK>
Nov 27 00:23:01 Tower kernel: dump_stack_lvl+0x44/0x5c
Nov 27 00:23:01 Tower kernel: __schedule_bug+0xa3/0xb6
Nov 27 00:23:01 Tower kernel: __schedule+0x59/0x5f6
Nov 27 00:23:01 Tower kernel: ? _printk+0x58/0x73
Nov 27 00:23:01 Tower kernel: do_task_dead+0x3f/0x41
Nov 27 00:23:01 Tower kernel: make_task_dead+0xb2/0xba
Nov 27 00:23:01 Tower kernel: rewind_stack_and_make_dead+0x17/0x17
Nov 27 00:23:01 Tower kernel: RIP: 0033:0x1459b808a7c9
Nov 27 00:23:01 Tower kernel: Code: Unable to access opcode bytes at RIP 0x1459b808a79f.
Nov 27 00:23:01 Tower kernel: RSP: 002b:00007ffdb5e3db28 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
Nov 27 00:23:01 Tower kernel: RAX: ffffffffffffffda RBX: 00001459b81929e0 RCX: 00001459b808a7c9
Nov 27 00:23:01 Tower kernel: RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
Nov 27 00:23:01 Tower kernel: RBP: 0000000000000000 R08: ffffffffffffff78 R09: 00001459b819dac0
Nov 27 00:23:01 Tower kernel: R10: 0000000000000003 R11: 0000000000000246 R12: 00001459b81929e0
Nov 27 00:23:01 Tower kernel: R13: 0000000000000000 R14: 00001459b81982c8 R15: 00001459b81982e0
Nov 27 00:23:01 Tower kernel: </TASK>
Nov 27 00:23:38 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 21446 jiffies s: 3525 root: 0x0/T
Nov 27 00:23:38 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:24:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:24:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:24:01 Tower kernel: #011(detected by 1, t=60002 jiffies, g=6087985, q=151701 ncpus=4)
Nov 27 00:24:43 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 86982 jiffies s: 3525 root: 0x0/T
Nov 27 00:24:43 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:25:49 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 152518 jiffies s: 3525 root: 0x0/T
Nov 27 00:25:49 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:26:54 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 218055 jiffies s: 3525 root: 0x0/T
Nov 27 00:26:54 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:27:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:27:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:27:01 Tower kernel: #011(detected by 0, t=240016 jiffies, g=6087985, q=592566 ncpus=4)
Nov 27 00:28:00 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 283590 jiffies s: 3525 root: 0x0/T
Nov 27 00:28:00 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:29:05 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 349127 jiffies s: 3525 root: 0x0/T
Nov 27 00:29:05 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:30:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:30:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:30:01 Tower kernel: #011(detected by 3, t=420027 jiffies, g=6087985, q=1056349 ncpus=4)
Nov 27 00:30:11 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 414662 jiffies s: 3525 root: 0x0/T
Nov 27 00:30:11 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:31:17 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 480198 jiffies s: 3525 root: 0x0/T
Nov 27 00:31:17 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:32:22 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 545735 jiffies s: 3525 root: 0x0/T
Nov 27 00:32:22 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:33:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:33:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:33:01 Tower kernel: #011(detected by 3, t=600038 jiffies, g=6087985, q=1513444 ncpus=4)
Nov 27 00:33:28 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 611271 jiffies s: 3525 root: 0x0/T
Nov 27 00:33:28 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:34:33 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 676806 jiffies s: 3525 root: 0x0/T
Nov 27 00:34:33 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:35:39 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 742342 jiffies s: 3525 root: 0x0/T
Nov 27 00:35:39 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:36:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:36:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:36:01 Tower kernel: #011(detected by 3, t=780049 jiffies, g=6087985, q=1974025 ncpus=4)
Nov 27 00:36:44 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 807878 jiffies s: 3525 root: 0x0/T
Nov 27 00:36:44 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:37:50 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 873414 jiffies s: 3525 root: 0x0/T
Nov 27 00:37:50 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:38:55 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 938951 jiffies s: 3525 root: 0x0/T
Nov 27 00:38:55 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:39:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:39:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:39:01 Tower kernel: #011(detected by 1, t=960066 jiffies, g=6087985, q=2433642 ncpus=4)
Nov 27 00:40:01 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1004486 jiffies s: 3525 root: 0x0/T
Nov 27 00:40:01 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:41:06 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1070022 jiffies s: 3525 root: 0x0/T
Nov 27 00:41:06 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:42:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:42:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:42:01 Tower kernel: #011(detected by 1, t=1140091 jiffies, g=6087985, q=2888874 ncpus=4)
Nov 27 00:42:12 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1135558 jiffies s: 3525 root: 0x0/T
Nov 27 00:42:12 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:43:17 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1201094 jiffies s: 3525 root: 0x0/T
Nov 27 00:43:17 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:44:23 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1266630 jiffies s: 3525 root: 0x0/T
Nov 27 00:44:23 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:45:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:45:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:45:01 Tower kernel: #011(detected by 2, t=1320120 jiffies, g=6087985, q=3346497 ncpus=4)
Nov 27 00:45:28 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1332167 jiffies s: 3525 root: 0x0/T
Nov 27 00:45:28 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:46:34 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1397702 jiffies s: 3525 root: 0x0/T
Nov 27 00:46:34 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:47:40 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1463238 jiffies s: 3525 root: 0x0/T
Nov 27 00:47:40 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:48:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:48:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:48:01 Tower kernel: #011(detected by 3, t=1500145 jiffies, g=6087985, q=3832495 ncpus=4)
Nov 27 00:48:45 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1528774 jiffies s: 3525 root: 0x0/T
Nov 27 00:48:45 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:49:51 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1594310 jiffies s: 3525 root: 0x0/T
Nov 27 00:49:51 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:50:56 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1659846 jiffies s: 3525 root: 0x0/T
Nov 27 00:50:56 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:51:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:51:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:51:01 Tower kernel: #011(detected by 2, t=1680170 jiffies, g=6087985, q=4339770 ncpus=4)
Nov 27 00:52:02 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1725382 jiffies s: 3525 root: 0x0/T
Nov 27 00:52:02 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:53:07 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1790919 jiffies s: 3525 root: 0x0/T
Nov 27 00:53:07 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:54:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:54:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:54:01 Tower kernel: #011(detected by 0, t=1860185 jiffies, g=6087985, q=4879672 ncpus=4)
Nov 27 00:54:13 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1856454 jiffies s: 3525 root: 0x0/T
Nov 27 00:54:13 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:54:48 Tower kernel: avahi-daemon invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Nov 27 00:54:48 Tower kernel: CPU: 3 PID: 9485 Comm: avahi-daemon Tainted: G     UD W         5.19.17-Unraid #2
Nov 27 00:54:48 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:54:48 Tower kernel: Call Trace:
Nov 27 00:54:48 Tower kernel: <TASK>
Nov 27 00:54:48 Tower kernel: dump_stack_lvl+0x44/0x5c
Nov 27 00:54:48 Tower kernel: dump_header+0x4a/0x1ff
Nov 27 00:54:48 Tower kernel: oom_kill_process+0x80/0x111
Nov 27 00:54:48 Tower kernel: out_of_memory+0x3e8/0x41a
Nov 27 00:54:48 Tower kernel: __alloc_pages_slowpath.constprop.0+0x6eb/0x8ee
Nov 27 00:54:48 Tower kernel: __alloc_pages+0x132/0x1e8
Nov 27 00:54:48 Tower kernel: __get_free_pages+0xd/0x3c
Nov 27 00:54:48 Tower kernel: __pollwait+0x57/0xc8
Nov 27 00:54:48 Tower kernel: sock_poll_wait+0x15/0x23 [unix]
Nov 27 00:54:48 Tower kernel: unix_poll+0x12/0xb0 [unix]
Nov 27 00:54:48 Tower kernel: sock_poll+0xb5/0xc5
Nov 27 00:54:48 Tower kernel: do_sys_poll+0x225/0x48b
Nov 27 00:54:48 Tower kernel: ? udp_recvmsg+0x319/0x453
Nov 27 00:54:48 Tower kernel: ? udp_recvmsg+0x3de/0x453
Nov 27 00:54:48 Tower kernel: ? poll_initwait+0x34/0x34
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: ? get_compat_sigset_argpack.constprop.0+0x3b/0x3b
Nov 27 00:54:48 Tower kernel: __do_sys_poll+0x51/0xdd
Nov 27 00:54:48 Tower kernel: do_syscall_64+0x68/0x81
Nov 27 00:54:48 Tower kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd
Nov 27 00:54:48 Tower kernel: RIP: 0033:0x1546465c24a0
Nov 27 00:54:48 Tower kernel: Code: 00 00 49 8b 45 10 48 83 c4 08 5b 5d 41 5c 41 5d c3 66 0f 1f 84 00 00 00 00 00 80 3d 41 71 0e 00 00 74 17 b8 07 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 89 54
Nov 27 00:54:48 Tower kernel: RSP: 002b:00007ffe3328fb98 EFLAGS: 00000202 ORIG_RAX: 0000000000000007
Nov 27 00:54:48 Tower kernel: RAX: ffffffffffffffda RBX: 0000000000427f20 RCX: 00001546465c24a0
Nov 27 00:54:48 Tower kernel: RDX: 0000000000000081 RSI: 000000000000000b RDI: 0000000000447370
Nov 27 00:54:48 Tower kernel: RBP: 0000154646324740 R08: 0000000000000000 R09: 0000000000476af0
Nov 27 00:54:48 Tower kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000000
Nov 27 00:54:48 Tower kernel: R13: 0000000000427fd0 R14: 000000000042c170 R15: 00001546467d7020
Nov 27 00:54:48 Tower kernel: </TASK>
Nov 27 00:54:48 Tower kernel: Mem-Info:
Nov 27 00:54:48 Tower kernel: active_anon:235054 inactive_anon:156596 isolated_anon:0
Nov 27 00:54:48 Tower kernel: active_file:358 inactive_file:2137 isolated_file:34
Nov 27 00:54:48 Tower kernel: unevictable:10473 dirty:92 writeback:0
Nov 27 00:54:48 Tower kernel: slab_reclaimable:102373 slab_unreclaimable:349425
Nov 27 00:54:48 Tower kernel: mapped:42392 shmem:261189 pagetables:3026 bounce:0
Nov 27 00:54:48 Tower kernel: kernel_misc_reclaimable:0
Nov 27 00:54:48 Tower kernel: free:23022 free_pcp:609 free_cma:0
Nov 27 00:54:48 Tower kernel: Node 0 active_anon:940216kB inactive_anon:626384kB active_file:800kB inactive_file:9256kB unevictable:41892kB isolated(anon):0kB isolated(file):128kB mapped:169568kB dirty:368kB writeback:0kB shmem:1044756kB shmem_thp: 38912kB shmem_pmdmapped: 0kB anon_thp: 178176kB writeback_tmp:0kB kernel_stack:7936kB pagetables:12104kB all_unreclaimable? no
Nov 27 00:54:48 Tower kernel: Node 0 DMA free:14356kB boost:0kB min:284kB low:352kB high:420kB reserved_highatomic:0KB active_anon:0kB inactive_anon:4kB active_file:0kB inactive_file:20kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Nov 27 00:54:48 Tower kernel: lowmem_reserve[]: 0 1537 3518 3518
Nov 27 00:54:48 Tower kernel: Node 0 DMA32 free:37256kB boost:0kB min:29416kB low:36768kB high:44120kB reserved_highatomic:0KB active_anon:15412kB inactive_anon:322328kB active_file:388kB inactive_file:1824kB unevictable:160kB writepending:56kB present:1828964kB managed:1740464kB mlocked:160kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Nov 27 00:54:48 Tower kernel: lowmem_reserve[]: 0 0 1980 1980
Nov 27 00:54:48 Tower kernel: Node 0 Normal free:40432kB boost:2048kB min:39924kB low:49392kB high:58860kB reserved_highatomic:2048KB active_anon:924804kB inactive_anon:304176kB active_file:868kB inactive_file:5692kB unevictable:41732kB writepending:312kB present:2101248kB managed:2027836kB mlocked:32kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Nov 27 00:54:48 Tower kernel: lowmem_reserve[]: 0 0 0 0
Nov 27 00:54:48 Tower kernel: Node 0 DMA: 3*4kB (UE) 3*8kB (UME) 1*16kB (U) 3*32kB (UME) 2*64kB (ME) 2*128kB (UM) 2*256kB (ME) 2*512kB (ME) 2*1024kB (ME) 1*2048kB (E) 2*4096kB (M) = 14356kB
Nov 27 00:54:48 Tower kernel: Node 0 DMA32: 1205*4kB (UME) 595*8kB (UME) 386*16kB (UME) 279*32kB (UME) 132*64kB (UME) 43*128kB (UME) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 38636kB
Nov 27 00:54:48 Tower kernel: Node 0 Normal: 4506*4kB (UM) 1205*8kB (UME) 267*16kB (UME) 88*32kB (UME) 44*64kB (UM) 25*128kB (UME) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 41024kB
Nov 27 00:54:48 Tower kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Nov 27 00:54:48 Tower kernel: 263772 total pagecache pages
Nov 27 00:54:48 Tower kernel: 0 pages in swap cache
Nov 27 00:54:48 Tower kernel: Swap cache stats: add 0, delete 0, find 0/0
Nov 27 00:54:48 Tower kernel: Free swap  = 0kB
Nov 27 00:54:48 Tower kernel: Total swap = 0kB
Nov 27 00:54:48 Tower kernel: 986551 pages RAM
Nov 27 00:54:48 Tower kernel: 0 pages HighMem/MovableOnly
Nov 27 00:54:48 Tower kernel: 40636 pages reserved
Nov 27 00:54:48 Tower kernel: 0 pages cma reserved
Nov 27 00:54:48 Tower kernel: Tasks state (memory values in pages):
Nov 27 00:54:48 Tower kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Nov 27 00:54:48 Tower kernel: [    632]     0   632     2595     1194    53248        0         -1000 udevd
Nov 27 00:54:48 Tower kernel: [    917]    68   917      777      607    40960        0             0 dhcpcd
Nov 27 00:54:48 Tower kernel: [    918]     0   918      764      527    40960        0             0 dhcpcd
Nov 27 00:54:48 Tower kernel: [    919]    68   919      758       69    40960        0             0 dhcpcd
Nov 27 00:54:48 Tower kernel: [   1009]     0  1009      667       38    40960        0             0 mcelog
Nov 27 00:54:48 Tower kernel: [   1019]    81  1019     1295      525    49152        0             0 dbus-daemon
Nov 27 00:54:48 Tower kernel: [   1032]     0  1032     1580      660    45056        0             0 elogind-daemon
Nov 27 00:54:48 Tower kernel: [   1066]    44  1066    18680     1038    57344        0             0 ntpd
Nov 27 00:54:48 Tower kernel: [   1073]     0  1073      650       25    40960        0             0 acpid
Nov 27 00:54:48 Tower kernel: [   1087]     0  1087      644      462    45056        0             0 crond
Nov 27 00:54:48 Tower kernel: [   1091]     0  1091      664      363    45056        0             0 atd
Nov 27 00:54:48 Tower kernel: [   1212]     0  1212     1003      628    40960        0             0 monitor_nchan
Nov 27 00:54:48 Tower kernel: [   1649]    68  1649      764       74    40960        0             0 dhcpcd
Nov 27 00:54:48 Tower kernel: [   2358]     0  2358      652      230    40960        0             0 agetty
Nov 27 00:54:48 Tower kernel: [   2359]     0  2359      652      228    40960        0             0 agetty
Nov 27 00:54:48 Tower kernel: [   2360]     0  2360      652      229    40960        0             0 agetty
Nov 27 00:54:48 Tower kernel: [   2361]     0  2361      652      218    45056        0             0 agetty
Nov 27 00:54:48 Tower kernel: [   2362]     0  2362      652      208    40960        0             0 agetty
Nov 27 00:54:48 Tower kernel: [   2363]     0  2363      652      230    40960        0             0 agetty
Nov 27 00:54:48 Tower kernel: [   2392]     0  2392     1086       61    45056        0             0 inetd
Nov 27 00:54:48 Tower kernel: [   2393]     0  2393    86868     1125    90112        0             0 emhttpd
Nov 27 00:54:48 Tower kernel: [   2479]    68  2479      764       74    40960        0             0 dhcpcd
Nov 27 00:54:48 Tower kernel: [   2599]     0  2599    22238     2502   147456        0             0 php-fpm
Nov 27 00:54:48 Tower kernel: [   2652]     0  2652    36865     1040    61440        0             0 nginx
Nov 27 00:54:48 Tower kernel: [   2653]     0  2653    37226     3109    77824        0             0 nginx
Nov 27 00:54:48 Tower kernel: [   2751]     0  2751     1170      865    40960        0             0 rc.flash_backup
Nov 27 00:54:48 Tower kernel: [   3229]     0  3229    35060       69    65536        0             0 shfs
Nov 27 00:54:48 Tower kernel: [   3240]     0  3240   103988     2342   102400        0             0 shfs
Nov 27 00:54:48 Tower kernel: [   3292]     0  3292    70331     1060    73728        0             0 rsyslogd
Nov 27 00:54:48 Tower kernel: [   9451]     0  9451    19876     3432   135168        0             0 smbd
Nov 27 00:54:48 Tower kernel: [   9456]     0  9456    19529     2202   131072        0             0 smbd-notifyd
Nov 27 00:54:48 Tower kernel: [   9457]     0  9457    19529     1079   126976        0             0 cleanupd
Nov 27 00:54:48 Tower kernel: [   9465]     0  9465      647      452    40960        0             0 wsdd2
Nov 27 00:54:48 Tower kernel: [   9468]     0  9468    19010     3062   135168        0             0 winbindd
Nov 27 00:54:48 Tower kernel: [   9470]     0  9470    19031     3708   135168        0             0 winbindd
Nov 27 00:54:48 Tower kernel: [   9485]    61  9485     1411      846    45056        0             0 avahi-daemon
Nov 27 00:54:48 Tower kernel: [   9486]    61  9486     1329       67    45056        0             0 avahi-daemon
Nov 27 00:54:48 Tower kernel: [   9494]     0  9494      669       29    45056        0             0 avahi-dnsconfd
Nov 27 00:54:48 Tower kernel: [   9564]     0  9564   680700    23308   921600        0             0 dockerd
Nov 27 00:54:48 Tower kernel: [   9578]     0  9578   355318    12180   290816        0             0 containerd
Nov 27 00:54:48 Tower kernel: [  10538]     0 10538    19118     3013   135168        0             0 winbindd
Nov 27 00:54:48 Tower kernel: [  11889]     0 11889     2564     1742    53248        0             0 ttyd
Nov 27 00:54:48 Tower kernel: [  22178]     0 22178   178180     4782   118784        0             1 containerd-shim
Nov 27 00:54:48 Tower kernel: [  22199]     0 22199       51        1    28672        0             0 s6-svscan
Nov 27 00:54:48 Tower kernel: [  22246]     0 22246       51        1    28672        0             0 s6-supervise
Nov 27 00:54:48 Tower kernel: [  22538]     0 22538       51        1    28672        0             0 s6-supervise
Nov 27 00:54:48 Tower kernel: [  22541]    99 22541    51988    11249   475136        0             0 Plex Media Serv
Nov 27 00:54:48 Tower kernel: [  22642]    99 22642    16284     8204   143360        0             0 Plex Script Hos
Nov 27 00:54:48 Tower kernel: [  22727]    99 22727    10723      500    94208        0             0 Plex Tuner Serv
Nov 27 00:54:48 Tower kernel: [   5236]     0  5236   178180     4320   118784        0             1 containerd-shim
Nov 27 00:54:48 Tower kernel: [   5256]   201  5256    51895    28255   495616        0             0 netdata
Nov 27 00:54:48 Tower kernel: [   5296]   201  5296     5424      785    81920        0             0 netdata
Nov 27 00:54:48 Tower kernel: [   5542]   201  5542     8189     4515    98304        0             0 python3
Nov 27 00:54:48 Tower kernel: [   5553]   201  5553     1400      890    45056        0             0 apps.plugin
Nov 27 00:54:48 Tower kernel: [   5565]   201  5565   322495     4756   237568        0             0 go.d.plugin
Nov 27 00:54:48 Tower kernel: [  20636]     0 20636    23019     7346   159744        0             0 notify_poller
Nov 27 00:54:48 Tower kernel: [  20639]     0 20639    23019     7249   163840        0             0 session_check
Nov 27 00:54:48 Tower kernel: [  20641]     0 20641    23035     7383   163840        0             0 system_temp
Nov 27 00:54:48 Tower kernel: [  20645]     0 20645    23019     7220   163840        0             0 wg_poller
Nov 27 00:54:48 Tower kernel: [  20650]     0 20650    23019     7267   163840        0             0 update_1
Nov 27 00:54:48 Tower kernel: [  20655]     0 20655    23069     7459   163840        0             0 update_2
Nov 27 00:54:48 Tower kernel: [  20658]     0 20658    23019     7292   163840        0             0 update_3
Nov 27 00:54:48 Tower kernel: [  23536]     0 23536     2564     1256    53248        0             0 ttyd
Nov 27 00:54:48 Tower kernel: [  31889]     0 31889    23079     7456   167936        0             0 device_list
Nov 27 00:54:48 Tower kernel: [  31891]     0 31891     1004      782    40960        0             0 disk_load
Nov 27 00:54:48 Tower kernel: [  31893]     0 31893    23019     7315   163840        0             0 parity_list
Nov 27 00:54:48 Tower kernel: [   4880]     0  4880  2851827    34388  3424256        0             0 unraid-api
Nov 27 00:54:48 Tower kernel: [   4833]     0  4833    23035     7328   163840        0             0 ups_status
Nov 27 00:54:48 Tower kernel: [   7106]     0  7106      992      692    45056        0             0 sh
Nov 27 00:54:48 Tower kernel: [   7107]     0  7107      993      690    45056        0             0 sh
Nov 27 00:54:48 Tower kernel: [   7108]     0  7108    22974     7088   155648        0             0 startCustom.php
Nov 27 00:54:48 Tower kernel: [   7127]     0  7127      994      720    45056        0             0 sh
Nov 27 00:54:48 Tower kernel: [   7189]     0  7189    17191     1798   118784        0             0 notify
Nov 27 00:54:48 Tower kernel: [   7298]     0  7298      994      748    40960        0             0 sh
Nov 27 00:54:48 Tower kernel: [   7299]     0  7299     1286      716    45056        0             0 lsof
Nov 27 00:54:48 Tower kernel: [   7300]     0  7300     1521      626    45056        0             0 awk
Nov 27 00:54:48 Tower kernel: [   7301]     0  7301     1488      635    45056        0             0 awk
Nov 27 00:54:48 Tower kernel: [   7611]     0  7611     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [   7697]     0  7697    17191     1806   114688        0             0 smartctl_type
Nov 27 00:54:48 Tower kernel: [   7698]     0  7698    17191     2377   114688        0             0 smartctl_type
Nov 27 00:54:48 Tower kernel: [   7699]     0  7699    17191     2347   110592        0             0 smartctl_type
Nov 27 00:54:48 Tower kernel: [   8736]     0  8736      992      709    40960        0             0 sh
Nov 27 00:54:48 Tower kernel: [   8737]     0  8737    17191     1794   110592        0             0 php
Nov 27 00:54:48 Tower kernel: [   8881]     0  8881     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  10213]     0 10213     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  11524]     0 11524     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  12782]     0 12782     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  14050]     0 14050     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  14207]   201 14207      638      116    40960        0             0 bash
Nov 27 00:54:48 Tower kernel: [  15331]     0 15331     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  16599]     0 16599     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  17857]     0 17857     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  19160]     0 19160     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  20414]     0 20414     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  21683]     0 21683     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  23018]     0 23018     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  24269]     0 24269     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  25547]     0 25547     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  26809]     0 26809     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  28081]     0 28081     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  29330]     0 29330     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  30604]     0 30604     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [  31853]     0 31853     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [    658]     0   658     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [   1124]     0  1124      992      688    40960        0             0 sh
Nov 27 00:54:48 Tower kernel: [   1141]     0  1141     1000      754    40960        0             0 run-parts
Nov 27 00:54:48 Tower kernel: [   1160]     0  1160      993      710    40960        0             0 user.script.sta
Nov 27 00:54:48 Tower kernel: [   1161]     0  1161    17191     1830   110592        0             0 startSchedule.p
Nov 27 00:54:48 Tower kernel: [   1982]     0  1982     2505        1    45056        0             1 exe
Nov 27 00:54:48 Tower kernel: [   3274]     0  3274     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [   3351]     0  3351      994      711    40960        0             0 sh
Nov 27 00:54:48 Tower kernel: [   3352]     0  3352    17191     1811   114688        0             0 php
Nov 27 00:54:48 Tower kernel: [   4506]     0  4506     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [   5846]     0  5846     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [   7051]     0  7051     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [   8283]     0  8283     2505        1    40960        0             1 exe
Nov 27 00:54:48 Tower kernel: [   9379]     0  9379      644      228    40960        0             0 sleep
Nov 27 00:54:48 Tower kernel: [   9460]     0  9460     1943     1050    49152        0             0 git
Nov 27 00:54:48 Tower kernel: [   9471]     0  9471      644      230    40960        0             0 sleep
Nov 27 00:54:48 Tower kernel: [   9480]     0  9480      994      758    40960        0             0 sh
Nov 27 00:54:48 Tower kernel: [   9482]     0  9482      820       89    36864        0             0 grep
Nov 27 00:54:48 Tower kernel: [   9483]     0  9483      650      236    40960        0             0 tr
Nov 27 00:54:48 Tower kernel: [   9488]     0  9488   262367     1312   139264        0             1 runc
Nov 27 00:54:48 Tower kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=unraid-api,pid=4880,uid=0
Nov 27 00:54:48 Tower kernel: Out of memory: Killed process 4880 (unraid-api) total-vm:11407308kB, anon-rss:90296kB, file-rss:4kB, shmem-rss:47516kB, UID:0 pgtables:3344kB oom_score_adj:0
Nov 27 00:54:56 Tower kernel: pgrep invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Nov 27 00:54:56 Tower kernel: CPU: 2 PID: 9668 Comm: pgrep Tainted: G     UD W         5.19.17-Unraid #2
Nov 27 00:54:56 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:54:56 Tower kernel: Call Trace:
Nov 27 00:54:56 Tower kernel: <TASK>
Nov 27 00:54:56 Tower kernel: dump_stack_lvl+0x44/0x5c
Nov 27 00:54:56 Tower kernel: dump_header+0x4a/0x1ff
Nov 27 00:54:56 Tower kernel: oom_kill_process+0x80/0x111
Nov 27 00:54:56 Tower kernel: out_of_memory+0x3e8/0x41a
Nov 27 00:54:56 Tower kernel: __alloc_pages_slowpath.constprop.0+0x6eb/0x8ee
Nov 27 00:54:56 Tower kernel: __alloc_pages+0x132/0x1e8
Nov 27 00:54:56 Tower kernel: alloc_slab_page+0x11/0x35
Nov 27 00:54:56 Tower kernel: new_slab+0xa3/0x1df
Nov 27 00:54:56 Tower kernel: ___slab_alloc+0x4a9/0x590
Nov 27 00:54:56 Tower kernel: ? __alloc_file+0x26/0x93
Nov 27 00:54:56 Tower kernel: ? __alloc_file+0x26/0x93
Nov 27 00:54:56 Tower kernel: __slab_alloc.constprop.0+0x4e/0x79
Nov 27 00:54:56 Tower kernel: ? __alloc_file+0x26/0x93
Nov 27 00:54:56 Tower kernel: kmem_cache_alloc+0x8e/0x143
Nov 27 00:54:56 Tower kernel: __alloc_file+0x26/0x93
Nov 27 00:54:56 Tower kernel: alloc_empty_file+0x7f/0xd5
Nov 27 00:54:56 Tower kernel: path_openat+0x4b/0xaa9
Nov 27 00:54:56 Tower kernel: ? proc_pid_status+0x92c/0x956
Nov 27 00:54:56 Tower kernel: ? _copy_to_iter+0x165/0x3f5
Nov 27 00:54:56 Tower kernel: do_filp_open+0x55/0xb8
Nov 27 00:54:56 Tower kernel: ? seq_read_iter+0x330/0x346
Nov 27 00:54:56 Tower kernel: ? slab_post_alloc_hook+0x4d/0x15e
Nov 27 00:54:56 Tower kernel: ? getname_flags+0x29/0x152
Nov 27 00:54:56 Tower kernel: ? kmem_cache_alloc+0x11a/0x143
Nov 27 00:54:56 Tower kernel: do_sys_openat2+0x6c/0xd9
Nov 27 00:54:56 Tower kernel: do_sys_open+0x3a/0x5a
Nov 27 00:54:56 Tower kernel: do_syscall_64+0x68/0x81
Nov 27 00:54:56 Tower kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd
Nov 27 00:54:56 Tower kernel: RIP: 0033:0x1489d0bf10f1
Nov 27 00:54:56 Tower kernel: Code: 75 37 89 f0 25 00 00 41 00 3d 00 00 41 00 74 29 80 3d fa b4 0e 00 00 74 4d 89 da 48 89 ee bf 9c ff ff ff b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 77 77 48 83 c4 68 5b 5d c3 48 8d 84 24 80 00 00
Nov 27 00:54:56 Tower kernel: RSP: 002b:00007ffd654d5b50 EFLAGS: 00000202 ORIG_RAX: 0000000000000101
Nov 27 00:54:56 Tower kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00001489d0bf10f1
Nov 27 00:54:56 Tower kernel: RDX: 0000000000000000 RSI: 00007ffd654d5be0 RDI: 00000000ffffff9c
Nov 27 00:54:56 Tower kernel: RBP: 00007ffd654d5be0 R08: 0000000000000035 R09: 0000000000000073
Nov 27 00:54:56 Tower kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffd654d65e0
Nov 27 00:54:56 Tower kernel: R13: 00007ffd654d5be0 R14: 0000000000000020 R15: 00007ffd654d65e0
Nov 27 00:54:56 Tower kernel: </TASK>
Nov 27 00:54:56 Tower kernel: Mem-Info:
Nov 27 00:54:56 Tower kernel: active_anon:235058 inactive_anon:134145 isolated_anon:0
Nov 27 00:54:56 Tower kernel: active_file:258 inactive_file:3018 isolated_file:49
Nov 27 00:54:56 Tower kernel: unevictable:10469 dirty:20 writeback:0
Nov 27 00:54:56 Tower kernel: slab_reclaimable:107030 slab_unreclaimable:366499
Nov 27 00:54:56 Tower kernel: mapped:34698 shmem:261195 pagetables:2325 bounce:0
Nov 27 00:54:56 Tower kernel: kernel_misc_reclaimable:0
Nov 27 00:54:56 Tower kernel: free:23347 free_pcp:522 free_cma:0
Nov 27 00:54:56 Tower kernel: Node 0 active_anon:940232kB inactive_anon:536580kB active_file:792kB inactive_file:12540kB unevictable:41876kB isolated(anon):0kB isolated(file):152kB mapped:138792kB dirty:80kB writeback:0kB shmem:1044780kB shmem_thp: 38912kB shmem_pmdmapped: 0kB anon_thp: 151552kB writeback_tmp:0kB kernel_stack:7840kB pagetables:9300kB all_unreclaimable? yes
Nov 27 00:54:56 Tower kernel: Node 0 DMA free:14356kB boost:0kB min:284kB low:352kB high:420kB reserved_highatomic:0KB active_anon:0kB inactive_anon:4kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Nov 27 00:54:56 Tower kernel: lowmem_reserve[]: 0 1537 3518 3518
Nov 27 00:54:56 Tower kernel: Node 0 DMA32 free:37328kB boost:0kB min:29416kB low:36768kB high:44120kB reserved_highatomic:0KB active_anon:15416kB inactive_anon:309788kB active_file:0kB inactive_file:4144kB unevictable:160kB writepending:56kB present:1828964kB managed:1740464kB mlocked:160kB bounce:0kB free_pcp:564kB local_pcp:8kB free_cma:0kB
Nov 27 00:54:56 Tower kernel: lowmem_reserve[]: 0 0 1980 1980
Nov 27 00:54:56 Tower kernel: Node 0 Normal free:41704kB boost:0kB min:37876kB low:47344kB high:56812kB reserved_highatomic:4096KB active_anon:924816kB inactive_anon:226664kB active_file:0kB inactive_file:8404kB unevictable:41716kB writepending:24kB present:2101248kB managed:2027836kB mlocked:16kB bounce:0kB free_pcp:1524kB local_pcp:636kB free_cma:0kB
Nov 27 00:54:56 Tower kernel: lowmem_reserve[]: 0 0 0 0
Nov 27 00:54:56 Tower kernel: Node 0 DMA: 2*4kB (ME) 2*8kB (ME) 2*16kB (UM) 3*32kB (UME) 2*64kB (ME) 2*128kB (UM) 2*256kB (ME) 2*512kB (ME) 2*1024kB (ME) 1*2048kB (E) 2*4096kB (M) = 14360kB
Nov 27 00:54:56 Tower kernel: Node 0 DMA32: 365*4kB (UME) 504*8kB (UME) 410*16kB (UM) 305*32kB (UME) 145*64kB (UME) 44*128kB (UME) 1*256kB (U) 0*512kB 1*1024kB (U) 0*2048kB 0*4096kB = 38004kB
Nov 27 00:54:56 Tower kernel: Node 0 Normal: 5229*4kB (M) 1315*8kB (ME) 352*16kB (ME) 75*32kB (ME) 15*64kB (M) 10*128kB (ME) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 41708kB
Nov 27 00:54:56 Tower kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Nov 27 00:54:56 Tower kernel: 264592 total pagecache pages
Nov 27 00:54:56 Tower kernel: 0 pages in swap cache
Nov 27 00:54:56 Tower kernel: Swap cache stats: add 0, delete 0, find 0/0
Nov 27 00:54:56 Tower kernel: Free swap  = 0kB
Nov 27 00:54:56 Tower kernel: Total swap = 0kB
Nov 27 00:54:56 Tower kernel: 986551 pages RAM
Nov 27 00:54:56 Tower kernel: 0 pages HighMem/MovableOnly
Nov 27 00:54:56 Tower kernel: 40636 pages reserved
Nov 27 00:54:56 Tower kernel: 0 pages cma reserved
Nov 27 00:54:56 Tower kernel: Tasks state (memory values in pages):
Nov 27 00:54:56 Tower kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Nov 27 00:54:56 Tower kernel: [    632]     0   632     2595     1194    53248        0         -1000 udevd
Nov 27 00:54:56 Tower kernel: [    917]    68   917      777      607    40960        0             0 dhcpcd
Nov 27 00:54:56 Tower kernel: [    918]     0   918      764      527    40960        0             0 dhcpcd
Nov 27 00:54:56 Tower kernel: [    919]    68   919      758       69    40960        0             0 dhcpcd
Nov 27 00:54:56 Tower kernel: [   1009]     0  1009      667       38    40960        0             0 mcelog
Nov 27 00:54:56 Tower kernel: [   1019]    81  1019     1295      525    49152        0             0 dbus-daemon
Nov 27 00:54:56 Tower kernel: [   1032]     0  1032     1580      660    45056        0             0 elogind-daemon
Nov 27 00:54:56 Tower kernel: [   1066]    44  1066    18680     1038    57344        0             0 ntpd
Nov 27 00:54:56 Tower kernel: [   1073]     0  1073      650       25    40960        0             0 acpid
Nov 27 00:54:56 Tower kernel: [   1087]     0  1087      644      462    45056        0             0 crond
Nov 27 00:54:56 Tower kernel: [   1091]     0  1091      664      363    45056        0             0 atd
Nov 27 00:54:56 Tower kernel: [   1212]     0  1212     1003      628    40960        0             0 monitor_nchan
Nov 27 00:54:56 Tower kernel: [   1649]    68  1649      764       74    40960        0             0 dhcpcd
Nov 27 00:54:56 Tower kernel: [   2358]     0  2358      652      230    40960        0             0 agetty
Nov 27 00:54:56 Tower kernel: [   2359]     0  2359      652      228    40960        0             0 agetty
Nov 27 00:54:56 Tower kernel: [   2360]     0  2360      652      229    40960        0             0 agetty
Nov 27 00:54:56 Tower kernel: [   2361]     0  2361      652      218    45056        0             0 agetty
Nov 27 00:54:56 Tower kernel: [   2362]     0  2362      652      208    40960        0             0 agetty
Nov 27 00:54:56 Tower kernel: [   2363]     0  2363      652      230    40960        0             0 agetty
Nov 27 00:54:56 Tower kernel: [   2392]     0  2392     1086       61    45056        0             0 inetd
Nov 27 00:54:56 Tower kernel: [   2393]     0  2393    86868     1125    90112        0             0 emhttpd
Nov 27 00:54:56 Tower kernel: [   2479]    68  2479      764       74    40960        0             0 dhcpcd
Nov 27 00:54:56 Tower kernel: [   2599]     0  2599    22238     2502   147456        0             0 php-fpm
Nov 27 00:54:56 Tower kernel: [   2652]     0  2652    36865     1040    61440        0             0 nginx
Nov 27 00:54:56 Tower kernel: [   2653]     0  2653    37226     3109    77824        0             0 nginx
Nov 27 00:54:56 Tower kernel: [   2751]     0  2751     1170      865    40960        0             0 rc.flash_backup
Nov 27 00:54:56 Tower kernel: [   3229]     0  3229    35060       69    65536        0             0 shfs
Nov 27 00:54:56 Tower kernel: [   3240]     0  3240   103988     2342   102400        0             0 shfs
Nov 27 00:54:56 Tower kernel: [   3292]     0  3292    70331     1060    73728        0             0 rsyslogd
Nov 27 00:54:56 Tower kernel: [   9451]     0  9451    19876     3432   135168        0             0 smbd
Nov 27 00:54:56 Tower kernel: [   9456]     0  9456    19529     2202   131072        0             0 smbd-notifyd
Nov 27 00:54:56 Tower kernel: [   9457]     0  9457    19529     1079   126976        0             0 cleanupd
Nov 27 00:54:56 Tower kernel: [   9465]     0  9465      647      452    40960        0             0 wsdd2
Nov 27 00:54:56 Tower kernel: [   9468]     0  9468    19010     3062   135168        0             0 winbindd
Nov 27 00:54:56 Tower kernel: [   9470]     0  9470    19031     3708   135168        0             0 winbindd
Nov 27 00:54:56 Tower kernel: [   9485]    61  9485     1411      846    45056        0             0 avahi-daemon
Nov 27 00:54:56 Tower kernel: [   9486]    61  9486     1329       67    45056        0             0 avahi-daemon
Nov 27 00:54:56 Tower kernel: [   9494]     0  9494      669       29    45056        0             0 avahi-dnsconfd
Nov 27 00:54:56 Tower kernel: [   9564]     0  9564   680700    23323   921600        0             0 dockerd
Nov 27 00:54:56 Tower kernel: [   9578]     0  9578   355318    12180   290816        0             0 containerd
Nov 27 00:54:56 Tower kernel: [  10538]     0 10538    19118     3013   135168        0             0 winbindd
Nov 27 00:54:56 Tower kernel: [  11889]     0 11889     2564     1742    53248        0             0 ttyd
Nov 27 00:54:56 Tower kernel: [  22178]     0 22178   178180     4782   118784        0             1 containerd-shim
Nov 27 00:54:56 Tower kernel: [  22199]     0 22199       51        1    28672        0             0 s6-svscan
Nov 27 00:54:56 Tower kernel: [  22246]     0 22246       51        1    28672        0             0 s6-supervise
Nov 27 00:54:56 Tower kernel: [  22538]     0 22538       51        1    28672        0             0 s6-supervise
Nov 27 00:54:56 Tower kernel: [  22541]    99 22541    51988    11247   475136        0             0 Plex Media Serv
Nov 27 00:54:56 Tower kernel: [  22642]    99 22642    16284     8204   143360        0             0 Plex Script Hos
Nov 27 00:54:56 Tower kernel: [  22727]    99 22727    10723      501    94208        0             0 Plex Tuner Serv
Nov 27 00:54:56 Tower kernel: [   5236]     0  5236   178180     4320   118784        0             1 containerd-shim
Nov 27 00:54:56 Tower kernel: [   5256]   201  5256    51942    28390   495616        0             0 netdata
Nov 27 00:54:56 Tower kernel: [   5296]   201  5296     5424      785    81920        0             0 netdata
Nov 27 00:54:56 Tower kernel: [   5542]   201  5542     8189     4515    98304        0             0 python3
Nov 27 00:54:56 Tower kernel: [   5553]   201  5553     1399      889    45056        0             0 apps.plugin
Nov 27 00:54:56 Tower kernel: [   5565]   201  5565   322495     4686   237568        0             0 go.d.plugin
Nov 27 00:54:56 Tower kernel: [  20636]     0 20636    23019     7346   159744        0             0 notify_poller
Nov 27 00:54:56 Tower kernel: [  20639]     0 20639    23019     7249   163840        0             0 session_check
Nov 27 00:54:56 Tower kernel: [  20641]     0 20641    23035     7383   163840        0             0 system_temp
Nov 27 00:54:56 Tower kernel: [  20645]     0 20645    23019     7220   163840        0             0 wg_poller
Nov 27 00:54:56 Tower kernel: [  20650]     0 20650    23019     7267   163840        0             0 update_1
Nov 27 00:54:56 Tower kernel: [  20655]     0 20655    23069     7459   163840        0             0 update_2
Nov 27 00:54:57 Tower kernel: [  20658]     0 20658    23019     7292   163840        0             0 update_3
Nov 27 00:54:57 Tower kernel: [  23536]     0 23536     2564     1256    53248        0             0 ttyd
Nov 27 00:54:57 Tower kernel: [  31889]     0 31889    23079     7456   167936        0             0 device_list
Nov 27 00:54:57 Tower kernel: [  31891]     0 31891     1004      782    40960        0             0 disk_load
Nov 27 00:54:57 Tower kernel: [  31893]     0 31893    23019     7315   163840        0             0 parity_list
Nov 27 00:54:57 Tower kernel: [   4833]     0  4833    23035     7328   163840        0             0 ups_status
Nov 27 00:54:57 Tower kernel: [   7106]     0  7106      992      692    45056        0             0 sh
Nov 27 00:54:57 Tower kernel: [   7107]     0  7107      993      690    45056        0             0 sh
Nov 27 00:54:57 Tower kernel: [   7108]     0  7108    22974     7088   155648        0             0 startCustom.php
Nov 27 00:54:57 Tower kernel: [   7127]     0  7127      994      720    45056        0             0 sh
Nov 27 00:54:57 Tower kernel: [   7189]     0  7189    17191     1798   118784        0             0 notify
Nov 27 00:54:57 Tower kernel: [   7298]     0  7298      994      748    40960        0             0 sh
Nov 27 00:54:57 Tower kernel: [   7299]     0  7299     1286      716    45056        0             0 lsof
Nov 27 00:54:57 Tower kernel: [   7300]     0  7300     1521      626    45056        0             0 awk
Nov 27 00:54:57 Tower kernel: [   7301]     0  7301     1488      635    45056        0             0 awk
Nov 27 00:54:57 Tower kernel: [   7611]     0  7611     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [   7697]     0  7697    17191     1806   114688        0             0 smartctl_type
Nov 27 00:54:57 Tower kernel: [   7698]     0  7698    17191     2377   114688        0             0 smartctl_type
Nov 27 00:54:57 Tower kernel: [   7699]     0  7699    17191     2347   110592        0             0 smartctl_type
Nov 27 00:54:57 Tower kernel: [   8736]     0  8736      992      709    40960        0             0 sh
Nov 27 00:54:57 Tower kernel: [   8737]     0  8737    17191     1794   110592        0             0 php
Nov 27 00:54:57 Tower kernel: [   8881]     0  8881     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  10213]     0 10213     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  11524]     0 11524     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  12782]     0 12782     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  14050]     0 14050     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  14207]   201 14207      638      116    40960        0             0 bash
Nov 27 00:54:57 Tower kernel: [  15331]     0 15331     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  16599]     0 16599     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  17857]     0 17857     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  19160]     0 19160     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  20414]     0 20414     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  21683]     0 21683     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  23018]     0 23018     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  24269]     0 24269     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  25547]     0 25547     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  26809]     0 26809     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  28081]     0 28081     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  29330]     0 29330     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  30604]     0 30604     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [  31853]     0 31853     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [    658]     0   658     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [   1124]     0  1124      992      688    40960        0             0 sh
Nov 27 00:54:57 Tower kernel: [   1141]     0  1141     1000      754    40960        0             0 run-parts
Nov 27 00:54:57 Tower kernel: [   1160]     0  1160      993      710    40960        0             0 user.script.sta
Nov 27 00:54:57 Tower kernel: [   1161]     0  1161    17191     1830   110592        0             0 startSchedule.p
Nov 27 00:54:57 Tower kernel: [   1982]     0  1982     2505        1    45056        0             1 exe
Nov 27 00:54:57 Tower kernel: [   3274]     0  3274     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [   3351]     0  3351      994      711    40960        0             0 sh
Nov 27 00:54:57 Tower kernel: [   3352]     0  3352    17191     1811   114688        0             0 php
Nov 27 00:54:57 Tower kernel: [   4506]     0  4506     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [   5846]     0  5846     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [   7051]     0  7051     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [   8283]     0  8283     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [   9379]     0  9379      644      228    40960        0             0 sleep
Nov 27 00:54:57 Tower kernel: [   9488]     0  9488   279328     2150   147456        0             1 runc
Nov 27 00:54:57 Tower kernel: [   9497]     0  9497     2505        1    40960        0             1 exe
Nov 27 00:54:57 Tower kernel: [   9506]     0  9506      644      228    40960        0             0 sleep
Nov 27 00:54:57 Tower kernel: [   9659]     0  9659      644      228    40960        0             0 sleep
Nov 27 00:54:57 Tower kernel: [   9660]   201  9660      397        7    40960        0             0 sleep
Nov 27 00:54:57 Tower kernel: [   9668]     0  9668     1058      560    45056        0             0 pgrep
Nov 27 00:54:57 Tower kernel: [   9669]     0  9669    23028     7220   163840        0             0 wg_poller
Nov 27 00:54:57 Tower kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/docker/7c19db787702b271bfea22be350873bce784f9f046369c3d3f9afb6c0c978a15,task=netdata,pid=5256,uid=201
Nov 27 00:54:57 Tower kernel: Out of memory: Killed process 5256 (netdata) total-vm:207768kB, anon-rss:113516kB, file-rss:0kB, shmem-rss:44kB, UID:201 pgtables:484kB oom_score_adj:0
Nov 27 00:55:18 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1921990 jiffies s: 3525 root: 0x0/T
Nov 27 00:55:18 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:56:24 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 1987527 jiffies s: 3525 root: 0x0/T
Nov 27 00:56:24 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:57:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 00:57:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 00:57:01 Tower kernel: #011(detected by 1, t=2040192 jiffies, g=6087985, q=5282216 ncpus=4)
Nov 27 00:57:29 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 2053062 jiffies s: 3525 root: 0x0/T
Nov 27 00:57:29 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:58:35 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 2118599 jiffies s: 3525 root: 0x0/T
Nov 27 00:58:35 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:59:40 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 2184135 jiffies s: 3525 root: 0x0/T
Nov 27 00:59:40 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 00:59:56 Tower kernel: disk_load invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=1, oom_score_adj=0
Nov 27 00:59:56 Tower kernel: CPU: 1 PID: 31891 Comm: disk_load Tainted: G     UD W         5.19.17-Unraid #2
Nov 27 00:59:56 Tower kernel: Hardware name: Intel Corporation Jasper Lake Client Platform/0123456789ABCDEF0123456789ABCDEF, BIOS V1.21 11/23/2022
Nov 27 00:59:56 Tower kernel: Call Trace:
Nov 27 00:59:56 Tower kernel: <TASK>
Nov 27 00:59:56 Tower kernel: dump_stack_lvl+0x44/0x5c
Nov 27 00:59:56 Tower kernel: dump_header+0x4a/0x1ff
Nov 27 00:59:56 Tower kernel: oom_kill_process+0x80/0x111
Nov 27 00:59:56 Tower kernel: out_of_memory+0x3e8/0x41a
Nov 27 00:59:56 Tower kernel: __alloc_pages_slowpath.constprop.0+0x6eb/0x8ee
Nov 27 00:59:56 Tower kernel: __alloc_pages+0x132/0x1e8
Nov 27 00:59:56 Tower kernel: __get_free_pages+0xd/0x3c
Nov 27 00:59:56 Tower kernel: pgd_alloc+0x2f/0x16d
Nov 27 00:59:56 Tower kernel: ? dup_mm+0x3a/0x524
Nov 27 00:59:56 Tower kernel: mm_init+0x161/0x218
Nov 27 00:59:56 Tower kernel: dup_mm+0x68/0x524
Nov 27 00:59:56 Tower kernel: ? copy_fs_struct+0x22/0x9c
Nov 27 00:59:56 Tower kernel: ? copy_process+0x916/0x15f0
Nov 27 00:59:56 Tower kernel: ? kmem_cache_alloc+0x11a/0x143
Nov 27 00:59:56 Tower kernel: copy_process+0xb19/0x15f0
Nov 27 00:59:56 Tower kernel: kernel_clone+0xa5/0x27c
Nov 27 00:59:56 Tower kernel: ? alloc_file+0x8f/0x109
Nov 27 00:59:56 Tower kernel: __do_sys_clone+0x65/0x8b
Nov 27 00:59:56 Tower kernel: do_syscall_64+0x68/0x81
Nov 27 00:59:56 Tower kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd
Nov 27 00:59:56 Tower kernel: RIP: 0033:0x14b8a0d05713
Nov 27 00:59:56 Tower kernel: Code: 00 e8 11 5e f5 ff 90 64 48 8b 04 25 10 00 00 00 45 31 c0 31 d2 31 f6 bf 11 00 20 01 4c 8d 90 d0 02 00 00 b8 38 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 35 89 c2 85 c0 75 2c 64 48 8b 04 25 10 00 00
Nov 27 00:59:56 Tower kernel: RSP: 002b:00007ffc46f35cd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
Nov 27 00:59:56 Tower kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 000014b8a0d05713
Nov 27 00:59:56 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
Nov 27 00:59:56 Tower kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
Nov 27 00:59:56 Tower kernel: R10: 000014b8a0c26a10 R11: 0000000000000246 R12: 0000000000000001
Nov 27 00:59:56 Tower kernel: R13: 00007ffc46f35f00 R14: 00000000004eebb7 R15: 0000000000560890
Nov 27 00:59:56 Tower kernel: </TASK>
Nov 27 00:59:56 Tower kernel: Mem-Info:
Nov 27 00:59:56 Tower kernel: active_anon:235054 inactive_anon:92231 isolated_anon:0
Nov 27 00:59:56 Tower kernel: active_file:297 inactive_file:1125 isolated_file:64
Nov 27 00:59:56 Tower kernel: unevictable:10469 dirty:68 writeback:0
Nov 27 00:59:56 Tower kernel: slab_reclaimable:114831 slab_unreclaimable:400667
Nov 27 00:59:56 Tower kernel: mapped:33038 shmem:261200 pagetables:2051 bounce:0
Nov 27 00:59:56 Tower kernel: kernel_misc_reclaimable:0
Nov 27 00:59:56 Tower kernel: free:23315 free_pcp:36 free_cma:0
Nov 27 00:59:56 Tower kernel: Node 0 active_anon:940216kB inactive_anon:368924kB active_file:968kB inactive_file:4636kB unevictable:41876kB isolated(anon):0kB isolated(file):256kB mapped:132152kB dirty:272kB writeback:0kB shmem:1044800kB shmem_thp: 38912kB shmem_pmdmapped: 0kB anon_thp: 94208kB writeback_tmp:0kB kernel_stack:6432kB pagetables:8204kB all_unreclaimable? yes
Nov 27 00:59:56 Tower kernel: Node 0 DMA free:14340kB boost:0kB min:284kB low:352kB high:420kB reserved_highatomic:0KB active_anon:0kB inactive_anon:4kB active_file:0kB inactive_file:20kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Nov 27 00:59:56 Tower kernel: lowmem_reserve[]: 0 1537 3518 3518
Nov 27 00:59:56 Tower kernel: Node 0 DMA32 free:37044kB boost:0kB min:29416kB low:36768kB high:44120kB reserved_highatomic:0KB active_anon:15432kB inactive_anon:190004kB active_file:0kB inactive_file:2528kB unevictable:160kB writepending:144kB present:1828964kB managed:1740464kB mlocked:160kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Nov 27 00:59:56 Tower kernel: lowmem_reserve[]: 0 0 1980 1980
Nov 27 00:59:56 Tower kernel: Node 0 Normal free:41876kB boost:0kB min:37876kB low:47344kB high:56812kB reserved_highatomic:4096KB active_anon:924784kB inactive_anon:178916kB active_file:840kB inactive_file:2144kB unevictable:41716kB writepending:128kB present:2101248kB managed:2027836kB mlocked:16kB bounce:0kB free_pcp:144kB local_pcp:0kB free_cma:0kB
Nov 27 00:59:56 Tower kernel: lowmem_reserve[]: 0 0 0 0
Nov 27 00:59:56 Tower kernel: Node 0 DMA: 1*4kB (E) 2*8kB (ME) 1*16kB (U) 3*32kB (UME) 2*64kB (ME) 2*128kB (UM) 2*256kB (ME) 2*512kB (ME) 2*1024kB (ME) 1*2048kB (E) 2*4096kB (M) = 14340kB
Nov 27 00:59:56 Tower kernel: Node 0 DMA32: 3597*4kB (UME) 526*8kB (UME) 246*16kB (UME) 172*32kB (UME) 92*64kB (UME) 25*128kB (ME) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 37124kB
Nov 27 00:59:56 Tower kernel: Node 0 Normal: 5233*4kB (UM) 1289*8kB (UME) 359*16kB (UME) 87*32kB (UME) 17*64kB (UM) 7*128kB (UM) 0*256kB 1*512kB (U) 0*1024kB 0*2048kB 0*4096kB = 42268kB
Nov 27 00:59:56 Tower kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Nov 27 00:59:56 Tower kernel: 262732 total pagecache pages
Nov 27 00:59:56 Tower kernel: 0 pages in swap cache
Nov 27 00:59:56 Tower kernel: Swap cache stats: add 0, delete 0, find 0/0
Nov 27 00:59:56 Tower kernel: Free swap  = 0kB
Nov 27 00:59:56 Tower kernel: Total swap = 0kB
Nov 27 00:59:56 Tower kernel: 986551 pages RAM
Nov 27 00:59:56 Tower kernel: 0 pages HighMem/MovableOnly
Nov 27 00:59:56 Tower kernel: 40636 pages reserved
Nov 27 00:59:56 Tower kernel: 0 pages cma reserved
Nov 27 00:59:56 Tower kernel: Unreclaimable slab info:
Nov 27 00:59:56 Tower kernel: Name                      Used          Total
Nov 27 00:59:56 Tower kernel: nf_conntrack              92KB         92KB
Nov 27 00:59:56 Tower kernel: xfs_efi_item              15KB         15KB
Nov 27 00:59:56 Tower kernel: xfs_efd_item              15KB         15KB
Nov 27 00:59:56 Tower kernel: xfs_buf_item              23KB         23KB
Nov 27 00:59:56 Tower kernel: xfs_extfree_intent          7KB          7KB
Nov 27 00:59:56 Tower kernel: unraid/md               1270KB       1270KB
Nov 27 00:59:56 Tower kernel: i915_dependency           16KB         16KB
Nov 27 00:59:56 Tower kernel: drm_i915_gem_object        383KB        510KB
Nov 27 00:59:56 Tower kernel: scsi_sense_cache          52KB         52KB
Nov 27 00:59:56 Tower kernel: btrfs_delayed_node        164KB        396KB
Nov 27 00:59:56 Tower kernel: btrfs_ordered_extent        108KB        108KB
Nov 27 00:59:56 Tower kernel: bio-304                  101KB        132KB
Nov 27 00:59:56 Tower kernel: bio-216                   84KB        112KB
Nov 27 00:59:56 Tower kernel: btrfs_free_space        1225KB       1279KB
Nov 27 00:59:56 Tower kernel: RAWv6                     92KB         92KB
Nov 27 00:59:56 Tower kernel: UDPv6                    126KB        126KB
Nov 27 00:59:56 Tower kernel: tw_sock_TCPv6             15KB         15KB
Nov 27 00:59:56 Tower kernel: request_sock_TCPv6         31KB         31KB
Nov 27 00:59:56 Tower kernel: TCPv6                    300KB        300KB
Nov 27 00:59:56 Tower kernel: bio-120                   32KB         32KB
Nov 27 00:59:56 Tower kernel: bfq_io_cq                 11KB         11KB
Nov 27 00:59:56 Tower kernel: mqueue_inode_cache         63KB         63KB
Nov 27 00:59:56 Tower kernel: fuse_request             718KB       2972KB
Nov 27 00:59:56 Tower kernel: kioctx                    15KB         15KB
Nov 27 00:59:56 Tower kernel: dio                      187KB        187KB
Nov 27 00:59:56 Tower kernel: ip4-frags                 15KB         15KB
Nov 27 00:59:56 Tower kernel: xfrm_dst_cache           109KB        109KB
Nov 27 00:59:56 Tower kernel: ip_fib_trie               15KB         15KB
Nov 27 00:59:56 Tower kernel: ip_fib_alias              15KB         15KB
Nov 27 00:59:56 Tower kernel: RAW                       80KB         80KB
Nov 27 00:59:56 Tower kernel: UDP                      157KB        157KB
Nov 27 00:59:56 Tower kernel: tw_sock_TCP               27KB         27KB
Nov 27 00:59:56 Tower kernel: request_sock_TCP          31KB         31KB
Nov 27 00:59:56 Tower kernel: TCP                      306KB        306KB
Nov 27 00:59:56 Tower kernel: hugetlbfs_inode_cache         31KB         31KB
Nov 27 00:59:56 Tower kernel: bio-248                   12KB         12KB
Nov 27 00:59:56 Tower kernel: ep_head                   16KB         16KB
Nov 27 00:59:56 Tower kernel: request_queue            123KB        123KB
Nov 27 00:59:56 Tower kernel: bio-184                   74KB         74KB
Nov 27 00:59:56 Tower kernel: biovec-max              1332KB       1472KB
Nov 27 00:59:56 Tower kernel: biovec-128               228KB        320KB
Nov 27 00:59:56 Tower kernel: biovec-64                 64KB         64KB
Nov 27 00:59:56 Tower kernel: khugepaged_mm_slot         33KB         74KB
Nov 27 00:59:56 Tower kernel: uid_cache               1815KB       1815KB
Nov 27 00:59:56 Tower kernel: skbuff_fclone_cache         96KB         96KB
Nov 27 00:59:56 Tower kernel: skbuff_head_cache         62KB         68KB
Nov 27 00:59:56 Tower kernel: file_lock_cache           41KB         41KB
Nov 27 00:59:56 Tower kernel: taskstats                810KB        810KB
Nov 27 00:59:56 Tower kernel: proc_dir_entry           173KB        173KB
Nov 27 00:59:56 Tower kernel: pde_opener                15KB         15KB
Nov 27 00:59:56 Tower kernel: seq_file                  19KB         19KB
Nov 27 00:59:56 Tower kernel: sigqueue                 151KB        167KB
Nov 27 00:59:56 Tower kernel: shmem_inode_cache      11273KB      11273KB
Nov 27 00:59:56 Tower kernel: kernfs_node_cache       3832KB       4036KB
Nov 27 00:59:56 Tower kernel: mnt_cache                129KB        164KB
Nov 27 00:59:56 Tower kernel: filp                 1093384KB    1093384KB
Nov 27 00:59:56 Tower kernel: names_cache              256KB        256KB
Nov 27 00:59:56 Tower kernel: net_namespace             94KB         94KB
Nov 27 00:59:57 Tower kernel: uts_namespace             30KB         30KB
Nov 27 00:59:57 Tower kernel: nsproxy                   15KB         15KB
Nov 27 00:59:57 Tower kernel: mm_struct                286KB        286KB
Nov 27 00:59:57 Tower kernel: files_cache              173KB        189KB
Nov 27 00:59:57 Tower kernel: signal_cache           27232KB      27232KB
Nov 27 00:59:57 Tower kernel: sighand_cache            670KB        742KB
Nov 27 00:59:57 Tower kernel: task_struct           139299KB     139299KB
Nov 27 00:59:57 Tower kernel: cred_jar               14529KB      14553KB
Nov 27 00:59:57 Tower kernel: anon_vma_chain           678KB        916KB
Nov 27 00:59:57 Tower kernel: anon_vma                 590KB        736KB
Nov 27 00:59:57 Tower kernel: pid                    13192KB      13192KB
Nov 27 00:59:57 Tower kernel: Acpi-Operand             941KB        941KB
Nov 27 00:59:57 Tower kernel: Acpi-Parse                58KB         71KB
Nov 27 00:59:57 Tower kernel: Acpi-State                87KB        103KB
Nov 27 00:59:57 Tower kernel: numa_policy               68KB         76KB
Nov 27 00:59:57 Tower kernel: trace_event_file         142KB        142KB
Nov 27 00:59:57 Tower kernel: ftrace_event_field        477KB        494KB
Nov 27 00:59:57 Tower kernel: pool_workqueue          3308KB       3308KB
Nov 27 00:59:57 Tower kernel: task_group                63KB         63KB
Nov 27 00:59:57 Tower kernel: vmap_area               1680KB       1680KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-8k            128KB        128KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-4k            392KB        480KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-2k            470KB        544KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-1k            228KB        256KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-512            80KB         80KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-256            20KB         20KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-192            78KB         78KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-128            20KB         20KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-96             15KB         15KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-64             16KB         16KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-32             83KB        296KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-16             16KB         16KB
Nov 27 00:59:57 Tower kernel: kmalloc-cg-8              16KB         16KB
Nov 27 00:59:57 Tower kernel: kmalloc-8k               728KB        864KB
Nov 27 00:59:57 Tower kernel: kmalloc-4k              4124KB       4224KB
Nov 27 00:59:57 Tower kernel: kmalloc-2k              6624KB       6624KB
Nov 27 00:59:57 Tower kernel: kmalloc-1k              1371KB       1488KB
Nov 27 00:59:57 Tower kernel: kmalloc-512             2995KB       3320KB
Nov 27 00:59:57 Tower kernel: kmalloc-256             2455KB       2624KB
Nov 27 00:59:57 Tower kernel: kmalloc-192             8497KB       8497KB
Nov 27 00:59:57 Tower kernel: kmalloc-128            34614KB      34624KB
Nov 27 00:59:57 Tower kernel: kmalloc-96               330KB        330KB
Nov 27 00:59:57 Tower kernel: kmalloc-64              1660KB       1660KB
Nov 27 00:59:57 Tower kernel: kmalloc-32               648KB        648KB
Nov 27 00:59:57 Tower kernel: kmalloc-16               121KB        132KB
Nov 27 00:59:57 Tower kernel: kmalloc-8                 47KB         48KB
Nov 27 00:59:57 Tower kernel: kmem_cache_node           20KB         20KB
Nov 27 00:59:57 Tower kernel: kmem_cache                52KB         52KB
Nov 27 00:59:57 Tower kernel: Tasks state (memory values in pages):
Nov 27 00:59:57 Tower kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Nov 27 00:59:57 Tower kernel: [    632]     0   632     2595     1194    53248        0         -1000 udevd
Nov 27 00:59:57 Tower kernel: [    917]    68   917      777      607    40960        0             0 dhcpcd
Nov 27 00:59:57 Tower kernel: [    918]     0   918      764      527    40960        0             0 dhcpcd
Nov 27 00:59:57 Tower kernel: [    919]    68   919      758       69    40960        0             0 dhcpcd
Nov 27 00:59:57 Tower kernel: [   1009]     0  1009      667       38    40960        0             0 mcelog
Nov 27 00:59:57 Tower kernel: [   1019]    81  1019     1295      525    49152        0             0 dbus-daemon
Nov 27 00:59:57 Tower kernel: [   1032]     0  1032     1580      660    45056        0             0 elogind-daemon
Nov 27 00:59:57 Tower kernel: [   1066]    44  1066    18680     1038    57344        0             0 ntpd
Nov 27 00:59:57 Tower kernel: [   1073]     0  1073      650       25    40960        0             0 acpid
Nov 27 00:59:57 Tower kernel: [   1087]     0  1087      644      462    45056        0             0 crond
Nov 27 00:59:57 Tower kernel: [   1091]     0  1091      664      363    45056        0             0 atd
Nov 27 00:59:57 Tower kernel: [   1212]     0  1212     1003      628    40960        0             0 monitor_nchan
Nov 27 00:59:57 Tower kernel: [   1649]    68  1649      764       74    40960        0             0 dhcpcd
Nov 27 00:59:57 Tower kernel: [   2358]     0  2358      652      230    40960        0             0 agetty
Nov 27 00:59:57 Tower kernel: [   2359]     0  2359      652      228    40960        0             0 agetty
Nov 27 00:59:57 Tower kernel: [   2360]     0  2360      652      229    40960        0             0 agetty
Nov 27 00:59:57 Tower kernel: [   2361]     0  2361      652      218    45056        0             0 agetty
Nov 27 00:59:57 Tower kernel: [   2362]     0  2362      652      208    40960        0             0 agetty
Nov 27 00:59:57 Tower kernel: [   2363]     0  2363      652      230    40960        0             0 agetty
Nov 27 00:59:57 Tower kernel: [   2392]     0  2392     1086       61    45056        0             0 inetd
Nov 27 00:59:57 Tower kernel: [   2393]     0  2393    86868     1125    90112        0             0 emhttpd
Nov 27 00:59:57 Tower kernel: [   2479]    68  2479      764       74    40960        0             0 dhcpcd
Nov 27 00:59:57 Tower kernel: [   2599]     0  2599    22238     2502   147456        0             0 php-fpm
Nov 27 00:59:57 Tower kernel: [   2652]     0  2652    36865     1040    61440        0             0 nginx
Nov 27 00:59:57 Tower kernel: [   2653]     0  2653    37226     3109    77824        0             0 nginx
Nov 27 00:59:57 Tower kernel: [   2751]     0  2751     1170      865    40960        0             0 rc.flash_backup
Nov 27 00:59:57 Tower kernel: [   3229]     0  3229    35060       69    65536        0             0 shfs
Nov 27 00:59:57 Tower kernel: [   3240]     0  3240   103988     2342   102400        0             0 shfs
Nov 27 00:59:57 Tower kernel: [   3292]     0  3292    70331     1125    73728        0             0 rsyslogd
Nov 27 00:59:57 Tower kernel: [   9451]     0  9451    19876     3432   135168        0             0 smbd
Nov 27 00:59:57 Tower kernel: [   9456]     0  9456    19529     2202   131072        0             0 smbd-notifyd
Nov 27 00:59:57 Tower kernel: [   9457]     0  9457    19529     1079   126976        0             0 cleanupd
Nov 27 00:59:57 Tower kernel: [   9465]     0  9465      647      452    40960        0             0 wsdd2
Nov 27 00:59:57 Tower kernel: [   9468]     0  9468    19010     3062   135168        0             0 winbindd
Nov 27 00:59:57 Tower kernel: [   9470]     0  9470    19031     3708   135168        0             0 winbindd
Nov 27 00:59:57 Tower kernel: [   9485]    61  9485     1411      846    45056        0             0 avahi-daemon
Nov 27 00:59:57 Tower kernel: [   9486]    61  9486     1329       67    45056        0             0 avahi-daemon
Nov 27 00:59:57 Tower kernel: [   9494]     0  9494      669       29    45056        0             0 avahi-dnsconfd
Nov 27 00:59:57 Tower kernel: [   9564]     0  9564   680700    21755   921600        0             0 dockerd
Nov 27 00:59:57 Tower kernel: [   9578]     0  9578   355318    12158   290816        0             0 containerd
Nov 27 00:59:57 Tower kernel: [  10538]     0 10538    19118     3013   135168        0             0 winbindd
Nov 27 00:59:57 Tower kernel: [  11889]     0 11889     2564     1742    53248        0             0 ttyd
Nov 27 00:59:57 Tower kernel: [  22178]     0 22178   178180     4750   118784        0             1 containerd-shim
Nov 27 00:59:57 Tower kernel: [  22199]     0 22199       51        1    28672        0             0 s6-svscan
Nov 27 00:59:57 Tower kernel: [  22246]     0 22246       51        1    28672        0             0 s6-supervise
Nov 27 00:59:57 Tower kernel: [  22538]     0 22538       51        1    28672        0             0 s6-supervise
Nov 27 00:59:57 Tower kernel: [  22541]    99 22541    53533    11252   479232        0             0 Plex Media Serv
Nov 27 00:59:57 Tower kernel: [  22642]    99 22642    16284     8203   143360        0             0 Plex Script Hos
Nov 27 00:59:57 Tower kernel: [  22727]    99 22727    10723      501    94208        0             0 Plex Tuner Serv
Nov 27 00:59:57 Tower kernel: [   5236]     0  5236   178180     4264   118784        0             1 containerd-shim
Nov 27 00:59:57 Tower kernel: [  20636]     0 20636    23019     7346   159744        0             0 notify_poller
Nov 27 00:59:57 Tower kernel: [  20639]     0 20639    23019     7249   163840        0             0 session_check
Nov 27 00:59:57 Tower kernel: [  20641]     0 20641    23035     7383   163840        0             0 system_temp
Nov 27 00:59:57 Tower kernel: [  20645]     0 20645    23028     7220   163840        0             0 wg_poller
Nov 27 00:59:57 Tower kernel: [  20650]     0 20650    23019     7267   163840        0             0 update_1
Nov 27 00:59:57 Tower kernel: [  20655]     0 20655    23069     7459   163840        0             0 update_2
Nov 27 00:59:57 Tower kernel: [  20658]     0 20658    23019     7292   163840        0             0 update_3
Nov 27 00:59:57 Tower kernel: [  23536]     0 23536     2564     1256    53248        0             0 ttyd
Nov 27 00:59:57 Tower kernel: [  31889]     0 31889    23079     7456   167936        0             0 device_list
Nov 27 00:59:57 Tower kernel: [  31891]     0 31891     1004      782    40960        0             0 disk_load
Nov 27 00:59:57 Tower kernel: [  31893]     0 31893    23019     7315   163840        0             0 parity_list
Nov 27 00:59:57 Tower kernel: [   4833]     0  4833    23035     7328   163840        0             0 ups_status
Nov 27 00:59:57 Tower kernel: [   7106]     0  7106      992      692    45056        0             0 sh
Nov 27 00:59:57 Tower kernel: [   7107]     0  7107      993      690    45056        0             0 sh
Nov 27 00:59:57 Tower kernel: [   7108]     0  7108    22974     7088   155648        0             0 startCustom.php
Nov 27 00:59:57 Tower kernel: [   7127]     0  7127      994      720    45056        0             0 sh
Nov 27 00:59:57 Tower kernel: [   7189]     0  7189    17191     1798   118784        0             0 notify
Nov 27 00:59:57 Tower kernel: [   7298]     0  7298      994      748    40960        0             0 sh
Nov 27 00:59:57 Tower kernel: [   7299]     0  7299     1286      716    45056        0             0 lsof
Nov 27 00:59:57 Tower kernel: [   7300]     0  7300     1521      626    45056        0             0 awk
Nov 27 00:59:57 Tower kernel: [   7301]     0  7301     1488      635    45056        0             0 awk
Nov 27 00:59:57 Tower kernel: [   7611]     0  7611     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [   7697]     0  7697    17191     1806   114688        0             0 smartctl_type
Nov 27 00:59:57 Tower kernel: [   7698]     0  7698    17191     2377   114688        0             0 smartctl_type
Nov 27 00:59:57 Tower kernel: [   7699]     0  7699    17191     2347   110592        0             0 smartctl_type
Nov 27 00:59:57 Tower kernel: [   8736]     0  8736      992      709    40960        0             0 sh
Nov 27 00:59:57 Tower kernel: [   8737]     0  8737    17191     1794   110592        0             0 php
Nov 27 00:59:57 Tower kernel: [   8881]     0  8881     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  10213]     0 10213     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  11524]     0 11524     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  12782]     0 12782     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  14050]     0 14050     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  15331]     0 15331     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  16599]     0 16599     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  17857]     0 17857     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  19160]     0 19160     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  20414]     0 20414     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  21683]     0 21683     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  23018]     0 23018     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  24269]     0 24269     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  25547]     0 25547     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  26809]     0 26809     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  28081]     0 28081     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  29330]     0 29330     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  30604]     0 30604     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  31853]     0 31853     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [    658]     0   658     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [   1124]     0  1124      992      688    40960        0             0 sh
Nov 27 00:59:57 Tower kernel: [   1141]     0  1141     1000      754    40960        0             0 run-parts
Nov 27 00:59:57 Tower kernel: [   1160]     0  1160      993      710    40960        0             0 user.script.sta
Nov 27 00:59:57 Tower kernel: [   1161]     0  1161    17191     1830   110592        0             0 startSchedule.p
Nov 27 00:59:57 Tower kernel: [   1982]     0  1982     2505        1    45056        0             1 exe
Nov 27 00:59:57 Tower kernel: [   3274]     0  3274     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [   3351]     0  3351      994      711    40960        0             0 sh
Nov 27 00:59:57 Tower kernel: [   3352]     0  3352    17191     1811   114688        0             0 php
Nov 27 00:59:57 Tower kernel: [   4506]     0  4506     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [   5846]     0  5846     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [   7051]     0  7051     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [   8283]     0  8283     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [   9497]     0  9497     2505        1    40960        0             1 exe
Nov 27 00:59:57 Tower kernel: [  12322]     0 12322      644      232    40960        0             0 sleep
Nov 27 00:59:57 Tower kernel: [  12424]     0 12424      644      207    40960        0             0 sleep
Nov 27 00:59:57 Tower kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=dockerd,pid=9564,uid=0
Nov 27 00:59:57 Tower kernel: Out of memory: Killed process 9564 (dockerd) total-vm:2722800kB, anon-rss:47972kB, file-rss:4kB, shmem-rss:39044kB, UID:0 pgtables:900kB oom_score_adj:0
Nov 27 01:00:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Nov 27 01:00:01 Tower kernel: rcu: #011Tasks blocked on level-0 rcu_node (CPUs 0-3): P7109/1:b..l
Nov 27 01:00:01 Tower kernel: #011(detected by 3, t=2220199 jiffies, g=6087985, q=5506866 ncpus=4)
Nov 27 01:00:46 Tower kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P7128 P7109 } 2249670 jiffies s: 3525 root: 0x0/T
Nov 27 01:00:46 Tower kernel: rcu: blocking rcu_node structures (internal RCU debug):
Nov 27 07:41:14 Tower root: Delaying execution of fix common problems scan for 10 minutes
Nov 27 07:41:14 Tower  emhttpd: Starting services...
Nov 27 07:41:14 Tower  emhttpd: shcmd (45): /etc/rc.d/rc.samba restart

Please God don't jynx me for making this post. 

 

I had a completely stable unraid server for months. Then mid-summer I started crashing nightly. Sometimes crashing mid-day.

I've tried so many things. Downgrades to 6.11.5. Downgrading plex versions. Dinking with so many settings.

 

As I was at my wits end, I did a completely fresh factory install of 6.12.4... And it still crashed

 

But coincidentally, on that very day, 6.12.5 came out. I followed poor scientific method, and did change a few things rapidly, so I'm not 100% sure on what the resolution is, But I haven't had a crash for 4 days now.

 

Upgrade to 6.12.5

 

disabled 2nd cache drive (btrfs raid1). (I've just re-enabled it today, to get protection back in cache, will see if crashes resume)

 

disabled drive spindown (if things stay stable from the prior step for a few days, ill turn this back on at 1 hour or something)

 

switched plex from plexinc to lsio (will probably keep this permanently, as most people seem to like it better)

 

applied the dc=0 i915 fix as recommended in the 6.12.5 release notes. (nothing in my logs indicated crashes were 915 related, and the "factory reset" crash didn't even have modprobe 915 turned on so I don't think its this - most of the crashes indicated either super high iowait, or something spawning hundreds of processes.)