embattled-solidarity5001

Hi all, A friend of mine is running Unraid 6.12.14, it seems a process is randomly deleting folders (with contents). I have attempted to use both inotify and the "file activity" plugin to troubleshoot, unfortunately both of these very annoyingly do not show the name of the process that triggered the deletion. I suspect either sab, sonarr, or radarr is triggering this deletion. They have processes setup to download files to .../mnt/disks/< .... nvme unassigned ...>/complete/tv. After download the files are moved to a share on the array. This part is working. Unfortunately something is causing the /tv dir to delete, resulting in having to manually recreate the dir every day or so. I encouraged them to place a random .txt file into the directory. The process deleted the folder along with this txt file. I would like to run a log to determine what process is causing the file to delete. As noted inotify and file activity both seem to be unhelpful - as they do not indicate the process that initiated the deletion. Thank-you all.

On an irregular (but consistent) pattern my Unraid server stops responding, it could happen after 5 days, 40 days, 33 days. Quite regularily, usually within 60 days) the server stops responding and enters a hanging state. I configured a remote syslog monitor, and pasted the server activity that occured immediately prior to the last hang. To add to the weirdness... The only way to recover from the hang is: force shutdown (hold power button). The system will then not start backup into the unraid boot menu. Instead it will simply present the bios flash startup screen, then enter a black screen of nothing. If I force restart it continually does this until I perform the following action: 1) Enter the system BIOS settings 2) Quite the BIOS setup utility without saving changes 3) like magic the system will now boot normall and bring me to the Unraid startup screen. Can someone please help me troubleshoot why this is happening? 2023-08-22 4:46:37 AMr.localdomainInformationaluser-level2023-08-22 4:46SServeremhttpdspinning down /dev/sdc 2023-08-22 5:06:13 AMr.localdomainInformationaluser-level2023-08-22 5:06SServeremhttpdread SMART /dev/sdc 2023-08-22 7:54:53 AMr.localdomainInformationaluser-level2023-08-22 7:54SServeremhttpdspinning down /dev/sdc 2023-08-22 8:02:49 AMr.localdomainInformationaluser-level2023-08-22 8:02SServeremhttpdread SMART /dev/sdc 2023-08-22 9:04:22 AMr.localdomainInformationaluser-level2023-08-22 9:04SServeremhttpdspinning down /dev/sdc 2023-08-22 9:35:27 AMr.localdomainAlertkernel2023-08-22 9:35SServerkernelBUG: kernel NULL pointer dereference, address: 0000000000000037 2023-08-22 9:35:27 AMr.localdomainAlertkernel2023-08-22 9:35SServerkernel#PF: supervisor read access in kernel mode 2023-08-22 9:35:27 AMr.localdomainAlertkernel2023-08-22 9:35SServerkernel#PF: error_code(0x0000) - not-present page 2023-08-22 9:35:27 AMr.localdomainInformationalkernel2023-08-22 9:35SServerkernelPGD 367947067 P4D 367947067 PUD 0 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelOops: 0000 [#1] SMP NOPTI 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCPU: 6 PID: 28198 Comm: kworker/u32:1 Tainted: P W O 5.10.28-Unraid #1 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelHardware name: Micro-Star International Co., Ltd. MS-7D08/MAG Z590 TOMAHAWK WIFI (MS-7D08), BIOS 2.40 01/13/2022 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelWorkqueue: events_power_efficient gc_worker [nf_conntrack] 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRIP: 0010:nf_ct_tuplehash_to_ctrack+0x0/0xe [nf_conntrack] 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCode: c3 48 8b 07 48 8b 57 08 a8 01 48 89 02 75 04 48 89 50 08 c3 48 8b 06 48 89 77 08 48 89 07 a8 01 48 89 3e 75 04 48 89 78 08 c3 <0f> b6 47 37 48 6b c0 c8 48 8d 44 07 f0 c3 48 8b 87 b8 00 00 00 48 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRSP: 0018:ffffc9000072fe40 EFLAGS: 00010206 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRAX: 0000000006053ec8 RBX: 0000000000000000 RCX: ffff88813f980000 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRDX: 00000001f9fac138 RSI: ffffc9000072fe5c RDI: 0000000000000000 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRBP: 000000000000ced0 R08: 0000000000000000 R09: 0000746e65696369 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelR10: 8080808080808080 R11: fefefefefefefeff R12: ffffffffa01cf5a0 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelR13: 0000000000000003 R14: 0000000000000000 R15: ffff88813c4c8b30 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelFS: 0000000000000000(0000) GS:ffff88903fb80000(0000) knlGS:0000000000000000 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCR2: 0000000000000037 CR3: 00000003659ea002 CR4: 0000000000770ee0 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelPKRU: 55555554 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCall Trace: 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelgc_worker+0x9a/0x240 [nf_conntrack] 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelprocess_one_work+0x13c/0x1d5 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelworker_thread+0x18b/0x22f 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernel? process_scheduled_works+0x27/0x27 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelkthread+0xe5/0xea 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernel? __kthread_bind_mask+0x57/0x57 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelret_from_fork+0x1f/0x30 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelModules linked in: nvidia_uvm(PO) veth xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle nf_tables vhost_net tun vhost vhost_iotlb tap macvlan xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xfs md_mod nvidia_drm(PO) nvidia_modeset(PO) drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops nvidia(PO) drm agpgart iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libblake2s blake2s_x86_64 libblake2s_generic libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables bonding x86_pkg_temp_thermal intel_powerclamp coretemp wmi_bmof mxm_wmi kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel mpt3sas aesni_intel crypto_simd nvme cryptd i2c_i801 nvme_core i2c_smbus video ahci glue_helper input_leds raid_class i2c_core 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelled_class scsi_transport_sas igc wmi libahci backlight thermal acpi_pad button fan 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCR2: 0000000000000037 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernel---[ end trace 5c65714ead4dcc94 ]--- 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRIP: 0010:nf_ct_tuplehash_to_ctrack+0x0/0xe [nf_conntrack] 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCode: c3 48 8b 07 48 8b 57 08 a8 01 48 89 02 75 04 48 89 50 08 c3 48 8b 06 48 89 77 08 48 89 07 a8 01 48 89 3e 75 04 48 89 78 08 c3 <0f> b6 47 37 48 6b c0 c8 48 8d 44 07 f0 c3 48 8b 87 b8 00 00 00 48 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRSP: 0018:ffffc9000072fe40 EFLAGS: 00010206 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRAX: 0000000006053ec8 RBX: 0000000000000000 RCX: ffff88813f980000 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRDX: 00000001f9fac138 RSI: ffffc9000072fe5c RDI: 0000000000000000 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelRBP: 000000000000ced0 R08: 0000000000000000 R09: 0000746e65696369 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelR10: 8080808080808080 R11: fefefefefefefeff R12: ffffffffa01cf5a0 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelR13: 0000000000000003 R14: 0000000000000000 R15: ffff88813c4c8b30 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelFS: 0000000000000000(0000) GS:ffff88903fb80000(0000) knlGS:0000000000000000 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelCR2: 0000000000000037 CR3: 00000003659ea002 CR4: 0000000000770ee0 2023-08-22 9:35:27 AMr.localdomainWarningkernel2023-08-22 9:35SServerkernelPKRU: 55555554