• [6.12.4] Server hangs once a day since updating to 6.12.4

    bastl

    By bastl

      • Urgent

    Hello everyone,

     

    coming from 6.12.2 with an stable server, the 6.12.4 update I did a week ago broke something. Once a day I find the server frozen, mostly in the morning. No WebUI, no SMB access, SSH or ping. No response. I have to force reboot the system.

     

    Main use for the server is for light media consumption with Jellyfin, Nextcloud sync from phone (CalDav, CardDav),
    Unifi etc. and from time to time some media conversation with Tdarr or Handbrake dockers, rarly some remote access with WG. Most dockers are running on idle also a VM or two doing nothing. Most time of the day the server is idle. No config changes on my side with the last update. No custom scripts running during this time.

     

    On 6.12.2 the server never had any issues or crashes. It started the night after the update.

     

    I activated the syslog server and catched the latest crash. 

    Sep 29 19:13:48 mini root: /mnt/cache: 284 GiB (304924037120 bytes) trimmed on /dev/nvme0n1p1
Sep 30 02:44:09 mini kernel: general protection fault, maybe for address 0xffffc900033abe6c: 0000 [#1] PREEMPT SMP NOPTI
Sep 30 02:44:09 mini kernel: CPU: 6 PID: 31855 Comm: ps Tainted: P           O       6.1.49-Unraid #1
Sep 30 02:44:09 mini kernel: Hardware name: BESSTAR TECH LIMITED HM90/HM90, BIOS 5.16 10/13/2021
Sep 30 02:44:09 mini kernel: RIP: 0010:mntput_no_expire+0x59/0x1f2
Sep 30 02:44:09 mini kernel: Code: 2e e7 ff 48 8b 83 e8 00 00 00 48 85 c0 74 16 48 8b 7b 50 83 ce ff e8 2f ef ff ff e8 cc 7a e7 ff e9 78 01 00 00 e8 91 ed ff ff <f0> 83 44 24 fc 00 48 8b 7b 50 83 ce ff e8 0e ef ff ff 48 89 df e8
Sep 30 02:44:09 mini kernel: RSP: 0018:ffffc900033abe70 EFLAGS: 00010286
Sep 30 02:44:09 mini kernel: RAX: 0000000000000000 RBX: ffff888134bf0838 RCX: 0000000000000064
Sep 30 02:44:09 mini kernel: RDX: 0000000000000001 RSI: 00000000ffffffff RDI: ffff888134bf09c8
Sep 30 02:44:09 mini kernel: RBP: ffff888106220b00 R08: 0000000000000000 R09: ffff888134bf0858
Sep 30 02:44:09 mini kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 00000000000a801d
Sep 30 02:44:09 mini kernel: R13: ffff888134bf0858 R14: ffff88818ab54e40 R15: 0000000000000000
Sep 30 02:44:09 mini kernel: FS:  0000147c21ef77c0(0000) GS:ffff888712d80000(0000) knlGS:0000000000000000
Sep 30 02:44:09 mini kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 30 02:44:09 mini kernel: CR2: 00000cdb8942f000 CR3: 000000033b1e2000 CR4: 0000000000350ee0
Sep 30 02:44:09 mini kernel: Call Trace:
Sep 30 02:44:09 mini kernel: <TASK>
Sep 30 02:44:09 mini kernel: ? __die_body+0x1a/0x5c
Sep 30 02:44:09 mini kernel: ? die_addr+0x38/0x51
Sep 30 02:44:09 mini kernel: ? exc_general_protection+0x30f/0x345
Sep 30 02:44:09 mini kernel: ? asm_exc_general_protection+0x22/0x30
Sep 30 02:44:09 mini kernel: ? mntput_no_expire+0x59/0x1f2
Sep 30 02:44:09 mini kernel: ? mntput_no_expire+0x6b/0x1f2
Sep 30 02:44:09 mini kernel: ? dput+0x39/0x17b
Sep 30 02:44:09 mini kernel: ? __fput+0x19f/0x1d2
Sep 30 02:44:09 mini kernel: ? task_work_run+0x6b/0x80
Sep 30 02:44:09 mini kernel: ? exit_to_user_mode_prepare+0x75/0x10d
Sep 30 02:44:09 mini kernel: ? syscall_exit_to_user_mode+0x18/0x2c
Sep 30 02:44:09 mini kernel: ? do_syscall_64+0x77/0x81
Sep 30 02:44:09 mini kernel: ? entry_SYSCALL_64_after_hwframe+0x64/0xce
Sep 30 02:44:09 mini kernel: </TASK>
Sep 30 02:44:09 mini kernel: Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap xt_nat xt_tcpudp veth macvlan xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter dm_crypt dm_mod xfs nfsd auth_rpcgss oid_registry lockd grace sunrpc md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) it87 tcp_diag inet_diag hwmon_vid vendor_reset(O) iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs bridge stp llc igc r8169 realtek amdgpu edac_mce_amd edac_core intel_rapl_msr intel_rapl_common iosf_mbi gpu_sched drm_buddy kvm_amd i2c_algo_bit drm_ttm_helper ttm drm_display_helper kvm
Sep 30 02:44:09 mini kernel: drm_kms_helper drm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 btusb btrtl aesni_intel btbcm btintel crypto_simd cryptd bluetooth agpgart i2c_piix4 syscopyarea rapl ahci ecdh_generic nvme sysfillrect i2c_core k10temp libahci amd_sfh ecc sysimgblt ccp fb_sys_fops nvme_core tpm_crb tpm_tis video tpm_tis_core wmi tpm backlight acpi_cpufreq button unix [last unloaded: igc]
Sep 30 02:44:09 mini kernel: ---[ end trace 0000000000000000 ]---
Sep 30 02:44:09 mini kernel: RIP: 0010:mntput_no_expire+0x59/0x1f2
Sep 30 02:44:09 mini kernel: Code: 2e e7 ff 48 8b 83 e8 00 00 00 48 85 c0 74 16 48 8b 7b 50 83 ce ff e8 2f ef ff ff e8 cc 7a e7 ff e9 78 01 00 00 e8 91 ed ff ff <f0> 83 44 24 fc 00 48 8b 7b 50 83 ce ff e8 0e ef ff ff 48 89 df e8
Sep 30 02:44:09 mini kernel: RSP: 0018:ffffc900033abe70 EFLAGS: 00010286
Sep 30 02:44:09 mini kernel: RAX: 0000000000000000 RBX: ffff888134bf0838 RCX: 0000000000000064
Sep 30 02:44:09 mini kernel: RDX: 0000000000000001 RSI: 00000000ffffffff RDI: ffff888134bf09c8
Sep 30 02:44:09 mini kernel: RBP: ffff888106220b00 R08: 0000000000000000 R09: ffff888134bf0858
Sep 30 02:44:09 mini kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 00000000000a801d
Sep 30 02:44:09 mini kernel: R13: ffff888134bf0858 R14: ffff88818ab54e40 R15: 0000000000000000
Sep 30 02:44:09 mini kernel: FS:  0000147c21ef77c0(0000) GS:ffff888712d80000(0000) knlGS:0000000000000000
Sep 30 02:44:09 mini kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 30 02:44:09 mini kernel: CR2: 00000cdb8942f000 CR3: 000000033b1e2000 CR4: 0000000000350ee0
Sep 30 02:44:09 mini kernel: note: ps[31855] exited with preempt_count 2

    mini-diagnostics-20230930-1231.zip

     

    No idea how to fix this issue. Any help is appreciated.

     

    syslog-10.0.0.4.log

    Go to reports Stable Releases

    User Feedback

    Recommended Comments



    bastl

    Small update. Server crashed again, yesterday in the evening. This time syslog didn't catched anything suspicious. I did a 14h memtest with no errors.

    Link to comment
    Admin9705

    Like you, I had this problem. What I did was just downgraded to 6.11.5 and uptime has been 17 days so far. On 6.12.1 - 12.4, always some random hang after several hours.

    Link to comment
    locmer

    Same here upgrade from 6.12.0 to 6.12.4. Hard hangs within 24 hours. No previous hangs ever occurred. System very very lighty loaded.

    Link to comment
    subven

    Most likely its the macvlan kernel issue. You have to switch from macvlan to ipvlan in Settings --> Docker. Look up the forum there are several issues regarding this topic. I have the same symptoms you described and hope this will fix it.

    Link to comment
    Juugis

    Never was running macvlan. By default ipvlan is enabled. tried fresh install of unraid 6.12.2 then upgraded to 6.12.4 on both versions getting system lock up. Seems that lockup happens on heavy cpu load after server has been running for more than 20 hours. 

    Link to comment
    bastl

    another small update:

     

    Still no idea, whats causing my server crashes on 6.12.4. I tried a couple things

    • disabled all dockers, only 2 VMs running: crash
    • no VMs, only a couple dockers running: random crash
    • no docker no VMs: crash
    • switched the Docker custom network type from macvlan to ipvlan even without having any macvlan call traces before: same issue: crash/freeze

     

    For all the crashes during the last couple of days the server was basically idle and the syslog server didn't catched anything.

     

    Any ideas?

     

    Link to comment
    JorgeB

    There are some reports of possible hangs due to OOM issues caused by the kernel failing to invoke the OOM killer, create the small script below with the user scripts plugin and schedule it to run hourly, it will output the memory stats to the syslog, then see if there's anything abnormal in the persistent syslog.

      

    #!/bin/bash
free -h |& logger &

     

     

    Link to comment
    bastl
    4 hours ago, JorgeB said:

    There are some reports of possible hangs due to OOM issues caused by the kernel failing to invoke the OOM killer, create the small script below with the user scripts plugin and schedule it to run hourly, it will output the memory stats to the syslog, then see if there's anything abnormal in the persistent syslog.

      

    #!/bin/bash
free -h |& logger &

     

    Thanks. Fresh start of the server and first run of the script: 

    Oct  8 16:36:20 mini emhttpd: cmd: /usr/local/emhttp/plugins/user.scripts/backgroundScript.sh /tmp/user.scripts/tmpScripts/OOM_test_script_testing_crashes/script
Oct  8 16:36:20 mini root:                total        used        free      shared  buff/cache   available
Oct  8 16:36:20 mini root: Mem:            27Gi       8.9Gi       4.3Gi       160Mi        14Gi        17Gi
Oct  8 16:36:20 mini root: Swap:             0B          0B          0B

    I will report back 👍

    Link to comment
    bmac6996

    just wanted to add, also having the same issue.  i usually wait for a while to do the next update, and jumped straight from 6.11.5 to 6.12.4.  Within 24 hours, i've have 2 crashes.. ip unpingable and can't wake server up.  Have to reboot.  

     

    I'l try to figure out turning on syslog to see if i can catch anything useful.

     

    If this is macvlan issue, i'll just have to roll back to 6.11.5.  Most of my dockers are running on the host network.

     

    Link to comment
    JorgeB
    1 hour ago, bmac6996 said:

    If this is macvlan issue, i'll just have to roll back to 6.11.5. 

    See the release notes, you can still use macvlan with v6.12.4

    Link to comment
    JorgeB
    55 minutes ago, bastl said:

    No unusual use of the server during that time. No plugin or docker updates, no media consumption, no restart.

    Looks like the RAM usage is not the problem, it's been pretty steady, unfortunately there's nothing else logged to give a clue, can you try rolling back to the previous known good release with your server to confirm the issue stops?

    Link to comment
    bastl
    1 hour ago, JorgeB said:

    rolling back to the previous known good release

    6.12.2 is the latest stable version i used. I`ll report back

    Link to comment
    bastl

    @JorgeB Ok, now server is also crashing on 6.12.2. I changed nothing else, only rolled back the update within Unraid itself.

     

    full syslog:  syslog.txt

     

    last couple lines: 

    Oct 11 16:41:06 mini kernel: divide error: 0000 [#1] PREEMPT SMP NOPTI
Oct 11 16:41:06 mini kernel: CPU: 7 PID: 0 Comm: swapper/7 Tainted: P           O       6.1.36-Unraid #1
Oct 11 16:41:06 mini kernel: Hardware name: BESSTAR TECH LIMITED HM90/HM90, BIOS 5.16 10/13/2021
Oct 11 16:41:06 mini kernel: RIP: 0010:flush_smp_call_function_queue+0x64/0x83
Oct 11 16:41:06 mini kernel: Code: e8 f6 f5 76 00 bf 01 00 00 00 e8 fb f4 ff ff 48 c7 c7 dd 59 10 82 e8 e0 f5 76 00 65 66 8b 05 ca e0 f2 7e 66 85 c0 74 05 e8 fd <45> f7 ff 0f ba e3 09 73 06 fb 0f 1f 44 00 00 5b e9 ae 34 b0 00 0f
Oct 11 16:41:06 mini kernel: RSP: 0018:ffffc9000019fee8 EFLAGS: 00010647
Oct 11 16:41:06 mini kernel: RAX: 0000000000000000 RBX: 0000000000000286 RCX: 00000000000f4240
Oct 11 16:41:06 mini kernel: RDX: 0000000000000002 RSI: ffffffff821059dd RDI: ffffffff820ba9d5
Oct 11 16:41:06 mini kernel: RBP: ffffffff823235a0 R08: ffff888712ded470 R09: ffff888712ded470
Oct 11 16:41:06 mini kernel: R10: 0000000000000000 R11: 0000000000000075 R12: 0000000000000007
Oct 11 16:41:06 mini kernel: R13: ffff88810090de80 R14: 0000000000000001 R15: 0000000000000000
Oct 11 16:41:06 mini kernel: FS:  0000000000000000(0000) GS:ffff888712dc0000(0000) knlGS:0000000000000000
Oct 11 16:41:06 mini kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 11 16:41:06 mini kernel: CR2: 0000001e6780b000 CR3: 0000000154e22000 CR4: 0000000000350ee0
Oct 11 16:41:06 mini kernel: Call Trace:
Oct 11 16:41:06 mini kernel: <TASK>
Oct 11 16:41:06 mini kernel: ? __die_body+0x1a/0x5c
Oct 11 16:41:06 mini kernel: ? die+0x30/0x49
Oct 11 16:41:06 mini kernel: ? do_trap+0x7b/0xfe
Oct 11 16:41:06 mini kernel: ? flush_smp_call_function_queue+0x64/0x83
Oct 11 16:41:06 mini kernel: ? flush_smp_call_function_queue+0x64/0x83
Oct 11 16:41:06 mini kernel: ? do_error_trap+0x6e/0x98
Oct 11 16:41:06 mini kernel: ? flush_smp_call_function_queue+0x64/0x83
Oct 11 16:41:06 mini kernel: ? exc_divide_error+0x34/0x41
Oct 11 16:41:06 mini kernel: ? flush_smp_call_function_queue+0x64/0x83
Oct 11 16:41:06 mini kernel: ? asm_exc_divide_error+0x16/0x20
Oct 11 16:41:06 mini kernel: ? flush_smp_call_function_queue+0x64/0x83
Oct 11 16:41:06 mini kernel: ? flush_smp_call_function_queue+0x55/0x83
Oct 11 16:41:06 mini kernel: do_idle+0x1d5/0x1fb
Oct 11 16:41:06 mini kernel: cpu_startup_entry+0x1d/0x1f
Oct 11 16:41:06 mini kernel: start_secondary+0xeb/0xeb
Oct 11 16:41:06 mini kernel: secondary_startup_64_no_verify+0xce/0xdb
Oct 11 16:41:06 mini kernel: </TASK>
Oct 11 16:41:06 mini kernel: Modules linked in: macvlan nfsv3 nfs xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap xt_nat xt_tcpudp veth ipvlan xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter dm_crypt dm_mod xfs nfsd auth_rpcgss oid_registry lockd grace sunrpc md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) it87 tcp_diag inet_diag hwmon_vid vendor_reset(O) iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs bridge stp llc igc r8169 realtek amdgpu edac_mce_amd edac_core gpu_sched drm_buddy i2c_algo_bit drm_ttm_helper ttm kvm_amd drm_display_helper drm_kms_helper kvm drm crct10dif_pclmul crc32_pclmul crc32c_intel
Oct 11 16:41:06 mini kernel: ghash_clmulni_intel sha512_ssse3 btusb btrtl btbcm aesni_intel btintel crypto_simd bluetooth cryptd agpgart ahci nvme i2c_piix4 ecdh_generic rapl i2c_core syscopyarea libahci k10temp ecc amd_sfh nvme_core sysfillrect ccp sysimgblt fb_sys_fops tpm_crb video tpm_tis tpm_tis_core wmi tpm backlight acpi_cpufreq button unix [last unloaded: igc]
Oct 11 16:41:06 mini kernel: ---[ end trace 0000000000000000 ]---
Oct 11 16:41:06 mini kernel: RIP: 0010:flush_smp_call_function_queue+0x64/0x83
Oct 11 16:41:06 mini kernel: Code: e8 f6 f5 76 00 bf 01 00 00 00 e8 fb f4 ff ff 48 c7 c7 dd 59 10 82 e8 e0 f5 76 00 65 66 8b 05 ca e0 f2 7e 66 85 c0 74 05 e8 fd <45> f7 ff 0f ba e3 09 73 06 fb 0f 1f 44 00 00 5b e9 ae 34 b0 00 0f
Oct 11 16:41:06 mini kernel: RSP: 0018:ffffc9000019fee8 EFLAGS: 00010647
Oct 11 16:41:06 mini kernel: RAX: 0000000000000000 RBX: 0000000000000286 RCX: 00000000000f4240
Oct 11 16:41:06 mini kernel: RDX: 0000000000000002 RSI: ffffffff821059dd RDI: ffffffff820ba9d5
Oct 11 16:41:06 mini kernel: RBP: ffffffff823235a0 R08: ffff888712ded470 R09: ffff888712ded470
Oct 11 16:41:06 mini kernel: R10: 0000000000000000 R11: 0000000000000075 R12: 0000000000000007
Oct 11 16:41:06 mini kernel: R13: ffff88810090de80 R14: 0000000000000001 R15: 0000000000000000
Oct 11 16:41:06 mini kernel: FS:  0000000000000000(0000) GS:ffff888712dc0000(0000) knlGS:0000000000000000
Oct 11 16:41:06 mini kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 11 16:41:06 mini kernel: CR2: 0000001e6780b000 CR3: 0000000154e22000 CR4: 0000000000350ee0

     

    Link to comment
    JorgeB
    4 minutes ago, bastl said:

    Ok, now server is also crashing on 6.12.2.

    That suggests a hardware issue that possibly coincided with the last update, if it was the release related problem downgrading should have fixed the issue.

    • Like 1
    Link to comment
    DeatheTongue

    I've been having a crash at around 36-48 hours uptime since upgrading to 6.12.4.  Upgraded to the latest motherboard BIOS.  When I first upgraded to 6.12.x back in August and had this trouble I reverted to 6.11.5 and the problem did go away, now that I have time to work the problem I've upgraded again and am looking for answers.  Diagnostics attached.

    kong-diagnostics-20231017-1023.zip

    Link to comment
    bastl

    Small update from my side. Tried a couple things with no luck on 6.12 builds. I'am back on 6.11.5 now for 3 and a half days and no issues so far.

    Link to comment
    DeatheTongue

    Diagnostics again, now with moar syslog.  Uptime of 4 days, I think.  The parity check had just finished the day before the hang.  The interesting thing is these kernel faults are at 1500-ish and I didn't notice the full hang until around 2200.

     

    I have to completely remove power from the system to get it to reboot from this hang.

     

    Oct 27 10:14:07 Kong kernel: veth8c9ca72: renamed from eth0
    Oct 27 10:14:09 Kong kernel: eth0: renamed from veth0df38af
    Oct 27 10:45:59 Kong kernel: docker0: port 19(veth05c1d17) entered disabled state
    Oct 27 10:45:59 Kong kernel: veth35adc40: renamed from eth0
    Oct 27 10:45:59 Kong kernel: docker0: port 19(veth05c1d17) entered disabled state
    Oct 27 10:45:59 Kong kernel: device veth05c1d17 left promiscuous mode
    Oct 27 10:45:59 Kong kernel: docker0: port 19(veth05c1d17) entered disabled state
    Oct 27 10:46:01 Kong kernel: docker0: port 19(veth50db4f8) entered blocking state
    Oct 27 10:46:01 Kong kernel: docker0: port 19(veth50db4f8) entered disabled state
    Oct 27 10:46:01 Kong kernel: device veth50db4f8 entered promiscuous mode
    Oct 27 10:46:02 Kong kernel: eth0: renamed from veth9a08974
    Oct 27 10:46:02 Kong kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
    Oct 27 10:46:02 Kong kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth50db4f8: link becomes ready
    Oct 27 10:46:02 Kong kernel: docker0: port 19(veth50db4f8) entered blocking state
    Oct 27 10:46:02 Kong kernel: docker0: port 19(veth50db4f8) entered forwarding state
    Oct 27 11:02:35 Kong monitor: Stop running nchan processes
    Oct 27 12:00:51 Kong rpc.mountd[12955]: v4.2 client detached: 0xa1fd6fb7653934a1 from "192.168.10.13:727"
    Oct 27 15:01:08 Kong kernel: ------------[ cut here ]------------
    Oct 27 15:01:08 Kong kernel: WARNING: CPU: 14 PID: 0 at kernel/softirq.c:415 __do_softirq+0x256/0x288
    Oct 27 15:01:08 Kong kernel: Modules linked in: rpcsec_gss_krb5 nvidia_uvm(PO) xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle iptable_mangle vhost_net tun vhost vhost_iotlb tap ipvlan veth xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs ip6table_nat nfsd auth_rpcgss oid_registry lockd grace sunrpc md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) algif_hash algif_skcipher af_alg cmac bnep tcp_diag inet_diag iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs 8021q garp mrp bridge stp llc mlx4_en mlx4_core igb i2c_algo_bit nvidia_drm(PO) nvidia_modeset(PO) edac_mce_amd edac_core intel_rapl_msr intel_rapl_common iosf_mbi kvm_amd nvidia(PO) kvm btusb btrtl video
    Oct 27 15:01:08 Kong kernel: drm_kms_helper btbcm crct10dif_pclmul crc32_pclmul btintel crc32c_intel ghash_clmulni_intel sha512_ssse3 drm bluetooth aesni_intel mpt3sas crypto_simd backlight cryptd syscopyarea sysfillrect wmi_bmof mxm_wmi raid_class ecdh_generic i2c_piix4 rapl sysimgblt tpm_crb ecc k10temp ccp scsi_transport_sas i2c_core fb_sys_fops nvme ahci cp210x tpm_tis input_leds tpm_tis_core nvme_core led_class usbserial libahci corsair_psu wmi tpm button acpi_cpufreq unix [last unloaded: mlx4_core]
    Oct 27 15:01:08 Kong kernel: CPU: 14 PID: 0 Comm: swapper/14 Tainted: P           O       6.1.49-Unraid #1
    Oct 27 15:01:08 Kong kernel: Hardware name: System manufacturer System Product Name/PRIME X370-PRO, BIOS 6203 07/27/2023
    Oct 27 15:01:08 Kong kernel: RIP: 0010:__do_softirq+0x256/0x288
    Oct 27 15:01:08 Kong kernel: Code: c0 75 09 41 ff cf 0f 85 2b fe ff ff e8 af 2d 47 ff 65 81 05 38 c9 41 7e 00 ff ff ff 65 8b 05 31 c9 41 7e a9 00 ff ff 00 74 02 <0f> 0b 8b 54 24 10 65 48 8b 04 25 c0 cb 01 00 81 60 2c ff f7 ff ff
    Oct 27 15:01:08 Kong kernel: RSP: 0018:ffffc90000534fa0 EFLAGS: 00010006
    Oct 27 15:01:08 Kong kernel: RAX: 0000000080010001 RBX: 0000000000000000 RCX: 000000000000000a
    Oct 27 15:01:08 Kong kernel: RDX: 0000000000010101 RSI: ffffffff821626da RDI: ffffffff82117638
    Oct 27 15:01:08 Kong kernel: RBP: ffffffff82206110 R08: 0000000000000000 R09: 0000000000010101
    Oct 27 15:01:08 Kong kernel: R10: 0000000000000000 R11: ffffc90000534ff8 R12: 0000000000000009
    Oct 27 15:01:08 Kong kernel: R13: 0000000000010101 R14: ffff8881003dde80 R15: 000000000000000a
    Oct 27 15:01:08 Kong kernel: FS:  0000000000000000(0000) GS:ffff888feeb80000(0000) knlGS:0000000000000000
    Oct 27 15:01:08 Kong kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    Oct 27 15:01:08 Kong kernel: CR2: 000014ccf21cf410 CR3: 0000000850d50000 CR4: 0000000000350ee0
    Oct 27 15:01:08 Kong kernel: Call Trace:
    Oct 27 15:01:08 Kong kernel: <IRQ>
    Oct 27 15:01:08 Kong kernel: ? __warn+0xab/0x122
    Oct 27 15:01:08 Kong kernel: ? report_bug+0x109/0x17e
    Oct 27 15:01:08 Kong kernel: ? __do_softirq+0x256/0x288
    Oct 27 15:01:08 Kong kernel: ? handle_bug+0x41/0x6f
    Oct 27 15:01:08 Kong kernel: ? exc_invalid_op+0x13/0x60
    Oct 27 15:01:08 Kong kernel: ? asm_exc_invalid_op+0x16/0x20
    Oct 27 15:01:08 Kong kernel: ? __do_softirq+0x256/0x288
    Oct 27 15:01:08 Kong kernel: __irq_exit_rcu+0x5e/0xb8
    Oct 27 15:01:08 Kong kernel: sysvec_apic_timer_interrupt+0x85/0xa6
    Oct 27 15:01:08 Kong kernel: </IRQ>
    Oct 27 15:01:08 Kong kernel: <TASK>
    Oct 27 15:01:08 Kong kernel: asm_sysvec_apic_timer_interrupt+0x16/0x20
    Oct 27 15:01:08 Kong kernel: RIP: 0010:cpuidle_enter_state+0x11d/0x202
    Oct 27 15:01:08 Kong kernel: Code: 20 22 a0 ff 45 84 ff 74 1b 9c 58 0f 1f 40 00 0f ba e0 09 73 08 0f 0b fa 0f 1f 44 00 00 31 ff e8 4c e3 a4 ff fb 0f 1f 44 00 00 <45> 85 e4 0f 88 ba 00 00 00 48 8b 04 24 49 63 cc 48 6b d1 68 49 29
    Oct 27 15:01:08 Kong kernel: RSP: 0018:ffffc900001c7e98 EFLAGS: 00000246
    Oct 27 15:01:08 Kong kernel: RAX: ffff888feeb80000 RBX: ffff888108042400 RCX: 0000000000000000
    Oct 27 15:01:08 Kong kernel: RDX: 0000a8bbb2035d2b RSI: ffffffff820ed4af RDI: ffffffff820ed9b8
    Oct 27 15:01:08 Kong kernel: RBP: 0000000000000002 R08: 0000000000000002 R09: 0000000000000002
    Oct 27 15:01:08 Kong kernel: R10: 0000000000000020 R11: 00000000000000ff R12: 0000000000000002
    Oct 27 15:01:08 Kong kernel: R13: ffffffff82323720 R14: 0000a8bbb2035d2b R15: 0000000000000000
    Oct 27 15:01:08 Kong kernel: ? cpuidle_enter_state+0xf7/0x202
    Oct 27 15:01:08 Kong kernel: cpuidle_enter+0x2a/0x38
    Oct 27 15:01:08 Kong kernel: do_idle+0x18d/0x1fb
    Oct 27 15:01:08 Kong kernel: cpu_startup_entry+0x1d/0x1f
    Oct 27 15:01:08 Kong kernel: start_secondary+0x101/0x101
    Oct 27 15:01:08 Kong kernel: secondary_startup_64_no_verify+0xce/0xdb
    Oct 27 15:01:08 Kong kernel: </TASK>
    Oct 27 15:01:08 Kong kernel: ---[ end trace 0000000000000000 ]---
    Oct 27 15:01:08 Kong kernel: BUG: scheduling while atomic: swapper/14/0/0x00010001
    Oct 27 15:01:08 Kong kernel: Modules linked in: rpcsec_gss_krb5 nvidia_uvm(PO) xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle iptable_mangle vhost_net tun vhost vhost_iotlb tap ipvlan veth xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs ip6table_nat nfsd auth_rpcgss oid_registry lockd grace sunrpc md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) algif_hash algif_skcipher af_alg cmac bnep tcp_diag inet_diag iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs 8021q garp mrp bridge stp llc mlx4_en mlx4_core igb i2c_algo_bit nvidia_drm(PO) nvidia_modeset(PO) edac_mce_amd edac_core intel_rapl_msr intel_rapl_common iosf_mbi kvm_amd nvidia(PO) kvm btusb btrtl video
    Oct 27 15:01:08 Kong kernel: drm_kms_helper btbcm crct10dif_pclmul crc32_pclmul btintel crc32c_intel ghash_clmulni_intel sha512_ssse3 drm bluetooth aesni_intel mpt3sas crypto_simd backlight cryptd syscopyarea sysfillrect wmi_bmof mxm_wmi raid_class ecdh_generic i2c_piix4 rapl sysimgblt tpm_crb ecc k10temp ccp scsi_transport_sas i2c_core fb_sys_fops nvme ahci cp210x tpm_tis input_leds tpm_tis_core nvme_core led_class usbserial libahci corsair_psu wmi tpm button acpi_cpufreq unix [last unloaded: mlx4_core]
    Oct 27 15:01:08 Kong kernel: Preemption disabled at:
    Oct 27 15:01:08 Kong kernel: [<0000000000000000>] 0x0
    Oct 27 15:01:08 Kong kernel: CPU: 14 PID: 0 Comm: swapper/14 Tainted: P        W  O       6.1.49-Unraid #1
    Oct 27 15:01:08 Kong kernel: Hardware name: System manufacturer System Product Name/PRIME X370-PRO, BIOS 6203 07/27/2023
    Oct 27 15:01:08 Kong kernel: Call Trace:
    Oct 27 15:01:08 Kong kernel: <TASK>
    Oct 27 15:01:08 Kong kernel: dump_stack_lvl+0x44/0x5c
    Oct 27 15:01:08 Kong kernel: __schedule_bug+0x9a/0xac
    Oct 27 15:01:08 Kong kernel: __schedule+0x59/0x612
    Oct 27 15:01:08 Kong kernel: ? flush_smp_call_function_queue+0x12/0x83
    Oct 27 15:01:08 Kong kernel: schedule_idle+0x27/0x3e
    Oct 27 15:01:08 Kong kernel: cpu_startup_entry+0x1d/0x1f
    Oct 27 15:01:08 Kong kernel: start_secondary+0x101/0x101
    Oct 27 15:01:08 Kong kernel: secondary_startup_64_no_verify+0xce/0xdb
    Oct 27 15:01:08 Kong kernel: </TASK>
    Oct 27 16:06:34 Kong smbd[31382]: [2023/10/27 16:06:34.798515,  0] ../../source3/smbd/open.c:3306(smbd_calculate_maximum_allowed_access_fsp)
    Oct 27 16:06:34 Kong smbd[31382]:   smbd_calculate_maximum_allowed_access_fsp: Could not get acl on file home-assistant/www/alerts/driveway.20231023_160000.1168020.3-1.jpg: NT_STATUS_ACCESS_DENIED

    kong-diagnostics-20231027-2351.zip

    Link to comment
    JorgeB
    5 hours ago, DeatheTongue said:

    Here it is.

    If you have already updated the BIOS I would try with v6.13 once it's out, it will have a much newer kernel and it might play better with your hardware.

    • Thanks 1
    Link to comment
    DeatheTongue
    On 10/29/2023 at 3:08 PM, JorgeB said:

    If you have already updated the BIOS I would try with v6.13 once it's out, it will have a much newer kernel and it might play better with your hardware.

    I reverted to 6.11.5 and continued to have crashes until I removed a bluetooth adapter.  BT was attached and bluez-5.66-x86_64-1.txz installed and running to share into Home Assistant docker.  At 1 month uptime on 6.11.5 and will probably be attempting 6.12.x again in the coming weeks.  Will follow up to close the loop on my report.

    Link to comment
    elgatobavaria
    On 1/6/2024 at 8:45 PM, DeatheTongue said:

    I reverted to 6.11.5 and continued to have crashes until I removed a bluetooth adapter.  BT was attached and bluez-5.66-x86_64-1.txz installed and running to share into Home Assistant docker.  At 1 month uptime on 6.11.5 and will probably be attempting 6.12.x again in the coming weeks.  Will follow up to close the loop on my report.


    Since a few days i try to redirect the USB BT500 Bluetooth dongle to a HASSIO Docker container, too. Since then i have a similar problem. The Unraid crashes random and is not able to start anymore (fans are spinning for a second and then repeat reboot).
    Before the idea of using docker for HASSIO i was running the whole thing for months without any issue.

    Update: Currently i have deactivated HASSIO VM and also disconnected the USB BT500 Dongle. Same behaviour.
     

    Edited by elgatobavaria
    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×

  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.

    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.