• VM Windows 11 Keeps Crashing Since Upgraded to 6.11.1


    Tucubanito07
    • Urgent

    Good morning all,

     

    I went through the first page of the stable releases, and I don't see anyone reporting issues with their VM's. Since upgrading to 6.11.1 my windows 11 for some reason keeps crashing. When i looked at the logs for the machine, i can see the below:

     

    text  error  warn  system  array  login  

    -device '{"driver":"vfio-pci","host":"0000:02:00.0","id":"hostdev2","bus":"pci.6","addr":"0x0"}' \
    -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
    -msg timestamp=on
    char device redirected to /dev/pts/0 (label charserial0)
    2022-10-20T11:08:34.025908Z qemu-system-x86_64: vfio: Cannot reset device 0000:02:00.0, depends on group 16 which is not owned.
    2022-10-20T11:08:35.054882Z qemu-system-x86_64: vfio: Cannot reset device 0000:02:00.0, depends on group 16 which is not owned.
    2022-10-21 00:57:48.271+0000: shutting down, reason=crashed
    2022-10-21 12:59:06.775+0000: Starting external device: TPM Emulator
    /usr/bin/swtpm socket --ctrl 'type=unixio,path=/run/libvirt/qemu/swtpm/2-Windows 11-swtpm.sock,mode=0600' --tpmstate dir=/var/lib/libvirt/swtpm/9b001939-8548-4888-72c7-9edef447e28c/tpm2,mode=0600 --log 'file=/var/log/swtpm/libvirt/qemu/Windows 11-swtpm.log' --terminate --tpm2
    2022-10-21 12:59:06.812+0000: starting up libvirt version: 8.7.0, qemu version: 7.1.0, kernel: 5.19.14-Unraid, hostname: Eleanor
    LC_ALL=C \
    PATH=/bin:/sbin:/usr/bin:/usr/sbin \
    HOME='/var/lib/libvirt/qemu/domain-2-Windows 11' \
    XDG_DATA_HOME='/var/lib/libvirt/qemu/domain-2-Windows 11/.local/share' \
    XDG_CACHE_HOME='/var/lib/libvirt/qemu/domain-2-Windows 11/.cache' \
    XDG_CONFIG_HOME='/var/lib/libvirt/qemu/domain-2-Windows 11/.config' \
    /usr/local/sbin/qemu \
    -name 'guest=Windows 11,debug-threads=on' \
    -S \
    -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-2-Windows 11/master-key.aes"}' \
    -blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
    -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
    -blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/9b001939-8548-4888-72c7-9edef447e28c_VARS-pure-efi-tpm.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \
    -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
    -machine pc-q35-6.2,usb=off,dump-guest-core=off,mem-merge=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \
    -accel kvm \
    -cpu host,migratable=on,topoext=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=none,host-cache-info=on,l3-cache=off \
    -m 4096 \
    -object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":4294967296}' \
    -overcommit mem-lock=off \
    -smp 2,sockets=1,dies=1,cores=1,threads=2 \
    -uuid 9b001939-8548-4888-72c7-9edef447e28c \
    -display none \
    -no-user-config \
    -nodefaults \
    -chardev socket,id=charmonitor,fd=36,server=on,wait=off \
    -mon chardev=charmonitor,id=monitor,mode=control \
    -rtc base=localtime \
    -no-hpet \
    -no-shutdown \
    -boot strict=on \
    -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
    -device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
    -device '{"driver":"pcie-root-port","port":15,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x7"}' \
    -device '{"driver":"pcie-root-port","port":10,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x2"}' \
    -device '{"driver":"pcie-root-port","port":11,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x3"}' \
    -device '{"driver":"pcie-root-port","port":12,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x1.0x4"}' \
    -device '{"driver":"pcie-root-port","port":13,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x1.0x5"}' \
    -device '{"driver":"pcie-root-port","port":14,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x1.0x6"}' \
    -device '{"driver":"pcie-root-port","port":16,"chassis":9,"id":"pci.9","bus":"pcie.0","addr":"0x2"}' \
    -device '{"driver":"pcie-pci-bridge","id":"pci.10","bus":"pci.1","addr":"0x0"}' \
    -device '{"driver":"ich9-usb-ehci1","id":"usb","bus":"pcie.0","addr":"0x7.0x7"}' \
    -device '{"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pcie.0","multifunction":true,"addr":"0x7"}' \
    -device '{"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pcie.0","addr":"0x7.0x1"}' \
    -device '{"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pcie.0","addr":"0x7.0x2"}' \
    -device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}' \
    -blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 11/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
    -blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
    -device '{"driver":"virtio-blk-pci","bus":"pci.2","addr":"0x0","drive":"libvirt-3-format","id":"virtio-disk2","bootindex":1,"write-cache":"on"}' \
    -blockdev '{"driver":"file","filename":"/mnt/user/isos/Win11_English_x64v.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
    -blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
    -device '{"driver":"ide-cd","bus":"ide.0","drive":"libvirt-2-format","id":"sata0-0-0","bootindex":2}' \
    -blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.208-1.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
    -blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
    -device '{"driver":"ide-cd","bus":"ide.1","drive":"libvirt-1-format","id":"sata0-0-1"}' \
    -netdev tap,fd=37,id=hostnet0 \
    -device '{"driver":"e1000","netdev":"hostnet0","id":"net0","mac":"52:54:00:24:5b:da","bus":"pci.10","addr":"0x1"}' \
    -chardev pty,id=charserial0 \
    -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
    -chardev socket,id=charchannel0,fd=35,server=on,wait=off \
    -device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
    -chardev 'socket,id=chrtpm,path=/run/libvirt/qemu/swtpm/2-Windows 11-swtpm.sock' \
    -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
    -device '{"driver":"tpm-tis","tpmdev":"tpm-tpm0","id":"tpm0"}' \
    -device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \
    -audiodev '{"id":"audio1","driver":"none"}' \
    -device '{"driver":"vfio-pci","host":"0000:12:00.0","id":"hostdev0","bus":"pci.4","addr":"0x0"}' \
    -device '{"driver":"vfio-pci","host":"0000:12:00.1","id":"hostdev1","bus":"pci.5","addr":"0x0"}' \
    -device '{"driver":"vfio-pci","host":"0000:02:00.0","id":"hostdev2","bus":"pci.6","addr":"0x0"}' \
    -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
    -msg timestamp=on
    char device redirected to /dev/pts/0 (label charserial0)
    2022-10-21T12:59:13.981913Z qemu-system-x86_64: vfio: Cannot reset device 0000:02:00.0, depends on group 16 which is not owned.
    2022-10-21T12:59:15.055882Z qemu-system-x86_64: vfio: Cannot reset device 0000:02:00.0, depends on group 16 which is not owned.
    2022-10-21T12:59:16.251148Z qemu-system-x86_64: vfio-pci: Cannot read device rom at 0000:12:00.0
    Device option ROM contents are probably invalid (check dmesg).
    Skip option ROM probe with rombar=0, or load from file with romfile=
    2022-10-21T13:00:26.888267Z qemu-system-x86_64: vfio: Cannot reset device 0000:02:00.0, depends on group 16 which is not owned.
    2022-10-21T13:00:27.950906Z qemu-system-x86_64: vfio: Cannot reset device 0000:02:00.0, depends on group 16 which is not owned.
    2022-10-21 23:46:06.664+0000: shutting down, reason=crashed
     

    Can someone help me figure out why this keeps happening please? This is my wife's work machine and would like to keep it from crashing if possible and/or at least figure what the issue is so I can fix it. I have also uploaded the support file. Any help is greatly appreciated.

     

    Thank you.

    eleanor-diagnostics-20221022-0916.zip




    User Feedback

    Recommended Comments

    Hello, I can confirm this error pattern. I have an Ubuntu VM where the same error pattern occurred twice in one week. Probably during shutdown inside the VM, Unraid crashed completely. Only a hard reset helped. However, I have no logs of the Crash. 
     

    Greetings

    Thoms

    • Thanks 1
    Link to comment

    i believe i have a issue very similar to this. i have pin point it to the virtio driver/ when another linux vm is running along side the windows vm. if anybody can help me fix this please i will

    appreciate it. thanks. 

    Link to comment
    3 hours ago, aniel said:

    i believe i have a issue very similar to this. i have pin point it to the virtio driver/ when another linux vm is running along side the windows vm. if anybody can help me fix this please i will

    appreciate it. thanks. 

    Post your XML, maybe start a general support thread. 

    Link to comment

    Got similar random reboots on Windows 11 / Windows 10 VM. It may happen in an hour or in 24 hours but always happens. The entire unraid server crashes. No errors in the log upon reboot, it happens silently. I ran memtest for few hours (zero errors) and ran bare metal windows 10 configuration for few days - no reboots. So this is not a hardware issue.

    I've no idea yet what causes it.

     

    I have an unraid 6.9.2 with similar conifg running aside for years with zero issues. 

    Link to comment

    I disabled my VM for a night. And caught an error in the logs (server didn't reboot this time):

    Jan  5 10:14:52 ares-unraid kernel: CPU: 6 PID: 0 Comm: swapper/6 Tainted: P           O      5.19.17-Unraid #2
    Jan  5 10:14:52 ares-unraid kernel: Hardware name: ASUS System Product Name/ProArt X570-CREATOR WIFI, BIOS 0801 04/26/2022
    Jan  5 10:14:52 ares-unraid kernel: RIP: 0010:__nf_conntrack_confirm+0xa5/0x2cb [nf_conntrack]
    Jan  5 10:14:52 ares-unraid kernel: Code: c6 48 89 44 24 10 e8 dd e2 ff ff 8b 7c 24 04 89 da 89 c6 89 04 24 e8 56 e6 ff ff 84 c0 75 a2 48 8b 85 80 00 00 00 a8 08 74 18 <0f> 0b 8b 34 24 8b 7c 24 04 e8 16 de ff ff e8 2c e3 ff ff e9 7e 01
    Jan  5 10:14:52 ares-unraid kernel: RSP: 0018:ffffc900003788c8 EFLAGS: 00010202
    Jan  5 10:14:52 ares-unraid kernel: RAX: 0000000000000188 RBX: 0000000000000000 RCX: 74f8a55f28104df6
    Jan  5 10:14:52 ares-unraid kernel: RDX: 0000000000000000 RSI: 00000000000001db RDI: ffffffffa035bdc0
    Jan  5 10:14:52 ares-unraid kernel: RBP: ffff8889f30caf00 R08: 47ddfafba8ac2d75 R09: 25dae199f6406014
    Jan  5 10:14:52 ares-unraid kernel: R10: 300a12ed70ccfb20 R11: 44f4d35e05496612 R12: ffffffff82909480
    Jan  5 10:14:52 ares-unraid kernel: R13: 0000000000001ee0 R14: ffff8882ad763f00 R15: 0000000000000000
    Jan  5 10:14:52 ares-unraid kernel: FS:  0000000000000000(0000) GS:ffff88900e980000(0000) knlGS:0000000000000000
    Jan  5 10:14:52 ares-unraid kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    Jan  5 10:14:52 ares-unraid kernel: CR2: 0000146d90e2fa60 CR3: 0000000109450000 CR4: 0000000000750ee0
    Jan  5 10:14:52 ares-unraid kernel: PKRU: 55555554
    Jan  5 10:14:52 ares-unraid kernel: Call Trace:
    Jan  5 10:14:52 ares-unraid kernel: <IRQ>
    Jan  5 10:14:52 ares-unraid kernel: nf_conntrack_confirm+0x25/0x54 [nf_conntrack]
    Jan  5 10:14:52 ares-unraid kernel: nf_hook_slow+0x3d/0x96
    Jan  5 10:14:52 ares-unraid kernel: ? ip_protocol_deliver_rcu+0x164/0x164
    Jan  5 10:14:52 ares-unraid kernel: NF_HOOK.constprop.0+0x79/0xd9
    Jan  5 10:14:52 ares-unraid kernel: ? ip_protocol_deliver_rcu+0x164/0x164
    Jan  5 10:14:52 ares-unraid kernel: ip_sabotage_in+0x4a/0x58 [br_netfilter]
    Jan  5 10:14:52 ares-unraid kernel: nf_hook_slow+0x3d/0x96
    Jan  5 10:14:52 ares-unraid kernel: ? ip_rcv_finish_core.constprop.0+0x3b7/0x3b7
    Jan  5 10:14:52 ares-unraid kernel: NF_HOOK.constprop.0+0x79/0xd9
    Jan  5 10:14:52 ares-unraid kernel: ? ip_rcv_finish_core.constprop.0+0x3b7/0x3b7
    Jan  5 10:14:52 ares-unraid kernel: __netif_receive_skb_one_core+0x77/0x9c
    Jan  5 10:14:52 ares-unraid kernel: netif_receive_skb+0xbf/0x127
    Jan  5 10:14:52 ares-unraid kernel: br_handle_frame_finish+0x476/0x4b0 [bridge]
    Jan  5 10:14:52 ares-unraid kernel: ? br_pass_frame_up+0xdd/0xdd [bridge]
    Jan  5 10:14:52 ares-unraid kernel: br_nf_hook_thresh+0xe5/0x109 [br_netfilter]
    Jan  5 10:14:52 ares-unraid kernel: ? br_pass_frame_up+0xdd/0xdd [bridge]
    Jan  5 10:14:52 ares-unraid kernel: br_nf_pre_routing_finish+0x2c1/0x2ec [br_netfilter]
    Jan  5 10:14:52 ares-unraid  rsyslogd: action 'action-3-builtin:omfwd' resumed (module 'builtin:omfwd') [v8.2102.0 try https://www.rsyslog.com/e/2359 ]
    Jan  5 10:14:52 ares-unraid kernel: ? br_pass_frame_up+0xdd/0xdd [bridge]
    Jan  5 10:14:52 ares-unraid kernel: ? NF_HOOK.isra.0+0xe4/0x140 [br_netfilter]
    Jan  5 10:14:52 ares-unraid kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter]
    Jan  5 10:14:52 ares-unraid kernel: br_nf_pre_routing+0x226/0x23a [br_netfilter]
    Jan  5 10:14:52 ares-unraid kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter]
    Jan  5 10:14:52 ares-unraid kernel: br_handle_frame+0x27f/0x2e7 [bridge]
    Jan  5 10:14:52 ares-unraid kernel: ? br_pass_frame_up+0xdd/0xdd [bridge]
    Jan  5 10:14:52 ares-unraid kernel: __netif_receive_skb_core.constprop.0+0x4f9/0x6e3
    Jan  5 10:14:52 ares-unraid kernel: ? __alloc_skb+0xb2/0x15e
    Jan  5 10:14:52 ares-unraid kernel: ? __kmalloc_node_track_caller+0x1ae/0x1d9
    Jan  5 10:14:52 ares-unraid kernel: ? udp4_gro_receive+0x1b/0x20c
    Jan  5 10:14:52 ares-unraid kernel: ? inet_gro_receive+0x234/0x254
    Jan  5 10:14:52 ares-unraid kernel: __netif_receive_skb_list_core+0x8a/0x11e
    Jan  5 10:14:52 ares-unraid kernel: netif_receive_skb_list_internal+0x1d7/0x210
    Jan  5 10:14:52 ares-unraid kernel: gro_normal_list+0x1d/0x3f
    Jan  5 10:14:52 ares-unraid kernel: napi_complete_done+0x7b/0x11a
    Jan  5 10:14:52 ares-unraid kernel: aq_vec_poll+0x13c/0x187 [atlantic]
    Jan  5 10:14:52 ares-unraid kernel: __napi_poll.constprop.0+0x2b/0x124
    Jan  5 10:14:52 ares-unraid kernel: net_rx_action+0x159/0x24f
    Jan  5 10:14:52 ares-unraid kernel: __do_softirq+0x129/0x288
    Jan  5 10:14:52 ares-unraid kernel: __irq_exit_rcu+0x79/0xb8
    Jan  5 10:14:52 ares-unraid kernel: common_interrupt+0x9b/0xc1
    Jan  5 10:14:52 ares-unraid kernel: </IRQ>
    Jan  5 10:14:52 ares-unraid kernel: <TASK>
    Jan  5 10:14:52 ares-unraid kernel: asm_common_interrupt+0x22/0x40
    Jan  5 10:14:52 ares-unraid kernel: RIP: 0010:cpuidle_enter_state+0x11b/0x1e4
    Jan  5 10:14:52 ares-unraid kernel: Code: 5b fa a1 ff 45 84 ff 74 1b 9c 58 0f 1f 40 00 0f ba e0 09 73 08 0f 0b fa 0f 1f 44 00 00 31 ff e8 9d a9 a6 ff fb 0f 1f 44 00 00 <45> 85 ed 0f 88 9e 00 00 00 48 8b 04 24 49 63 cd 48 6b d1 68 49 29
    Jan  5 10:14:52 ares-unraid kernel: RSP: 0018:ffffc90000197e98 EFLAGS: 00000246
    Jan  5 10:14:52 ares-unraid kernel: RAX: ffff88900e980000 RBX: 0000000000000002 RCX: 0000000000000000
    Jan  5 10:14:52 ares-unraid kernel: RDX: 0000000000000006 RSI: ffffffff820d7be1 RDI: ffffffff820d80c1
    Jan  5 10:14:52 ares-unraid kernel: RBP: ffff888108fbac00 R08: 0000000000000002 R09: 0000000000000002
    Jan  5 10:14:52 ares-unraid kernel: R10: 0000000000000020 R11: 000000000001295c R12: ffffffff82318880
    Jan  5 10:14:52 ares-unraid kernel: R13: 0000000000000002 R14: 00001b8038bae1e9 R15: 0000000000000000
    Jan  5 10:14:52 ares-unraid kernel: ? cpuidle_enter_state+0xf5/0x1e4
    Jan  5 10:14:52 ares-unraid kernel: cpuidle_enter+0x2a/0x38
    Jan  5 10:14:52 ares-unraid kernel: do_idle+0x187/0x1f5
    Jan  5 10:14:52 ares-unraid kernel: cpu_startup_entry+0x1d/0x1f
    Jan  5 10:14:52 ares-unraid kernel: start_secondary+0xeb/0xeb
    Jan  5 10:14:52 ares-unraid kernel: secondary_startup_64_no_verify+0xce/0xdb
    Jan  5 10:14:52 ares-unraid kernel: </TASK>
    Jan  5 10:14:52 ares-unraid kernel: ---[ end trace 0000000000000000 ]---

     

    ares-unraid-diagnostics-20230105-1311.zip

    Link to comment
    55 minutes ago, zeus83 said:

    And caught an error in the logs

    Try switching to ipvlan (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right))

    Link to comment

    I replaced memory modules and no reboots now. No kernel errors as well.

     

    I'm testing old modules via memtest still don't show any errors, possible some hardware incompatibility...

     

    Therefore I must admit in my case this is not related to unraid.

    Edited by zeus83
    • Like 1
    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.