I had a Win11 VM for gaming and I passed though a 4080. Everything was good in August. Acturally I finished BMW on it. I remember I updated Geforce Game Ready in Aug. 22 (560.94) and then the VM still worked well for about a week.

Then one day suddenly the VM was "paused". And if I click "resume", it reports:

so I have to "force stop" it. The log of the VM is like:

-vnc 0.0.0.0:0,websocket=5700,audiodev=audio1 \
-k en-us \
-device '{"driver":"qxl-vga","id":"video0","max_outputs":1,"ram_size":67108864,"vram_size":67108864,"vram64_size_mb":0,"vgamem_mb":16,"bus":"pcie.0","addr":"0x1e"}' \
-device '{"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.3","multifunction":true,"addr":"0x0","romfile":"/mnt/user/isos/ROM/Zotac4080-20230910.rom"}' \
-device '{"driver":"vfio-pci","host":"0000:01:00.1","id":"hostdev1","bus":"pci.3","addr":"0x0.0x1"}' \
-device '{"driver":"vfio-pci","host":"0000:05:00.0","id":"hostdev2","bus":"pci.4","addr":"0x0"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/004","id":"hostdev3","bus":"usb.0","port":"2"}' \
-device '{"driver":"usb-host","id":"hostdev4","bus":"usb.0","port":"3"}' \
-device '{"driver":"usb-host","id":"hostdev5","bus":"usb.0","port":"4"}' \
-device '{"driver":"usb-host","id":"hostdev6","bus":"usb.0","port":"5"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/0 (label charserial0)
qxl_send_events: spice-server bug: guest stopped, ignoring
2024-10-06T12:31:28.074101Z qemu-system-x86_64: vfio_err_notifier_handler(0000:01:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest
2024-10-06T12:31:28.114218Z qemu-system-x86_64: vfio_err_notifier_handler(0000:01:00.1) Unrecoverable error detected. Please collect any data possible and then kill the guest
2024-10-06T12:31:28.200916Z qemu-system-x86_64: vfio_err_notifier_handler(0000:01:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest
2024-10-06T12:31:28.200973Z qemu-system-x86_64: vfio_err_notifier_handler(0000:01:00.1) Unrecoverable error detected. Please collect any data possible and then kill the guest

 

 

I have been trying to fix it these days, and now I am quite sure that the problem comes from nvidia driver Geforce Game Ready. The VM will keep working for minutes after updating and then be "paused". Sometimes the VM usually can not be started again.

Both win10 VM and win11 VM have the same problem.

 

I have create a new VM and it works with no probem with its builtin nvidia driver (536.23 for win10VM) untill I update the nvidia driver to 560.81 or newer (I have not tried older version because I am quite sure the VM have been run on it for weeks).

 

And today I tried to update nvidia driver through windows update. The VM reboot successfully, so I can check the nvidia driver version which shows to be 560.94. Then it "paused" without surprise.

 

 

 

 

I tried to change the xml from

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/ROM/Zotac4080-20230910.rom'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
    </hostdev>

to

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/ROM/Zotac4080-20230910.rom'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>

 

It seems that the VM works looger. I watch videoes for about 10 minutes. But it still paused about 2 minutes after I started a Game (God of War).

But this time I get something new from the log:

-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/0 (label charserial0)
qxl_send_events: spice-server bug: guest stopped, ignoring
2024-10-06T13:19:33.731078Z qemu-system-x86_64: libusb_release_interface: -99 [OTHER]
2024-10-06T13:19:33.731112Z qemu-system-x86_64: libusb_release_interface: -99 [OTHER]
2024-10-06T13:19:33.731115Z qemu-system-x86_64: libusb_release_interface: -99 [OTHER]
2024-10-06T13:19:35.761051Z qemu-system-x86_64: libusb_release_interface: -99 [OTHER]
2024-10-06T13:19:35.761068Z qemu-system-x86_64: libusb_release_interface: -99 [OTHER]
2024-10-06T13:19:35.761073Z qemu-system-x86_64: libusb_release_interface: -99 [OTHER]
2024-10-06T13:32:51.266412Z qemu-system-x86_64: vfio_err_notifier_handler(0000:01:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest
2024-10-06T13:32:51.346018Z qemu-system-x86_64: vfio_err_notifier_handler(0000:01:00.1) Unrecoverable error detected. Please collect any data possible and then kill the guest

 

I found that the problem is associated with the Sound Card. The VM would not be "paused" if I remove it in VM's setting (of course the VM cannot be used as a gaming VM without sound). Then I add the sound card and the VM was paused very soon after start.

 

But I am confused again. Without the sound card, I can install the newest version of Nvidia driver (565.90), but the gaming performance is very bad. I use the same setting but I can only get 1 FPS in Black Myth (RTX4080, 2560x1440, no Ray Tracing).