Win10 VM only starts after Force Start

[iltisdaniel]

Hey Everyone. 

 

Kinda new here. Evaluating unraid as we speak. I've been loving it so far. Fixed a ton of problems thanks to this forum, but there's one problem I can't seem to find a fix for. 

 

My VM only starts after I've force stopped it. 

 

I go to my PC, use a phone to login to unraid and then start my VM and it doesn't do anything. The monitor doesn't turn on, the keyboard doesn't light up. Nothing. 

 

So I force stop it (as I've tried just restarting, or hibernating and it does nothing) and start it agian and voila, it starts like nothing happened. 

 

It's kinda annoying that I have to do it like this and I'm sure there's a FIX for this issue. 

 

Let me know what I need to share to give you all a better understanding of my issue. 

 

Cheers,

Daniel. 

[iltisdaniel]

Am I doing something wrong that only my post isn`t getting a single reply? Anyone? Any ideas. I searched the forum top to bottom and couldn`t find anything. Nor did I manage to fix the problem. 

 

Here`s the VM`s log

 

ErrorWarningSystemArrayLogin

-overcommit mem-lock=off \
-smp 2,sockets=1,dies=1,cores=2,threads=1 \
-uuid f2c7f993-49ad-6949-700a-44f8f90aca56 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=31,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.0,addr=0x7 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
-blockdev '{"driver":"file","filename":"/mnt/systems/vDisks/WS01DIL/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
-device virtio-blk-pci,bus=pci.0,addr=0x4,drive=libvirt-3-format,id=virtio-disk2,bootindex=1,write-cache=on \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Windows10.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device ide-cd,bus=ide.0,unit=0,drive=libvirt-2-format,id=ide0-0-0,bootindex=2 \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.190-1.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device ide-cd,bus=ide.0,unit=1,drive=libvirt-1-format,id=ide0-0-1 \
-netdev tap,fd=33,id=hostnet0 \
-device virtio-net,netdev=hostnet0,id=net0,mac=52:54:00:42:a4:49,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=34,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0,bus=usb.0,port=2 \
-device vfio-pci,host=0000:01:00.0,id=hostdev0,bus=pci.0,addr=0x5,romfile=/mnt/user/isos/vbios/nvidia1050ti.rom \
-device vfio-pci,host=0000:01:00.1,id=hostdev1,bus=pci.0,addr=0x6 \
-device vfio-pci,host=0000:03:00.0,id=hostdev2,bus=pci.0,addr=0x8 \
-device vfio-pci,host=0000:04:00.0,id=hostdev3,bus=pci.0,addr=0x9 \
-device usb-host,hostbus=1,hostaddr=2,id=hostdev4,bus=usb.0,port=1 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2021-07-19 16:45:26.996+0000: Domain id=8 is tainted: high-privileges
2021-07-19 16:45:26.996+0000: Domain id=8 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)

[ghost82]

On 7/19/2021 at 6:51 PM, iltisdaniel said:

Am I doing something wrong that only my post isn`t getting a single reply?

 

No, you don't, but it may be difficult to solve the issue with the description you provided.

First things I would check is the gpu, and more into details the rom you are passing (dump your own rom if you downloaded from internet) and the topology in the xml: create a multifunction device for the gpu, now you are passing through the video at target address 0:05.0 and the audio at 0:06.0, you need to pass the audio at the target address 0:05.1

 

You should have something like this in your xml:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/nvidia1050ti.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

 

Replace with this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/nvidia1050ti.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>

 

Edited July 23, 2021 by ghost82

[iltisdaniel]

Hello Ghost. Big thanks for you reply and steering me in some direction. 

 

Just so I understand, I modify my xml directly with what you've sent? 

 

I'm happy to give additional data, just very new to unraid so don't know what is needed and where to get. 

 

Also, I struggled in the beginning to get the pass through working but this forum helped a lot. I never changed any iommu settings apart from my USB PCI card. So I just went with the default settings for the sound card. 

 

I'll make a clone of my VM ND make the adjusments there. To see if that helps. 

 

I did some troubleshooting and I'm quite sure the problem is with the gpu not giving any output on startup and sudenly working after a force start. 

 

Daniel. 

[ghost82]

41 minutes ago, iltisdaniel said:

Hello Ghost. Big thanks for you reply and steering me in some direction. 

 

Just so I understand, I modify my xml directly with what you've sent? 

 

I'm happy to give additional data, just very new to unraid so don't know what is needed and where to get. 

 

Also, I struggled in the beginning to get the pass through working but this forum helped a lot. I never changed any iommu settings apart from my USB PCI card. So I just went with the default settings for the sound card. 

 

I'll make a clone of my VM ND make the adjusments there. To see if that helps. 

 

I did some troubleshooting and I'm quite sure the problem is with the gpu not giving any output on startup and sudenly working after a force start. 

 

Daniel. 

Share you diagnostic file: Tools tab, click on the Diagnostics icon, then click on the Download button, save the file and attach it here; so we can see full xml of your vm, your iommu groups and hardware.

You can start by dumping your own rom (gpu-z in windows if you have it installed, or unraid), again, do not download it from internet.

Edited July 22, 2021 by ghost82

[iltisdaniel]

Hey again, 

 

here's the diagnostics fileuns01dil-diagnostics-20210723-1349.zip.  I did use some tricks from this forum and space invaders to get my own GPU BIOS (if that's what you mean by ROM (sorry not familiar with the term). 

 

Should I share it here? I then linked to following his manual. 

 

Cheers for taking the time.

Daniel.

[ghost82]

2 hours ago, iltisdaniel said:

here's the diagnostics file

It doesn't seem you are binding to vfio the gpu, in your vfio-pci.cfg there's only:

BIND=0000:03:00.0|1b21:2142 0000:04:00.0|1106:3483

which is something related to usb controllers.

In unraid you need to bind to vfio the gpu and the audio part, from your diagnostic the iommu group should include also a pci bridge which should not be an issue to passthrough with the gpu (video+audio). If the bridge is an issue, use pcie acs override downstream,multifunction ("both") to split more your iommu groups.

You can do it from the unraid gui.

Once you bind the gpu this will not be available for unraid, so unraid will output nothing on that gpu, which will be dedicated to your vm.

Then, replace the whole xml of the vm with the attached one.

Next, restart unraid and try to boot the vm.

If this still doesn't work, you may need to switch your vm from i440fx to q35.

WS01DIL.txt

Edited July 23, 2021 by ghost82

[iltisdaniel]

Hey Ghost!

 

So I finally had a chance to test this out. Tbh, it was mainly my ignorance as I didn't know I have to bind the devices in the Device List. I saw them as available when making the VM and assumed that was it. 

 

I bound them, and it seem to be working. But It's still he same issue. The problem is the bridge. I cannot bind it. I've tried the pcie-acs override trick, that scattered the IOMMU groups but the bridge still remained unbindable. 

 

And I've tried Space INvaders trick of vfio_bind.id=*id of item*. That unfortunatelly also didn't help. Any ideas on how to move this stubborn bridge?

 

Also, what is the difference between Q51 and the default one I'm using?

 

Cheers,
Daniel.