[GUIDE] Windows 10 VM Q35-5.x with Nvidia GPU passtrough


twiikker

Recommended Posts

Hi.

 

Here is how i did it with official drivers.

 

My setup.

Unraid 6.9.0-beta30

AMD 3700x / GT1030

ASUS TUF B550M PLUS / BIOS:0805 / AGESA combo V2 PI 1.0.0.2

Latest bios with AGESA combo V2 PI 1.0.8.0 did not work. got same error43 with it.

 

Requirements.

1. Dumped vbios rom with GPUZ and removed nvflash header with hexeditor.

I had to dump mine even though techpowerup had exact model but i have 2 years newer revision.

Guide for making that dump: https://www.youtube.com/watch?v=1IP-h9IKof0

 

2. Having your GPU in own IOMMU group

Info about that: https://www.youtube.com/watch?v=qQiMMeVNw-o

My example at Tools>System Devices

IOMMU group 18: [1022:43ea] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43ea

                         [10de:1d01] 04:00.0 VGA compatible controller: NVIDIA Corporation GP108 [GeForce GT 1030] (rev a1)

                         [10de:0fb8] 04:00.1 Audio device: NVIDIA Corporation GP108 High Definition Audio Controller (rev a1)

 

3. UEFI / BIOS Settings:

Have virtualizion enabled in bios and other setting to help separating IOMMU groups.

My example.

 

Tweaker -> Advanced CPU Settings -> SVM Mode -> Enable

Settings -> Miscellaneous -> IOMMU -> Enable

Settings -> AMD CBS -> ACS Enable -> Enable

Settings -> AMD CBS -> Enable AER Cap -> Enable

There is also SR-IOV setting somewhere.

 

There is also PCIe ACS override setting in unraid to help separate IOMMU groups.

Setting>VM Manager and select advanced view.

 

 

Installation

 

Create VM:

Add VM>Windows 10

 

CPU Mode: Host Passtrough

Machine: Q35-5.x

BIOS: OVMF

Select OS install ISO. I used Windows 10 1909 version.

Download and select Virtio ISO. I used virtio-win-0.1.185.iso.

Select suitable disk size. 50G on mine.

Graphics as VNC.

 

Uncheck start VM after creation.

Create.

 

Edit and change to xml view.

 

Add this under <apic/> tag

<ioapic driver='kvm'/>

 

Change hyper-V settings to look like this and add kvm setting after it:

  <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
  </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>

 

In my case for ryzen i need to apply fix for bluescreen KERNEL SECURITY CHECK ISSUE.

Add this to end of xml between   </devices> and </domain>

  <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off,hv_vendor_id=null,topoext=on,invtsc=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vpindex,hv-synic,hv-stimer,hv-reset,hv-frequencies,host-cache-info=on,l3-cache=off,-amd-stibp'/>
    <qemu:arg value='-machine'/>
    <qemu:arg value='q35,kernel_irqchip=on'/>
  </qemu:commandline>

 

As for others you can try these lines if those are enough.

<qemu:commandline> 
     <qemu:arg value='-cpu'/> 
     <qemu:arg value='host,kvm=off,hv_vendor_id=null'/>
     <qemu:arg value='-machine'/> 
     <qemu:arg value='q35,kernel_irqchip=on'/> 
   </qemu:commandline>

 

UPDATE

 

Start VM and take VNC connection ASAP to be able to press any key.

 

Select language preferences next and install.

I don't have product key (set it after install)

Select your windows version. (My case 10 Pro)

Accept terms.

Custom: Install Windows only (advanced)

Load driver>Browse>virtio-win-0.1.185>amd64>w10

Red Hat VirtIO SCSI controller>Next

Load driver>Browse>virtio-win-0.1.185>NetKVM>w10>amd64

Red Hat VirtIO Ethernet Adapter>Next

(You can add network later after installation also)

Select drive and next.

 

Set your personal account and privacy settings

 

You should be at your windows 10 desktop now.

Open start menu and type compmgmt.msc and open.

Open Device Manager.

On my case there are 2 devices without drivers.

Right click on driverless device and select update driver>Browse my computer>select whole virtio disk.

 

Enable RDP

Write "This PC" on start menu > right click on it > properties

From right press "Change settings" > Take note on Full Computer Name > remote tab

Check "Allow remote connections" > uncheck tab below it > OK.

 

Test rdp connection to VM.

If it doesnt work. Fix it before you continue.

Shutdown VM.

 

Edit VM in form view.

Press + on VNC selection to add another graphics card.

select your GPU and select vbios for your card.

select nvidia audio controller as sound card if it is in same IOMMU group.

(Optionally select attached KB+mouse)

Update

 

Now go back to edit and change to xml view.

Add those qemu commandlines again which we did earlier as basic view gets rid of them.

Disable VNC by removing <graphics> and </graphics> and everything between them.

Same on <video> and </video>.

Update.

 

Start VM

 

You should see it boot on attached monitor.

Use RDP or kb+mouse to install gpu drivers.

I use 399.24 version. Haven't tested on later versions.

I get error after installation that there drivers are not suitable but seems to work fine.

 

You should have now Windows 10 VM with gpu passtrough.

 

Hope this helps!

 

Twiikker

 

TROUBLESHOOTING:

qemu-system-x86_64: vfio_region_write(0000:04:00.0:region1+0xa8ec8, 0x0,8) failed: Device or resource busy

vfio-pci 0000:04:00.0: BAR 1: can't reserve [mem 0xd0000000-0xdfffffff 64bit pref]

 

make this type of script and run everytime array starts or manually run from console.

 

#!/bin/bash
echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

 

 

 

 

 

Edited by twiikker
added troubleshooting for one problem i had.
  • Like 1
  • Thanks 5
Link to comment
  • 1 month later...

I've been getting random blue screens. Could the follow fix my issue?

 

In my case for ryzen i need to apply fix for bluescreen KERNEL SECURITY CHECK ISSUE.

Add this to end of xml between   </devices> and </domain>

  <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off,hv_vendor_id=null,topoext=on,invtsc=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vpindex,hv-synic,hv-stimer,hv-reset,hv-frequencies,host-cache-info=on,l3-cache=off,-amd-stibp'/>
    <qemu:arg value='-machine'/>
    <qemu:arg value='q35,kernel_irqchip=on'/>
  </qemu:commandline>

Link to comment
5 hours ago, phat_cow said:

I've been getting random blue screens. Could the follow fix my issue?

 

In my case for ryzen i need to apply fix for bluescreen KERNEL SECURITY CHECK ISSUE.

Add this to end of xml between   </devices> and </domain>

  <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off,hv_vendor_id=null,topoext=on,invtsc=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vpindex,hv-synic,hv-stimer,hv-reset,hv-frequencies,host-cache-info=on,l3-cache=off,-amd-stibp'/>
    <qemu:arg value='-machine'/>
    <qemu:arg value='q35,kernel_irqchip=on'/>
  </qemu:commandline>

My bluescreen problem was fixed by those qemu commands.

 

My bluescreens wasnt just random. I couldnt even boot into install media fully without those commands.

Edited by twiikker
Link to comment
On 8/14/2020 at 9:47 AM, twiikker said:

Hi.

 

Here is how i did it with official drivers.

 

My setup.

Unraid 6.9.0-beta25

AMD 3700x / GT1030

 

Requirements.

1. Dumped vbios rom with GPUZ and removed nvflash header with hexeditor.

I had to dump mine even though techpowerup had exact model but i have 2 years newer revision.

Guide for making that dump: https://www.youtube.com/watch?v=1IP-h9IKof0

 

2. Having your GPU in own IOMMU group

Info about that: https://www.youtube.com/watch?v=qQiMMeVNw-o

My example at Tools>System Devices

IOMMU group 18: [1022:43ea] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43ea

                         [10de:1d01] 04:00.0 VGA compatible controller: NVIDIA Corporation GP108 [GeForce GT 1030] (rev a1)

                         [10de:0fb8] 04:00.1 Audio device: NVIDIA Corporation GP108 High Definition Audio Controller (rev a1)

 

3. UEFI / BIOS Settings:

Have virtualizion enabled in bios and other setting to help separating IOMMU groups.

My example.

 

Tweaker -> Advanced CPU Settings -> SVM Mode -> Enable

Settings -> Miscellaneous -> IOMMU -> Enable

Settings -> AMD CBS -> ACS Enable -> Enable

Settings -> AMD CBS -> Enable AER Cap -> Enable

There is also SR-IOV setting somewhere.

 

There is also PCIe ACS override setting in unraid to help separate IOMMU groups.

Setting>VM Manager and select advanced view.

 

 

Installation

 

Create VM:

Add VM>Windows 10

 

CPU Mode: Host Passtrough

Machine: Q35-5.0

BIOS: OVMF

Select OS install ISO. I used Windows 10 1909 version.

Download and select Virtio ISO. I used virtio-win-0.1.185.iso.

Select suitable disk size. 50G on mine.

Graphics as VNC.

 

Uncheck start VM after creation.

Create.

 

Edit and change to xml view.

 

Add this under <apic/> tag

<ioapic driver='kvm'/>

 

Change hyper-V settings to look like this and add kvm setting after it:

  <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
  </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>

 

In my case for ryzen i need to apply fix for bluescreen KERNEL SECURITY CHECK ISSUE.

Add this to end of xml between   </devices> and </domain>

  <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off,hv_vendor_id=null,topoext=on,invtsc=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vpindex,hv-synic,hv-stimer,hv-reset,hv-frequencies,host-cache-info=on,l3-cache=off,-amd-stibp'/>
    <qemu:arg value='-machine'/>
    <qemu:arg value='q35,kernel_irqchip=on'/>
  </qemu:commandline>

 

As for others you can try these lines if those are enough.

<qemu:commandline> 
     <qemu:arg value='-cpu'/> 
     <qemu:arg value='host,kvm=off,hv_vendor_id=null'/>
     <qemu:arg value='-machine'/> 
     <qemu:arg value='q35,kernel_irqchip=on'/> 
   </qemu:commandline>

 

UPDATE

 

Start VM and take VNC connection ASAP to be able to press any key.

 

Select language preferences next and install.

I don't have product key (set it after install)

Select your windows version. (My case 10 Pro)

Accept terms.

Custon: Install Windows only (advanced)

Load driver>Browse>virtio-win-0.1.185>amd64>w10

Red Hat VirtIO SCSI controller>Next

Load driver>Browse>virtio-win-0.1.185>NetKVM>w10>amd64

Red Hat VirtIO Ethernet Adapter>Next

(You can add network later after installation also)

Select drive and next.

 

Set your personal account and privacy settings

 

You should be at your windows 10 desktop now.

Open start menu and type compmgmt.msc and open.

Open Device Manager.

On my case there are 2 devices without drivers.

Right click on driverless device and select update driver>Browse my computer>select whole virtio disk.

 

Enable RDP

Write "This PC" on start menu > right click on it > properties

From right press "Change settings" > Take note on Full Computer Name > remote tab

Check "Allow remote connections" > uncheck tab below it > OK.

 

Test rdp connection to VM.

If it doesnt work. Fix it before you continue.

Shutdown VM.

 

Edit VM in form view.

Press + on VNC selection to add another graphics card.

select your GPU and select vbios for your card.

select nvidia audio controller as sound card if it is in same IOMMU group.

(Optionally select attached KB+mouse)

Update

 

Now go back to edit and change to xml view.

Add those qemu commandlines again which we did earlier as basic view gets rid of them.

Disable VNC by removing <graphics> and </graphics> and everything between them.

Same on <video> and </video>.

Update.

 

Start VM

 

You should see it boot on attached monitor.

Use RDP or kb+mouse to install gpu drivers.

I use 399.24 version. Haven't tested on later versions.

I get error after installation that there drivers are not suitable but seems to work fine.

 

You should have now Windows 10 VM with gpu passtrough.

 

Hope this helps!

 

Twiikker

 

 

Hi, what motherboard are you using? and what firmware version?

Link to comment

Actually dont know if latest bios is the root cause since i cant get rid of error 43 currently.

I though it was since my installation was old and i dont have extra activation key for that test machine which i made this guide.

I don't think that i have changed anything else which would affect that. Have to double check bios settings if something in there has been changed.

 

Agesa 1.0.8.0 firmware.

Edited by twiikker
Link to comment
  • 4 weeks later...

I also have a problem with passing my GPU to my windows 10. runnin unraid 6.8.3

 

when i do all the above i get

2020-11-14 21:29:10.154+0000: Domain id=16 is tainted: high-privileges
2020-11-14 21:29:10.154+0000: Domain id=16 is tainted: custom-argv
2020-11-14 21:29:10.154+0000: Domain id=16 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2020-11-14T21:29:10.225526Z qemu-system-x86_64: warning: This family of AMD CPU doesn't support hyperthreading(2)
Please configure -smp options properly or try enabling topoext feature.
libusb: error [udev_hotplug_event] ignoring udev action bind

in the log file.

 

before when i didnt do that i got that

vfio-pci 0000:0b:00.0: BAR 3: can't reserve [mem 0xd0000000-0xdfffffff 64bit pref]

 

 

i can do the same for ubuntu VM without any of these things. It just works.

 

Edit:

It works it just Takes a very long time to get into windows. and it does not feel smooth. I gave it 12 core and 16 gb so should be smooth.

Edited by KoNeko
extra info
Link to comment
12 hours ago, KoNeko said:

I also have a problem with passing my GPU to my windows 10. runnin unraid 6.8.3

 

when i do all the above i get

2020-11-14 21:29:10.154+0000: Domain id=16 is tainted: high-privileges
2020-11-14 21:29:10.154+0000: Domain id=16 is tainted: custom-argv
2020-11-14 21:29:10.154+0000: Domain id=16 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2020-11-14T21:29:10.225526Z qemu-system-x86_64: warning: This family of AMD CPU doesn't support hyperthreading(2)
Please configure -smp options properly or try enabling topoext feature.
libusb: error [udev_hotplug_event] ignoring udev action bind

in the log file.

 

before when i didnt do that i got that

vfio-pci 0000:0b:00.0: BAR 3: can't reserve [mem 0xd0000000-0xdfffffff 64bit pref]

 

 

i can do the same for ubuntu VM without any of these things. It just works.

 

Edit:

It works it just Takes a very long time to get into windows. and it does not feel smooth. I gave it 12 core and 16 gb so should be smooth.

what cpu you have? Googled a bit that those tainted errors shouldnt matter but not sure about those other errors.

 

My log also gets there errors at end but my boot time to login screen is ~25 seconds.

So errors after that is propably what you should research.

 

2020-11-15 10:16:20.255+0000: Domain id=3 is tainted: high-privileges
2020-11-15 10:16:20.255+0000: Domain id=3 is tainted: custom-argv
2020-11-15 10:16:20.255+0000: Domain id=3 is tainted: host-cpu
char device redirected to /dev/pts/1 (label charserial0)

 

Link to comment
1 hour ago, twiikker said:

what cpu you have? Googled a bit that those tainted errors shouldnt matter but not sure about those other errors.

 

My log also gets there errors at end but my boot time to login screen is ~25 seconds.

So errors after that is propably what you should research.

 

2020-11-15 10:16:20.255+0000: Domain id=3 is tainted: high-privileges
2020-11-15 10:16:20.255+0000: Domain id=3 is tainted: custom-argv
2020-11-15 10:16:20.255+0000: Domain id=3 is tainted: host-cpu
char device redirected to /dev/pts/1 (label charserial0)

 

i got a threadripper 2950x

Link to comment
1 hour ago, KoNeko said:

i got a threadripper 2950x

Does that windows 10 vm work normally without gpu passtrough? Im pretty sure that its upto something with that cpu and you need to figure some workaround over that. Many people seems to have same problem. Like by just googling kvm threadripper and that error.

Link to comment
12 minutes ago, twiikker said:

Does that windows 10 vm work normally without gpu passtrough? Im pretty sure that its upto something with that cpu and you need to figure some workaround over that. Many people seems to have same problem. Like by just googling kvm threadripper and that error.

with vnc it works normaly. With the gpu passthrough it does work but it seems a little bit laggy. even when i just click on file explorer.

Link to comment

i tried it again. but all i got now is

 

2020-11-16T18:50:19.072503Z qemu-system-x86_64: -device vfio-pci,host=0000:0b:00.0,id=hostdev0,bus=pci.3,addr=0x0,romfile=/mnt/cache/system/vbios/Palit.GTX980.4096.rom: Failed to mmap 0000:0b:00.0 BAR 3. Performance may be slow
2020-11-16T18:50:19.386211Z qemu-system-x86_64: vfio: Cannot reset device 0000:0d:00.3, depends on group 21 which is not owned.
2020-11-16T18:50:20.438181Z qemu-system-x86_64: vfio: Cannot reset device 0000:0d:00.3, depends on group 21 which is not owned.

 

and the log is filling up VERY quickly with these lines. like matter of seconds and the /var/log/syslog is full.  Need to reboot to get it cleared.

2020-11-16T19:36:52.808331Z qemu-system-x86_64: vfio_region_write(0000:0b:00.0:region3+0x10eb41c, 0x0,4) failed: Device or resource busy
2020-11-16T19:36:52.808344Z qemu-system-x86_64: vfio_region_write(0000:0b:00.0:region3+0x10eb420, 0x0,4) failed: Device or resource busy
2020-11-16T19:36:52.808356Z qemu-system-x86_64: vfio_region_write(0000:0b:00.0:region3+0x10eb424, 0x0,4) failed: Device or resource busy
2020-11-16T19:36:52.808368Z qemu-system-x86_64: vfio_region_write(0000:0b:00.0:region3+0x10eb428, 0x0,4) failed: Device or resource busy

 

 

Edited by KoNeko
Link to comment
7 hours ago, KoNeko said:

i tried it again. but all i got now is

 

2020-11-16T18:50:19.072503Z qemu-system-x86_64: -device vfio-pci,host=0000:0b:00.0,id=hostdev0,bus=pci.3,addr=0x0,romfile=/mnt/cache/system/vbios/Palit.GTX980.4096.rom: Failed to mmap 0000:0b:00.0 BAR 3. Performance may be slow
2020-11-16T18:50:19.386211Z qemu-system-x86_64: vfio: Cannot reset device 0000:0d:00.3, depends on group 21 which is not owned.
2020-11-16T18:50:20.438181Z qemu-system-x86_64: vfio: Cannot reset device 0000:0d:00.3, depends on group 21 which is not owned.

 

and the log is filling up VERY quickly with these lines. like matter of seconds and the /var/log/syslog is full.  Need to reboot to get it cleared.

2020-11-16T19:36:52.808331Z qemu-system-x86_64: vfio_region_write(0000:0b:00.0:region3+0x10eb41c, 0x0,4) failed: Device or resource busy
2020-11-16T19:36:52.808344Z qemu-system-x86_64: vfio_region_write(0000:0b:00.0:region3+0x10eb420, 0x0,4) failed: Device or resource busy
2020-11-16T19:36:52.808356Z qemu-system-x86_64: vfio_region_write(0000:0b:00.0:region3+0x10eb424, 0x0,4) failed: Device or resource busy
2020-11-16T19:36:52.808368Z qemu-system-x86_64: vfio_region_write(0000:0b:00.0:region3+0x10eb428, 0x0,4) failed: Device or resource busy

 

 

did you map gpu soundcard also and are gpu and gpu soundcard on same iommu group?

 

Link to comment
15 hours ago, twiikker said:

did you map gpu soundcard also and are gpu and gpu soundcard on same iommu group?

 

Well what i did was did a factory reset on my mainboard. mine also has a BMC vga output port i put that as the main videocard of the computer. Enabled the iommu stuff again.

 

did a reinstall of the VM

image.png.ffa7fcd648f36e0b10f1ba65fe400893.png

 

and its working again. and i dont see any lag that i can notice. still have to do more tests and install steam etc on it to test. but it looks nicely.

 

i didnt change anything in the XML for this.

 

Only problem that i have i cant put tho my keyboard

Nov 17 19:35:39 kernel: input: Corsair Corsair Gaming K95 RGB PLATINUM Keyboard Keyboard as /devices/pci0000:00/0000:00:01.1/0000:01:00.0/usb1/1-1/1-1.4/1-1.4:1.0/0003:1B1C:1B2D.0025/input/input66
Nov 17 19:35:39 kernel: input: Corsair Corsair Gaming K95 RGB PLATINUM Keyboard Consumer Control as /devices/pci0000:00/0000:00:01.1/0000:01:00.0/usb1/1-1/1-1.4/1-1.4:1.0/0003:1B1C:1B2D.0025/input/input67
Nov 17 19:35:39 kernel: input: Corsair Corsair Gaming K95 RGB PLATINUM Keyboard as /devices/pci0000:00/0000:00:01.1/0000:01:00.0/usb1/1-1/1-1.4/1-1.4:1.0/0003:1B1C:1B2D.0025/input/input68
Nov 17 19:35:39 kernel: input: Corsair Corsair Gaming K95 RGB PLATINUM Keyboard as /devices/pci0000:00/0000:00:01.1/0000:01:00.0/usb1/1-1/1-1.4/1-1.4:1.0/0003:1B1C:1B2D.0025/input/input69
Nov 17 19:35:39 kernel: input: Corsair Corsair Gaming K95 RGB PLATINUM Keyboard Mouse as /devices/pci0000:00/0000:00:01.1/0000:01:00.0/usb1/1-1/1-1.4/1-1.4:1.0/0003:1B1C:1B2D.0025/input/input70
Nov 17 19:35:39 kernel: hid-generic 0003:1B1C:1B2D.0025: input,hiddev97,hidraw1: USB HID v1.11 Keyboard [Corsair Corsair Gaming K95 RGB PLATINUM Keyboard] on usb-0000:01:00.0-1.4/input0
Nov 17 19:35:39 kernel: hid-generic 0003:1B1C:1B2D.0026: hiddev98,hidraw2: USB HID v1.11 Device [Corsair Corsair Gaming K95 RGB PLATINUM Keyboard] on usb-0000:01:00.0-1.4/input1

 

i use a HDMI switch that has my Mouse and Keyboard. It does see my mouse but not my K95

Edited by KoNeko
  • Like 1
Link to comment
  • 1 year later...
  • 2 weeks later...
On 8/14/2020 at 8:47 AM, twiikker said:

Hi.

 

Here is how i did it with official drivers.

 

My setup.

Unraid 6.9.0-beta30

AMD 3700x / GT1030

ASUS TUF B550M PLUS / BIOS:0805 / AGESA combo V2 PI 1.0.0.2

Latest bios with AGESA combo V2 PI 1.0.8.0 did not work. got same error43 with it.

 

Requirements.

1. Dumped vbios rom with GPUZ and removed nvflash header with hexeditor.

I had to dump mine even though techpowerup had exact model but i have 2 years newer revision.

Guide for making that dump: https://www.youtube.com/watch?v=1IP-h9IKof0

 

2. Having your GPU in own IOMMU group

Info about that: https://www.youtube.com/watch?v=qQiMMeVNw-o

My example at Tools>System Devices

IOMMU group 18: [1022:43ea] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43ea

                         [10de:1d01] 04:00.0 VGA compatible controller: NVIDIA Corporation GP108 [GeForce GT 1030] (rev a1)

                         [10de:0fb8] 04:00.1 Audio device: NVIDIA Corporation GP108 High Definition Audio Controller (rev a1)

 

3. UEFI / BIOS Settings:

Have virtualizion enabled in bios and other setting to help separating IOMMU groups.

My example.

 

Tweaker -> Advanced CPU Settings -> SVM Mode -> Enable

Settings -> Miscellaneous -> IOMMU -> Enable

Settings -> AMD CBS -> ACS Enable -> Enable

Settings -> AMD CBS -> Enable AER Cap -> Enable

There is also SR-IOV setting somewhere.

 

There is also PCIe ACS override setting in unraid to help separate IOMMU groups.

Setting>VM Manager and select advanced view.

 

 

Installation

 

Create VM:

Add VM>Windows 10

 

CPU Mode: Host Passtrough

Machine: Q35-5.x

BIOS: OVMF

Select OS install ISO. I used Windows 10 1909 version.

Download and select Virtio ISO. I used virtio-win-0.1.185.iso.

Select suitable disk size. 50G on mine.

Graphics as VNC.

 

Uncheck start VM after creation.

Create.

 

Edit and change to xml view.

 

Add this under <apic/> tag

<ioapic driver='kvm'/>

 

Change hyper-V settings to look like this and add kvm setting after it:

  <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
  </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>

 

In my case for ryzen i need to apply fix for bluescreen KERNEL SECURITY CHECK ISSUE.

Add this to end of xml between   </devices> and </domain>

  <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off,hv_vendor_id=null,topoext=on,invtsc=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vpindex,hv-synic,hv-stimer,hv-reset,hv-frequencies,host-cache-info=on,l3-cache=off,-amd-stibp'/>
    <qemu:arg value='-machine'/>
    <qemu:arg value='q35,kernel_irqchip=on'/>
  </qemu:commandline>

 

As for others you can try these lines if those are enough.

<qemu:commandline> 
     <qemu:arg value='-cpu'/> 
     <qemu:arg value='host,kvm=off,hv_vendor_id=null'/>
     <qemu:arg value='-machine'/> 
     <qemu:arg value='q35,kernel_irqchip=on'/> 
   </qemu:commandline>

 

UPDATE

 

Start VM and take VNC connection ASAP to be able to press any key.

 

Select language preferences next and install.

I don't have product key (set it after install)

Select your windows version. (My case 10 Pro)

Accept terms.

Custom: Install Windows only (advanced)

Load driver>Browse>virtio-win-0.1.185>amd64>w10

Red Hat VirtIO SCSI controller>Next

Load driver>Browse>virtio-win-0.1.185>NetKVM>w10>amd64

Red Hat VirtIO Ethernet Adapter>Next

(You can add network later after installation also)

Select drive and next.

 

Set your personal account and privacy settings

 

You should be at your windows 10 desktop now.

Open start menu and type compmgmt.msc and open.

Open Device Manager.

On my case there are 2 devices without drivers.

Right click on driverless device and select update driver>Browse my computer>select whole virtio disk.

 

Enable RDP

Write "This PC" on start menu > right click on it > properties

From right press "Change settings" > Take note on Full Computer Name > remote tab

Check "Allow remote connections" > uncheck tab below it > OK.

 

Test rdp connection to VM.

If it doesnt work. Fix it before you continue.

Shutdown VM.

 

Edit VM in form view.

Press + on VNC selection to add another graphics card.

select your GPU and select vbios for your card.

select nvidia audio controller as sound card if it is in same IOMMU group.

(Optionally select attached KB+mouse)

Update

 

Now go back to edit and change to xml view.

Add those qemu commandlines again which we did earlier as basic view gets rid of them.

Disable VNC by removing <graphics> and </graphics> and everything between them.

Same on <video> and </video>.

Update.

 

Start VM

 

You should see it boot on attached monitor.

Use RDP or kb+mouse to install gpu drivers.

I use 399.24 version. Haven't tested on later versions.

I get error after installation that there drivers are not suitable but seems to work fine.

 

You should have now Windows 10 VM with gpu passtrough.

 

Hope this helps!

 

Twiikker

 

TROUBLESHOOTING:

qemu-system-x86_64: vfio_region_write(0000:04:00.0:region1+0xa8ec8, 0x0,8) failed: Device or resource busy

vfio-pci 0000:04:00.0: BAR 1: can't reserve [mem 0xd0000000-0xdfffffff 64bit pref]

 

make this type of script and run everytime array starts or manually run from console.

 

#!/bin/bash
echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

 

 

 

 

 

 

You're legend. Thank you!

  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.