Skip to content
View in the app

A better way to browse. Learn more.
Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.
Unraid Summer Sale in live: 20% off Starter and Unleashed + buy one, get one 50% off

[Solved] GPU Passthrough black screen - gtx 1660

Lynxphp
in VM Engine (KVM)

Featured Replies

Hi

 

I just bought a Gigabyte GTX 1660 OC 6G for my VM.
I have been watching SI1's videos about how to passthroug a vm.

I had already previously setup my windows vm which works fine with VNC. I had to turn acs override mode on (both) to get my VM to start. My mobo is mini-itx so no second pcie slot here.

I got the vbios from techpowerup. I also dumped the actual bios of my card just to make sure, and it is the same as techpowerup's. I edited the vbios following SI1's video guide. I noticed that the bios contains twice the "Utm ek7400....." line. I tried with two edited versions, one where i only deleted the part above the first "Utm ek7400....." line and one where i deleted everything above the second "Utm ek7400....." line. When added to the vm template, both allow me to boot the vm, but with both i end up with a black screen in my vnc viewer (not unraid's).

 

 

I read in a forum post that the virtio may be the problem. I tried to uninstall it through unraid's VNC, i get a black screen and when i reboot it is back. Should i disable de the display adapter in the device manager? Im afraid that i might loose access to my VM's display if i do that.

 

I would very much appreciate any suggestions!

Thanks in advance.

 

I tried to add as much information as possible below.

 

My setup:

Case: Node 304 | Mobo: C236 WSI | CPU: E3-1245v6 | PSU: Corsair SF450 | Cache: 2x 850 pro 256Gb | HDDs: WD RED 8Tb (parity) & 2x WD RED 8Tb (data) | RAM 2x 8g 2133Mhz ECC

VM drive: 850 evo passedthrough.

 

VM's power mode set to high performance

 

 

Default IOMMU groups: 

IOMMU group 0:	[8086:5918] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers (rev 05)
IOMMU group 1:	[8086:1901] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (rev 05)
	[10de:2184] 01:00.0 VGA compatible controller: NVIDIA Corporation Device 2184 (rev a1)
	[10de:1aeb] 01:00.1 Audio device: NVIDIA Corporation Device 1aeb (rev a1)
	[10de:1aec] 01:00.2 USB controller: NVIDIA Corporation Device 1aec (rev a1)
	[10de:1aed] 01:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1aed (rev a1)
IOMMU group 2:	[8086:a12f] 00:14.0 USB controller: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller (rev 31)
	[8086:a131] 00:14.2 Signal processing controller: Intel Corporation 100 Series/C230 Series Chipset Family Thermal Subsystem (rev 31)
IOMMU group 3:	[8086:a13a] 00:16.0 Communication controller: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #1 (rev 31)
	[8086:a13d] 00:16.3 Serial controller: Intel Corporation 100 Series/C230 Series Chipset Family KT Redirection (rev 31)
IOMMU group 4:	[8086:a102] 00:17.0 SATA controller: Intel Corporation Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] (rev 31)
IOMMU group 5:	[8086:a110] 00:1c.0 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #1 (rev f1)
IOMMU group 6:	[8086:a117] 00:1c.7 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #8 (rev f1)
IOMMU group 7:	[8086:a149] 00:1f.0 ISA bridge: Intel Corporation C236 Chipset LPC/eSPI Controller (rev 31)
	[8086:a121] 00:1f.2 Memory controller: Intel Corporation 100 Series/C230 Series Chipset Family Power Management Controller (rev 31)
	[8086:a170] 00:1f.3 Audio device: Intel Corporation 100 Series/C230 Series Chipset Family HD Audio Controller (rev 31)
	[8086:a123] 00:1f.4 SMBus: Intel Corporation 100 Series/C230 Series Chipset Family SMBus (rev 31)
IOMMU group 8:	[8086:15b7] 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-LM (rev 31)
IOMMU group 9:	[8086:1533] 03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

 

 

IOMMU groups with acs override to both: 

IOMMU group 0:	[8086:5918] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers (rev 05)
IOMMU group 1:	[8086:1901] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (rev 05)
IOMMU group 2:	[8086:a12f] 00:14.0 USB controller: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller (rev 31)
	[8086:a131] 00:14.2 Signal processing controller: Intel Corporation 100 Series/C230 Series Chipset Family Thermal Subsystem (rev 31)
IOMMU group 3:	[8086:a13a] 00:16.0 Communication controller: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #1 (rev 31)
	[8086:a13d] 00:16.3 Serial controller: Intel Corporation 100 Series/C230 Series Chipset Family KT Redirection (rev 31)
IOMMU group 4:	[8086:a102] 00:17.0 SATA controller: Intel Corporation Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] (rev 31)
IOMMU group 5:	[8086:a110] 00:1c.0 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #1 (rev f1)
IOMMU group 6:	[8086:a117] 00:1c.7 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #8 (rev f1)
IOMMU group 7:	[8086:a149] 00:1f.0 ISA bridge: Intel Corporation C236 Chipset LPC/eSPI Controller (rev 31)
	[8086:a121] 00:1f.2 Memory controller: Intel Corporation 100 Series/C230 Series Chipset Family Power Management Controller (rev 31)
	[8086:a170] 00:1f.3 Audio device: Intel Corporation 100 Series/C230 Series Chipset Family HD Audio Controller (rev 31)
	[8086:a123] 00:1f.4 SMBus: Intel Corporation 100 Series/C230 Series Chipset Family SMBus (rev 31)
IOMMU group 8:	[8086:15b7] 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-LM (rev 31)
IOMMU group 9:	[10de:2184] 01:00.0 VGA compatible controller: NVIDIA Corporation Device 2184 (rev a1)
IOMMU group 10:	[10de:1aeb] 01:00.1 Audio device: NVIDIA Corporation Device 1aeb (rev a1)
IOMMU group 11:	[10de:1aec] 01:00.2 USB controller: NVIDIA Corporation Device 1aec (rev a1)
IOMMU group 12:	[10de:1aed] 01:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1aed (rev a1)
IOMMU group 13:	[8086:1533] 03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

Here's the log i get when i start the VM with acs_override to both and get the black screen: 


2019-04-25 20:16:35.558+0000: starting up libvirt version: 4.7.0, qemu version: 3.0.0, kernel: 4.18.20-unRAID, hostname: Tower
LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-Windows 10/master-key.aes' -machine pc-i440fx-3.0,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=none -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/b503c30d-ee23-37c2-aa85-b30b04595f33_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 8704 -realtime mlock=off -smp 6,sockets=1,cores=3,threads=2 -uuid b503c30d-ee23-37c2-aa85-b30b04595f33 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=24,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-hpet -no-shutdown -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 -device ich9-usb-4:00:10:d3:bd,bus=pci.0,addr=0x2 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=28,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3,romfile=/mnt/user/isos/Gigabyte.GTX1660.edited.rom -device vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x6 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
2019-04-25 20:16:35.558+0000: Domain id=1 is tainted: high-privileges
2019-04-25 20:16:35.558+0000: Domain id=1 is tainted: host-cpu
2019-04-25T20:16:35.596913Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/0 (label charserial0)
2019-04-25T20:16:54.621995Z qemu-system-x86_64: vfio: Cannot reset device 0000:01:00.1, depends on group 11 which is not owned.

 

 

 

IOMMU groups with acs override to downstream and the returned error when i try to start the vm:

Quote

internal error: qemu unexpectedly closed the monitor: 2019-04-25T19:54:04.479393Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3,romfile=/mnt/disk3/isos/Gigabyte.GTX1660.edited.rom: vfio error: 0000:01:00.0: group 9 is not viable Please ensure all devices within the iommu_group are bound to their vfio bus driver. 

IOMMU group 0:	[8086:5918] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers (rev 05)
IOMMU group 1:	[8086:1901] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (rev 05)
IOMMU group 2:	[8086:a12f] 00:14.0 USB controller: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller (rev 31)
	[8086:a131] 00:14.2 Signal processing controller: Intel Corporation 100 Series/C230 Series Chipset Family Thermal Subsystem (rev 31)
IOMMU group 3:	[8086:a13a] 00:16.0 Communication controller: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #1 (rev 31)
	[8086:a13d] 00:16.3 Serial controller: Intel Corporation 100 Series/C230 Series Chipset Family KT Redirection (rev 31)
IOMMU group 4:	[8086:a102] 00:17.0 SATA controller: Intel Corporation Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] (rev 31)
IOMMU group 5:	[8086:a110] 00:1c.0 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #1 (rev f1)
IOMMU group 6:	[8086:a117] 00:1c.7 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #8 (rev f1)
IOMMU group 7:	[8086:a149] 00:1f.0 ISA bridge: Intel Corporation C236 Chipset LPC/eSPI Controller (rev 31)
	[8086:a121] 00:1f.2 Memory controller: Intel Corporation 100 Series/C230 Series Chipset Family Power Management Controller (rev 31)
	[8086:a170] 00:1f.3 Audio device: Intel Corporation 100 Series/C230 Series Chipset Family HD Audio Controller (rev 31)
	[8086:a123] 00:1f.4 SMBus: Intel Corporation 100 Series/C230 Series Chipset Family SMBus (rev 31)
IOMMU group 8:	[8086:15b7] 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-LM (rev 31)
IOMMU group 9:	[10de:2184] 01:00.0 VGA compatible controller: NVIDIA Corporation Device 2184 (rev a1)
	[10de:1aeb] 01:00.1 Audio device: NVIDIA Corporation Device 1aeb (rev a1)
	[10de:1aec] 01:00.2 USB controller: NVIDIA Corporation Device 1aec (rev a1)
	[10de:1aed] 01:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1aed (rev a1)
IOMMU group 10:	[8086:1533] 03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

 

The VM XML's 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 10</name>
  <uuid>b503c30d-ee23-37c2-aa85-b30b04595f33</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>8912896</memory>
  <currentMemory unit='KiB'>8912896</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='5'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='6'/>
    <vcpupin vcpu='4' cpuset='3'/>
    <vcpupin vcpu='5' cpuset='7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/b503c30d-ee23-37c2-aa85-b30b04595f33_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='3' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source dev='/dev/disk/by-id/ata-Samsung_SSD_850_EVO_250GB_S3R0NF0J857389E'/>
      <target dev='hdc' bus='sata'/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/en_windows_10_multiple_editions_version_1703_updated_march_2017_x64_dvd_10189288.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:10:d3:bd'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/Gigabyte.GTX1660.edited.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

gigabyitegtx1660unedited.rom

Edited by Lynxphp

  • Quote
    • Author

    Just tried with RDP instead of VNC server and viewer and i got acces to my vm!!

    Everything works just fine!

     

  • Quote

    • Archived

    This topic is now archived and is closed to further replies.

    Go to topic listing

    Account

    Navigation

    Search

    Search

    Configure browser push notifications
    Chrome (Android)
    1. Tap the lock icon next to the address bar.
    2. Tap Permissions → Notifications.
    3. Adjust your preference.
    Chrome (Desktop)
    1. Click the padlock icon in the address bar.
    2. Select Site settings.
    3. Find Notifications and adjust your preference.