Weird behavior with graphics passthrough


Recommended Posts

Long time lurker, first time poster,

 

I started with unraid on the 31st of December, jumping through hoops, and eventually got things kind of working? my vm was running, passthrough was working, drivers and all, but there were performance issues. long story short my hardware is as follows:

unRAID Version: 6.4.0_rc21b

 

Mainboard: 970 PRO GAMING AURA

CPU: AMD FX-6300

Memory: 2 sets of G.SKILL Ares Series 8GB (4x4) 16GB

GPU1: PowerColor R9 270x 2GB

GPU2: MSI GTX1050ti 4GB 

 

In this configuration I eventually got windows to accept the 1050ti, but at a performance hit compared to bare metal,

so that got me thinking, could it be the difference in the slots? (8x compared to 16x) I did some research and found negligible results, but because it was a virtual machine, I decided to check anyway. Switching those two GPUs unraid would boot, everything would be fine until I started the virtual machine, then it would lock up the entire machine (still trying to passthrough the 1050ti, resetting the config after switching to make sure). so some how this resolved itself? and then I ran into another problem where the output would go black when the machine would turn on (as expected) but then the vm would not take the GPU. I read into this as well and found a guide by hupster here. this helped to a point when I could finally get the rom to dump, although his guide was not 100% helpful to me because of my unraid version. (The vm XML files were different) SO I continued searching and found something useful for actually using the dumped vbios.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <rom file='/boot/vbios.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

For me version of the xml, this at least got the vm to boot, so that was something. Now we run into a different problem, error code 43. So i tried disabling all hyper-v extensions to no avail, I used Display Driver Uninstaller to attempt to start fresh, still no luck.

 

So the question is, Why does it work in one slot but refuses to in another? and is there a way around the error code 43 when using the gpu in the top most slot? i found this to be a weird experience seeing how everything is behaving. Any help would be greatly appreciated!

 

 

Bellow and attached is everything that might just be helpful.

 

Windows 10 XML: (started with windows 8 template because of 10 upgrade, doubt that would make a difference but who knows)

<domain type='kvm' id='1'>
  <name>Windows 10</name>
  <uuid>04d3019b-b971-5192-49bf-8d4c2169c54a</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 8.x" icon="windows.png" os="windows"/>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.10'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/04d3019b-b971-5192-49bf-8d4c2169c54a_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>
  <clock offset='utc'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='nec-xhci'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:93:8f:a1'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-Windows 10/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0909'/>
        <product id='0x001a'/>
        <address bus='4' device='2'/>
      </source>
      <alias name='hostdev2'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1038'/>
        <product id='0x1710'/>
        <address bus='8' device='2'/>
      </source>
      <alias name='hostdev3'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x413c'/>
        <product id='0x2107'/>
        <address bus='8' device='3'/>
      </source>
      <alias name='hostdev4'/>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

IOMMU Groups:

IOMMU group 0:	[1002:5a14] 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD9x0/RX980 Host Bridge (rev 02)
IOMMU group 1:	[1002:5a16] 00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GFX port 0)
IOMMU group 2:	[1002:5a17] 00:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0 PCI to PCI bridge (PCI Express GFX port 1)
IOMMU group 3:	[1002:5a18] 00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 0)
IOMMU group 4:	[1002:5a1a] 00:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 2)
IOMMU group 5:	[1002:5a1b] 00:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 3)
IOMMU group 6:	[1002:4391] 00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
IOMMU group 7:	[1002:4397] 00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
[1002:4396] 00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
IOMMU group 8:	[1002:4397] 00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
[1002:4396] 00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
IOMMU group 9:	[1002:4385] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 42)
IOMMU group 10:	[1002:4383] 00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) (rev 40)
IOMMU group 11:	[1002:439d] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
IOMMU group 12:	[1002:4384] 00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40)
IOMMU group 13:	[1002:4399] 00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
IOMMU group 14:	[1002:4397] 00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
[1002:4396] 00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
IOMMU group 15:	[1002:6810] 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Curacao XT / Trinidad XT [Radeon R7 370 / R9 270X/370X]
[1002:aab0] 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
IOMMU group 16:	[10de:1c82] 02:00.0 VGA compatible controller: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] (rev a1)
[10de:0fb9] 02:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1)
IOMMU group 17:	[1b21:1343] 03:00.0 USB controller: ASMedia Technology Inc. Device 1343
IOMMU group 18:	[1b21:1142] 04:00.0 USB controller: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller
IOMMU group 19:	[8086:1539] 05:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)

CPU Thread Pairings:

Pair 1:	cpu 0 / cpu 1
Pair 2:	cpu 2 / cpu 3
Pair 3:	cpu 4 / cpu 5

USB Devices:

Bus 001 Device 001:	ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001:	ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002:	ID 154b:00d2 PNY
Bus 003 Device 001:	ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001:	ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 002:	ID 0909:001a Audio-Technica Corp.
Bus 005 Device 001:	ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001:	ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001:	ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 008 Device 001:	ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 008 Device 002:	ID 1038:1710 SteelSeries ApS
Bus 008 Device 003:	ID 413c:2107 Dell Computer Corp.
Bus 009 Device 001:	ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 010 Device 001:	ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 011 Device 001:	ID 1d6b:0003 Linux Foundation 3.0 root hub

SCSI Devices:

[0:0:0:0]	disk    PNY      USB 3.0 FD       PMAP  /dev/sda   15.5GB
[1:0:0:0]	disk    ATA      SEAGATE ST375033 NA02  /dev/sdb    750GB
[2:0:0:0]	disk    ATA      WDC WD3200KS-00P 0M21  /dev/sdc    320GB
[3:0:0:0]	disk    ATA      WDC WD10EZEX-75Z 0A80  /dev/sdd   1.00TB
[4:0:0:0]	disk    ATA      KINGSTON SV300S3 BBF0  /dev/sde    120GB
[5:0:0:0]	disk    ATA      SAMSUNG MMCRE64G AH1Q  /dev/sdf   64.0GB
[6:0:0:0]	disk    ATA      WDC WD5000AAKX-0 1H15  /dev/sdg    500GB

Hardware Profile and Diagnostics are attached.

 

(Sorry for such a long post, but id rather be verbose than have someone need to ask about something I forgot to include.)

 

HardwareProfile.xml

cas-diagnostics-20180114-1624.zip

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.