Hello guys,

 

I am having problems with GPU passthrough to my VM. It won't boot and I'm getting the "Unknown PCI header type '127'" error.

After a restart it "boots" but I get "qemu-system-x86_64: vfio: Unable to power on device, stuck in D3". But after some time it goes back to error 127

I have tried finding help through threads here but can't seem to find any solutions to my problem.

 

What I have tried:

Changing OS from win10 to Linux

changing i440fx-4.2 to Q35

Turning off Hyper-V

Following Spaceinvaders Videos

 

Hardware:

CPU: AMD Ryzen 1700

GPU: MSI GTX970 

Motherboard: B450 GAMING PLUS MAX

 

Here is my VM log:

Quote

ErrorWarningSystemArrayLogin

-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 \
-device ahci,id=sata0,bus=pci.0,addr=0x3 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Windows 10.iso","node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw","file":"libvirt-3-storage"}' \
-device ide-cd,bus=sata0.0,drive=libvirt-3-format,id=sata0-0-0,bootindex=2 \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.173-2.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device ide-cd,bus=sata0.1,drive=libvirt-2-format,id=sata0-0-1 \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 10 Game/vdisk1.img","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \
-device ide-hd,bus=sata0.2,drive=libvirt-1-format,id=sata0-0-2,bootindex=1,write-cache=on \
-netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:38:f5:48,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=38,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0,bus=usb.0,port=1 \
-device vfio-pci,host=0000:26:00.0,id=hostdev0,bus=pci.0,addr=0x5 \
-device vfio-pci,host=0000:26:00.1,id=hostdev1,bus=pci.0,addr=0x6 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2020-03-19 18:19:43.744+0000: Domain id=1 is tainted: high-privileges
2020-03-19 18:19:43.744+0000: Domain id=1 is tainted: host-cpu
char device redirected to /dev/pts/1 (label charserial0)
2020-03-19T18:19:46.462957Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3
2020-03-19T18:19:46.463215Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3
2020-03-19T18:20:05.050233Z qemu-system-x86_64: terminating on signal 15 from pid 7989 (/usr/sbin/libvirtd)
2020-03-19 18:20:07.251+0000: shutting down, reason=destroyed
2020-03-19 18:38:29.935+0000: shutting down, reason=failed
2020-03-19 18:38:35.546+0000: shutting down, reason=failed

 

and is my VM xml:

      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.173-2.iso'/>
      <target dev='hdb' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10 Game/vdisk1.img'/>
      <target dev='hdc' bus='sata'/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:38:f5:48'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x26' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x26' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

Hopefully someone smarter than me can help me resolve the issue.

Thanks

Here is also the syslog

syslog

Do you have the latest BIOS for the motherboard?  There are issues with passthrough that was only fixed recently

 

Edited March 20, 20206 yr by toastman

8 hours ago, toastman said:

Do you have the latest BIOS for the motherboard?  There are issues with passthrough that was only fixed recently

 

Thanks! Just updated my motherboard and it boots but without a screen output. So I got my vbios and tried it out and... IT WORKS i am getting an display output. Trying to install Windows now and see how it does.

So I installed Windows and everything worked like a charm until I needed to reboot the VM to add mouse and keyboard passthrough. Now I'm back to the 127 error and the "stuck in D3" error. I remove keyboard and mouse passthrough but that did not fix the problem.

I tried adding a new VM but got the same error.

That's "normal".  unRAID can't reset the card after the VM stops using it.  But it can reset it after a full server reboot.  So you're stuck in the situation where anytime you power down the VM, you need to reboot the server before powering on a VM that uses the card.

 

If your BIOS is up to date, do a search for 127 errors.  There are a LOT of posts on here about that

Good to know. I tried rebooting and it did not work. I rebooted again with autoboot on and it worked. Seems a little bit unstable but it gets the job done.

Thanks for all the help toastman