No bootable device when attempting GPU passthrough


Recommended Posts

I am trying to load windows 8.1 through KVM. I originally set it up with the VM Manager plugin and everything loaded up fine through VNC. I installed a few windows updates and decided to attempt GPU passthrough. I feel like I am close because I can get my screen to display when I start the VM, but I get an error:

 

Booting from Hard Disk...
Boot failed: not a bootable disk

No bootable device.

 

So far I have done the following:

 

  • Ensured VT-d is enabled in the BIOS
  • Changed my primary display to the integrated GPU (Xeon E3 1276 v3)
  • Updated my go script to include: /usr/local/sbin/vfio-bind 0000:01:00.0 0000:01:00.1
  • Updated syslinux.cfg to include: append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot
  • Updated my XML file for gpu passthrough instead of vnc

 

Here is my xml:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>windows81pro</name>
  <uuid>d2f62486-79d7-1d78-145a-70e63187f893</uuid>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-2.1'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough'>
  </cpu>
  <clock offset='localtime'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/mnt/disk/appdisk/vm/windows81pro/windows81pro.qcow2'/>
      <target dev='hda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/disk/appdisk/vm/windows81pro/os_image/Win81.iso'/>
      <target dev='hdc' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pci-bridge'>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:76:56:bd'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/>
  </qemu:commandline>
</domain>

 

What am I missing? Its late and I'm sure its something easy I'm just not seeing.

Link to comment

I have recently seen the same issue...numerous times.  Every time it was also when attempting GPU passthrough.  Whether it was passed through from the start or added after the OS install didn't matter.  I was only trying to build a test VM so I gave up but I find it odd since I have never had this issue before.

 

I'll try again today and see if it happens. Can others also try to see if we can replicate the issue?

 

EDIT:  I have even tried to use one of my existing, working XMLs and ran into the same issue.

 

John

Link to comment

I still haven't fixed this, but here is what I tried:

 

- Reverted my syslinux.cfg and go scripts

- Rebooted

- Created a new vanilla test VM with VM Manager and chose to create a new disk

- Started VNC - Success.

 

This worked, however I tried forcing the domain to shutdown with the eject button in VM Manager. This shut down the domain, but now it will not start back up and I get the same boot device error. Why does forcing the domain to shutdown cause this to happen?

 

This persists through a reboot. It looks like forcing the domain to shutdown possibly corrupted the disk image. Can anyone else confirm?

 

Is there a better way to shutdown when the standard graceful shutdown in VM Manager does not work?

Link to comment

I still haven't fixed this, but here is what I tried:

 

- Reverted my syslinux.cfg and go scripts

- Rebooted

- Created a new vanilla test VM with VM Manager and chose to create a new disk

- Started VNC - Success.

 

This worked, however I tried forcing the domain to shutdown with the eject button in VM Manager. This shut down the domain, but now it will not start back up and I get the same boot device error. Why does forcing the domain to shutdown cause this to happen?

 

This persists through a reboot. It looks like forcing the domain to shutdown possibly corrupted the disk image. Can anyone else confirm?

 

Is there a better way to shutdown when the standard graceful shutdown in VM Manager does not work?

The way the Create VM works is, it creates 2 separate xml's at first.  A running/install xml and a permanent xml.  The initial xml has the cdrom as a boot device and if selected an extra cdrom with the virtio drivers loaded. The saved xml has the hard drive as the boot device pointing to your vm's image and the driver cdrom removed.

 

If you destroy the vm before an os is installed then the next start will try to use the empty hard drive to boot. 

 

The best thing would be to have 2 separate configs pointing to the same vm image.  One for vnc/boot problems and another for the passthrough/tinkering.

Link to comment

I have seen the GUI non-responsive couple of times when attempting to clean stop, maybe something in my VM settings could be the cause. I am more comfortable with the terminal and hence I have used it to stop and start the VM. The Win 8.1 installation on my system behaves pleasantly strange. The VM goes into a 'suspend' mode. When I destroy and restart it, it seems the VM starts from a suspend state, all the previous windows are back on. Copied below is my terminal script. I also use this script through my Android phone using "Script Kitty" and it works perfectly.

 

root@Tower:~# cat /boot/bin/restartvm.sh

virsh destroy win81pci2
virsh start win81pci2

 

As for the installation part, I have edited the xml's manually and used the GUI only for creating the VM's and hence cannot help much there.

 

I still haven't fixed this, but here is what I tried:

 

- Reverted my syslinux.cfg and go scripts

- Rebooted

- Created a new vanilla test VM with VM Manager and chose to create a new disk

- Started VNC - Success.

 

This worked, however I tried forcing the domain to shutdown with the eject button in VM Manager. This shut down the domain, but now it will not start back up and I get the same boot device error. Why does forcing the domain to shutdown cause this to happen?

 

This persists through a reboot. It looks like forcing the domain to shutdown possibly corrupted the disk image. Can anyone else confirm?

 

Is there a better way to shutdown when the standard graceful shutdown in VM Manager does not work?

Link to comment

Thanks for the clarification. That was indeed the problem. I didn't even realize it until I compared what the XML looked like by "view xml" feature compared to "edit xml" when the domain was not running. I added the cdrom back in as the first boot device and just reinstalled windows 8.1 since it didn't detect any partitions or anything. So far so good...

 

I swear I had Windows installed and running properly though prior to attempting my GPU passthrough... so that doesn't really explain why I was getting the boot device error. I will be careful to make sure I gracefully shut everything down from now on.

 

Maybe I will test forcing the domain to stop after I back everything up to see if I can recreate the issue.

 

The good news is that I now have my GTX970 properly passed through with audio. Windows Media Center and Kodi are smooth as silk.

Link to comment

I swear I had Windows installed and running properly though prior to attempting my GPU passthrough... so that doesn't really explain why I was getting the boot device error.

 

This is exactly what I experienced.  I had a fully functional WIN8.1 VM using native graphics (VNC).  As soon as I added the passed through GPU, I got the boot error.  I thought I just had too many beers and kept trying to rebuild the same VM over and over.  :)

 

John

Link to comment
  • 4 years later...

Having the same issue in 2019 with Ubuntu Linux, I tried adding a boot element to the OS section of the XML, but what options are available, it didnt like when I put cd. - Nevermind just recreated and this time picked scsi as the cdrom type now I'm getting the install from ubuntu

Edited by CodeSuth
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.