Sample xml file for windows 8.1 and GPU/USB passtrough


Recommended Posts

  • Replies 110
  • Created
  • Last Reply

Top Posters In This Topic

I did not get anything out of my hdmi? when using VNC all looks good in the VM? maybe we need LT to release new beta with all patches added.

 

@LT , is it possible to release a new beta ?

 

//Peter

 

Peter,

 

So when you said you got iGPU working, you actually never got the output from HDMI or any output to actually display anything on your monitor, you just got the VM to start and then via VNC, you were able to install the drivers and whatnot, yes?  We have gotten that far in our internal testing as well, but if the display isn't outputting to the monitor IMMEDIATELY after creating the VM, it's not actually working.

 

Let me also add that we have actually spoken with Alex Williamson from Red Hat on this as well.  The bottom line is that iGPU pass through is a real b#*% (pardon my french ;-).  iGPUs don't adhere to the same rules as an atypical PCI/PCIe device and come with a number of additional challenges.

 

In our own internal alpha development build at this point, we have tried a number of various patches to see if we could get this to work, but even Alex said that those patches are very selective in who they will actually work for.

 

Bottom line:  don't get your hopes up on iGPU pass through.  It just isn't an area that anyone in the development community for KVM or even VFIO are focusing on as a concentrated effort because quite simply, there are too many complexities due to the different types of iGPUs that exist and how they work.

Link to comment

Ok, I've researched silently now for quite a while and I'm tired of banging my head repeatedly.  :o

 

The graphics card I want to pass is in IOMMU group 1 along with other things that I do NOT want to pass.

However I'm reading I have to pass all the hardware in the group in order for it to work.

Which I'm pretty sure is why I'm getting the error below when I  run "virsh create win8kvm.xml"

 

error: Failed to create domain from win8kvm.xml

error: internal error: early end of file from monitor: possible problem:

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error opening /dev/vfio/1: Operation not permitted

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 1

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed.

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

 

Is that right?

Is there an easy way to move the graphics card to another IOMMU group?

 

Qemu.conf is edited accordingly, my xml looks to be correct (per here and other places), and the card shows the kernel driver in use is "vfio-pci". Also syslinux.cfg has "intel_iommu=on" and hardware is all vt-d capable and properly enabled.

 

IOMMU Group 1 for me has all this in it:

### Group 1 ###

    00:01.0 PCI bridge: Intel Corporation Haswell PCI Express x16 Controller (rev 06)

    00:01.1 PCI bridge: Intel Corporation Haswell PCI Express x8 Controller (rev 06)

    00:01.2 PCI bridge: Intel Corporation Haswell PCI Express x4 Controller (rev 06)

    02:00.0 VGA compatible controller: AMD/ATI [Advanced Micro Devices, Inc.] Park [Mobility Radeon HD 5430]

    02:00.1 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] Cedar HDMI Audio [Radeon HD 5400/6300 Series]

    03:00.0 RAID bus controller: Adaptec Serial ATA II RAID 1430SA (rev 02)

 

Link to comment

I did not get anything out of my hdmi? when using VNC all looks good in the VM? maybe we need LT to release new beta with all patches added.

 

@LT , is it possible to release a new beta ?

 

//Peter

 

 

Let me also add that we have actually spoken with Alex Williamson from Red Hat on this as well.  The bottom line is that iGPU pass through is a real b#*% (pardon my french ;-).  iGPUs don't adhere to the same rules as an atypical PCI/PCIe device and come with a number of additional challenges.

 

In our own internal alpha development build at this point, we have tried a number of various patches to see if we could get this to work, but even Alex said that those patches are very selective in who they will actually work for.

 

Bottom line:  don't get your hopes up on iGPU pass through.  It just isn't an area that anyone in the development community for KVM or even VFIO are focusing on as a concentrated effort because quite simply, there are too many complexities due to the different types of iGPUs that exist and how they work.

 

Hi Jomp,

 

I'm fine with my Radeon GPU that works fine to passtrough, will not give the iGPU more try. since it needs to much patches to work, maybe if this patches are in next beta I can try again, anyway, I'm OK with my Radeon card. And I don't think you should put to much work on this iGPU, KVM works great anyway!

 

//Peter

Link to comment

 

Hi Jomp,

 

I'm fine with my Radeon GPU that works fine to passtrough, will not give the iGPU more try. since it needs to much patches to work, maybe if this patches are in next beta I can try again, anyway, I'm OK with my Radeon card. And I don't think you should put to much work on this iGPU, KVM works great anyway!

 

//Peter

 

Thanks Peter.  I can also confirm that the i915 patch isn't needed for iGPU pass through in unRAID (even though we tried it out nonetheless ;-).  i915 is only needed if you need to unbind the GPU from your host and rebind it to a guest.  We never bind the GPU to the host in the first place, so this isn't necessary (same reason why we don't need to use pci-stub either).

Link to comment

In case anyone else here gets the same error as me when executing "virsh create inputwhatever.xml"

(My error)

"error: Failed to create domain from win8kvm.xml

error: internal error: early end of file from monitor: possible problem:

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error opening /dev/vfio/1: Operation not permitted

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 1

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed.

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized"

 

(From what I have read and discussed with help from individuals; thank you!) This is related to how vfio-bind works in relation to the IOMMU grouping.

To say this quickly (and not technical), you MUST pass all devices within an IOMMU group in order to successfully pass that device to a VM..

So if you have a GPU and a SATA card in the same grouping (as I do) you cannot just pick out the GPU, bind it, and pass it.

If you bind all in the group, then you'd be golden (however I need the SATA to stay right where it is for UnRaid).

 

Here's a good snippet from here: https://lwn.net/Articles/474088/

"One problem with KVM's device assignment is that it assumes that all devices are fully independent of each other. In particular, groups of devices may be connected through the same IOMMU; that means that any device can access any memory regions made available to any other devices in the same group. That, in turn, implies that the group of devices must be assigned as a unit; if any of those devices are assigned separately, the isolation of the group as a whole can be broken.

 

So the first thing a VFIO driver writer will encounter is the group mechanism. The VFIO code creates the groups to match the hardware topology. It then ensures that every device in a group is controlled by a VFIO driver; if any device is unavailable, then the group as a whole cannot be used. Most devices on a typical system are unlikely to be bound to VFIO drivers at boot, so the system administrator must explicitly unbind them and tell VFIO to claim them. This is probably a good thing; exposing groups of devices to user space is best not done by default."

 

There is a patch to the kernel that when enabled is intended to move individual devices to separate IOMMU groups, it can be set with "pcie_acs_override=downstream" when it is part of the kernel.

It is not currently in the 6.0b6 kernel, but may come in a future release.

 

Hope this helps others having the same issue and sorry if I didn't explain this perfectly, Linux noob.

For now, no hardware pass-thru for me..  :(

Link to comment

In case anyone else here gets the same error as me...

Hope this helps others having the same issue and sorry if I didn't explain this perfectly, Linux noob.

 

Thanks for posting the results of your research on this.  I got tripped up by this also, but since my card died, I've not done any research into it since then.

 

Hopefully this useful information can be used by jonp et al to make progress on getting iGPU working.  It sounds like they've not had any luck at all with this, so I'm not expecting miracles, but I really, really do hope iGPU passthru can be made to work eventually, since I just don't want to purchase a new card if I don't have to.

Link to comment

 

 

Bind the PCI device to vfio-pc ( I have a script for that)

 

save below as vfio-bind

#!/bin/bash

modprobe vfio-pci

for dev in "$@"; do
        vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
        device=$(cat /sys/bus/pci/devices/$dev/device)
        if [ -e /sys/bus/pci/devices/$dev/driver ]; then
                echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
        fi
        echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id
done

 

Pls, I am stuck here. How do I "as vfio-bind"? What is the extension? Where should it be placed?

Link to comment
  • 3 weeks later...

I've finally made some progress with KVM and successfully passed through my first device, a PCIe dual Sat tuner. My next task is to pass through the GPU but when i look at the IOMMU groups i get this.

 

 

### Group 1 ###

    00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)

    01:00.0 VGA compatible controller: AMD/ATI [Advanced Micro Devices, Inc.] Caicos [Radeon HD 6450]

    01:00.1 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] Caicos HDMI Audio [Radeon HD 6400 Series]

 

Every other device in my machine is in its own group apart from the graphics card i want to pass through. So i have 2 questions.

 

Do you think i can pas through the PCIe Root port without it causing issues?

 

If i move the graphics card to a different PCIe slot will it move groups?

 

 

Link to comment

Hi Peter,

 

I am trying to follow your guide. When you say the following ...

 

Then run ....

 

00:1d.0 0c03: 8086:1e26 (rev 04)
        Subsystem: 1849:1e26
        Kernel driver in use: vfio-pci
04:00.0 0300: 1002:68f9
        Subsystem: 1682:304e
        Kernel driver in use: vfio-pci
04:00.1 0403: 1002:aa68
        Subsystem: 1682:aa68
        Kernel driver in use: vfio-pci

 

...you mean to copy this code -modified with my desired options to be passthroughed (I look for them with "lspci -knn")- and paste it in the command line?

 

If I do that I get this oputput and I think it is not ok....pls let me know what am i missing:

root@Tower2:/boot# vfio-bind 0000:03:00.0 0000:03:00.1 0000:00:16.0 0000:07:00.0           0000:08:00.0
root@Tower2:/boot# 03:00.0 0300: 1002:554f
-bash: 03:00.0: command not found
00:16.0 0c03: 1002:4397
root@Tower2:/boot#         Subsystem: 196d:1089
-bash: $'\302\240': command not found
root@Tower2:/boot#         Kernel driver in use: vfio-pci
-bash: $'\302\240': command not found
root@Tower2:/boot# 03:00.1 0380: 1002:556f
-bash: 03:00.1: command not found
root@Tower2:/boot#         Subsystem: 196d:1088
-bash: $'\302\240': command not found
root@Tower2:/boot#         Kernel driver in use: vfio-pci
-bash: $'\302\240': command not found
root@Tower2:/boot# 00:16.0 0c03: 1002:4397
-bash: 00:16.0: command not found
root@Tower2:/boot#         Subsystem: 1043:85bf
-bash: $'\302\240': command not found
root@Tower2:/boot#         Kernel driver in use: vfio-pci
-bash: $'\302\240': command not found
root@Tower2:/boot# 07:00.0 0c03:  1b21:1142
-bash: 07:00.0: command not found
root@Tower2:/boot#         Subsystem: 1043:85bf
-bash: $'\302\240': command not found
        Subsystem: 1043:85bf
        Kernel driver in use: vfio-pciroot@Tower2:/boot#         Kernel driver i         n use: vfio-pci
-bash: $'\302\240': command not found
root@Tower2:/boot# 08:00.0 0c03:  1b21:1142
-bash: 08:00.0: command not found
root@Tower2:/boot#         Subsystem: 1043:85bf
-bash: $'\302\240': command not found
root@Tower2:/boot#         Kernel driver in use: vfio-pci
-bash: $'\302\240': command not found

Rgds.

Link to comment

Hi again, after following the whole steps (including what I posted before) This is what I get:

root@Tower2:/mnt/btrfs/VM# virsh create w81kvm1.xml
error: failed to connect to the hypervisor
error: no valid connection
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory

root@Tower2:/mnt/btrfs/VM#

I need some hint!

Edit: Let me know if you need me to post anything else.

My syslinux file:

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest
label Xen/unRAID OS
  kernel /syslinux/mboot.c32
  append /xen --- /bzimage --- /bzroot
label Xen/unRAID OS Safe Mode (no plugins)
  kernel /syslinux/mboot.c32
  append /xen --- /bzimage --- /bzroot unraidsafemode
label KVM unRAID OS
  kernel /bzimage
  append intel_iommu=on quiet vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot acpi_sleep=s3_bios

 

And my w81kvm1.xml:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>w81kvm1</name>
  <uuid>cc411d70-4463-4db7-bf36-d364c0cdaa9d</uuid>
  <memory unit='GB'>2</memory>
  <currentMemory unit='GB'>2</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='q35'>hvm</type>
    <loader>/usr/share/qemu/bios-256k.bin</loader>
    <boot dev='cdrom'/>
  </os>
<features>
    <acpi/>
    <apic/>
</features>
  <cpu mode='host-passthrough'>
  </cpu>
  <clock offset='localtime'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/mnt/btrfs/VM/w81kvm1.qcow2'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/btrfs/VM/Win_8_1_64BIT.iso'/>
      <target dev='sdc' bus='sata'/>
      <readonly/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/btrfs/VM/virtio-win-0.1-81.iso'/>
      <target dev='sdd' bus='sata'/>
      <readonly/>
    </disk>
    <controller type='sata' index='0'>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
    </controller>
    <controller type='pci' index='2' model='pci-bridge'>
    </controller>
    <controller type='usb' index='0' model='none'>
    </controller>
<graphics type='vnc' port='5902'>
<listen type='address' address='0.0.0.0' />
</graphics>
  <interface type='bridge'>
  <source bridge='br0'/>
  <mac address='52:54:00:00:00:00'/>  
  <model type='virtio'/>
  </interface>
    <input type='mouse' bus='ps2'/>
    <memballoon model='virtio'/>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-device'/>
    [color=red]<qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=03:00.1,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=07:00.0,bus=root.1,addr=00.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=00:1d.0,bus=pcie.0'/>
  </qemu:commandline>[/color]
</domain>

EDIT: While reviewing my w81kvm1.xml file I noticed I posted the wrong one. Already modified!

Anyway that the last paragraph is wrong, but I have no idea how it should look like. I attach the devices I want to be passthroughed...maybe someone can help:

 

03:00.0 VGA compatible controller [0300]: AMD/ATI [Advanced Micro Devices, Inc.] R430 [Radeon X800] [1002:554f]
        Subsystem: Club-3D BV Device [196d:1089]
03:00.1 Display controller [0380]: AMD/ATI [Advanced Micro Devices, Inc.] R430 [Radeon X800] (Secondary) [1002:556f]
        Subsystem: Club-3D BV Device [196d:1088]

00:16.0 USB controller [0c03]: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
        Subsystem: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
        Kernel driver in use: ohci-pci

07:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:1142]
        Subsystem: ASUSTeK Computer Inc. Device [1043:85bf]
        Kernel driver in use: xhci_hcd
08:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:1142]
        Subsystem: ASUSTeK Computer Inc. Device [1043:85bf]
        Kernel driver in use: xhci_hcd

 

### Group 13 ###
    00:15.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
    00:15.1 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1)
    00:15.2 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB900 PCI to PCI bridge (PCIE port 2)
    00:15.3 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB900 PCI to PCI bridge (PCIE port 3)
    06:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
    07:00.0 USB controller: ASMedia Technology Inc. Device 1142
    08:00.0 USB controller: ASMedia Technology Inc. Device 1142
    09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller (rev 09)
### Group 14 ###
    00:16.0 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
    00:16.2 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB EHCI Controller
### Group 15 ###
    01:00.0 VGA compatible controller: NVIDIA Corporation G92 [GeForce 9800 GT] (rev a2)
### Group 16 ###
    03:00.0 VGA compatible controller: AMD/ATI [Advanced Micro Devices, Inc.] R430 [Radeon X800]
    03:00.1 Display controller: AMD/ATI [Advanced Micro Devices, Inc.] R430 [Radeon X800] (Secondary)

Link to comment
  • 2 weeks later...

I seem to be in the same boat as luca2. Would really like to get this working but I have a few questions.

 

1. With the newest beta has anyone been able to successfully pass through the iGPU in a usable manner (i.e. outputting to a display) or are we still stuck with having to use a discrete graphics card?

2. Still confused about editing quemu.conf. I know which groups the devices I want to passthrough belong to but not sure how to "add" them as they all appear to be contained in the file. See below. Should I remove any groups that I don't want pass through or is it simply a matter of removing the quotation marks?

 

cgroup_device_acl = [
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc","/dev/hpet", "/dev/vfio/vfio",
    "/dev/vfio/1","/dev/vfio/2","/dev/vfio/3",
    "/dev/vfio/4","/dev/vfio/5","/dev/vfio/6",
    "/dev/vfio/7","/dev/vfio/8","/dev/vfio/9",
    "/dev/vfio/10","/dev/vfio/11","/dev/vfio/12",
    "/dev/vfio/13","/dev/vfio/14","/dev/vfio/15",
    "/dev/vfio/16","/dev/vfio/17","/dev/vfio/18",
    "/dev/vfio/19","/dev/vfio/20","/dev/vfio/21",
    "/dev/vfio/22","/dev/vfio/23","/dev/vfio/24",
    "/dev/vfio/25","/dev/vfio/26","/dev/vfio/27",
    "/dev/vfio/28","/dev/vfio/29","/dev/vfio/30",
    "/dev/vfio/31","/dev/vfio/32"

 

Any help is, as always, appreciated!

Link to comment
  • 3 weeks later...

Any chance you could give us some pointers on on the go script thing.

 

Currently when restarting i have to re create vfio-bind, chmod the file etc and then edit and restart qemu.conf before i can start the machines. I'd love them to auto start with the host.

 

Other than recreating the files after a reboot I've found my KVM VM's with pass through to be rock solid. I've now got 2 VM's one running ARGUS TV and passing through a SAT tuner and the other running XBMC with a GPU and soundcard passed through. I'm using the XBMC VM as my main HTPC running the TV in the living room and its great. My old HTPC has been sat there switched off for the last 5 weeks or more.

Link to comment

Any chance you could give us some pointers on on the go script thing.

 

Currently when restarting i have to re create vfio-bind, chmod the file etc and then edit and restart qemu.conf before i can start the machines. I'd love them to auto start with the host.

 

So, turns out it lives in /usr/local/sbin so...

 

# Script to enable passthrough
/usr/local/sbin/vfio_bind <pci devices>

 

Where <pci devices> are your actual devices you want to pass.

 

Is there a way to confirm what vfio_bind has currently passed?  I would like to confirm 100% before rebooting.  I had changed so much I'd sure feel better being able to validate whats there now.

 

edited: To change the built in vfio_bind location

 

 

Link to comment

Vfio-bind should be built into /usr/local/sbin. You don't need to recreate the script file.

 

We included this in an earlier beta.

 

Right you are!

root@NAS1:/boot/config# ls -l /usr/local/sbin/vfio-bind

-rwxr-xr-x 1 root root 552 Sep 11 22:22 /usr/local/sbin/vfio-bind*

 

 

So I'll edit my above statement

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.