Simpler / Easier PCI Device Pass Through for NON-GPUs


jonp

Recommended Posts

I have problem with my ASUS Essence STX II Soundcard. I cant passthrough it to VM.

I tested it with this script but it doesnt work for me :/

 

Can you guys help me ?

IOMMU group 31
	[1b21:1080] 04:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 04)
	[13f6:8788] 05:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 [Oxygen HD Audio]

i tried this in my XML but i think this is wrong :D

 <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
    </hostdev>

 <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x04' function='0x0'/>
      </source>
    </hostdev>

 

Link to comment
  • 2 months later...

Hi i wonder could you help me, 

I'm attempting to pass through a DVBSky T9580 PCIe card with no joy.

So when setting up VM, you can see PCI is not available.

1.JPG.23498402d98458bf865710e25466d87f.JPG

I do as suggested, find the bus vendor id etc, then modify the XML

2.JPG.953095f3fe13e5488a8ca145e72da09f.JPG

So original xml will be changed from this;

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
    </hostdev>

and will be edited to;
 

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
      </source>
    </hostdev>

----------------------------------------------------------

Modify my XML now, 
 

Quote

<domain type='kvm'>
  <name>Windows 10</name>
  <uuid>df1513bd-105f-6d26-6fcf-47a832f1054b</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>7340032</memory>
  <currentMemory unit='KiB'>7340032</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='5'/>
    <vcpupin vcpu='2' cpuset='6'/>
    <vcpupin vcpu='3' cpuset='7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/df1513bd-105f-6d26-6fcf-47a832f1054b_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Windows.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.126-2.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:f0:17:1c'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
      </source>
    </hostdev>

  </devices>
</domain>



And when i go back to edit, you can see the pci card available. 

3.JPG.bd4f981e1b5d2666d1f12cecde0925b5.JPG

Now when i select Update then run VM, i get this prompt


5.JPG.214813404dcbebb0c0039973ad040a21.JPG

 

 

And if i got back to the edit XML, the code has changed;

 

Quote

<domain type='kvm'>
  <name>Windows 10</name>
  <uuid>df1513bd-105f-6d26-6fcf-47a832f1054b</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>7340032</memory>
  <currentMemory unit='KiB'>7340032</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='5'/>
    <vcpupin vcpu='2' cpuset='6'/>
    <vcpupin vcpu='3' cpuset='7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/df1513bd-105f-6d26-6fcf-47a832f1054b_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Windows.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.126-2.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:f0:17:1c'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

 

Edited by foster182
Link to comment
1 hour ago, saarg said:

You should stub the device in the syslinux.cfg. Then it shows up in the other devices list and will not dissappear when you reboot your server. 

 

Is the card in its own iommu group? 

Your dealing with a novice mate, can you explain a little further. 
Or explain how to set this up as i'm sure I've overlooked this if it doesn't sound familiar.

Thank you

 

Link to comment
12 hours ago, foster182 said:

Your dealing with a novice mate, can you explain a little further. 
Or explain how to set this up as i'm sure I've overlooked this if it doesn't sound familiar.

Thank you

 

 

Away from home this week so limited time to a computer, so you'll have to figure this one out by yourself. There should be numerous threads about how to stub the device if you search the forum. It might even be in the wiki and I'm sure @gridrunner. Have made a video about it also. 

It's also in one of the unraid versions announcement threads. Should be 6.2 if I remember correctly. 

Link to comment
19 hours ago, foster182 said:

Your dealing with a novice mate, can you explain a little further. 
Or explain how to set this up as i'm sure I've overlooked this if it doesn't sound familiar.

Thank you

 

Hi, I have had a read through your post.

As  @saarg says it is best to stub the device in the syslinux config. You will be able to edit the syslinux file from the main tab. Click main then flash. About half way down you will see

the following

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append  initrd=/bzroot  
label unRAID OS GUI Mode
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui
label unRAID OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label unRAID OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

you need to add your device to be stubbed/isolated your device id is 1ade:3038

So we add this to the syslinux config

vfio-pci.ids=1ade:3038

so syslinux should look like this

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append  vfio-pci.ids=1ade:3038 initrd=/bzroot  
label unRAID OS GUI Mode
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui
label unRAID OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label unRAID OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

If you have enabled acs override then the above syslinux will already have extra into it so you would add it after

append pcie_acs_override=downstream  vfio-pci.ids=1ade:3038 initrd=/bzroot 

 

After stubbing a reboot is required

 

But

 

I think your problem is due to the iommu groups. Please check the iommu group (group one your card is in) has only itself in that group.

Please post the iommu groups here

If other devices are present you will need to try and split them up. And if you haven't already you can try enabling the acs override I mentioned above.

For this, go to settings, vm manager,  (toggle Advanced view) then for,  Enable PCIe ACS Override: select  Yes

Then reboot the server and see if the device is now in its own iommu group.

Best of luck and please report back :)

 

 

Link to comment
  • 1 month later...

I've scoured the internets and this thread has come the closest to what I'm seeing. However, I am still unable to pass any NIC through to a Win10 VM. I get the same error as noted above (couple times), but here is my specific error:

 

internal error: qemu unexpectedly closed the monitor: 2017-09-01T21:46:07.720078Z qemu-system-x86_64: -device vfio-pci,host=09:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to set iommu for container: Operation not permitted
2017-09-01T21:46:07.720112Z qemu-system-x86_64: -device vfio-pci,host=09:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to setup container for group 21
2017-09-01T21:46:07.720118Z qemu-system-x86_64: -device vfio-pci,host=09:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to get group 21
2017-09-01T21:46:07.720134Z qemu-system-x86_64: -device vfio-pci,host=09:00.0,id=hostdev0,bus=pci.0,addr=0x6: Device initialization failed

I'm running dual Xeon 5650s on a Dell P19C9 (C2100 Server) with VT-d (and VT-x) enabled. IOMMU shows 'Enabled' in the 'Info' output of Unraid (running version 6.3.5). I have enabled the PCIe ACS Override to split my IOMMU groups but I continue to get the error above. I'm currently trying to pass through the onboard dual NICs, but I do have a 4-port NIC that fails as well. Attached is my full output of 'lspci' but the relevant section is:

 

//4-Port NIC
06:00.0 0200: 8086:10e8 (rev 01)
06:00.1 0200: 8086:10e8 (rev 01)
07:00.0 0200: 8086:10e8 (rev 01)
07:00.1 0200: 8086:10e8 (rev 01)

08:00.0 0107: 1000:0072 (rev 03)

//On-Board NIC
09:00.0 0200: 8086:10c9 (rev 01)
09:00.1 0200: 8086:10c9 (rev 01)

I have tried the vfio-pci.ids in my syslinux.cfg file, just trying to further isolate things (not sure if that was necessary, but I've tried to apply each recommendation from the various threads). This continues to result in same error.

 

Full XML is attached, but relevant snippet from VM is:

 

 <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>

I'm hoping this is something dumb I'm overlooking so I'm reaching out for additional feedback. Thanks in advance!

VM_XML.txt

lspci.txt

Link to comment

Thanks 1812. Group 21 only has the one device. However, I followed your link to 'HP Proliant & unRaid Information Thread' and started going through your steps just to see if anything would help. I am embarrassed to admit that I didn't go to the system log during all this testing, but right at the bottom after trying to dupe the problem I found this little gem

 

No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

So I've added the following snippet

vfio_iommu_type1.allow_unsafe_interrupts=1

to my syslinux.cfg file and am rebooting.

 

I'll post my logs if this still doesn't work.

Link to comment
9 minutes ago, dharrah said:

Thanks 1812. Group 21 only has the one device. However, I followed your link to 'HP Proliant & unRaid Information Thread' and started going through your steps just to see if anything would help. I am embarrassed to admit that I didn't go to the system log during all this testing, but right at the bottom after trying to dupe the problem I found this little gem

 


No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

So I've added the following snippet


vfio_iommu_type1.allow_unsafe_interrupts=1

to my syslinux.cfg file and am rebooting.

 

I'll post my logs if this still doesn't work.

 

 

That should fix thta problem. Hopefully there won't be more!

Link to comment
1 minute ago, dharrah said:

 

That was it! Thanks again. It was the kick in the pants I needed

 

I saw it was on a Dell, and while I have unRaid on a couple older Dells, I didn't do any passthrough so I wasn't aware that it appears some do not support interrupt remapping. I'll have to file that one away in the brain somewhere for future use.

 

Have fun playing around!

Link to comment
  • 2 months later...

Had to rebuild a Win10 VM and trying to figure out why my USB 3.0 controller card (click here for specs) is not working.  

 

Here is the USB Card that VM doesn't see, as well as the video card that it DOES see:

USB3.0.PNG.074bfcad8d0c73c296c27716f2d88f6e.PNG

 

Here is a Hauppauge Colossus that is working fine:

Colossus.PNG.c6ffa23c32d04220cf148634cd2991fc.PNG

 

Here is the XML where I have attempted to make the changes recommended in this thread:

syslinux.PNG.f743032c71f2f0d85ad18494cceefe3f.PNG

 

What am I not seeing?

unraid-diagnostics-20171113-1125.zip

Link to comment
  • 2 weeks later...

Hi, may anyone has an idea about cannot reset device ...

 

example from VM log

 

2017-11-26T08:10:14.310180Z qemu-system-x86_64: vfio: Cannot reset device 0000:00:14.0, no available reset mechanism.

 

sometimes when my VM crashes the USB controller is dead and i have to reboot server to get it working again (like its locked or similar) ...

i see that from RDP to VM (after hard reboot VM) in device manager that the USB controller is triangle marked, a server reboot fixes this issue ...

i already tested forward / backward with vfio-pci.ids=8086:a2af (wich is 00:14:0) in syslinux to see if that makes any difference, same result ...

 

its one of the onboard USB devices, it works as it should usually, just sometimes when VM crashes its like described above.

from my xml

 

    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x14' function='0x0'/>
      </source>

 

for any tip, thanks ahead ;)

Edited by alturismo
Link to comment
2 hours ago, alturismo said:

Hi, may anyone has an idea about cannot reset device ...

 

example from VM log

 

2017-11-26T08:10:14.310180Z qemu-system-x86_64: vfio: Cannot reset device 0000:00:14.0, no available reset mechanism.

 

sometimes when my VM crashes the USB controller is dead and i have to reboot server to get it working again (like its locked or similar) ...

i see that from RDP to VM (after hard reboot VM) in device manager that the USB controller is triangle marked, a server reboot fixes this issue ...

i already tested forward / backward with vfio-pci.ids=8086:a2af (wich is 00:14:0) in syslinux to see if that makes any difference, same result ...

 

its one of the onboard USB devices, it works as it should usually, just sometimes when VM crashes its like described above.

from my xml

 

    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x14' function='0x0'/>
      </source>

 

for any tip, thanks ahead ;)

 

I don't think there are much you can do about it. There are some workarounds for AMD cards having problems reseting, but that is in the vfio module. 

So you could ask in the vfio mailing list if it's possible or if it's impossible to fix. 

 

Link to comment
  • 10 months later...
  • 7 months later...
On 2/19/2015 at 12:15 PM, jonp said:

If you have a non-GPU device that you wish to assign to a guest in KVM and are using the machine type of PC (not Q35), you can use this XML and skip the <qemu:cmd> arguments at the end.  This goes in between the <devices> and </devices> tags in your domain XML.

 

 


    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
    </hostdev>
 

 

 

For "bus", "slot", and "function", this should be your PCI device address.  The above XML will pass through PCI device 02:00.1.

 

Again, this only applies to non-graphics PCI devices (NICs, USB controllers, capture cards, tuners, etc.)

 

Happy virtualizing!!

 

 

Thank you. I managed to run my machine with a 1660ti which came with some sort of additional controllers like usb controller and serial bus controller, now I passed the usb bus using your method:

 

 

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x27' slot='0x00' function='0x2'/>
      </source>

 

 

Now I would like to know how do you pass two devices (I already got 27:00.2, also need 27:00.3)

 

 

IOMMU group 15:	[10de:2182] 27:00.0 VGA compatible controller: NVIDIA Corporation Device 2182 (rev a1)
		[10de:1aeb] 27:00.1 Audio device: NVIDIA Corporation Device 1aeb (rev a1)
		[10de:1aec] 27:00.2 USB controller: NVIDIA Corporation Device 1aec (rev a1)
		[10de:1aed] 27:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1aed (rev a1)

 

Thanks in advance.

Link to comment
  • 8 months later...

Hello all!!

 

So, using the below,

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
    </hostdev>
	<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
    </hostdev>

 

and stubbing out the two ports i wish to use, I can see both ports that i wish to use while creating a VM. However, pfSense is only seeing one port. I am confused and have searched quite a bit, Sorry if this has been covered somewhere else. my GoogleFu has failed me today.

 

Please let know know another information that is needed.

 

Thanks!

 

 

Screenshot (2).png

Screenshot (3).png

Link to comment
  • 2 months later...
On 5/8/2020 at 11:29 PM, saarg said:

There is already support for it. All you have to do is stub them.

When I am in the GUI for VM creation I can't select any of my PCIe devices (USB show up fine). I have to manually "virsh edit <vm_name>" and manually add the device/function assignments. Not the worst thing in the world. Am I doing something wrong? I am also on the free trial at the moment, kicking the tires.

Link to comment
4 hours ago, idlacrosseplayer said:

When I am in the GUI for VM creation I can't select any of my PCIe devices (USB show up fine). I have to manually "virsh edit <vm_name>" and manually add the device/function assignments. Not the worst thing in the world. Am I doing something wrong? I am also on the free trial at the moment, kicking the tires.

You need to stub them before they appear in the list.

Install the VFIO PCI Config plugin and stub the devices there. You can also append the ID to the syslinux.conf file, but the plugin is probably easier.

Link to comment
On 5/14/2020 at 1:43 AM, saarg said:

You need to stub them before they appear in the list.

Install the VFIO PCI Config plugin and stub the devices there. You can also append the ID to the syslinux.conf file, but the plugin is probably easier.

Cool! That subbed it correctly.

 

Is there a place I could request a "multifunction" checkbox be added when selecting multiple functions of the same device? Within my VM, the drivers are expecting two functions on one device (I don't ask why), and by default without the "multifunction" flag, they will show up as two unique devices within the Guest....great for SRIOV where the functions are usually two "devices", but in my case they're not :(

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.