Hello,

I am new to KVM and not sure if I am doing anything wrong, but I have two 7870s and I would like to give one to each of my VMs.  I am just using the GUI in unraid to do this and if I have one of the VMs shutdown the other works great.  When I try to turn on the other VM with the other one already running I get a device busy error.

 

I looked at the XML and tested both VMs are using different cards so I am not sure where the problem is or if this is a limitation of KVM.  Belowe is what unraid put in for the GPU passthrough.

 

  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=04:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=04:00.1,bus=root.1,addr=00.1'/>
  </qemu:commandline>

  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=03:00.1,bus=root.1,addr=00.1'/>
  </qemu:commandline>

 

Thank you for any help

That looks right to me.  I am currently passing through 3 HD6450's and all are running at the same time:

 

  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=84:00.0,bus=pcie.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=84:00.1,bus=pcie.0'/>
  </qemu:commandline>

 

  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=83:00.0,bus=pcie.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=83:00.1,bus=pcie.0'/>
  </qemu:commandline>

 

  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=86:00.0,bus=pcie.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=86:00.1,bus=pcie.0'/>
  </qemu:commandline>

 

Is it possible that they are not getting enough power?

 

John

Don't think so, its a 750W with the video cards and 4x500GB and 2xSSDs

Here is the error I am getting.

but I have to 7870s and I would like to give one to two of my VMs.

 

OK wait...I am re-reading your OP.

 

You want to share one of the cards between 2 different VMs and both VMs powered on at the same time?  If so, I do not think it is possible to share a single vid card between two active VMs.  But, wait for jonp to chime in to confirm.

 

John

Sorry keyboard error.

 

I have two 7870s and I have each one setup to two different VMs.

Sorry for the multiple posts, as I said I am still learning KVM and where everything is.  Here is what the log file says on the second VM when I try to start it.

 

2015-07-16 14:43:40.206+0000: starting up libvirt version: 1.2.15, qemu version: 2.3.0
Domain id=40 is tainted: high-privileges
Domain id=40 is tainted: custom-argv
Domain id=40 is tainted: host-cpu
char device redirected to /dev/pts/1 (label charserial0)
2015-07-16T14:43:41.403077Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=pcie.0,multifunction=on,x-vga=on: vfio: error opening /dev/vfio/1: Device or resource busy
2015-07-16T14:43:41.403102Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=pcie.0,multifunction=on,x-vga=on: vfio: failed to get group 1
2015-07-16T14:43:41.403126Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=pcie.0,multifunction=on,x-vga=on: Device initialization failed
2015-07-16T14:43:41.403141Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=pcie.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized
2015-07-16 14:43:41.845+0000: shutting down

 

in the unRAID webui, go to Tools --> System Devices

 

Copy and paste the contents of IOMMU Groups here.

 

John

Here is what you requested.

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/1/devices/0000:01:00.0
/sys/kernel/iommu_groups/1/devices/0000:02:08.0
/sys/kernel/iommu_groups/1/devices/0000:02:10.0
/sys/kernel/iommu_groups/1/devices/0000:03:00.0
/sys/kernel/iommu_groups/1/devices/0000:03:00.1
/sys/kernel/iommu_groups/1/devices/0000:04:00.0
/sys/kernel/iommu_groups/1/devices/0000:04:00.1
/sys/kernel/iommu_groups/2/devices/0000:00:02.0
/sys/kernel/iommu_groups/3/devices/0000:00:14.0
/sys/kernel/iommu_groups/4/devices/0000:00:16.0
/sys/kernel/iommu_groups/5/devices/0000:00:1a.0
/sys/kernel/iommu_groups/6/devices/0000:00:1b.0
/sys/kernel/iommu_groups/7/devices/0000:00:1c.0
/sys/kernel/iommu_groups/8/devices/0000:00:1c.2
/sys/kernel/iommu_groups/9/devices/0000:00:1c.4
/sys/kernel/iommu_groups/10/devices/0000:00:1c.5
/sys/kernel/iommu_groups/11/devices/0000:00:1c.6
/sys/kernel/iommu_groups/12/devices/0000:00:1c.7
/sys/kernel/iommu_groups/13/devices/0000:00:1d.0
/sys/kernel/iommu_groups/14/devices/0000:00:1f.0
/sys/kernel/iommu_groups/14/devices/0000:00:1f.2
/sys/kernel/iommu_groups/14/devices/0000:00:1f.3
/sys/kernel/iommu_groups/15/devices/0000:06:00.0
/sys/kernel/iommu_groups/16/devices/0000:07:00.0
/sys/kernel/iommu_groups/17/devices/0000:08:00.0
/sys/kernel/iommu_groups/18/devices/0000:09:00.0
/sys/kernel/iommu_groups/19/devices/0000:0a:00.0

 

When you asked about the IOMMU I remembered reading an older post about devices on the same IOMMU cant be passed through to defferent VM.  Could this still be the case? If so is my only solution is to try changing slots and get the cards on different IOMMUs?