Hi

 

I have just built a new system to run UnRaid with the intention of running two separate VM's with their own graphics cards.

 

Sysetm Specs:

MSI X99A SLI Plus

i7-5820K CPU

32G DDR4 Memory

Asus HD5450 GFX

GTX 970 GFX

GTX 970 GFX

2 xSSD 500G

2 xHDD 2TB

 

My motherboard does not have onboard graphics so I have purchased a cheap one and placed in PCI slot 1. I then placed two Nvidia GTX 970's into PCI slots 2 & 3

 

My first VM works fine using the the 970 in slot 2 but when I try the a second VM and use the 970 in slot three I receive the following error

 

I have tried moving all the graphics cards around the pci slots, but even if a place the cheap card in the third slot I receive the same error. If i place a graphics card in slot 1 and slot 3 but leave slot 2 empty the VM works.

 

It would seem that the problem is with running 3 cards.

2016-02-06T19:19:20.475517Z qemu-system-x86_64: -device vfio-pci,host=06:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error opening /dev/vfio/33: Operation not permitted

 

Any help would be appreciated

 

Beskee

 

I would think its your IOMMU Groups. When you have the 3 cards plugged in is anything else grouped in the same IOMMU group? Please post your IOMMU groups and your pci devices

Have you tried the PCIe ACS Override setting?

You have to apply the hotfix found here

The hot fix did the trick and I'm now able to launch two VMs using separate GFX cards.

 

Thanks saarg.

 

Beskee

Good you got it working.