unRAID 6.0 Beta 7 - KVM-Specific Updates

[jonp]

Thought it would be helpful if I came in here to provide some insights on what exactly went into the beta 7 release and what specifically went into this with respect to KVM-based virtualization:

 

1:  QEMU 2.1 Upgrade

A full copy of the 2.1 changelog can be found here, but some of the highlights that we're hoping to test out (and hoping you guys will as well):

• Full support for USB3 passthrough (including streams).
  
• VFIO support for RTL8168 NICs.

There are changes that are somewhat vaguely described, but could result in better support and stability for key features.  All we can do is test and see.

 

2:  QEMU Patch for nVIDIA GPUs / VFIO

For those that don't know, nVIDIA updated their drivers recently to do something somewhat "odd."  If you run a Windows VM running under KVM and have an nVIDIA GPU installed for pass through, nVIDIA detects the fact that it is running it's GPU on a VM and disables the GPU in Windows under device manager with a "Code 43" error.  Alex Williamson of Red Hat has provided a custom patch for this to hide the KVM flag from nVIDIA's detection to prevent the disabling of the card.  I've been testing using this patch on my GTX 780 for a few months now as we've been going through the paces of testing and through multiple driver updates, the patch has continued to prove rock-solid.

 

There are numerous theories on why nVIDIA choose to implement this "detect and disable" method for Windows VMs and there is always the possibility that they find another way to detect and disable again in the future, but for now, it seems to be working just fine.  Bottom line:  if you have an nVIDIA GPU that you wish to pass through, this patch will help you.

 

3:  ACS Override Patch for PCI Pass Through

When passing through a PCI device to a VM using VFIO, the very nature of IOMMU tends to group multiple PCI devices together (even different ones).  A perfect example we incurred during testing on this was with a GPU we wanted to pass through.  Even though we just wanted to pass through the GPU, the IOMMU grouping of the pass through devices was trying to force us to pass through both the GPU and a SATA controller we had on the system (for no rhyme or reason).  Another wonderful patch from Mr. Alex Williamson that we've incorporated will eliminate this and allow users to pass through individual VFIO devices to individual virtual machines without being grouped to other devices.

 

With these three major changes to our build, KVM becomes far more useful.  Stay tuned as more content is shared on how to make use of these capabilities to deliver powerful new features in unRAID...

[wewantrice]

Still getting group passthrough errors when trying to pass through USB ports. Is the new beta supposed to be get rid of the requirement to pass through all devices in a group?

 

error: Failed to create domain from Win8.1.xml
error: internal error: early end of file from monitor: possible problem:
2014-08-30T20:19:52.049460Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,bus=root.1,addr=00.0: vfio: error, group 9 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
2014-08-30T20:19:52.049478Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,bus=root.1,addr=00.0: vfio: failed to get group 9
2014-08-30T20:19:52.049485Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,bus=root.1,addr=00.0: Device initialization failed.
2014-08-30T20:19:52.049491Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,bus=root.1,addr=00.0: Device 'vfio-pci' could not be initialized

 

Here is the group the USB controller belongs to

 

### Group 9 ###
    00:1c.0 PCI bridge: Intel Corporation Device 8c90 (rev d0)
    00:1c.2 PCI bridge: Intel Corporation Device 8c94 (rev d0)
    00:1c.3 PCI bridge: Intel Corporation Device 8c96 (rev d0)
    00:1c.6 PCI bridge: Intel Corporation Device 8c9c (rev d0)
    03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller (rev 11)
    04:00.0 PCI bridge: ASMedia Technology Inc. Device 1184
    05:01.0 PCI bridge: ASMedia Technology Inc. Device 1184
    05:03.0 PCI bridge: ASMedia Technology Inc. Device 1184
    05:05.0 PCI bridge: ASMedia Technology Inc. Device 1184
    05:07.0 PCI bridge: ASMedia Technology Inc. Device 1184
    07:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
    09:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
    0a:00.0 USB controller: ASMedia Technology Inc. Device 1142

 

This is an onboard USB controller if it makes any difference.

 

[bungee91]

Did you happen to add any settings to your syslinux.cfg file?

 

For the Unraid one without Xen (so with KVM) I added this to mine, grouping is exactly what it should be now afterwards.

 

 

This is the last line of that entry, the most important for this is the "pcie_acs_override=downstream"

 

append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot

 

[wewantrice]

Awesome thanks so much, worked like a charm! Looks like all of the onboard ports on my mobo are passed through at once so I may have to invest in a USB PCI card.