Windows 10 VM creation fail - Nvidia GT 710 - "vfio: failed to set iommu for container: Operation not permitted"


Recommended Posts

Hi, hope someone can help.

 

On Windows 10 VM creation, I see the following error:

 

internal error: process exited while connecting to monitor: 2017-05-06T18:47:37.955194Z qemu-system-x86_64: -device vfio-pci,host=0b:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to set iommu for container: Operation not permitted
2017-05-06T18:47:37.955252Z qemu-system-x86_64: -device vfio-pci,host=0b:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to setup container for group 16
2017-05-06T18:47:37.955262Z qemu-system-x86_64: -device vfio-pci,host=0b:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to get group 16
2017-05-06T18:47:37.955289Z qemu-system-x86_64: -device vfio-pci,host=0b:00.0,id=hostdev0,bus=pci.0,addr=0x6: Device initialization failed

 

The following is set:
BIOS: OVFM
Hyper-V: No (as it's Nvidia, though I've not yet seen why it's recommended to be off)

 

The groups for the two cards are:
IOMMU group 16
    [10de:128b] 0b:00.0 VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 710B] (rev a1)
    [10de:0e0f] 0b:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1)

 

IOMMU group 19
    [10de:128b] 05:00.0 VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 710B] (rev a1)
    [10de:0e0f] 05:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1)

 

Selecting both the video and audio for VM creation on either the card in group 16 or 19 generates the above error. There is nothing else in these groups, and the cards are not being used by another other VM or Docker. Each card is intended to be used in separate VM's.

 

I've tried the Enable PCIe ACS Override, which didn't make any difference to the error, as I expected since the groups seem fine.

 

I've also tried the following in syslinux.cfg:
label unRAID OS
  menu default
  kernel /bzimage
  append vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot

 

The cards are both NVIDIA GT 710, one Asus and one MSI, which I'm sure that I read someplace that 700 series upwards should be fine with UNRAID.

 

Does anyone else have any ideas?

Link to comment
  • 2 weeks later...

The cards are fine, i have several 710's.

 

you need to put this back in your syslinux.cfg, as it does not show up in your system log:

 

append vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot

 

then reboot, and try again. You have a proliant, this should resolve the issue.  If you get another error, post it and diagnostics again. Don't reboot and post "fresh" diagnostics as it is missing information that may be helpful.

Link to comment

Thanks for the quick response.

 

I tried your suggestion previously (see original post), it's missing from the last diagnostics since it was a re-install. I've re-applied the configuration and am still getting the same error, but for group 19 as you'd expect as that's where the 710 is.

 

Still no luck though. Please see the attached new diagnostics.

 

Thanks. 

grid-diagnostics-20170516-0647.zip

Link to comment
2 hours ago, dorian said:

Thanks for the quick response.

 

I tried your suggestion previously (see original post), it's missing from the last diagnostics since it was a re-install. I've re-applied the configuration and am still getting the same error, but for group 19 as you'd expect as that's where the 710 is.

 

Still no luck though. Please see the attached new diagnostics.

 

Thanks. 

grid-diagnostics-20170516-0647.zip

 

 

Quick question while I look through the logs: did you specify onboard video as primary on your server bios? doesn't matter, read next post.

Edited by 1812
Link to comment

This is your problem:

 

May 16 06:46:30 Tower kernel: vfio-pci 0000:05:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement.  Contact your platform vendor.

 

This is the reason/workaround:

 

 

You may be able to blacklist the card and get it to use audio and video, but I haven't fooled with that yet.

 

Edited by 1812
  • Upvote 1
Link to comment

I've never had to set a video card for the primary BIOS, so haven't looked as explicitly stating that it be used.

 

At this moment there are two video outputs:

1) Onboard ATI VGA, which is connected to a screen to I can see the lower login, and the server posting.

2) NVIDIA 710, which is HDMI. 

 

Given that fact that the VGA is outputting the post and tower login, I'd assumed that the onboard ATI would not be an issue.

Link to comment
2 minutes ago, dorian said:

Thanks......

 

I'll play around with that tonight and see what can be done. It just seems that UNRAID and HP servers don't like to play nice together. ¬¬ 

 

Guessing it'll be Dell next time.

 

There are certainly quirks to using software that wasn't intended to be used on enterprise servers actually on enterprise servers. 

  • Upvote 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.