• [6.9.0-beta1] Internal Error/IOMMU group container failure for USB PCIe card passthrough / unRAID itself is unable to use the card


    fireflower
    • Closed

    The problematic card:
    Card: Allegro USB-C 4-port PCIe
    https://www.sonnettech.com/product/allegro-usbc-4port-pcie.html
    Use case for card: 4x 10Gbit USB-C connections for cold backup drives / USB nics / etc. 

    I have waited pretty patiently for unRAID to have 5.X kernel with drivers to use this card but then turns out unRAID has a problem with this card. Because unRAID refused to use the card itself, I tried to passthrough it to VM to test some of my USB-C cards but I get the error below. I tried first XML editing for passthrough and then blacklisting the device. Got the same error with both ways.

    internal error: qemu unexpectedly closed the monitor: 2020-03-20T17:17:14.681758Z qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio 0000:05:00.0: failed to setup container for group 15: Failed to set iommu for container: Operation not permitted
    internal error: qemu unexpectedly closed the monitor: 2020-03-20T17:31:44.966753Z qemu-system-x86_64: -device vfio-pci,host=0000:06:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio 0000:06:00.0: failed to setup container for group 16: Failed to set iommu for container: Operation not permitted
    


    I am happy to try any possible fixes.

    rs1-diagnostics-20200320-1933.zip




    User Feedback

    Recommended Comments

    Tried to disable IOMMU (Virtualization support) groups in bios like somebody suggested, well the card works now in unraid with keyboards at least.

    Cannot say it is a solution though because doing this practically makes unRAID then unable to run virtual machines or containers, which then again is like why even bother using unRAID?

    Wanted to check the card actually works properly in other distros, so took Linux Mint 19.3 and it was able to use the card with IOMMU groups enabled and create VM's and give access to one of the controllers on the card for that VM.


    The fix would be very much appreciated. 

    Link to comment

    Reporting:

    This card still has issues with unRAID 6.9 beta22 release.

    Somebody asked me to try ACS patch boot options and that way I got card's both hosts or single host passed to VM or let unRAID use other but the problem is the card does not work in unRAID or VM's guest OS (tried Fedora 31)

    Which is expected result because IOMMU groups are not anymore respected but the card still won't work under unRAID with IOMMU on. I think I will next try the card with newer hardware and see might this just be somekind mobo / bios incompatibility which I hear are a rare thing.

    Link to comment

    Okay, I found a workaround for me that lets me use VM's, dockers and I don't have to go into bios to disable IOMMU which disables virtualization (Thanks Dell!)

    Boot option: iommu=soft and you can use the USB device on Dell R410. Confirmed the card working newer hardware correctly with unRAID so it was a host hardware problem all along. 

    I don't need IOMMU isolation so desperately for PCIE passthrough on my machine but it would have been nice but this should work also because I can still run dockers & VM's.

    • Thanks 1
    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.