I have a Asus x99 Deluxe ii motherboard and I am on current Unraid 7.2.3

I have Intel VM-X and VT-D enabled. I have fast boot disabled. I have secure boot disabled. I have CSM disabled. I also enabled 4G decoding since Google said that may help. I have Unraid booting up in UEIF with no legacy type settings active. In Unraid in the "Info" card I have HVM enabled but IOMMU disabled. .

I have read that with some systems its necessary to manually enable IOMMU by adding a line iommu=on and iommu=pt to the kernal, however I do not want to do that without someone here that knows what they are doing walking me through that. Also, before doing that I feel like I need to verify if Unraid can see that VT-d is enabled on the bios side, it appears it is but that Unraid just cant see that. Hoping someone can walk me through some of this.

Thanks!

Stephen

Edited January 7Jan 7 by SPOautos

whats does info say:

please post a diagnostic file...

intel or amd?

some amd requrie going deper into bios to enable iommu not set to auto and enable memory remaping.

may require a mother board bios firmware update as well...

Edited January 8Jan 8 by bmartino1

Here is the "Info" and the diagnostic file. The motherboard is a Asus x99 Deluxe II that had the latest bios update done yesterday and the cpu is a Intel Xeon E5-2690 v3 and both are old but still run great and are listed as supporting IOMMU and grouping.

tower-diagnostics-20260107-1958.zip

Edited January 8Jan 8 by SPOautos

so yes there is adational bios otions that need enabled and set your syslog is showing

Jan 7 13:02:29 Tower kernel: iommu: Default domain type: Passthrough
and sone't have any thing with DMAR nor remaping which is required for iommu support at the linmux qemu software layer and vfio for sriov devices.

Verifying IOMMU parameters

Verify IOMMU is enabled

Reboot, then run:

dmesg | grep -e DMAR -e IOMMU

There should be a line that looks like "DMAR: IOMMU enabled". If there is no output, something is wrong.

Verify IOMMU interrupt remapping is enabled

It is not possible to use PCI passthrough without interrupt remapping. Device assignment will fail with 'Failed to assign device "[device name]": Operation not permitted' or 'Interrupt Remapping hardware not found, passing devices to unprivileged domains is insecure.'.

All systems using an Intel processor and chipset that have support for Intel Virtualization Technology for Directed I/O (VT-d), but do not have support for interrupt remapping will see such an error. Interrupt remapping support is provided in newer processors and chipsets (both AMD and Intel).

To identify if your system has support for interrupt remapping:

dmesg | grep 'remapping'

If you see one of the following lines:

• AMD-Vi: Interrupt remapping enabled

• DMAR-IR: Enabled IRQ remapping in x2apic mode ('x2apic' can be different on old CPUs, but should still work)

then remapping is supported.

If your system doesn't support interrupt remapping, you can allow unsafe interrupts with:

but atm this looks to be bios level settings...

13 hours ago, bmartino1 said:

so yes there is adational bios otions that need enabled and set your syslog is showing

Jan 7 13:02:29 Tower kernel: iommu: Default domain type: Passthrough
and sone't have any thing with DMAR nor remaping which is required for iommu support at the linmux qemu software layer and vfio for sriov devices.

Verifying IOMMU parameters

Verify IOMMU is enabled

Reboot, then run:

dmesg | grep -e DMAR -e IOMMU

There should be a line that looks like "DMAR: IOMMU enabled". If there is no output, something is wrong.

Verify IOMMU interrupt remapping is enabled

It is not possible to use PCI passthrough without interrupt remapping. Device assignment will fail with 'Failed to assign device "[device name]": Operation not permitted' or 'Interrupt Remapping hardware not found, passing devices to unprivileged domains is insecure.'.

All systems using an Intel processor and chipset that have support for Intel Virtualization Technology for Directed I/O (VT-d), but do not have support for interrupt remapping will see such an error. Interrupt remapping support is provided in newer processors and chipsets (both AMD and Intel).

To identify if your system has support for interrupt remapping:

dmesg | grep 'remapping'

If you see one of the following lines:

• AMD-Vi: Interrupt remapping enabled

• DMAR-IR: Enabled IRQ remapping in x2apic mode ('x2apic' can be different on old CPUs, but should still work)

then remapping is supported.

If your system doesn't support interrupt remapping, you can allow unsafe interrupts with:

but atm this looks to be bios level settings...

okay, I rebooted the server and went into BIOS to verify that the VT-d settings were all enabled, along with ACS. Then logged back into Unraid and ran the two lines you gave me....the first one turned up nothing....here are the results of both.

n

Now I spoke with someone who used to run one of these Asus x99 boards with passthrough on a Linux server...NOT Unraid.....so not sure if it will work or not. But here is what he told me is required for this older setup with these Asus x99 boards and old Xeon E5 processors. (I also found this same info in some real old posts online). I don't know enough about this to know if this is something I can try in Unraid or if its only helpful in Linux....

BUT, this is what he did get IOMMU grouping and maybe you can share your thoughts about it......

"I used kernel command line arguments...... intel_iommu=on iommu=pt which enable VT-d (I/O virtualization) code in the Linux kernel for Intel processers. This would be necessary for e.g. PCIe passthrough. I don't know enough about Unraid to directly say if this will work or where exactly to add it. Generally speaking, the bootloader executes the kernel and this is where command line arguments should be specified. On Debian/Ubuntu systems that use Grub for example this is typically /etc/default/grub. You add the arguments to the value of GRUB_CMDLINE_LINUX_DEFAULT in that file, run update-grub (which writes a new /boot/grub/grub.cfg), then reboot.

So you'll need to research the Unraid way to do all that. And be warned: you can easily make your system un-bootable with a bad kernel command line argument or syntax error, but it's usually easy to fix by editing the command line at boot or by going into recovery mode. Good luck!"

*** END OF WHAT HE TOLD ME***

Do you have any thoughts on that or do you think its worth a shot to try it? Since the boot files are in the USB, if it doesn't work I think I could just open the usb on my laptop and edit it back and put it back in the server like nothing ever happened. I don't think there's any real risk to try is there? If there's no real risk in trying, could you tell me how to go about putting that in and seeing what happens? Like I said, In Linux he gets full grouping using this, he said with this older versions of VT-d IOMMU wont show up in the OS unless these kernel commands are added.

Again you need to edit bios to enable iommu and memory remapping... U road already passed that at the kernel level...

In Debian and other distro you have to edit grub/cmdkernwl commands to pass the options...

Example proxmox:

https://pve.proxmox.com/wiki/PCI_Passthrough

But if bios settings is not set right it will now work send picture of you in BIOS I can help guid then but until you fix your BIOS settings even Debian/proxmox won't work...

7 hours ago, bmartino1 said:

Again you need to edit bios to enable iommu and memory remapping... U road already passed that at the kernel level...

In Debian and other distro you have to edit grub/cmdkernwl commands to pass the options...

Example proxmox:

https://pve.proxmox.com/wiki/PCI_Passthrough

But if bios settings is not set right it will now work send picture of you in BIOS I can help guid then but until you fix your BIOS settings even Debian/proxmox won't work...

Here are some images of the bios settings. Now theres a LOT of settings in the bios, I just took picture of things I've changed. I have left most of the bios on factory default except for what I've read may be helpful for IOMMU and for unraid bootup (Intel Virtual Tech called it VT-X in the manual, VT-d/ACS, and 4G decoding). I have taken pictures of what I have changed. I have "Above 4G decoding" enabled because I read that sometimes its beneficial for passthrough but I've tried it on and off and it didn't make a difference. You will notice in one picture I disabled Secure Boot that is because it had some kind of key issue so I just cleared the key and disabled it. One the picture that has VT-d sub settings, I have ACS enabled, I have MCTP disabled because I read it's not applicable to IOMMU, that said I have tried then both on and off. I've tried many combinations with some on and some off, rebooted probably 20 times trying different things...... but this is how is currently sets and this is how is was when I tested the terminal lines you gave me to try out.

This is all I could find online that is relative but maybe there's something else that you can direct me too.

Edited January 9Jan 9 by SPOautos

Is the vt-d in image #2 enabled?

may have adataional options to set

52 minutes ago, bmartino1 said:

may have adataional options to set

56 minutes ago, Veah said:

Is the vt-d in image #2 enabled?

@bmartino1 @Veah

What I've read online is that in these early versions of the Intel Virtualization Tech that the VT-d comes on automatically with the "Intel Virtualization Technology" setting, you cant turn it on or off separately, it just activated with the other setting. I searched through every menu to make sure, but it makes sense since it actually has settings related to VT-d but no enable/disable box. I read that the enable/disable of it by itself came in the next version of the virtualization technology, at this earlier stage it apparently was automatic. I've found some real old posts on Reddit where they said it didn't have a enable/disable but that they were getting good Iommu grouping so it was working. Also in case you're wondering....the cpu Xeon E-5 2690 v3 also says it also supports VT-d. If there might be some other setting somewhere that applies I'm not sure what it is, but I'm glad to look if you guys have an idea.

I'd love to get this figured out, I'd really like it to work and there's people using this board and getting excellent IOMMU grouping on Linux systems, so I'm not sure why this has been so hard.

BUT at the end of the day, all I really really really need is to get my Windows 10 VM to output sound through the motherboards onboard sound and maybe network adapter since in the VM I can see the network and internet even though there it says there's no network adapter, but it seems somewhat unstable so I might need to just get the network adapter to show. Is that something that can be done in a stable way without causing server or VM instability? If so, can you give me some guidance with that? Ultimately what I'm trying to do is run a VM that stays up all the time for PlexAmp and run a optical cable from the server to stereo so other Plex devices such as the phone app can render to it. From my phone I can see the VM PlexAmp, however when I try to play to it, it looses connection, and of course, its doesn't have audio out yet.

I appreciate your help, maybe we can get this figured out....

Edited January 9Jan 9 by SPOautos

I had a break through!!! That bios line for VT-d that didn't have the ability to enable/disable, welllllll I changed the pci speed of the slot my video card is in from Auto to Gen 3 and then it changed the VT-d and allowed me to enable/disable it. Now I have it enabled and I now show IOMMU groups in Unraid!!!

Now I have audio passing through my VM....BUT quick question....how can I get it showing up on the network and stable? I need to keep another device connected to it over the network but i cant seem to get the VM to show up on the network, or at least it did a few times but then goes away, its not stable. I am able to see other devices on the network and access internet, however other devices cant see the VM. I only have one network adapter so I don't guess I will be able to pass it through because then my Unraid would loose it.

23 minutes ago, SPOautos said:

I had a break through!!! That bios line for VT-d that didn't have the ability to enable/disable, welllllll I changed the pci speed of the slot my video card is in from Auto to Gen 3 and then it changed the VT-d and allowed me to enable/disable it. Now I have it enabled and I now show IOMMU groups in Unraid!!!

Now I have audio passing through my VM....BUT quick question....how can I get it showing up on the network and stable? I need to keep another device connected to it over the network but i cant seem to get the VM to show up on the network, or at least it did a few times but then goes away, its not stable. I am able to see other devices on the network and access internet, however other devices cant see the VM. I only have one network adapter so I don't guess I will be able to pass it through because then my Unraid would loose it.

What bridge is defined on the VM network?

7 hours ago, SimonF said:

What bridge is defined on the VM network?

Its br0 but I think I have it figured out, I changed some Win 10 network settings to give full access to anything on the network and in my router I gave it "high priority" and it seems to have stabilized. My router doesn't seem to like that it has "Random MAC address detected" but I'm not sure what to do about that....I think in Win 10 I can create it a MAC address, I'll have to look into that. Also, after rebooting a few times it looks like Win 10 has automatically started seeing the ethernet even though I didn't pass it through and even shows that its a 10GB connection. I passed through the audio card and even installed the original Asus driver for it and set its audio out to high resolution direct output to the optical port and its passing through 192/24 resolution to my stereo perfectly. So it looks like I have a good HIFI streaming setup to my stereo now using PlexAmp and I can control it all from my phone PlexAmp app to select music, playlists, and control volume! It's working great!

It looks like everything is running great at the moment lol