AMD AM1/FM2+ Mobos that support IOMMU?


Recommended Posts

Hello,

 

I'm interested in combining my HTPC and UNRAID box into one unit with the latest virtualization feature of 6.0.  I'm looking at using either the AMD AM1 or FM2+ style CPU's which support HVM, but I'm having a hard time figuring out which motherboards have a BIOS that has IOMMU support?

 

Thanks

Link to comment

I am using an ASRock FM2A88M Extreme 4+ with an A8-6500T, I also have an ASUS HD6450 1GB graphics card which I am using to virtualise Kodi.

 

I was using Citrix Xenserver 6.2 and virtualising UNRAID 5, Ubuntu and Windows 7 (for Kodi). As off this week I have installed UNRAID 6.1.2 and I am using it to virtualise OpenELEC via the HD6450 and am passing through a keyboard/mouse to it and a Pulse-Eight USB-CEC adapter. It is all working great, I just had to add some kernel arguments I found on the forum to get the IOMMU working. The MoBo has 8 SATA3 ports natively which was also a big selling point to me. 

Link to comment

Are you able to pass through the graphics of the APU or just the HD6450?

 

I have done some experiments this morning with a Windows 7 VM I set up. I have been able to pass through the HD6450 to it and display the output of the VM on my TV but when the APU is selected I get no output, just a black screen. I tried changing the primary graphics adapter in the BIOS but this had no effect on the result.

Link to comment

I do not have first hand experience, but what you describe is what I've heard is the norm. so for OP I feel like the better route is AM3+ w/ FX cpu ... unless there is something else the A-series brings to the game. I suppose all else being equal, letting the built in graphics do console work and the discrete gpu service HTPC makes sense. I just remember when i was looking 2 years ago an FX cpu with a very cheap PCI card for console duties was a better bang for the buck proposition especially for Plex transcoding. But my unraid doesn't serve as an HTPC, I just push streams out of it to my roku and my tablet.

Link to comment
  • 2 weeks later...

ntphil, do you remember what the kernel arguments you used were? I also sent you a PM.

 

I have a Gigabyte GIGABYTE GA-F2A88X-D3H with a A4-6300 that I'm having trouble getting working. I get errors about IOMMU groups when trying to start a machine with something passed through.

 

The reason I don't simply believe I'm stuck is I've had passthrough working with ESXI 5.5 using this hardware. I've passed through the pci-e TV Tuner, several PCI devices like sound cards, USB cards, a PCI-e USB3 card, and a couple old video cards and got them to work. It could be particular about hardware (I went through 3-4 USB3 cards before I found one that worked) it seemed pretty stable. I'm not at all a fan of ESXI admin tools for the most part and its really not the right tool for my job. Also, none of the AMD components were available to mark for passthrough and I'd like to at least access the onboard sound and maybe one of the USB controllers.

Link to comment

ntphil, do you remember what the kernel arguments you used were? I also sent you a PM.

 

I have a Gigabyte GIGABYTE GA-F2A88X-D3H with a A4-6300 that I'm having trouble getting working. I get errors about IOMMU groups when trying to start a machine with something passed through.

 

The reason I don't simply believe I'm stuck is I've had passthrough working with ESXI 5.5 using this hardware. I've passed through the pci-e TV Tuner, several PCI devices like sound cards, USB cards, a PCI-e USB3 card, and a couple old video cards and got them to work. It could be particular about hardware (I went through 3-4 USB3 cards before I found one that worked) it seemed pretty stable. I'm not at all a fan of ESXI admin tools for the most part and its really not the right tool for my job. Also, none of the AMD components were available to mark for passthrough and I'd like to at least access the onboard sound and maybe one of the USB controllers.

 

I have the same board and I have hat the same issue. I could solve it after I have edited my  syslinux.cfg in syslinux folder on the flash drive.

 

My syslinux.cfg now looks like this

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

The only issue that I know notice some problems with USB 3.0 in the VMs because unraid-OS finds it but the VM has problems to accept them.

Link to comment

ntphil, do you remember what the kernel arguments you used were? I also sent you a PM.

 

I have a Gigabyte GIGABYTE GA-F2A88X-D3H with a A4-6300 that I'm having trouble getting working. I get errors about IOMMU groups when trying to start a machine with something passed through.

 

The reason I don't simply believe I'm stuck is I've had passthrough working with ESXI 5.5 using this hardware. I've passed through the pci-e TV Tuner, several PCI devices like sound cards, USB cards, a PCI-e USB3 card, and a couple old video cards and got them to work. It could be particular about hardware (I went through 3-4 USB3 cards before I found one that worked) it seemed pretty stable. I'm not at all a fan of ESXI admin tools for the most part and its really not the right tool for my job. Also, none of the AMD components were available to mark for passthrough and I'd like to at least access the onboard sound and maybe one of the USB controllers.

 

I have the same board and I have hat the same issue. I could solve it after I have edited my  syslinux.cfg in syslinux folder on the flash drive.

 

My syslinux.cfg now looks like this

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

The only issue that I know notice some problems with USB 3.0 in the VMs because unraid-OS finds it but the VM has problems to accept them.

 

Keep in mind those that first one is an intel specific parameters which I'm pretty sure will not apply to AMD world.

 

I have found this useful in the past http://redsymbol.net/linux-kernel-boot-parameters/

Link to comment

Thanks for the ideas, I did some reading around this morning and came across a few of those things (probably following the same trail ntphil had).

 

I had tried some of the options but not the allow unsafe interrupts one I don't think. I'm glad to see you don't have the acs option in your file, it didn't seem like I would need that. When I looked at the list of devices and iommu groups everything seemed to be cleanly separated and I have a lot of cards currently installed.

 

akira62:

 

Are the problems with the USB3.0 when doing a passthrough of the whole controller or just the USB device?

 

I also think I might want to update the bios since there's a newer version from April.

 

There was an option on ESXI I had to use for some devices...it was called something like MSIEnabled=false. I saw yesterday that something very similar sounded can be configured.

Link to comment

I have the problems only with the USB 3.0 but when I put it in again in the USB 3 it works, only if it stay put over the boot up sequence of the unraid-os I have the problem with the VM and the device hosted at USB 3.

 

@jumperalex:

 

It may be that it only works for "Intel" CPUs or Systems, but the fact is, after and before I have these parameters not in the syslinux.cfg I can not run any VMs with PCI-E passthrough only with VNC with this parameter my HD7850 with my Board and the A10-6800 work fine. It may also be that it works the same as amd_iommu, I do not know. I have not really thought about the "Intel", because I am not very into Linux I only try to solve my problems with the solution other people with more knowledge has found. But thank you for the hint I will give it a trry when I need to reboot my system.

Link to comment

Just to add a data point.  Tonight I setup an F2A88XM-D3H mobo with an A8 7600 and I've literally done almost nothing and it works.  I enabled IOMMU in the BIOS as it was set to disabled.  That's it.  I've got Kodi running in a VM using my CPUs GPU as the passthru.  I don't even have an add on video card in there.  It just takes over the console when it boots.  Passed the VM template my USB Logitech dongle that my wireless keyboard talks to and my HDMI ports "sound card" and I'm watching movies.  Info reports HVM & IOMMU as enabled.

 

The movies are dropping every other frame and the CPU is at 100% during playback but that's probably related to not having an add-on card.

 

root@nas:/boot/syslinux# cat syslinux.cfg

default /syslinux/menu.c32

menu title Lime Technology

prompt 0

timeout 50

label unRAID OS

  menu default

  kernel /bzimage

  append initrd=/bzroot irqpoll <-- This was added from my old board. I should probably take it out...

label unRAID OS Safe Mode (no plugins)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

 

Link to comment

Thanks for the data point. What bios version are you running if you just happen to know? I'm going to update mine tonight.

 

With the allow unsafe interrupts option I passed through a 3450 video card and I got no video when I swapped the monitor to that card. I've gotten the card to work as a secondary adapter under esxi before (there is no way I believe to remove the ESXI adapter from virtual machines) but I know that's a different setup. I'm almost positive that the same test failed without the interrupts thing the previous night but maybe not. I tried passing through the onboard video combined with the hdmi audio and it started but it seemed like I was still getting the unraid console. I'm not sure why that would be. I didn't have much time to play with this unfortunately.

 

I still got errors when trying to run a VM with only the onboard AMD (not hdmi) sound and it failed with a message about iommu groups. But if I look at the groups I see that in addition an ISA bridge there is a SM bus controller in the group which of course aren't passed through. I haven't really gotten a good idea what this is from googling around but some suggested its a legacy interface. I could just pass it through if nothing is using it.

 

I think my best bet is to update the bios (release notes say nothing about this, but last time I updated I fixed a shutdown issue that the release notes also said nothing about), drop the unsafe interrupts and try again.

 

Honestly, the VGA card passthrough is academic for me. I need VMs with sound cards, TV cards and USB controllers. Unfortunately, two other sound cards I have installed (a crappy c-media pci-e one and an ancient SB live PCI one) do not show up in the drop down at all. They wouldn't work that way anyway since it seems that the UI doesn't automatically add grouped devices based on the AMD one. They're behind bridges. I'll have to hand edit the XML and I think there is a place you modify to blacklist devices? More research I guess.

Link to comment

The movies are dropping every other frame and the CPU is at 100% during playback but that's probably related to not having an add-on card.

 

  append initrd=/bzroot irqpoll <-- This was added from my old board. I should probably take it out...

 

Definitely remove that irqpoll, significantly hurting performance.  You only want that if you can't run at all without it.

Link to comment

Thanks for the data point. What bios version are you running if you just happen to know? I'm going to update mine tonight.

 

F8 from 4/09/2015.  I was using the 6.0.0b3 OpenELEC template.

 

Definitely remove that irqpoll, significantly hurting performance.  You only want that if you can't run at all without it.

 

Done! Thanks for the heads up.

Link to comment

It looks like the M and non-M board I have actually have different bios histories. The latest on mine is F6.

 

Unfortunately, the newer bios from gigabyte returned a restart instead of shutdown on soft shutdown issue I had with the board with the F4 bios. That was a pretty unacceptable issue for a board that I was going to have hitched to a UPS so I can't use F6.

 

I can't get a machine to start with passed through hardware without the allow_unsafe_interrupts option on.

 

I never actually got to test hardware. XP is actually kind of a pain to install. You have to hand set the disk to IDE from virto to get the installer to see it. I tried the add a second disk and install the virto drivers before switching option, but windows never starts if that driver is running. The driver install seems to take forever and then when windows never boots, it just sits at the starting windows screen getting nowhere. The IDE problem might not be a problem since I could load the controller from the floppy virto driver image, but the guide suggests you should load other drivers first. There's no way to do that short of slipstreaming perhaps in the XP installer. Regardless, when the XP driver polls the disks before asking for a install location with the driver installed it also just gets stuck forever.

 

Its up and running in IDE mode though. I just want to see if I can get a few pieces of hardware running that I had working with esxi.

Link to comment

Are you able to pass through the graphics of the APU or just the HD6450?

 

I have had some success that I thought you might be interested in; I have successfully passed through the on board graphics and HDMI audio to an OpenELEC VM! Video playback including 1080p blu-ray rip play back with no problems! I am having an issue with DTS-HD and TrueHD passthrough but I suspect it is an OpenELEC or driver issue rather than a VM specific one.

 

Check out your system log and look for anything similar to this (what I found in my log!):

 

Oct  3 17:32:30 Tower kernel: [Firmware Bug]: AMD-Vi: IOAPIC[5] not in IVRS table
Oct  3 17:32:30 Tower kernel: [Firmware Bug]: AMD-Vi: No southbridge IOAPIC found
Oct  3 17:32:30 Tower kernel: AMD-Vi: Disabling interrupt remapping

 

This alerted me to the fact that all was not well with my AMD-Vi. I did some googling and came across this article:

 

http://ubuntuforums.org/showthread.php?t=2254677

 

Using the information from the above article I made some amendments to my syslinux.cfg which now looks like this:

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  kernel /bzimage
  append initrd=/bzroot
label KVM/unRAID OS (AMD)
  menu default
  kernel /bzimage
  append amd_iommu=on ivrs_ioapic[5]=00:14.0 initrd=/bzroot
label KVM/unRAID OS (AMD Test)
  kernel /bzimage
  append iommu=pt iommu=1 ivrs_ioapic[5]=00:14.0 initrd=/bzroot
label KVM/unRAID OS (Intel)
  kernel /bzimage
  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest
label Xen/unRAID OS
  kernel /syslinux/mboot.c32
  append /xen --- /bzimage --- /bzroot
label Xen/unRAID OS Safe Mode (no plugins)
  kernel /syslinux/mboot.c32
  append /xen --- /bzimage --- /bzroot unraidsafemode

 

Using the "AMD Test" menu selection on boot I am able to run my VMs without unsafe interrupts and I am also able to passthrough the on board graphics!!

 

Hope this helps anyone who is looking for a solution!

Link to comment
  • 2 weeks later...

Just to add a data point.  Tonight I setup an F2A88XM-D3H mobo with an A8 7600 and I've literally done almost nothing and it works.  I enabled IOMMU in the BIOS as it was set to disabled.  That's it.  I've got Kodi running in a VM using my CPUs GPU as the passthru.  I don't even have an add on video card in there.  It just takes over the console when it boots.  Passed the VM template my USB Logitech dongle that my wireless keyboard talks to and my HDMI ports "sound card" and I'm watching movies.  Info reports HVM & IOMMU as enabled.

 

The movies are dropping every other frame and the CPU is at 100% during playback but that's probably related to not having an add-on card.

 

root@nas:/boot/syslinux# cat syslinux.cfg

default /syslinux/menu.c32

menu title Lime Technology

prompt 0

timeout 50

label unRAID OS

  menu default

  kernel /bzimage

  append initrd=/bzroot irqpoll <-- This was added from my old board. I should probably take it out...

label unRAID OS Safe Mode (no plugins)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

 

 

Definitely remove that irqpoll, significantly hurting performance.  You only want that if you can't run at all without it.

 

Done! Thanks for the heads up.

 

How did removing that irqpoll affect movie playback with the A8-7600?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.