mr_cb7 Posted September 20, 2015 Share Posted September 20, 2015 Hello, I'm interested in combining my HTPC and UNRAID box into one unit with the latest virtualization feature of 6.0. I'm looking at using either the AMD AM1 or FM2+ style CPU's which support HVM, but I'm having a hard time figuring out which motherboards have a BIOS that has IOMMU support? Thanks Quote Link to comment
jumperalex Posted September 20, 2015 Share Posted September 20, 2015 I have heard rumors that the APU's can be passed through, but I believe right now the best option is an affordable discreet card coupled with an FX cpu vice an APU. See my sig for my iommu setup. I used a very cheap gpu just for monitor support, you can step it up just a few bucks for decent htpc performance Quote Link to comment
ntphil Posted September 20, 2015 Share Posted September 20, 2015 I am using an ASRock FM2A88M Extreme 4+ with an A8-6500T, I also have an ASUS HD6450 1GB graphics card which I am using to virtualise Kodi. I was using Citrix Xenserver 6.2 and virtualising UNRAID 5, Ubuntu and Windows 7 (for Kodi). As off this week I have installed UNRAID 6.1.2 and I am using it to virtualise OpenELEC via the HD6450 and am passing through a keyboard/mouse to it and a Pulse-Eight USB-CEC adapter. It is all working great, I just had to add some kernel arguments I found on the forum to get the IOMMU working. The MoBo has 8 SATA3 ports natively which was also a big selling point to me. Quote Link to comment
jumperalex Posted September 20, 2015 Share Posted September 20, 2015 Are you able to pass through the graphics of the APU or just the HD6450? Quote Link to comment
ntphil Posted September 21, 2015 Share Posted September 21, 2015 Are you able to pass through the graphics of the APU or just the HD6450? I have done some experiments this morning with a Windows 7 VM I set up. I have been able to pass through the HD6450 to it and display the output of the VM on my TV but when the APU is selected I get no output, just a black screen. I tried changing the primary graphics adapter in the BIOS but this had no effect on the result. Quote Link to comment
jumperalex Posted September 21, 2015 Share Posted September 21, 2015 I do not have first hand experience, but what you describe is what I've heard is the norm. so for OP I feel like the better route is AM3+ w/ FX cpu ... unless there is something else the A-series brings to the game. I suppose all else being equal, letting the built in graphics do console work and the discrete gpu service HTPC makes sense. I just remember when i was looking 2 years ago an FX cpu with a very cheap PCI card for console duties was a better bang for the buck proposition especially for Plex transcoding. But my unraid doesn't serve as an HTPC, I just push streams out of it to my roku and my tablet. Quote Link to comment
scorcho99 Posted September 30, 2015 Share Posted September 30, 2015 ntphil, do you remember what the kernel arguments you used were? I also sent you a PM. I have a Gigabyte GIGABYTE GA-F2A88X-D3H with a A4-6300 that I'm having trouble getting working. I get errors about IOMMU groups when trying to start a machine with something passed through. The reason I don't simply believe I'm stuck is I've had passthrough working with ESXI 5.5 using this hardware. I've passed through the pci-e TV Tuner, several PCI devices like sound cards, USB cards, a PCI-e USB3 card, and a couple old video cards and got them to work. It could be particular about hardware (I went through 3-4 USB3 cards before I found one that worked) it seemed pretty stable. I'm not at all a fan of ESXI admin tools for the most part and its really not the right tool for my job. Also, none of the AMD components were available to mark for passthrough and I'd like to at least access the onboard sound and maybe one of the USB controllers. Quote Link to comment
jumperalex Posted September 30, 2015 Share Posted September 30, 2015 I'm gonna guess at least one of them is iommu=pt appended to syslinux; that is what helped me get unraid to report that my machine supported VM. Quote Link to comment
akira62 Posted September 30, 2015 Share Posted September 30, 2015 ntphil, do you remember what the kernel arguments you used were? I also sent you a PM. I have a Gigabyte GIGABYTE GA-F2A88X-D3H with a A4-6300 that I'm having trouble getting working. I get errors about IOMMU groups when trying to start a machine with something passed through. The reason I don't simply believe I'm stuck is I've had passthrough working with ESXI 5.5 using this hardware. I've passed through the pci-e TV Tuner, several PCI devices like sound cards, USB cards, a PCI-e USB3 card, and a couple old video cards and got them to work. It could be particular about hardware (I went through 3-4 USB3 cards before I found one that worked) it seemed pretty stable. I'm not at all a fan of ESXI admin tools for the most part and its really not the right tool for my job. Also, none of the AMD components were available to mark for passthrough and I'd like to at least access the onboard sound and maybe one of the USB controllers. I have the same board and I have hat the same issue. I could solve it after I have edited my syslinux.cfg in syslinux folder on the flash drive. My syslinux.cfg now looks like this default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest The only issue that I know notice some problems with USB 3.0 in the VMs because unraid-OS finds it but the VM has problems to accept them. Quote Link to comment
jumperalex Posted September 30, 2015 Share Posted September 30, 2015 ntphil, do you remember what the kernel arguments you used were? I also sent you a PM. I have a Gigabyte GIGABYTE GA-F2A88X-D3H with a A4-6300 that I'm having trouble getting working. I get errors about IOMMU groups when trying to start a machine with something passed through. The reason I don't simply believe I'm stuck is I've had passthrough working with ESXI 5.5 using this hardware. I've passed through the pci-e TV Tuner, several PCI devices like sound cards, USB cards, a PCI-e USB3 card, and a couple old video cards and got them to work. It could be particular about hardware (I went through 3-4 USB3 cards before I found one that worked) it seemed pretty stable. I'm not at all a fan of ESXI admin tools for the most part and its really not the right tool for my job. Also, none of the AMD components were available to mark for passthrough and I'd like to at least access the onboard sound and maybe one of the USB controllers. I have the same board and I have hat the same issue. I could solve it after I have edited my syslinux.cfg in syslinux folder on the flash drive. My syslinux.cfg now looks like this default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest The only issue that I know notice some problems with USB 3.0 in the VMs because unraid-OS finds it but the VM has problems to accept them. Keep in mind those that first one is an intel specific parameters which I'm pretty sure will not apply to AMD world. I have found this useful in the past http://redsymbol.net/linux-kernel-boot-parameters/ Quote Link to comment
scorcho99 Posted September 30, 2015 Share Posted September 30, 2015 Thanks for the ideas, I did some reading around this morning and came across a few of those things (probably following the same trail ntphil had). I had tried some of the options but not the allow unsafe interrupts one I don't think. I'm glad to see you don't have the acs option in your file, it didn't seem like I would need that. When I looked at the list of devices and iommu groups everything seemed to be cleanly separated and I have a lot of cards currently installed. akira62: Are the problems with the USB3.0 when doing a passthrough of the whole controller or just the USB device? I also think I might want to update the bios since there's a newer version from April. There was an option on ESXI I had to use for some devices...it was called something like MSIEnabled=false. I saw yesterday that something very similar sounded can be configured. Quote Link to comment
akira62 Posted September 30, 2015 Share Posted September 30, 2015 I have the problems only with the USB 3.0 but when I put it in again in the USB 3 it works, only if it stay put over the boot up sequence of the unraid-os I have the problem with the VM and the device hosted at USB 3. @jumperalex: It may be that it only works for "Intel" CPUs or Systems, but the fact is, after and before I have these parameters not in the syslinux.cfg I can not run any VMs with PCI-E passthrough only with VNC with this parameter my HD7850 with my Board and the A10-6800 work fine. It may also be that it works the same as amd_iommu, I do not know. I have not really thought about the "Intel", because I am not very into Linux I only try to solve my problems with the solution other people with more knowledge has found. But thank you for the hint I will give it a trry when I need to reboot my system. Quote Link to comment
Cmc0619 Posted October 1, 2015 Share Posted October 1, 2015 Just to add a data point. Tonight I setup an F2A88XM-D3H mobo with an A8 7600 and I've literally done almost nothing and it works. I enabled IOMMU in the BIOS as it was set to disabled. That's it. I've got Kodi running in a VM using my CPUs GPU as the passthru. I don't even have an add on video card in there. It just takes over the console when it boots. Passed the VM template my USB Logitech dongle that my wireless keyboard talks to and my HDMI ports "sound card" and I'm watching movies. Info reports HVM & IOMMU as enabled. The movies are dropping every other frame and the CPU is at 100% during playback but that's probably related to not having an add-on card. root@nas:/boot/syslinux# cat syslinux.cfg default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append initrd=/bzroot irqpoll <-- This was added from my old board. I should probably take it out... label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest Quote Link to comment
scorcho99 Posted October 1, 2015 Share Posted October 1, 2015 Thanks for the data point. What bios version are you running if you just happen to know? I'm going to update mine tonight. With the allow unsafe interrupts option I passed through a 3450 video card and I got no video when I swapped the monitor to that card. I've gotten the card to work as a secondary adapter under esxi before (there is no way I believe to remove the ESXI adapter from virtual machines) but I know that's a different setup. I'm almost positive that the same test failed without the interrupts thing the previous night but maybe not. I tried passing through the onboard video combined with the hdmi audio and it started but it seemed like I was still getting the unraid console. I'm not sure why that would be. I didn't have much time to play with this unfortunately. I still got errors when trying to run a VM with only the onboard AMD (not hdmi) sound and it failed with a message about iommu groups. But if I look at the groups I see that in addition an ISA bridge there is a SM bus controller in the group which of course aren't passed through. I haven't really gotten a good idea what this is from googling around but some suggested its a legacy interface. I could just pass it through if nothing is using it. I think my best bet is to update the bios (release notes say nothing about this, but last time I updated I fixed a shutdown issue that the release notes also said nothing about), drop the unsafe interrupts and try again. Honestly, the VGA card passthrough is academic for me. I need VMs with sound cards, TV cards and USB controllers. Unfortunately, two other sound cards I have installed (a crappy c-media pci-e one and an ancient SB live PCI one) do not show up in the drop down at all. They wouldn't work that way anyway since it seems that the UI doesn't automatically add grouped devices based on the AMD one. They're behind bridges. I'll have to hand edit the XML and I think there is a place you modify to blacklist devices? More research I guess. Quote Link to comment
RobJ Posted October 1, 2015 Share Posted October 1, 2015 The movies are dropping every other frame and the CPU is at 100% during playback but that's probably related to not having an add-on card. append initrd=/bzroot irqpoll <-- This was added from my old board. I should probably take it out... Definitely remove that irqpoll, significantly hurting performance. You only want that if you can't run at all without it. Quote Link to comment
Cmc0619 Posted October 1, 2015 Share Posted October 1, 2015 Thanks for the data point. What bios version are you running if you just happen to know? I'm going to update mine tonight. F8 from 4/09/2015. I was using the 6.0.0b3 OpenELEC template. Definitely remove that irqpoll, significantly hurting performance. You only want that if you can't run at all without it. Done! Thanks for the heads up. Quote Link to comment
scorcho99 Posted October 5, 2015 Share Posted October 5, 2015 It looks like the M and non-M board I have actually have different bios histories. The latest on mine is F6. Unfortunately, the newer bios from gigabyte returned a restart instead of shutdown on soft shutdown issue I had with the board with the F4 bios. That was a pretty unacceptable issue for a board that I was going to have hitched to a UPS so I can't use F6. I can't get a machine to start with passed through hardware without the allow_unsafe_interrupts option on. I never actually got to test hardware. XP is actually kind of a pain to install. You have to hand set the disk to IDE from virto to get the installer to see it. I tried the add a second disk and install the virto drivers before switching option, but windows never starts if that driver is running. The driver install seems to take forever and then when windows never boots, it just sits at the starting windows screen getting nowhere. The IDE problem might not be a problem since I could load the controller from the floppy virto driver image, but the guide suggests you should load other drivers first. There's no way to do that short of slipstreaming perhaps in the XP installer. Regardless, when the XP driver polls the disks before asking for a install location with the driver installed it also just gets stuck forever. Its up and running in IDE mode though. I just want to see if I can get a few pieces of hardware running that I had working with esxi. Quote Link to comment
ntphil Posted October 10, 2015 Share Posted October 10, 2015 Are you able to pass through the graphics of the APU or just the HD6450? I have had some success that I thought you might be interested in; I have successfully passed through the on board graphics and HDMI audio to an OpenELEC VM! Video playback including 1080p blu-ray rip play back with no problems! I am having an issue with DTS-HD and TrueHD passthrough but I suspect it is an OpenELEC or driver issue rather than a VM specific one. Check out your system log and look for anything similar to this (what I found in my log!): Oct 3 17:32:30 Tower kernel: [Firmware Bug]: AMD-Vi: IOAPIC[5] not in IVRS table Oct 3 17:32:30 Tower kernel: [Firmware Bug]: AMD-Vi: No southbridge IOAPIC found Oct 3 17:32:30 Tower kernel: AMD-Vi: Disabling interrupt remapping This alerted me to the fact that all was not well with my AMD-Vi. I did some googling and came across this article: http://ubuntuforums.org/showthread.php?t=2254677 Using the information from the above article I made some amendments to my syslinux.cfg which now looks like this: default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot label KVM/unRAID OS (AMD) menu default kernel /bzimage append amd_iommu=on ivrs_ioapic[5]=00:14.0 initrd=/bzroot label KVM/unRAID OS (AMD Test) kernel /bzimage append iommu=pt iommu=1 ivrs_ioapic[5]=00:14.0 initrd=/bzroot label KVM/unRAID OS (Intel) kernel /bzimage append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest label Xen/unRAID OS kernel /syslinux/mboot.c32 append /xen --- /bzimage --- /bzroot label Xen/unRAID OS Safe Mode (no plugins) kernel /syslinux/mboot.c32 append /xen --- /bzimage --- /bzroot unraidsafemode Using the "AMD Test" menu selection on boot I am able to run my VMs without unsafe interrupts and I am also able to passthrough the on board graphics!! Hope this helps anyone who is looking for a solution! Quote Link to comment
cbn Posted October 23, 2015 Share Posted October 23, 2015 Just to add a data point. Tonight I setup an F2A88XM-D3H mobo with an A8 7600 and I've literally done almost nothing and it works. I enabled IOMMU in the BIOS as it was set to disabled. That's it. I've got Kodi running in a VM using my CPUs GPU as the passthru. I don't even have an add on video card in there. It just takes over the console when it boots. Passed the VM template my USB Logitech dongle that my wireless keyboard talks to and my HDMI ports "sound card" and I'm watching movies. Info reports HVM & IOMMU as enabled. The movies are dropping every other frame and the CPU is at 100% during playback but that's probably related to not having an add-on card. root@nas:/boot/syslinux# cat syslinux.cfg default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append initrd=/bzroot irqpoll <-- This was added from my old board. I should probably take it out... label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest Definitely remove that irqpoll, significantly hurting performance. You only want that if you can't run at all without it. Done! Thanks for the heads up. How did removing that irqpoll affect movie playback with the A8-7600? Quote Link to comment
Cmc0619 Posted October 24, 2015 Share Posted October 24, 2015 No change. I ended up returning the AMD setup for an Intel i5 for unrelated reasons (wanted different PCI slot layout). I'm sure it would've been fine with a discreet card; it looked like the hardware acceleration wasn't being detected for some reason. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.