February 7, 201412 yr root@Fileserver02:/mnt/cache/Xen/xbmcbuntu# grep -E "(vmx|svm)" --color=always /proc/cpuinfo root@Fileserver02:/mnt/cache/Xen/xbmcbuntu# 04:00.0 VGA compatible controller: AMD/ATI [Advanced Micro Devices, Inc.] Caicos [Radeon HD 6450] 04:00.1 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] Caicos HDMI Audio [Radeon HD 6400 Series] added to syslinux append /xen dom0_mem=2097152 --- /bzimage xen-pciback.hide=(04:00.0)(04:00.1) --- /bzroot added to config pci = ['04:00.0','04:00.1'] root@Fileserver02:/mnt/cache/Xen/xbmcbuntu# xl create xbmcbuntu.cfg Parsing config from xbmcbuntu.cfg xc: info: VIRTUAL MEMORY ARRANGEMENT: Loader: 0000000000100000->000000000019ec84 Modules: 0000000000000000->0000000000000000 TOTAL: 0000000000000000->000000003f800000 ENTRY ADDRESS: 0000000000100000 xc: info: PHYSICAL MEMORY ALLOCATION: 4KB PAGES: 0x0000000000000200 2MB PAGES: 0x00000000000001fb 1GB PAGES: 0x0000000000000000 libxl: error: libxl_pci.c:1046:libxl__device_pci_add: PCI device 0000:04:00.0 cannot be assigned - no IOMMU? libxl: error: libxl_pci.c:1046:libxl__device_pci_add: PCI device 0000:04:00.1 cannot be assigned - no IOMMU? Daemon running with PID 2348 Upon some googling around I tried putting intel_iommu=on in the boot, no change append /xen dom0_mem=2097152 --- /bzimage xen-pciback.hide=(04:00.0)(04:00.1) intel_iommu=on --- /bzroot This is a Fujitsu Primergy TX200 S5 Dual Xeon 5520L they support vt-x/vt-d and its enabled in the bios. Also passthrough works in ESXi and Xenserver 6.2 syslog.txt
February 7, 201412 yr This is taken from the Arch Wiki Xen page. https://wiki.archlinux.org/index.php/xen#System_requirements System requirements The Xen hypervisor requires kernel level support which is included in recent Linux kernels and is built into the linux and linux-lts Arch kernel packages. To run HVM domUs, the physical hardware must have either Intel VT-x or AMD-V (SVM) virtualization support. In order to verify this, run the following command when the Xen hypervisor is not running: $ grep -E "(vmx|svm)" --color=always /proc/cpuinfo If the above command does not produce output, then hardware virtualization support is unavailable and your hardware is unable to run HVM domUs. If you believe the CPU supports one of these features you should access the host system's BIOS configuration menu during the boot process and look if options related to virtualization support have been disabled. If such an option exists and is disabled, then enable it, boot the system and repeat the above command. The Xen hypervisor also supports PCI passthrough where PCI devices can be passed directly to the domU even in the absence of dom0 support for the device. In order to use PCI passthrough, the CPU must support IOMMU/VT-d. If you get no output from that grep -E command, hardware virtualization support isn't available or configured correctly. Check your BIOS is the only help I can offer.
February 7, 201412 yr Author It must not be reading the bios correctly. I'll have to switch back to esx
February 7, 201412 yr Do me a favour and boot into standard unRAID (without the Xen option enabled) and run the same command.
February 7, 201412 yr Author root@Fileserver02:~# grep -E "(vmx|svm)" --color=always /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
February 7, 201412 yr Ok, that's probably a good sign although I'm not 100% sure. Reboot into unRAID/Xen and type xl pci-assignable-list
February 7, 201412 yr Author root@Fileserver02:~# xl pci-assignable-list 0000:04:00.0 0000:04:00.1 builder="hvm" name = "xbmcbuntu" memory = 1024 vcpus = '2' disk = [ 'phy:/mnt/cache/Xen/xbmcbuntu/xbmcbuntu.img,xvda,w', 'file:/mnt/cache/Xen/xbmcbuntu/xbmcbuntu-12.00.Intel-AMD.iso,hdc:cdrom,r' ] boot="dc" vif = [ 'mac=00:16:3e:xx:xx:xx,bridge=br0' ] usb = '1' usbdevice = 'tablet' vnc = '1' vnclisten = '0.0.0.0' vncpasswd = '1' pci = ['04:00.0','04:00.1']
February 7, 201412 yr If you're at your wits end you might want to review this article from RedHat on interrupt remapping. https://access.redhat.com/site/articles/66747 The issue cropped up in GrumpyButFun's "KVM on OpenSUSE" thread, http://lime-technology.com/forum/index.php?topic=30715.msg280950#msg280950. But it might apply to this case with Xen. The interesting part: Environment: Xen hypervisor running on Red Hat Enterprise Linux 5 Passing a PCI device to a para-virtualized Xen hypervisor guest always works and is therefore vulnerable to CVE-2011-1898. Red Hat recommends that users of para-virtualized Xen hypervisor guests on Red Hat Enterprise Linux 5 hosts only use PCI passthrough with trusted guests. For users of fully-virtualized Xen hypervisor guests on Red Hat Enterprise Linux 5 hosts that have the RHSA-2011:1479 or RHSA-2012:0358 update installed, the regression affects systems using an Intel processor and chipset that have support for Intel Virtualization Technology for Directed I/O (VT-d), but do not have support for interrupt remapping. Interrupt remapping support is provided in newer processors and chipsets. To identify if your system has support for interrupt remapping: Reboot the host and set the log level to info (using the loglvl=info kernel boot option). Run the xm dmesg | grep "Interrupt Remapping hardware not found" command. If this message is found, there is no interrupt remapping support and PCI passthrough will be disabled for security reasons (PCI devices will not be able to be passed to fully-virtualized guests). Workaround If you wish to use PCI passthrough for fully-virtualized Xen hypervisor guests on systems that do not have interrupt remapping, the previous, vulnerable behavior can be restored by rebooting the Xen hypervisor host and using the iommu=no-intremap kernel boot option. Using this option reintroduces CVE-2011-1898. Note that if your hardware does not support interrupt remapping, and the RHSA-2011:1479 or RHSA-2012:0358 update has been installed, using the iommu=on kernel boot option prevents PCI passthrough to fully-virtualized Xen hypervisor guests (PCI devices will not be able to be assigned to the guest). Run the cat /proc/cmdline command to view the options the kernel was booted with. Users of GRUB can use the /etc/grub.conf file to make kernel boot options persist across reboots. I hope this helps! Edit: I also found this: nointremap [X86-64, Intel-IOMMU] Do not enable interrupt remapping. [Deprecated - use intremap=off]
February 7, 201412 yr root@Fileserver02:~# xl pci-assignable-list 0000:04:00.0 0000:04:00.1 I'm out generalz. Xen is clearly grabbing the devices as they show up under pci-assignable-list. I have never had it not work for me if I get that far. Your config file looks good, the configuration of pciback looks good. One quick thought, please post the output for the 04:00.1 device from the following command. You're looking for the pciback driver to be in use. lspci -k Enabling unsafe interrupt mapping may help, I have seen this suggested before too. So follow that rabbit hole. But I'm out of ideas!
February 7, 201412 yr Author thanks for looking that up. I tried both commands still didn't want to do it. Upon further reading http://support.citrix.com/article/CTX136517 Intel 5500/5520/X58 chipset revision 0x13 has an errata (#47 and #53) which makes the IOMMU interrupt remapping unit unreliable. This erratum causes interruptions and the interrupt remapping invalidations become unresponsive. *sigh*
February 7, 201412 yr Author root@Fileserver02:~# xl pci-assignable-list 0000:04:00.0 0000:04:00.1 I'm out generalz. Xen is clearly grabbing the devices as they show up under pci-assignable-list. I have never had it not work for me if I get that far. Your config file looks good, the configuration of pciback looks good. One quick thought, please post the output for the 04:00.1 device from the following command. You're looking for the pciback driver to be in use. lspci -k Enabling unsafe interrupt mapping may help, I have seen this suggested before too. So follow that rabbit hole. But I'm out of ideas! 04:00.0 VGA compatible controller: AMD/ATI [Advanced Micro Devices, Inc.] Caicos [Radeon HD 6450] Subsystem: ASUSTeK Computer Inc. Device 047b Kernel driver in use: pciback 04:00.1 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] Caicos HDMI Audio [Radeon HD 6400 Series] Subsystem: ASUSTeK Computer Inc. Device aa98 Kernel driver in use: pciback
February 7, 201412 yr Author root@Fileserver02:~# xl pci-assignable-list 0000:04:00.0 0000:04:00.1 I'm out generalz. Xen is clearly grabbing the devices as they show up under pci-assignable-list. I have never had it not work for me if I get that far. Your config file looks good, the configuration of pciback looks good. One quick thought, please post the output for the 04:00.1 device from the following command. You're looking for the pciback driver to be in use. lspci -k Enabling unsafe interrupt mapping may help, I have seen this suggested before too. So follow that rabbit hole. But I'm out of ideas! 04:00.0 VGA compatible controller: AMD/ATI [Advanced Micro Devices, Inc.] Caicos [Radeon HD 6450] Subsystem: ASUSTeK Computer Inc. Device 047b Kernel driver in use: pciback 04:00.1 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] Caicos HDMI Audio [Radeon HD 6400 Series] Subsystem: ASUSTeK Computer Inc. Device aa98 Kernel driver in use: pciback thanks for looking into it anyways... much appreciated.
February 7, 201412 yr Just another thought, are you using any additional syslinux boot time options on your DOM0 (host) unRAID server?
February 7, 201412 yr Author nothing other then what was tried above. Always xen-pciback.hide=(04:00.0)(04:00.1) tried these individually iommu=no-intremap intremap=off I'm pretty sure its the version of Xen or the Kernel thats being used. I switched back to Xenserver 6.2 and during bootup seen "Applying CPU micocode update" Everything is working, xbmcbuntu and the unraid vm have multiple things passed through, did a pairty check and watched a movie at the same time.
February 8, 201412 yr Your syslog states the kernel command line as the pciback line only as opposed to all the options passed. It should be stating the entire line as far I remember so that suggests it is not picking up the option at all for some reason. For example 2.206899] Kernel command line: placeholder root=UUID=01935248-f777-455b-9fac-4e398112c8d7 ro quiet intel_iommu=on 2.206930] Intel-IOMMU: enabled This is not related to the microcode update which is a separate post boot package.
February 9, 201412 yr nothing other then what was tried above. Always xen-pciback.hide=(04:00.0)(04:00.1) tried these individually iommu=no-intremap intremap=off I'm pretty sure its the version of Xen or the Kernel thats being used. I switched back to Xenserver 6.2 and during bootup seen "Applying CPU micocode update" Everything is working, xbmcbuntu and the unraid vm have multiple things passed through, did a pairty check and watched a movie at the same time. So I have been successful passtrough some USB ports to a Windows 8.1 VM and a SATA card to a Arch VM, no problem, it works great! BUT passtrough my GPU (Radeon HD 5450) to my Windows 8.1 VM is a pain in the .... !! no way I have get it to work. I have also tested onboard intel hd graphic ,but no success, I hope these issues are in the version of Xen and the kernel we are using. since there was a cake walk to passtrough my USB and SATA card
February 9, 201412 yr Yes, that was the reason a bought it but have anyone get it to work on unraid ? if so, how did you do? //Peter
February 10, 201412 yr Author yea that syslog was from before I found those commands. I didn't think to grab the ones from with the commands. Your syslog states the kernel command line as the pciback line only as opposed to all the options passed. It should be stating the entire line as far I remember so that suggests it is not picking up the option at all for some reason. For example 2.206899] Kernel command line: placeholder root=UUID=01935248-f777-455b-9fac-4e398112c8d7 ro quiet intel_iommu=on 2.206930] Intel-IOMMU: enabled This is not related to the microcode update which is a separate post boot package.
March 15, 201412 yr Any progress? I'm having similar problems. I'm running a Gigabyte GA-990FXA-UD3 with an AMD FX-4100 cpu, virtualization and IOMMU definitely enabled in BIOS. With my Radeon HD 5450 everything works up to about here: root@unRAID:/mnt/cache/Apps/Windows8# xl pci-assignable-list 0000:01:00.0 0000:01:00.1 Then I try to do PCI passthrough to Windows 8.1 and I get this error (same with XBMCbuntu): root@unRAID:/mnt/cache/Apps/Windows8# xl create windows.cfg Parsing config from windows.cfg xc: info: VIRTUAL MEMORY ARRANGEMENT: Loader: 0000000000100000->000000000019ec84 Modules: 0000000000000000->0000000000000000 TOTAL: 0000000000000000->00000000e6c00000 ENTRY ADDRESS: 0000000000100000 xc: info: PHYSICAL MEMORY ALLOCATION: 4KB PAGES: 0x0000000000000200 2MB PAGES: 0x0000000000000335 1GB PAGES: 0x0000000000000002 libxl: error: libxl_pci.c:1046:libxl__device_pci_add: PCI device 0000:01:00.0 cannot be assigned - no IOMMU? libxl: error: libxl_pci.c:1046:libxl__device_pci_add: PCI device 0000:01:00.1 cannot be assigned - no IOMMU? Daemon running with PID 4088
March 15, 201412 yr Author i gave up. the kernel used does not have the patch for x58 chipset/ vt-d for passthrough to work
March 15, 201412 yr Any progress? I'm having similar problems. I'm running a Gigabyte GA-990FXA-UD3 with an AMD FX-4100 cpu, virtualization and IOMMU definitely enabled in BIOS. With my Radeon HD 5450 everything works up to about here: root@unRAID:/mnt/cache/Apps/Windows8# xl pci-assignable-list 0000:01:00.0 0000:01:00.1 Then I try to do PCI passthrough to Windows 8.1 and I get this error (same with XBMCbuntu): root@unRAID:/mnt/cache/Apps/Windows8# xl create windows.cfg Parsing config from windows.cfg xc: info: VIRTUAL MEMORY ARRANGEMENT: Loader: 0000000000100000->000000000019ec84 Modules: 0000000000000000->0000000000000000 TOTAL: 0000000000000000->00000000e6c00000 ENTRY ADDRESS: 0000000000100000 xc: info: PHYSICAL MEMORY ALLOCATION: 4KB PAGES: 0x0000000000000200 2MB PAGES: 0x0000000000000335 1GB PAGES: 0x0000000000000002 libxl: error: libxl_pci.c:1046:libxl__device_pci_add: PCI device 0000:01:00.0 cannot be assigned - no IOMMU? libxl: error: libxl_pci.c:1046:libxl__device_pci_add: PCI device 0000:01:00.1 cannot be assigned - no IOMMU? Daemon running with PID 4088 I had the same problem on my arch install. I had to add the "intel_iommu=on" to my grub config. Not sure how to do that on selinux, but your error is the exact one that I had and fixed. Edit: You can read a bit more about it here. One thing to note I'm using KVM so for Xen it looks like you need to add the "iommu=on" parameter.
March 15, 201412 yr I had the same problem on my arch install. I had to add the "intel_iommu=on" to my grub config. Not sure how to do that on selinux, but your error is the exact one that I had and fixed. Edit: You can read a bit more about it here. One thing to note I'm using KVM so for Xen it looks like you need to add the "iommu=on" parameter. I tried quite a few iommu arguments in grub with no dice. Apparently this is a problem specific to Xen 4.3.1 with the application of XSA-36 which picked up on the motherboard's broken IVRS ACPI table and disables virtualization . This issue isn't there in Xen 4.2.1 and I'm not sure if the issue is resolved in 4.4. I guess I'll have to wait for unraid to move to a new version to see. Here's a patch which could fix it: http://lists.xen.org/archives/html/xen-devel/2013-05/msg00448.html i think i'd need to get into unraid's bzimage to apply it
Archived
This topic is now archived and is closed to further replies.