February 5, 201412 yr I've seen a fair amount of mis-information on this topic over last few days, and seeing as I've spent the last 4 months refining the process on my own Xen setup I thought you might appreciate a defacto guide on the process. I have written extensively on this subject elsewhere but think it needs a home here too. Why would you want to do this? Running Windows as a VM with Xen and GPU passthrough allows you to use the same box for gaming as for storing TBs worth of media. I have done this for months and it works absolutely brilliantly 98% of the time providing you set things up in a very specific way. With this setup you still have the host graphics showing a blinking cursor and connect your HDMI cable (or DVI or whatever) to the port directly of the passed through graphics card. Want an example of performance? See the video below (spoiler: it's the same as bare metal to the naked eye!!!!!). Hardware requirements To make this work there are some pretty specific hardware requirements. I am only 100% sure of the requirements for Intel, so if you're an AMD guy feel free to post below and help a brother out. CPU / Motherboard For GPU passthrough your CPU, Motherboard and more specifically your exact BIOS revision must support Intel's vt-d technology. No if's, no buts. Sorry. Intel K series processors are generally not supported. I will be starting a known-good hardware sticky thread soon so people can post their working hardware configs for all to see to give you a better idea of what works and what doesn't. Graphics Card In addition to the CPU / Motherboard your graphics card must be one of an elite club! AMD cards generally have a better time passing through than their nVidia cousins. I have successfully passed through a Radeon HD 6450 and HD 7970. Nvidia cards may only be passed through if you flash them to be a 'quaddro' series card, I have no experience of this but would welcome others that have contributing (and I'll edit this section). I have never tried passing through the Intel graphics of the CPU to Windows, if you have let me know and I'll update this section with the results. Installation of Windows I have had very little success with Windows 7. I would recommend only using Windows 8. YMMV. I refer specifically to GPU passthrough only here, 7 works as a VM without GPU just fine. You will be running Windows as a PV on HVM guest. Your Xen .cfg file dictates everything that the VM is / does, and requires all the below parameters to be left in tact to work. This includes the tablet bit as this refers to the VNC viewer and should be left as is apart from the disk section which you'll need to configure. I have seen some crazy people add gfx_passthrough=1 etc, I can tell you this makes no difference and the config provided below is sold, tested and works 100%. Do not change it. For the best performance, ideally, you want to run the Windows VM from within an LVM container but at the moment unRAID does not support this. So for now, we'll just use a .img container despite the performance hit. To create an .img use the truncate command (the example below creates a 30gb file, change it to suit your needs): truncate -s 30G /mnt/cache/whateveryouwant/windows.img Installation of Windows itself is conducted over VNC. On Windows use TightVNC viewer, Mac use Chicken of the VNC however you must make sure each VNC viewer is set to raw encoding or you will have issues. The VNC viewer will reset on reboots or at random occasionally, this is normal. Also normal, is the fact that once your GPU passthrough is working is that the VNC viewer will just be a blank black screen. windows.cfg file example, save this wherever you like. name = 'windows' builder = 'hvm' vcpus = '4' memory = '8192' device_model_version="qemu-xen-traditional" disk = [ 'phy:/mnt/cache/whateveryouwant/windows.img,hda,w', # 'phy:/dev/cache/secondexample/example.img,hdb,w', 'file:/mnt/user/unraidarray/Software/Windows/Windows8_1Pro.iso,hdc:cdrom,r' ] vif = [ 'mac=00:16:3E:51:20:4C,bridge=br0,model=e1000' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' boot = 'dc' acpi = '1' apic = '1' viridian = '1' xen_platform_pci='1' sdl = '0' vnc = '1' vnclisten = '0.0.0.0' vncpasswd = '' stdvga = '0' usb = '1' usbdevice = 'tablet' #pci = ['01:00.0','01:00.1','00:1d.0'] Once you've created your .img and .cfg files, you are ready. We will install Windows entirely over VNC without any devices passed through and get to a desktop before doing anything tricky. Run the following command to create the VM, updating the path to the .cfg file as appropriate: xl create /boot/vm/windows.cfg Then fire up your VNC viewer and type in the IP address of the unRAID server and connect, the credentials for VNC can be adjusted in the .cfg file but are blank by default. Conduct your Windows installation as normal from this point over VNC. Reboot as normal, install any and all updates before attempting passthrough. Once your install is complete, updates are installed, shutdown the Windows VM completely. This is where the passthrough foo begins. **stopping here for tonight. will update this later when I have more time **
February 5, 201412 yr Disclaimer: I'm not an AMD guy and I haven't done passthrough on an AMD system, but I have researched it a lot, so thought I would share. That said, the requirements for IOMMU with AMD are really no different than they are for Intel, only the acronyms are different. You still need a CPU, mobo, and BIOS that support it. Intel VT-x == AMD-V -> required for HVM Intel VT-d == AMD-Vi -> required for IOMMU (passthrough)
February 5, 201412 yr I just successfully installed Vista* 64bit in a VM with no apparent issues. Intel i7-4820K supports vt-x * It was my only full-blown Windows install not being utilized or deactivated due to upgrade
February 5, 201412 yr Forgot to mention that I had no issues installing Windows 7 Home Premium using an ISO created with ImgBurn.
February 5, 201412 yr Author Forgot to mention that I had no issues installing Windows 7 Home Premium using an ISO created with ImgBurn. What about the passthrough though? That's where it always fell down for me. If it worked, which GPU, CPU etc?
February 5, 201412 yr What about the passthrough though? That's where it always fell down for me. If it worked, which GPU, CPU etc? Unfortunately, my MB doesn't seem to support passing video through or at least not with Radeon HD 8500 Series and fails on XL create. I haven't tried yet to pass through any other PCI ID's such as USB controllers to see if it is indeed the MB.
February 5, 201412 yr Author What about the passthrough though? That's where it always fell down for me. If it worked, which GPU, CPU etc? Unfortunately, my MB doesn't seem to support passing video through or at least not with Radeon HD 8500 Series and fails on XL create. I haven't tried yet to pass through any other PCI ID's such as USB controllers to see if it is indeed the MB. Cool. Just to be clear I was referring to win7 issues specifically with passthrough. I stand by my claims you will not get it to work in 7, only 8 and up. Sent from my Nexus 5 using Tapatalk
February 6, 201412 yr What about the passthrough though? That's where it always fell down for me. If it worked, which GPU, CPU etc? Unfortunately, my MB doesn't seem to support passing video through or at least not with Radeon HD 8500 Series and fails on XL create. I haven't tried yet to pass through any other PCI ID's such as USB controllers to see if it is indeed the MB. Cool. Just to be clear I was referring to win7 issues specifically with passthrough. I stand by my claims you will not get it to work in 7, only 8 and up. Sent from my Nexus 5 using Tapatalk I hope not because my plans were to drop the 3 ESXi servers to unRAID. Each ESXi box has 1 unRAID VM and 1 Win7 x64 VM each. They are headless in that I just RDP into them (Win7 VMs) but I have Tuners passed through to each in ESXi so I wouldn't want to loose that. I will not install Win8 on them. One and most important I don't have the licenses. Two I hate Metro and the new FLAT look that goes with it for all of the apps. I even prefer the look of XP over Win7 but it wasn't the big departure from XP that Win8 is.
February 6, 201412 yr * GPU passthrough Sent from my Nexus 5 using Tapatalk Thanks. That was what I was hoping you would say.
February 6, 201412 yr What about the passthrough though? That's where it always fell down for me. If it worked, which GPU, CPU etc? Unfortunately, my MB doesn't seem to support passing video through or at least not with Radeon HD 8500 Series and fails on XL create. I haven't tried yet to pass through any other PCI ID's such as USB controllers to see if it is indeed the MB. Cool. Just to be clear I was referring to win7 issues specifically with passthrough. I stand by my claims you will not get it to work in 7, only 8 and up. Sent from my Nexus 5 using Tapatalk I hope not because my plans were to drop the 3 ESXi servers to unRAID. Each ESXi box has 1 unRAID VM and 1 Win7 x64 VM each. They are headless in that I just RDP into them (Win7 VMs) but I have Tuners passed through to each in ESXi so I wouldn't want to loose that. I will not install Win8 on them. One and most important I don't have the licenses. Two I hate Metro and the new FLAT look that goes with it for all of the apps. I even prefer the look of XP over Win7 but it wasn't the big departure from XP that Win8 is. For the sake of argument, Microsoft dropped the name metro when referring to its UI, second, 8.1 also when a ways in reducing the need to look at it by allowing you to boot to a desktop instead. Your reasoning was solid when you just mentioned the licensing issue you face. There are also free apps out there that give you a real start menu.
February 25, 201412 yr I just successfully installed Vista* 64bit in a VM with no apparent issues. Intel i7-4820K supports vt-x * It was my only full-blown Windows install not being utilized or deactivated due to upgrade jbartlett, would you be so kind to share your cfg file? Quite some of us do have issues with getting an IP as you can see here: http://lime-technology.com/forum/index.php?topic=31674.75 I attached my cfg that I changed back to the original as proposed by Ironicbadger. Thanks a lot
February 25, 201412 yr Author I just successfully installed Vista* 64bit in a VM with no apparent issues. Intel i7-4820K supports vt-x * It was my only full-blown Windows install not being utilized or deactivated due to upgrade jbartlett, would you be so kind to share your cfg file? Quite some of us do have issues with getting an IP as you can see here: http://lime-technology.com/forum/index.php?topic=31674.75 I attached my cfg that I changed back to the original as proposed by Ironicbadger. Thanks a lot It's not your config file. The one I've posted works 100% of the time here, using a gargoyle based router. But it works at my parents house too, no issues. It's either going to be your router or bridge-utils config in unRaid. Please don't hijack this thread, and leave your issues to your own thread. Thanks!
February 25, 201412 yr I just successfully installed Vista* 64bit in a VM with no apparent issues. Intel i7-4820K supports vt-x * It was my only full-blown Windows install not being utilized or deactivated due to upgrade jbartlett, would you be so kind to share your cfg file? Quite some of us do have issues with getting an IP as you can see here: http://lime-technology.com/forum/index.php?topic=31674.75 I attached my cfg that I changed back to the original as proposed by Ironicbadger. Thanks a lot How many VMs have you created with this config? Have you tried changing the MAC? I have never heard of a router not giving an IP because it didnt like the MAC, unless it was a duplicate MAC.
February 25, 201412 yr this is great, im just waiting for the "DA" guide, i got 6 beta 3, and my windows 8 iso, now what?
February 25, 201412 yr Author Dude, most of what you need is in post #1... The rest you'll have to figure out from my blog. The info is all there, you need just a little bit of time / effort to implement it.
February 25, 201412 yr Dude, most of what you need is in post #1... The rest you'll have to figure out from my blog. The info is all there, you need just a little bit of time / effort to implement it. Good setting up my test raid..
March 12, 201412 yr Hey, been trying this for a few days now and seem to have run into quite some issues. Firstly my build; Mainboard: Asus Sabertooth FX990 R2.0 Memory: 8GB Crucial Ballistix Sport Drives: 2x4TB WD Red Running: unRaid 6 beta 3 Gpu: GTX260 (Primary) AMD HD6970 (Secondary) No Raid Controller, using onboard My syslinux.cfg; root@Tower:/mnt/user/VI# cat /boot/syslinux/syslinux.cfg default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest label Xen/unRAID OS kernel /syslinux/mboot.c32 append /xen dom0_mem=2097152 --- /bzimage --- /bzroot label Xen/unRAID Passthrough Test 01000 00142 menu default kernel /syslinux/mboot.c32 append /xen dom0_mem=2097152 --- /bzimage xen-pciback.hide=(07:00.0)(07:00.1)(09:05.0)(09:05.1)(09:05.2) --- /bzroot label Xen/unRAID OS Safe Mode (no plugins) kernel /syslinux/mboot.c32 append /xen dom0_mem=2097152 --- /bzimage --- /bzroot unraidsafemode My VI Config; name = 'windows' builder = 'hvm' vcpus = '4' memory = '6144' device_model_version="qemu-xen-traditional" disk = [ 'phy:/mnt/user/VI/windows.img,hda,w', # 'phy:/dev/cache/secondexample/example.img,hdb,w', # 'file:/mnt/user/VI/Win_Ent_8.1_64BIT.ISO,hdc:cdrom,r' ] vif = [ 'mac=00:16:3E:51:20:4C,bridge=br0,model=e1000' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' boot = 'dc' acpi = '1' apic = '1' viridian = '1' xen_platform_pci='1' sdl = '0' vnc = '1' vnclisten = '0.0.0.0' vncpasswd = '' stdvga = '0' usb = '1' usbdevice = 'tablet' pci = ['07:00.0','07:00.1','09:05.0','09:05.1','09:05.2'] lspci (devices I need, NEC is a PCI USB addon card for keyboard & mouse); root@Tower:/mnt/user/VI# lspci 00:00.0 Host bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02) 00:00.2 IOMMU: AMD/ATI [Advanced Micro Devices, Inc.] RD990 I/O Memory Management Unit (IOMMU) 00:02.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (PCI express gpp port B) 00:04.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (PCI express gpp port D) 00:05.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (PCI express gpp port E) 00:09.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (PCI express gpp port H) 00:0a.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (external gfx1 port A) 00:0b.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (NB-SB link) 00:0c.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890S PCI Express bridge for GPP2 port 1 00:0d.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (external gfx1 port B) 00:11.0 SATA controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40) 00:12.0 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller 00:12.2 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB EHCI Controller 00:13.0 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller 00:13.2 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB EHCI Controller 00:14.0 SMBus: AMD/ATI [Advanced Micro Devices, Inc.] SBx00 SMBus Controller (rev 42) 00:14.2 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] SBx00 Azalia (Intel HDA) (rev 40) 00:14.3 ISA bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40) 00:14.4 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SBx00 PCI to PCI Bridge (rev 40) 00:14.5 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller 00:15.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0) 00:15.1 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1) 00:15.2 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB900 PCI to PCI bridge (PCIE port 2) 00:15.3 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB900 PCI to PCI bridge (PCIE port 3) 00:16.0 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller 00:16.2 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB EHCI Controller 00:18.0 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 0 00:18.1 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 1 00:18.2 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 2 00:18.3 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 3 00:18.4 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 4 00:18.5 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 5 01:00.0 VGA compatible controller: NVIDIA Corporation GT200 [GeForce GTX 260] (rev a1) 02:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) 03:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) 04:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller 07:00.0 VGA compatible controller: AMD/ATI [Advanced Micro Devices, Inc.] Cayman XT [Radeon HD 6970] 07:00.1 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] Cayman/Antilles HDMI Audio [Radeon HD 6900 Series] 09:05.0 USB controller: NEC Corporation OHCI USB Controller (rev 43) 09:05.1 USB controller: NEC Corporation OHCI USB Controller (rev 43) 09:05.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 04) 0b:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller (rev 09) 0c:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller 0d:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller xl pci-assignable-list: root@Tower:/mnt/user/VI# xl pci-assignable-list 0000:07:00.0 0000:07:00.1 0000:09:05.0 0000:09:05.1 0000:09:05.2 Note, this is running on the raid as of now (without a parity drive) because I'm migrating data from my current storage to the new unRaid box before I start using it. So to the issues; 1. No picture on secondary card when starting the Windows VM, the system gets unresponsive and reboots the entire system. I had picture at start, but windows crashed (because I didn't install the gpu driver first), now its gone. 2. I tried creating a identical VI (except names of VM and .cfg), Install, reboot after initial windows install, then boots the entire system (pci devices are commented out in .cfg and at boot) 3. I get into my shell after install, try to boot the VM i just tried installing, system crashes and reboots. Cant really see why it behaves like this. Nothing has really changed. IOMMU is turned on and dmesg shows it working (I'm on the v2005 bios version were this now works correctly) Any ideas?
March 13, 201412 yr Hey, been trying this for a few days now and seem to have run into quite some issues. Firstly my build; Mainboard: Asus Sabertooth FX990 R2.0 Memory: 8GB Crucial Ballistix Sport Drives: 2x4TB WD Red Running: unRaid 6 beta 3 Gpu: GTX260 (Primary) AMD HD6970 (Secondary) No Raid Controller, using onboard My syslinux.cfg; root@Tower:/mnt/user/VI# cat /boot/syslinux/syslinux.cfg default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest label Xen/unRAID OS kernel /syslinux/mboot.c32 append /xen dom0_mem=2097152 --- /bzimage --- /bzroot label Xen/unRAID Passthrough Test 01000 00142 menu default kernel /syslinux/mboot.c32 append /xen dom0_mem=2097152 --- /bzimage xen-pciback.hide=(07:00.0)(07:00.1)(09:05.0)(09:05.1)(09:05.2) --- /bzroot label Xen/unRAID OS Safe Mode (no plugins) kernel /syslinux/mboot.c32 append /xen dom0_mem=2097152 --- /bzimage --- /bzroot unraidsafemode My VI Config; name = 'windows' builder = 'hvm' vcpus = '4' memory = '6144' device_model_version="qemu-xen-traditional" disk = [ 'phy:/mnt/user/VI/windows.img,hda,w', # 'phy:/dev/cache/secondexample/example.img,hdb,w', # 'file:/mnt/user/VI/Win_Ent_8.1_64BIT.ISO,hdc:cdrom,r' ] vif = [ 'mac=00:16:3E:51:20:4C,bridge=br0,model=e1000' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' boot = 'dc' acpi = '1' apic = '1' viridian = '1' xen_platform_pci='1' sdl = '0' vnc = '1' vnclisten = '0.0.0.0' vncpasswd = '' stdvga = '0' usb = '1' usbdevice = 'tablet' pci = ['07:00.0','07:00.1','09:05.0','09:05.1','09:05.2'] lspci (devices I need, NEC is a PCI USB addon card for keyboard & mouse); root@Tower:/mnt/user/VI# lspci 00:00.0 Host bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02) 00:00.2 IOMMU: AMD/ATI [Advanced Micro Devices, Inc.] RD990 I/O Memory Management Unit (IOMMU) 00:02.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (PCI express gpp port B) 00:04.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (PCI express gpp port D) 00:05.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (PCI express gpp port E) 00:09.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (PCI express gpp port H) 00:0a.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (external gfx1 port A) 00:0b.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (NB-SB link) 00:0c.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890S PCI Express bridge for GPP2 port 1 00:0d.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] RD890 PCI to PCI bridge (external gfx1 port B) 00:11.0 SATA controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40) 00:12.0 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller 00:12.2 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB EHCI Controller 00:13.0 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller 00:13.2 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB EHCI Controller 00:14.0 SMBus: AMD/ATI [Advanced Micro Devices, Inc.] SBx00 SMBus Controller (rev 42) 00:14.2 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] SBx00 Azalia (Intel HDA) (rev 40) 00:14.3 ISA bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40) 00:14.4 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SBx00 PCI to PCI Bridge (rev 40) 00:14.5 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller 00:15.0 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0) 00:15.1 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1) 00:15.2 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB900 PCI to PCI bridge (PCIE port 2) 00:15.3 PCI bridge: AMD/ATI [Advanced Micro Devices, Inc.] SB900 PCI to PCI bridge (PCIE port 3) 00:16.0 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller 00:16.2 USB controller: AMD/ATI [Advanced Micro Devices, Inc.] SB7x0/SB8x0/SB9x0 USB EHCI Controller 00:18.0 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 0 00:18.1 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 1 00:18.2 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 2 00:18.3 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 3 00:18.4 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 4 00:18.5 Host bridge: AMD [Advanced Micro Devices, Inc.] Family 15h Processor Function 5 01:00.0 VGA compatible controller: NVIDIA Corporation GT200 [GeForce GTX 260] (rev a1) 02:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) 03:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) 04:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller 07:00.0 VGA compatible controller: AMD/ATI [Advanced Micro Devices, Inc.] Cayman XT [Radeon HD 6970] 07:00.1 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] Cayman/Antilles HDMI Audio [Radeon HD 6900 Series] 09:05.0 USB controller: NEC Corporation OHCI USB Controller (rev 43) 09:05.1 USB controller: NEC Corporation OHCI USB Controller (rev 43) 09:05.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 04) 0b:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller (rev 09) 0c:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller 0d:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller xl pci-assignable-list: root@Tower:/mnt/user/VI# xl pci-assignable-list 0000:07:00.0 0000:07:00.1 0000:09:05.0 0000:09:05.1 0000:09:05.2 Note, this is running on the raid as of now (without a parity drive) because I'm migrating data from my current storage to the new unRaid box before I start using it. So to the issues; 1. No picture on secondary card when starting the Windows VM, the system gets unresponsive and reboots the entire system. I had picture at start, but windows crashed (because I didn't install the gpu driver first), now its gone. 2. I tried creating a identical VI (except names of VM and .cfg), Install, reboot after initial windows install, then boots the entire system (pci devices are commented out in .cfg and at boot) 3. I get into my shell after install, try to boot the VM i just tried installing, system crashes and reboots. Cant really see why it behaves like this. Nothing has really changed. IOMMU is turned on and dmesg shows it working (I'm on the v2005 bios version were this now works correctly) Any ideas? Have you tried shutting down the machine, switching off the psu and removing the power cable for 20 seconds? I've found that if my win vm with gpu passthrough does not shutdown properly / crashes, a full power cycle is required to 'reset' the gpu. It's a royal pain. You'll also need to set up ironics start / stop scripts to ensure ti can restart cleanly in future. Peter
March 13, 201412 yr Author I've never needed a full power cycle. A reboot is enough to preform the flr reset. Sent from my S4 via Tapatalk
March 13, 201412 yr Have you tried shutting down the machine, switching off the psu and removing the power cable for 20 seconds? I've found that if my win vm with gpu passthrough does not shutdown properly / crashes, a full power cycle is required to 'reset' the gpu. It's a royal pain. You'll also need to set up ironics start / stop scripts to ensure ti can restart cleanly in future. Peter Hi Peter, I havent tried removing the powercord but I'll give it a go just to make sure. Thanks for the tip! And I have not setup the start / stop scripts, but I'll give it a go this afternoon. As Ironicbadger mentions he never needed to do this, so could be it's needed for my build for some reason. There has on the other hand been some developments, I was fiddling about with this last night and got some progress out of it. I did a review of my BIOS settings, and found some extra functions I then disabled (mostly powersaving functions), and my host stopped rebooting when I shut down a VM, profit! I also use Ironicbadgers ArchVM appliance as a MySQL server for my XBMC library, ran this at the same time as I installed a new VM (below), just to see what happened. I did a complete reinstall of a windows 8.1 VM, I made a new .cfg and a new .img, Installed perfectly, rebooted, host didn't reboot. After the reboot I installed the AMD Catalyst driver, then I shut down the VM, stopped my array and rebooted the host, started the Host with the pci-hide options. Checked xl pci-assignable-list, and double checked my .cfg for spelling errors. Then I started the VM, removed my video cable from my primary card (the gtx260) and put it in my secondary (HD6970). Just a black screen, so I started Chicken of the VNC, connected and the VM was up. I went to device manager, and I had a GPU with no drivers, I checked the PCI_VEN via the device info and it is indeed an AMD HD69xx device. I tried installing the drivers manually by searching for driver in C:\AMD (which is the location the catalyst installer extracts them). It went ok for a while, and suddenly my mousepointer showed up on my display, some more time went by and I got a BSOD on my VNC session, reported as "video_tdr_failure atikmpag.sys". System rebooted after a little while and screen is black again, in VNC it seems windows does a rollback. Getting there it seems, thanks again for you input and suggestions
Archived
This topic is now archived and is closed to further replies.