November 5, 201411 yr unRAID OS Version: v6b10a Description: Xen GPU pass through not working with xen 4.4 and other issues reported - BSOD on Vms that were working with Xen 4.3 How to reproduce: Can't get GPU pass through to work. Drivers in Windows do not install properly. Expected results: GPU pass through to work. Actual results: GPU pass through not working. Other information: Xen 4.3 was working for many unRAID users, but is no longer working. Many reports on the Internet of broken GPU pass through in Xen 4.4. Solution: Roll back Xen to previous 4.3 version until Xen works out GPU pass through.
November 5, 201411 yr This may be resolved with Xen 4.5 which is due out soon, but driver issues could also be the result of newer version drivers not being compatible. Can you get this working with a Linux-based VM? Does the GPU never pass through, or does it pass through ok, but when you apply drivers, you get the BSOD?
November 5, 201411 yr This is not a global issue as I have no problems with GPU passthrough on UnRaid 6b10, multiple cards to multiple VMs, both windows 8 and Linux (net runner/manjaro). Cards are AMD HD5xxxx and HD 6xxx series. It's important that you install ONLY the drivers, not the full catalyst control suite (there's an option in the CCC installer to do this) It's important that you 'eject' the GPU before Windows VM shutdown or restart. If you don't, the GPU likely won't work on subsequent VM boots until you power cycle the entire host system (which resets the GPU). There are scripts available be to do this automagically on startup and shutdown, or you can do it manually via device manager while testing. It may be that some of the 'pass through not working'' reports are related to this. Peter
November 5, 201411 yr Author The issue is exactly this. Some have it working, most have issues. There seems to be some magic combination of hardware/software that gets it working. I've read that Hyperthreading being on could be an issue. I may try to turn that off and try again when I have the time to invest. Refer to this: http://lime-technology.com/forum/index.php?topic=35223.0. Reverting back to Xen 4.3 solved the GPU pass through for them. I have put way too many hours into this. Look at my signature for my hardware configuration. Yes, I have vt-d turned on in the bios, yes my CPU is vt-d capable. The GPU passes through and is recognized and the driver loads . The driver says there are not enough resources to get the driver working properly. I've disabled other GPU drivers and looked for conflicts. Still no go. I've tried an Intel integrated GPU, and an AMD HD5450 with no luck. There are documented issues with Xen 4.4 and these driver issues. I have not tried Linux, only Windows. I have no interest in a Linux GPU pass through and won't invest any time into it. If Xen 4.5 doesn't solve the problem, LT should revert back to 4.3 and not try to troubleshoot Xen 4.5.
November 6, 201411 yr Author meep, Can you post your config file so I can see what works for you? Can you describe your processor? I find that the amount of memory for the vm also affects the pass through. If I set 3072 for memory I get a page fault. This is a Windows 7 VM I am working with.
November 17, 201411 yr Author I read in a Xen blog post that the problem with vga pass through with 4.4 is recognized. It appears that 4.5 may fix the problem. Hopefully LT can get Xen 4.5 included before v6 final. I have multiple Xen VMs running perfectly and would like to get a vga pass through for my Windows 7 Media Center VM. EDIT: Here is the blog talking about the vga pass through issue: http://www.gossamer-threads.com/lists/xen/devel/351336?page=last
November 17, 201411 yr Interesting find. It seems the issue is limited only to Xen 4.4.1, so either updating to 4.5 or downgrading back to 4.4.0 or 4.3 would solve the issue.
November 21, 201411 yr Yes. I think that is correct. http://wiki.xenproject.org/wiki/Xen_Project_Hypervisor_Roadmap/4.5 When it hits, you can expect that we will add it, but note the release time-frame is December right now. In the meantime, I'd suggest trying out KVM. I'm not sure if Xen is going to convince me they can do better than KVM + VirtIO + VirtFS + VFIO. I'm a really big fan of where that technology has been and is headed. It is a more hardware-native solution. That said, integrated graphics pass through would be a really big win for the Xen team if they could get that in there and successful on a wide array of chipsets. I'd also like to see them move to OVMF with UEFI support for VM booting, and if we could get that in there, Xen may have a long-term home with unRAID. For now, we patch and maintain it, but we are not spending our efforts as much there as we have been with KVM. We're seeing near bare-metal performance for the most intensive workloads and will be featuring demos in an upcoming blog post on the matter, all featuring Docker and KVM.
November 21, 201411 yr Author Waiting for Xen to get pass through working and just maintaining it for us is reasonable. Pass through is not important enough for me to invest time in learning KVM and moving VMs from Xen to KVM. My Xen VMs are working perfect. In fact my Windows 7 Media Center with my Extenders has never worked better.
November 21, 201411 yr Yeah, outside this version specific hiccup, my Xen Windows VM (and Arch before I made the Docker transition) have been absolutely rock-solid. There is just minimal motivation to learn a new hypervisor when I can't see any room for improvement. I expect I'll make the transition one day - the focus on KVM by the unRAID team will likely dictate it. I'm just in no hurry until the detailed guides are posted to make it idiot-proof. For now, I've got a 24/7 Windows VM w/GPU passthrough that has had months with no downtime outside the short period before I rolled back to the 4.4.0 package.
November 23, 201411 yr Author I was able to get an Intel IGD to pass through. See here: http://lime-technology.com/forum/index.php?topic=36406.0 What I've learned: - The upstream device model is not working fully. I had to use the traditional model to get it to work. - The device model has to be set when setting up the VM and cannot be changed after the fact. - The traditional model does not recognize a qcow2 image. I had to use a raw image. - Xen 4.4.1 did work, but had performance issues. e.g Jerkey video and choppy sound. Xen 4.4.0 worked better. - I was able to increase memory for the VM to 3072G without issues and will try increasing to 4096G. Memory size has been reported as an issue. EDIT: I did end up with a video issue when I plugged in a USB hard drive for a SNAP backup. I don't fully understand this issue, but I backed off the memory to 2048G. It appears some of the video memory got corrupted. - The GPU has to be the default in the BIOS for a primary pass through. i.e. the BIOS initializes the GPU when unRAID is booted. - The GPU resets itself properly on reboots. Hope this helps LT understand what the GPU pass through issues are on Xen.
Archived
This topic is now archived and is closed to further replies.