[SOLVED] Xen GPU Passthrough -- Solution: IOMMU disabled in BIOS

[m4f1050]

Howdy...  I modified my Xen syslinux.cfg and added xen-pciback.hide=(01:00.0)(01:00.1) and on my vm .cfg file I added gfx_passthru = 1 and pci = ['01:00.0','01:00.1] and I am getting under Device Manager a "Xen pci device #0"

 

When I run my GTS 450 installation it does not detect my video card, how can I get it to work?

 

Thanks!

[meep]

Sounds like you're doing it right.

 

Can you post the full contents of both Config files and the output of lspci?

 

Also, I use AMD cards myself wih great success. There's something in the corner of my memory telling me there might be an issue passing through some NVIDIA cards and that they need firmware flashing to work? Not an expert here so someone else might chime in.

[m4f1050]

Thanks for the advice, I found this:

 

http://www.altechnative.net/2013/11/25/virtualized-gaming-nvidia-cards-part-3-how-to-modify-a-fermi-based-geforce-into-a-quadro-geforce-gts450gtx470gtx480-to-quadro-200050006000/

 

Looks like I have to hack the firmware and set it as a Quadro 2000 before I can use it.  Oh boy.....  Time to pull out the card and an do some hacking and sawing! 

[m4f1050]

Still a no-go....

 

On VNC I get:

serial0 console

 

Here is my lspci -k with the PCI's I'm passing:

 

00:14.2 Audio device: AMD/ATI [Advanced Micro Devices, Inc.] SBx00 Azalia (Intel HDA) (rev 40)
        Subsystem: ASRock Incorporation Device 7892
01:00.0 VGA compatible controller: NVIDIA Corporation GF106GL [Quadro 2000] (rev a1)
        Subsystem: eVga.com. Corp. Device 1352
        Kernel driver in use: pciback
01:00.1 Audio device: NVIDIA Corporation GF106 High Definition Audio Controller (rev a1)
        Subsystem: eVga.com. Corp. Device 1352
        Kernel driver in use: pciback
06:06.0 USB controller: NEC Corporation OHCI USB Controller (rev 43)
        Subsystem: Belkin Root Hub
        Kernel driver in use: pciback
06:06.1 USB controller: NEC Corporation OHCI USB Controller (rev 43)
        Subsystem: Belkin Root Hub
        Kernel driver in use: pciback
06:06.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 04)
        Subsystem: Belkin Root Hub
        Kernel driver in use: pciback
08:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6315 Series Firewire Controller (rev 01)
        Subsystem: ASRock Incorporation Device 3403
        Kernel driver in use: pciback

 

I converted my GTS450 into a Quadro 2000 with a USB dos boot disk and nvflash:

nvflash --index=X --straps 0x7FFC2FFF 0x10006000 0x7fffffff 0x00000000

 

Is there a problem with "pciback"?  How can I fix this?

[meep]

from command line, before booting VM, try this

 

xl pci-assignable-list

 

this will let you know if the device has been correctly hidden and is available for assignment. If it's not listed, the problem is with the    pciback setting is syslinux.cfg. If it does appear, the problem is with the VM config / passthrough / hardware.

 

If it does appear, start the VM and run that command again. If it's getting assigned to the VM, it should disappear from the reported assignable list.

 

The '"Xen pci device #0" is NOT your GPU. This is some kind of default device that always shows up but never gets a driver. In my experience, when a GPU is correctly assigned but has no drivers loaded, it shows up under graphics cards in device manager (assuming windows).

 

Can't help much more beyond that as I've only ever passed through AMD cards.

 

Peter

 

 

 

 

[m4f1050]

Here is what I get:

 

root@MMPC:~# xl pci-assignable-list
0000:00:14.2
0000:01:00.0
0000:01:00.1
0000:06:06.0
0000:06:06.1
0000:06:06.2
0000:08:00.0
root@MMPC:~#

 

When I ran it after starting the VM I got:

 

root@MMPC:~# xl pci-assignable-list
root@MMPC:~#

[meep]

That's MOST unusual

 

As I understand it, xl pci-assignable-list should show only those devices that may be assigned.

 

In your first case, it should show only those devices that have been hidden using pciback.hide, not the full list of all PCI devices.

 

In the second case, it should show the list of assignable devices, less those that have been passed through to an active VM.

 

Can you post your full syslinux.cfg and CM.cfg, if you can?

 

Peter

 

[m4f1050]

That's MOST unusual

 

As I understand it, xl pci-assignable-list should show only those devices that may be assigned.

 

In your first case, it should show only those devices that have been hidden using pciback.hide, not the full list of all PCI devices.

 

In the second case, it should show the list of assignable devices, less those that have been passed through to an active VM.

 

Can you post your full syslinux.cfg and CM.cfg, if you can?

 

Peter

 

Nothing unusual, I'm hiding audio, video, video's audio, a pci usb card and firewire.

 

How exactly does "pciback" drivers work or how do I pass these to the VM?  What am I missing?

 

Here is my syslinux.cfg and my VM .cfg files:

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  kernel /bzimage
  append initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest
label Xen/unRAID OS
  menu default
  kernel /syslinux/mboot.c32
  append /xen dom0_mem=8192000 --- /bzimage xen-pciback.hide=(00:14.2)(01:00.0)(01:00.1)(06:06.0)(06:06.1)(06:06.2)(
08:00.0) --- /bzroot
(06:06.2)(08:00.0) /bzroot
label Xen/unRAID OS Safe Mode (no plugins)
  kernel /syslinux/mboot.c32
  append /xen --- /bzimage --- /bzroot unraidsafemode

 

kernel = 'hvmloader'
builder = 'hvm'
vcpus = '8'
memory = '8000'
device_model_version="qemu-xen-traditional"
disk = [
        'file:/mnt/cache/custom/Windows8.img,hda,w',
#        'phy:/dev/vg1/winDATA,hdb,w',
#        'file:/mnt/user0/Data/Install/Operating Systems/Windows/Win8_x64/Win8Pro-x64-en-US_Jan2013.iso,hdc:cdrom,r'
]
name = 'Windows8'
vif = [ 'mac=00:16:3e:41:fc:4a,bridge=br0,model=e1000' ]
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
boot = 'dc'
acpi = '1'
apic = '1'
viridian = '1'
xen_platform_pci='1'
sdl = '0'
vnc = '1'
vnclisten = '0.0.0.0'
vncdisplay = '1'
vncpasswd = 'password'
stdvga = '0'
usb = '1'
usbdevice = 'tablet'
gfx_passthru = '1'
pci = ['00:14.2','01:00.0','01:00.1','06:06.0','06:06.1','06:06.2','08:00.0']

[dlandon]

The most obvious issue to me is the memory.  Cut down the memory to 2048MB.  There are known issues with GPU pass through over 4096MB.  I have a VM running with GPU pass through and 3072MB for memory.

 

You are giving Xen 8192MB also,  That's quite a bit.  I've found 4096MB is more than enough.  This keeps Xen from ballooning down memory that can also cause issues.

 

Assign 2 VCPUS to the VM.  8 is unnecessary.

 

You can always adjust later once it us running.

 

See if this doesn't help:  http://lime-technology.com/forum/index.php?topic=36406.0

[meep]

 

 

Nothing unusual, I'm hiding audio, video, video's audio, a pci usb card and firewire.

 

How exactly does "pciback" drivers work or how do I pass these to the VM?  What am I missing?

 

Here is my syslinux.cfg and my VM .cfg files:

 

<snip>

 

 

Sorry, I understood from OP that you had hidden and were passing through (01:00.0)(01:00.1)  only.

 

What you're seeing from xl-assignable-list therefore matches what you describe.

 

On drivers, there are no drivers required in the host system (unraid) for devices hidden using pciback.hide.

 

Passing these devices through to the guest OS, they should appear there and prompt driver installation. So, in Windows, you should get either a default driver assigned or the 'found new hardware' process kicks off.

 

Can I ask, do any of your other devices that you're passing through show up in your VM? (pci USB and firewire?)

 

It just may be that the nvidia card is not working with Xen - I've had great success with AMD HD5xxx cards, regardles of memory allocation - I'm running several Win8 VMs with 4096 memory and GPUs passed through. I also have a Win7 VM with 12GB memory and a satellite tuner passed through (no VGA though - not required).

 

Those HD5xxx cards are very inexpensive if you wanted to source one to try out.

 

Finally, I don't include the 'gfx_passthrough' in my config files, if that's any help?

 

Peter

[m4f1050]

They are now.  Here is what happened:  I switched my motherboard to an ASRock 970 Extreme4.  The 1st option related to VM is security vm, which I enabled and I skipped the IOMMU (it was disabled.)

 

To my understanding I thought you couldn't run VM's if you didn't have IOMMU (I now know you can, but no passthrough, just VNC)

 

Upon enabling IOMMU on the BIOS everything worked, I saw the blue Windows 8 icon upon starting the VM and found new hardware popped up, I copied my USB 3.0 drivers to the desktop and installed them, then plugged in my CM15A USB X10 controller and got it working via my VPN, I can turn on/off all my lights from outside my house when I get home late at night.

 

Thanks for your help!

[meep]

Ha!

 

I just ordered a CM15!

 

I've had X10 for >10 years now and for some reason in the past 12 months, my wireless switches have stared acting up (singles not getting through). I'm planning to set up a VM with intelliremote and a CN15 to traffic network signals instead and will build a simple web server to drive chap android touchscreen for x10 and whole house audio control. (as well as dim the lights when the movies start ;-) )

 

Sounds like we have similar ideas!

 

Peter

[m4f1050]

You don't need to change your adapter.  You need line noise filters around the house where you have appliances that cause noise in the electric lines.  I have a CM11 and a CM15a one is always on for my X10 Commander (play store has the app to control the home remotely) but it needs an X10 Commander server running on a machine which I have a Windows 2008 R2 running all my Windows related services.  I have another VM (Windows I use the CM11 with a USB-to-serial adapter for when im on the machine I can turn on/off lights with the software, if you need the software I have them all, shoot me a PM with your email addy and I can send them to you.  I think you only have to buy X10 Commander.

[meep]

Well, I just can't figure out what's causing the interference. We did a major upgrade on our heating system about a year ago, put in solar panels and all kinds of control gubbins. It's about that time the problems with X10 RF started but it's all hard-wired and the thought of trying to figure out where to put filters boggles me.

 

I'm actually planning to write my own home-control server and front-end in php/html as I want to integrate lighting, security and whole-house audio in a single UI. I have a somewhat custom setup in that I use X10 for lights & appliances, Comfort for security and Logitech Media Server for audio - now that I have an always on server in an unRaid VM (Win 7 for TV/PVR), it should be straightforward enough to put together a few pages and pop some cheap android tabs as controllers about the house.

 

That being the case, I do need the CM15 to get all this patched together.

 

Way off topic here, sorry. I'll pop you a PM - would be interested in anything you have to share.

 

Peter

[m4f1050]

Sorry for the late reply.  I just started having issues with my proximity sensors and all I did was replace battery (which I also had to reprogram the house/unit codes) and all started working again.  Are your affected devices battery powered in any way?  I know one of the CM controllers has battery, not sure which....