Nvidia GPU passthrough and code 43 errors


Recommended Posts

I thought it was worth starting a new thread with this vital info.

 

Below is a portion of Alex Williamson's blog which discusses code 43 errors when passing through Nvidia GPU's to Windows VM's

 

link to page

Code: [select]

http://vfio.blogspot.ca/2014/08/vfiovga-faq.html

Quote

Question 10:

 

I'm assigning an Nvidia card to a Windows guest and get a Code 43 error in device manager.

 

Answer:

 

The Nvidia driver, starting with 337.88 identifies the hypervisor and disables the driver when KVM is found.  Nvidia claims this is an unintentional bug, but has no plans to fix it.  To work around the problem, we can hide the hypervisor by adding kvm=off to the list of cpu options provided (QEMU 2.1+ required).  libvirt support for this option is currently upstream.

 

Note that -cpu kvm=off is not a valid incantation of the cpu parameter, a CPU model such as host, or SandyBridge must also be provided, ex: -cpu host,kvm=off.

 

Update: The above workaround is sufficient for drivers 337.88 and 340.52.  With 344.11 and presumably later, the Hyper-V CPUID extensions supported by KVM also trigger the Code 43 error.  Disabling these extensions appears to be sufficient to allow the 344.11 driver to work.  This includes all of the hv_* options to -cpu.  In libvirt, this includes:

 

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

 

    </hyperv>

 

and

 

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

  </clock>

 

Unfortunately removing these options will impose a performance penalty as these paravirtual interfaces are designed to improve the efficiency of virtual machines.

Link to comment

The one thing I want to comment on is this:

 

Unfortunately removing these options will impose a performance penalty as these paravirtual interfaces are designed to improve the efficiency of virtual machines.

 

I am not 100% convinced the performance penalty here is enough for anyone to notice or care for our type of usage.  I've run with the 340.52 driver with hyperv optimizations and I've rolled with newer driver versions and removed those optimizations.  I've noticed little to no difference myself in terms of VM usability and general performance.  Maybe under benchmarking situations you'd notice something, but there's a big difference between noticing #'s change in a benchmark test and noticing app performance differences when you're using the system.

 

The biggest thing that I'm curious about is why NVIDIA hasn't used other detection mechanisms to cause code 43 errors.  If they wanted, they could be detecting more than just the hyper-v extensions.  I have a few theories on this, but they are still a bit premature to discuss.

 

The short term solution for folks passing through NVIDIA GPUs is simple:  either stick with the 340.52 driver or do not use Hyper-V extensions in your VMs.  Problem solved.

Link to comment
  • 4 months later...

Sorry for bumping an old thread.  I'm currently running in to this issue.

 

Where exactly do I put:

 

The Nvidia driver, starting with 337.88 identifies the hypervisor and disables the driver when KVM is found.  Nvidia claims this is an unintentional bug, but has no plans to fix it.  To work around the problem, we can hide the hypervisor by adding kvm=off to the list of cpu options provided (QEMU 2.1+ required).  libvirt support for this option is currently upstream.

 

And do I just remove this from my xml?

 

 

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

 

    </hyperv>

 

and

 

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

  </clock>

 

Thanks in advance for your help.  I can also provide a copy of my XML and logs if that would help.

 

Sincerely,

 

Rob

Link to comment

Sorry for bumping an old thread.  I'm currently running in to this issue.

 

Where exactly do I put:

 

The Nvidia driver, starting with 337.88 identifies the hypervisor and disables the driver when KVM is found.  Nvidia claims this is an unintentional bug, but has no plans to fix it.  To work around the problem, we can hide the hypervisor by adding kvm=off to the list of cpu options provided (QEMU 2.1+ required).  libvirt support for this option is currently upstream.

 

And do I just remove this from my xml?

 

 

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

 

    </hyperv>

 

and

 

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

  </clock>

 

Thanks in advance for your help.  I can also provide a copy of my XML and logs if that would help.

 

Sincerely,

 

Rob

 

If you used unRAID to create your VM and assign an NVIDIA GPU to it, we automatically take care of all this for you.

Link to comment

Sorry for bumping an old thread.  I'm currently running in to this issue.

 

Where exactly do I put:

 

The Nvidia driver, starting with 337.88 identifies the hypervisor and disables the driver when KVM is found.  Nvidia claims this is an unintentional bug, but has no plans to fix it.  To work around the problem, we can hide the hypervisor by adding kvm=off to the list of cpu options provided (QEMU 2.1+ required).  libvirt support for this option is currently upstream.

 

And do I just remove this from my xml?

 

 

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

 

    </hyperv>

 

and

 

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

  </clock>

 

Thanks in advance for your help.  I can also provide a copy of my XML and logs if that would help.

 

Sincerely,

 

Rob

 

If you used unRAID to create your VM and assign an NVIDIA GPU to it, we automatically take care of all this for you.

Well pewp....  I'm getting absolutely nothing out of my card.  No flickers.  Just a "no signal" from my monitor.  Doesnt matter what guest is I'm using.  I'm pretty sure I'm overlooking something. 

 

I'll post XML and logs when I get home tonight.

Link to comment

Sorry for bumping an old thread.  I'm currently running in to this issue.

 

Where exactly do I put:

 

The Nvidia driver, starting with 337.88 identifies the hypervisor and disables the driver when KVM is found.  Nvidia claims this is an unintentional bug, but has no plans to fix it.  To work around the problem, we can hide the hypervisor by adding kvm=off to the list of cpu options provided (QEMU 2.1+ required).  libvirt support for this option is currently upstream.

 

And do I just remove this from my xml?

 

 

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

 

    </hyperv>

 

and

 

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

  </clock>

 

Thanks in advance for your help.  I can also provide a copy of my XML and logs if that would help.

 

Sincerely,

 

Rob

 

If you used unRAID to create your VM and assign an NVIDIA GPU to it, we automatically take care of all this for you.

Well pewp....  I'm getting absolutely nothing out of my card.  No flickers.  Just a "no signal" from my monitor.  Doesnt matter what guest is I'm using.  I'm pretty sure I'm overlooking something. 

 

I'll post XML and logs when I get home tonight.

Have you tried all the suggestions in the wiki (see link in my signature for the manual)?

Link to comment

Sorry for bumping an old thread.  I'm currently running in to this issue.

 

Where exactly do I put:

 

The Nvidia driver, starting with 337.88 identifies the hypervisor and disables the driver when KVM is found.  Nvidia claims this is an unintentional bug, but has no plans to fix it.  To work around the problem, we can hide the hypervisor by adding kvm=off to the list of cpu options provided (QEMU 2.1+ required).  libvirt support for this option is currently upstream.

 

And do I just remove this from my xml?

 

 

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

 

    </hyperv>

 

and

 

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

  </clock>

 

Thanks in advance for your help.  I can also provide a copy of my XML and logs if that would help.

 

Sincerely,

 

Rob

 

If you used unRAID to create your VM and assign an NVIDIA GPU to it, we automatically take care of all this for you.

Well pewp....  I'm getting absolutely nothing out of my card.  No flickers.  Just a "no signal" from my monitor.  Doesnt matter what guest is I'm using.  I'm pretty sure I'm overlooking something. 

 

I'll post XML and logs when I get home tonight.

Have you tried all the suggestions in the wiki (see link in my signature for the manual)?

Going through that, nothing is standing out that I missed.  I'll go through it again step by step tonight

Link to comment

Sorry for bumping an old thread.  I'm currently running in to this issue.

 

Where exactly do I put:

 

The Nvidia driver, starting with 337.88 identifies the hypervisor and disables the driver when KVM is found.  Nvidia claims this is an unintentional bug, but has no plans to fix it.  To work around the problem, we can hide the hypervisor by adding kvm=off to the list of cpu options provided (QEMU 2.1+ required).  libvirt support for this option is currently upstream.

 

And do I just remove this from my xml?

 

 

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

 

    </hyperv>

 

and

 

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

  </clock>

 

Thanks in advance for your help.  I can also provide a copy of my XML and logs if that would help.

 

Sincerely,

 

Rob

 

If you used unRAID to create your VM and assign an NVIDIA GPU to it, we automatically take care of all this for you.

Well pewp....  I'm getting absolutely nothing out of my card.  No flickers.  Just a "no signal" from my monitor.  Doesnt matter what guest is I'm using.  I'm pretty sure I'm overlooking something. 

 

I'll post XML and logs when I get home tonight.

Have you tried all the suggestions in the wiki (see link in my signature for the manual)?

Going through that, nothing is standing out that I missed.  I'll go through it again step by step tonight

Have you tried both OVMF and SeaBIOS?  Have you tried loading the GPU ROM?

Link to comment

Sorry for bumping an old thread.  I'm currently running in to this issue.

 

Where exactly do I put:

 

The Nvidia driver, starting with 337.88 identifies the hypervisor and disables the driver when KVM is found.  Nvidia claims this is an unintentional bug, but has no plans to fix it.  To work around the problem, we can hide the hypervisor by adding kvm=off to the list of cpu options provided (QEMU 2.1+ required).  libvirt support for this option is currently upstream.

 

And do I just remove this from my xml?

 

 

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

 

    </hyperv>

 

and

 

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

  </clock>

 

Thanks in advance for your help.  I can also provide a copy of my XML and logs if that would help.

 

Sincerely,

 

Rob

 

If you used unRAID to create your VM and assign an NVIDIA GPU to it, we automatically take care of all this for you.

Well pewp....  I'm getting absolutely nothing out of my card.  No flickers.  Just a "no signal" from my monitor.  Doesnt matter what guest is I'm using.  I'm pretty sure I'm overlooking something. 

 

I'll post XML and logs when I get home tonight.

Have you tried all the suggestions in the wiki (see link in my signature for the manual)?

Going through that, nothing is standing out that I missed.  I'll go through it again step by step tonight

Have you tried both OVMF and SeaBIOS?  Have you tried loading the GPU ROM?

Yeah, I have a windows based seabios and a Ubuntu based ovfm.  Can access both with rdp and vnc respectively.  I have the ROM passed through on the windows vm but not the Ubuntu. That got rid of the "invalid" rom contents error in the log. 

 

Both are still exhibiting the same behavior.  No signal like the HDMI cable isn't even plugged in.

Link to comment

Odd. In device manager in windows, is the device showing up?

Well I got it to work.  The work around is not ideal.  But it's up and running.  Basically, my motherboard is one of the ones that does not like to pass through primary PCI-E slot. 

 

When I had my EVGA 750ti in the primary PCI-E Slot I had all the problems as above.

 

I plugged in an old ati 5750 in the pci express 2.0 Slot and created a new VM. Boom, booted right up when using the secondary GPU.

 

Switched GPU's.  Put the ati in PCI-E 3.0 and the EVGA in PCI-E 2.0 and then the EVGA worked fine and the ati had no screen.

 

Tried booting with the EVGA in PCI-E 2.0 and nothing in PCI-E 3.0. And no dice.

 

So basically I have to have a GPU that I'm not going to use in the PCI-E 3.0 slot and the GPU I'm going to pass through in the PCI-E 2.0 slot and everything works great.  Although now I'm out of PCI-E slots which is going to affect future upgrade paths but I'm

Just glad it's working now.

 

Should be mentioned for anyone who comes across this later I'm running an ASRock H97M Pro4 with a Intel Xeon E3-1231v3 (With no integrated GPU) which may or may not be the issue. And a EVGA 750Ti SC GPU.

 

Link to comment
  • 3 years later...

Please help,I have a problem with my machine,nvidia passtrough and code error 43.

The driver is old 340.52-desktop-win8-win7-winvista-64bit-international-whql.exe

qemu is 2.11

host is slackware 14.2

libvirt is 4.8

host-gpu(not passed) amd ati

host-gpu(passed) nvidia GT218 gf 10 10de:0a65

This is my xml

 

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>Windows81</name>
  <uuid>1cdd1a64-bd6f-4cf7-891e-819b02656468</uuid>
  <title>windows81</title>
  <memory unit='KiB'>4240384</memory>
  <currentMemory unit='KiB'>4240384</currentMemory>
  <vcpu placement='static'>3</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-2.10'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/generic-2_VARS.fd</nvram>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state='off'/>
  </features>
  <cpu mode='host-model' check='partial'>
    <model fallback='allow'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/user/drivers/windows/windows vari/virtio/virtio-win-0.1.141.iso'/>
      <target dev='sda' bus='sata'/>
      <readonly/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='unsafe' io='threads'/>
      <source file='/home/user/.local/share/libvirt/images/windows81.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <boot order='2'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
      <model name='i82801b11-bridge'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pci-bridge'>
      <model name='pci-bridge'/>
      <target chassisNr='2'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x16'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
    </controller>
    <controller type='pci' index='10' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='10' port='0x17'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='**:**:**:**:**:**'/>
      <source bridge='bridge0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <input type='tablet' bus='virtio'>
      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
    </input>
    <input type='keyboard' bus='virtio'>
      <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
    </input>
    <graphics type='spice' autoport='yes' keymap='it'>
      <listen type='address'/>
      <image compression='off'/>
      <gl enable='no' rendernode='/dev/dri/renderD128'/>
    </graphics>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
    </hostdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='1'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=mouse1,evdev=/dev/input/by-id/usb-Logitech_USB_Receiver-event-mouse'/>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=kbd1,evdev=/dev/input/by-id/usb-_USB_Keyboard-event-kbd,grab_all=on,repeat=on'/>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=joypad1,evdev=/dev/input/by-id/usb-Logitech_Logitech_R__Precision_TM__Gamepad-event-joystick'/>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off'/>
  </qemu:commandline>
</domain>

 

Installing the driver,reboot and..code 43.

Remove the device,reinstall driver,reboot and..code 43.

No way?

Please help,thanks.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.