Help with GPU passthrough

[Nanobug]

I'm having trouble making the GPU passthrough work.

So far, I can see the GPU in the Windows VM I can install the driver, but right after I've installed it, I can install it again.

And it doesn't use it either.

 

I tried following different guides, but I can't make it work.

 

Does anyone have a guide or can tell me the exact requirements to make it work?

[ghost82]

16 hours ago, Nanobug said:

I'm having trouble making the GPU passthrough work.

So far, I can see the GPU in the Windows VM I can install the driver, but right after I've installed it, I can install it again.

And it doesn't use it either.

 

I tried following different guides, but I can't make it work.

 

Does anyone have a guide or can tell me the exact requirements to make it work?

 

Sorry but your description says nothing

Imagine a user reading what you wrote, how can be possible to help?

Which troubles? Which gpu, which driver?

If I have an installer with the gpu driver and I install it, then I can click again on it and reinstall it again, I see no issue.

Be more specific, and attach diagnostics file!

[Nanobug]

1 hour ago, ghost82 said:

 

Sorry but your description says nothing

Imagine a user reading what you wrote, how can be possible to help?

Which troubles? Which gpu, which driver?

If I have an installer with the gpu driver and I install it, then I can click again on it and reinstall it again, I see no issue.

Be more specific, and attach diagnostics file!

 

I haven't told it in details, but I asked if anyone has a guide, or can guide me through it from start to finish.

I described the problem;

I can see the GPU, I can install the driver, and right after, I can install it again. And it's not using the GPU.

It's a P2200, and its the newest in Nvidia GeForce Experience which is version 496.49.

Diagnostics are added as well.

 

 

nanostorage-diagnostics-20211206-2032.zip

[ghost82]

Try to properly set the multifunction device, replace this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

 

With this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>

 

Sometimes drivers can have issues if the gpu in the target is not multifunction.

[Nanobug]

58 minutes ago, ghost82 said:

Try to properly set the multifunction device, replace this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

 

With this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>

 

Sometimes drivers can have issues if the gpu in the target is not multifunction.

Where do I change that?

I tried looking for it in the VM, but there's no hostdev section.

[ghost82]

From your diagnostics file you attached you have 2 vms: remote and server.

the server VM has the gpu passthrough: edit that in the xml mode.

Edited December 7, 2021 by ghost82

[Nanobug]

20 minutes ago, ghost82 said:

From your diagnostics file you attached you have 2 vms: remote and server.

the server VM has the gpu passthrough: edit that in the xml mode.

I'm such an idiot, sorry.

Trying it now.

 

Will get back to you after with the result.

[Nanobug]

Same result.
Device manager says:

 

I tried downloading a driver and save it on the desktop as well and install that.

This is version 472.47 (had it from the first time I tried this) of the Nvidia driver.

 

When I tried from Nvidia Geforce Experience I can install it again right after, so there's never an installed driver. I always do a clean install.

It's version 496.49 here.

And right after:

 

If I check on my desktop (not the VM):

 

[Nanobug]

For some reason, it can see the GPU, but it can't use it, and I don't get why.

 

I don't know if it has any relevance, but I had to install Windows in the VM without a GPU before I even got to this point.

[ghost82]

That's really odd...I have no idea of what is happening...If I search in google it seems you are not alone, so I'm not sure this issue is related to virtualization.

What happens if you uninstall the drivers from the device manager, reboot and run windows update? Does it install any driver (not with the geforce experience app, nor manually)?

 

What does the yellow mark say in device manager?Any error code?

Edited December 7, 2021 by ghost82

[Nanobug]

13 hours ago, ghost82 said:

That's really odd...I have no idea of what is happening...If I search in google it seems you are not alone, so I'm not sure this issue is related to virtualization.

What happens if you uninstall the drivers from the device manager, reboot and run windows update? Does it install any driver (not with the geforce experience app, nor manually)?

 

What does the yellow mark say in device manager?Any error code?

I just thought I did it the wrong way. Because all I've heard, is that it's supposed to be really easy to set up GPU passthrough in unRAID.

 

I'll check that later when I'm home, and give the information.

[ghost82]

It could be error 43 because nvidia detects a virtual machine.

It is true that nvidia now allows passthrough of its gpus, from march 2021?, but this should only be with its consumer cards.

From Nvidia:

Quote

The feature is enabled on all GeForce/TITAN GPUs supported in the R465 driver (Kepler and later for Desktop; Maxwell and later for Notebook) for Windows 10.

 

It could be that quadro cards are not "consumer gpus", so you need to apply all the fixes to hide the hypervisor from nvidia.

If this is the case I would try to hide the hypervisor by modifying the xml.

 

Replace this:

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>

 

With this:

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>

 

[Nanobug]

2 hours ago, ghost82 said:

It could be error 43 because nvidia detects a virtual machine.

It is true that nvidia now allows passthrough of its gpus, from march 2021?, but this should only be with its consumer cards.

From Nvidia:

 

It could be that quadro cards are not "consumer gpus", so you need to apply all the fixes to hide the hypervisor from nvidia.

If this is the case I would try to hide the hypervisor by modifying the xml.

 

Replace this:

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>

 

With this:

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>

 

 

I'll try that later when I'm home.
Thank you so far.

I'll let you know what happens.

[Nanobug]

21 hours ago, ghost82 said:

It could be error 43 because nvidia detects a virtual machine.

It is true that nvidia now allows passthrough of its gpus, from march 2021?, but this should only be with its consumer cards.

From Nvidia:

 

It could be that quadro cards are not "consumer gpus", so you need to apply all the fixes to hide the hypervisor from nvidia.

If this is the case I would try to hide the hypervisor by modifying the xml.

 

Replace this:

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>

 

With this:

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>

 

This didn't work. I'll double check it though, since I only had 10 minutes yesterday to do it. Just in case.

 

Haven't had time to try removing the driver and reboot Windows.

I'll try to get it done during today at work.

[Nanobug]

You were right about the code 43:

 

It's a bit quiet here, so I've had time to check the rest.

Removing it from Device Management and rebooting it didn't help.

 

Found this one in the updates:

 

Didn't work either.

I've tried disabling the graphics card, reboot the VM and enable it again.

 

 

I did mod the XML file with the addition you made, and it didn't help either.

Any other ideas I could try, or should I just start over?

[ghost82]

I don't have any more ideas apart passing the vbios too and try to switch from i440fx+ovmf to q35+ovmf, maybe it could help.