Hello, 

I hope everyone is doing well! 

 

I have been experiencing an error (which has been driving me crazy since last month and is making me consider moving away from Unraid!) when launching a Windows or Ubuntu VM in Unraid with an Nvidia GPU. The VM works fine when launching it on a Virtual Graphic Card but not when selecting the physical Nvidia RTX A2000 GPU and its sound card.

 

I tried everything recommended online, but nothing worked. Can anyone please help me with this? 🤞

 

Please find the information below: 

 

1.   Error code:

 

2025-02-03T12:03:08.605185Z qemu-system-x86_64: VFIO_MAP_DMA failed: Invalid argument 2025-02-03T12:03:08.605205Z qemu-system-x86_64: vfio_container_dma_map(0x55e9884c7260, 0x383800000000, 0x200000000, 0x154a3b000000) = -22 

(Invalid argument) 2025-02-03T12:03:08.606773Z qemu-system-x86_64: VFIO_MAP_DMA failed: Invalid argument 2025-02-03T12:03:08.606787Z qemu-system-x86_64: vfio_container_dma_map(0x55e9884c7260, 0x383a00000000, 0x2000000, 0x154a39000000) = -22

 

 

 

2.    I also checked that the graphic and sound cards are grouped together and utilised in the VM (I tried binding and unbinding, and it didn't make a difference). Please see below:

 

 

 

 

3.  I changed the PCIe ACS override in all states (disabled, downstream, multi-function, and both), but nothing changed. 

 

4.  I also checked:

 

IOMMU: Enabled

Resizable BAR and Above 4G Decoding: Enabled

VT-d: Enabled 

 

5. I attempted to grab the vbios of the GPU and boot from it, but nothing made a difference.

 

6.  The VM template is the following: 
 

 

 

Thanks!

 

Edited February 4, 20251 yr by Jawad

You can try this. It sometimes works.

add maxphysaddr to cpu block.

<cpu mode="host-passthrough" check="none" migratable="on">
    <topology sockets="1" dies="1" clusters="1" cores="6" threads="2"/>
    <maxphysaddr mode="passthrough" limit="39"/>
</cpu>

 This method is from this webpage. 

Thank you so much for responding! 

 

Adding "maxphysaddr" seems to solve the VFIO_MAP_DMA failed error in logs, but the VM is still not outputting to the screen as there's no signal and and I got this new error:

 

2025-02-04 14:07:57.611+0000: Domain id=7 is tainted: high-privileges
char device redirected to /dev/pts/2 (label charserial0)
2025-02-04 14:08:00.358+0000: Domain id=7 is tainted: custom-monitor

 

 

Would you, by any chance, have any idea of how you might be able to solve this error?

 

Thanks!

Edited February 4, 20251 yr by Jawad

48 minutes ago, Jawad said:

2025-02-04 14:07:57.611+0000: Domain id=7 is tainted: high-privileges
char device redirected to /dev/pts/2 (label charserial0)
2025-02-04 14:08:00.358+0000: Domain id=7 is tainted: custom-monitor

I don't think there is something wrong with the output. My output is as follows, and it works well.

2025-02-04 14:52:49.810+0000: Domain id=1 is tainted: high-privileges
char device redirected to /dev/pts/0 (label charserial0)

RTX3080 works well on my gaming vm. But today I received RTX 5080 fe, It shows a black screen and the VFIO_MAP_DMA error.

After I tried maxphysaddr, everything is back to normal now.

As for signal missing, maybe you can try to reinstall the graphic driver.

Hiya, 

 

I tried to install the drivers, but they won't install as Windows isn't detecting the GPU when selecting VNC. I also tried binding the GPU, but still, nothing changed. 

 

Am I doing something wrong here ?

 

Thanks again!

 

 

 

Make vnc the first gpu and A2000 the second in your vm config.

Then reinstall the nvidia drivers.

This is great! I didn't know this is possible... Getting there and now I can see the GPU! Thank you! 

 

The only issue I have now is that every time I attempt to install the drivers, the screen gets stuck on the below: 

 

 

Then it goes black and then gets stuck on this and the only way around it is to force shutdown ... and then same thing keeps happening

 

 

Did you have this issue when installing the drivers? 

 

This is my XML just incase I did something wrong

 

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 11</name>
  <uuid>be36c243-aec9-2840-678d-2be8756cb635</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm" webui=""/>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>2621440</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='8'/>
    <vcpupin vcpu='1' cpuset='9'/>
    <vcpupin vcpu='2' cpuset='10'/>
    <vcpupin vcpu='3' cpuset='11'/>
    <vcpupin vcpu='4' cpuset='12'/>
    <vcpupin vcpu='5' cpuset='13'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-9.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/be36c243-aec9-2840-678d-2be8756cb635_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' clusters='1' cores='6' threads='1'/>
    <maxphysaddr mode='passthrough' limit='39'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='ignore'/>
      <source file='/mnt/user/domains/Windows 11/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <serial>vdisk1</serial>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Windows 11/Win11_23H2_EnglishInternational_x64.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.240-1.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='piix3-uhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:95:fd:95'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
    </tpm>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' sharePolicy='ignore'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0e' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'/>
</domain>

 

Create a new vm and use ISO image to test basic display output.

add maxphysaddr, I see a little difference with yours.

<cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' clusters='1' cores='6' threads='2'/>
    <cache mode='passthrough'/>
    <maxphysaddr mode='passthrough' limit='39'/>
</cpu>

Start vm, If there's no signal, there are still some unknown issues in your unraid settings or vm template settings. 

If you can see some ouput on your physical monitor, i think there's no problem with vm config.

Try to use Display Driver Uninstaller (DDU) to uninstall Nvidia driver with vnc(Graphic card set to vnc only).

Then edit vm config, set graphic card to your gpu only. Start and wait windows to autoinstall graphic drivers.

PS: when you restart unraid server, try to do a complete shutdown then press the power button. This can do a reset with your GPU card.

This is all the solutions I can think of so far. Good luck!

vm.xml

What are you seeing on the monitor directly plugged in to the video card?

Hiya I tried all the options above but non worked.

 

I am getting black screen on VNC and no signal on the monitor that is directly plugged into the video card.

 

I also tested the graphic card in a pc build and it ran just fine.

I started having this same issue this week as well.

Encountering the same as well. My setup and options are the same as OP, and while adding the maxphysaddr code, the VFIO_MAP_DMA errors disappear from the log, but there's no video output.

It seems that I'm not the only one experiencing this issue. It looks like the problem is related to the Unraid patch/kernel, but I'm unsure how to resolve it.

 

Does anyone have any other suggestions?

 

I am having this exact issue. Removing my GPU and only using virtual works just fine, as soon as I passthrough a GPU as a second Graphics card it gets stuck on that Unraid spash screen. Currently it is working with the machine set to q35 and a few minor changes to some random usb controllers I saw. Here is my xml in case it helps anyone:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='4'>
  <name>thirdtry</name>
  <uuid>b32a46aa-fcf2-66bd-cc9c-f6ea6e130220</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm" webui="" storage="default"/>
  </metadata>
  <memory unit='KiB'>18874368</memory>
  <currentMemory unit='KiB'>18874368</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <source type='memfd'/>
    <access mode='shared'/>
  </memoryBacking>
  <vcpu placement='static'>17</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='7'/>
    <vcpupin vcpu='6' cpuset='8'/>
    <vcpupin vcpu='7' cpuset='9'/>
    <vcpupin vcpu='8' cpuset='10'/>
    <vcpupin vcpu='9' cpuset='11'/>
    <vcpupin vcpu='10' cpuset='12'/>
    <vcpupin vcpu='11' cpuset='13'/>
    <vcpupin vcpu='12' cpuset='14'/>
    <vcpupin vcpu='13' cpuset='15'/>
    <vcpupin vcpu='14' cpuset='16'/>
    <vcpupin vcpu='15' cpuset='17'/>
    <vcpupin vcpu='16' cpuset='18'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-9.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/b32a46aa-fcf2-66bd-cc9c-f6ea6e130220_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' clusters='1' cores='17' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
      <source file='/mnt/user/domains/thirdtry/vdisk1.img' index='1'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <serial>vdisk1</serial>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:55:93:c9'/>
      <source bridge='br0'/>
      <target dev='vnet3'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/run/libvirt/qemu/channel/4-thirdtry/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
      <alias name='tpm0'/>
    </tpm>
    <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' sharePolicy='ignore'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <watchdog model='itco' action='reset'>
      <alias name='watchdog0'/>
    </watchdog>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

 

On 2/4/2025 at 2:07 PM, neoherozzz said:

You can try this. It sometimes works.

add maxphysaddr to cpu block.

<cpu mode="host-passthrough" check="none" migratable="on">
    <topology sockets="1" dies="1" clusters="1" cores="6" threads="2"/>
    <maxphysaddr mode="passthrough" limit="39"/></cpu>

 This method is from this webpage. 

Thank you so much for this tip and link!

I was going crazy with my gaming VM crashing under heavy load (e.g. DX12 titles with high/ultra settings). I have been troubleshooting this for weeks now and this finally seems to fix the issue :-)

For reference: I am using Unraid 7.1.2 with a RTX 5070 Ti and a NVME drive bound to vfio and passed through and 7 p-cores (isolated) and 4 e-cores (shared) of my i7-13700 for the Win11 VM. I also made sure that reBAR is enabled, TSC timer used by the VM, and some hyper-v parameters suggested in the forums. But the maxphysaddr final did the trick to stabilize the VM. Performance is superb!

Try removing the sound card, only passthrough the GPU.