Unraid version: 6.9.2

 

My motherboard is:

ASUSTeK COMPUTER INC. TUF GAMING B560M-PLUS WIFI, Version Rev 1.xx
American Megatrends Inc., Version 1203
BIOS dated: Thu 28 Oct 2021 12:00:00 AM PDT

11th Gen Intel® Core™ i7-11700K @ 3.60GHz

Nvidia card; https://www.techpowerup.com/gpu-specs/gigabyte-rtx-3060-gaming-oc-rev-2-0.b9471

 

0. I am trying to rum a Windows 10 VM with passthrough Nvidia (tried with both SeaBIOS and OVMF).

 

1. The card is bound to VFI successfully:

 

IOMMU group 13: [10de:2487] 01:00.0 VGA compatible controller: NVIDIA Corporation Device 2487 (rev a1)

IOMMU group 13: [10de:228b] 01:00.1 Audio device: NVIDIA Corporation GA104 High Definition Audio Controller (rev a1)

 

2. I have added the those 2 items settings to the os config:

 

video=vesafb:off video=efifb:off

 

3. The XML config of the VM is set as:

 

<hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/user/isos/vbios/GA104.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>

 

4. I have extracted the vbios via GPU-Z. And tried with that version and also stripped the Nvidia header. Both same result (attached here)

 

5. This is the result I get when I start the VM:

-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device vfio-pci,host=0000:01:00.0,id=hostdev0,x-vga=on,bus=pci.0,multifunction=on,addr=0x5,romfile=/mnt/user/isos/vbios/GA104.rom \
-device vfio-pci,host=0000:01:00.1,id=hostdev1,bus=pci.0,addr=0x5.0x1 \
-device usb-host,hostbus=1,hostaddr=5,id=hostdev2,bus=usb.0,port=1 \
-device usb-host,hostbus=1,hostaddr=11,id=hostdev3,bus=usb.0,port=2 \
-device usb-host,hostbus=2,hostaddr=7,id=hostdev4,bus=usb.0,port=3 \
-device usb-host,hostbus=1,hostaddr=4,id=hostdev5,bus=usb.0,port=4 \
-device usb-host,hostbus=1,hostaddr=14,id=hostdev6,bus=usb.0,port=5 \
-device usb-host,hostbus=2,hostaddr=4,id=hostdev7,bus=usb.0,port=6 \
-device usb-host,hostbus=1,hostaddr=12,id=hostdev8,bus=usb.0,port=7 \
-device usb-host,hostbus=1,hostaddr=9,id=hostdev9,bus=usb.0,port=8 \
-device usb-host,hostbus=1,hostaddr=8,id=hostdev10,bus=usb.0,port=9 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2022-03-03 11:02:45.887+0000: Domain id=19 is tainted: high-privileges
2022-03-03 11:02:45.887+0000: Domain id=19 is tainted: host-cpu
char device redirected to /dev/pts/1 (label charserial0)
2022-03-03T11:02:46.538185Z qemu-system-x86_64: -device vfio-pci,host=0000:01:00.0,id=hostdev0,x-vga=on,bus=pci.0,multifunction=on,addr=0x5,romfile=/mnt/disk1/isos/vbios/GA104.rom: Failed to mmap 0000:01:00.0 BAR 1. Performance may be slow

 

Anything comes to mind?

 

GA104_modified.rom GA104.rom

Edited March 3, 20224 yr by atraikov

diagnostics

Sharing.

Currently, the monitor is turning on, but no image comes through. When I stop the VM, signal also stops.

 

On windows 11 I get this: 

 "2022-03-03T13:55:00.782938Z qemu-system-x86_64: vfio_region_write(0000:01:00.0:region1+0x298c8, 0x0,8) failed: Device or resource busy"

 

On windows 10 I don't get anything, but there might also be a problem with displaying log information currenty.

thewall-diagnostics-20220303-1805.zip

2 minutes ago, atraikov said:

Currently, the monitor is turning on, but no image comes through. When I stop the VM, signal also stops.

Have you tried the other outputs on the card?

5 hours ago, atraikov said:

2. I have added the those 2 items settings to the os config:

 

video=vesafb:off video=efifb:off

From your diagnostics you didn't apply this, or you applied them wrong, try to apply or post a screenshot of your changes to syslinux config.

 

Syslog is empty (???)

Edited March 3, 20224 yr by ghost82

Yeah, syslog got 128 mb of this same message, I dumped it and now it is not running. Restarting...

This is syslinux.cfg contents (/boot/syslinux/syslinux.cfg😞

 

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label Unraid OS
  menu default
  kernel /bzimage
  append initrd=/bzroot
  video=vesafb:off video=efifb:off
label Unraid OS GUI Mode
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui
label Unraid OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Unraid OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

9 minutes ago, atraikov said:

  append initrd=/bzroot
  video=vesafb:off video=efifb:off

This is wrong, it must be on the same line:

  append initrd=/bzroot video=vesafb:off video=efifb:off

Then reboot.

 

PS: most probably you don't need video=vesafb:off, unless you are booting unraid in non uefi mode

Edited March 3, 20224 yr by ghost82

Understood, will make the ammendments.

Attaching syslog.

thewall-syslog-20220303-1659.zip

Yes, as you can see:

Mar  3 08:50:30 TheWall kernel: pci 0000:01:00.0: BAR 1: assigned to efifb

Mar  3 08:52:02 TheWall kernel: vfio-pci 0000:01:00.0: BAR 1: can't reserve [mem 0x4000000000-0x43ffffffff 64bit pref]
...
...

BAR 1 is already in use by efifb, so you need to not make it to attach, or detach on the fly.

 

append initrd=/bzroot video=efifb:off

 

Check also your vbios, best way is to dump from linux, but gpu-z is also fine, the important thing is that you do not download one from internet.

If dumped with gpu-z remove the header.

Edited March 3, 20224 yr by ghost82

@ghost82

Moving to a single line worked. Much appreaciated! I will see if all is  good now.