[SOLVED] 2060 super single GPU passthrough troubleshooting


Recommended Posts

Unraid 6.8.3

Motherboard: MSI B450M Mortar Max

CPU: AMD 3600x

GPU: Nvidia 2060 super

RAM: 64GB

 

I'm trying to passthrough a single GPU to a Windows 10 guest. I tried every possibility solution I can find on the internet. I think I'm pretty near to success but there is something that I still can't fix.

 

First, I enabled IOMMU and SVM in BIOS and Checked in Unraid system information that.

HVM: Enabled
IOMMU: Enabled

After setting PCIe ACS override to multi function, the IOMMU group for my gpu is now, 

(Downstream makes my unraid unbootable. I don't know why.)

IOMMU group 23:	[10de:1f06] 26:00.0 VGA compatible controller: NVIDIA Corporation TU106 [GeForce RTX 2060 SUPER] (rev a1)
IOMMU group 24:	[10de:10f9] 26:00.1 Audio device: NVIDIA Corporation TU106 High Definition Audio Controller (rev a1)
IOMMU group 25:	[10de:1ada] 26:00.2 USB controller: NVIDIA Corporation TU106 USB 3.1 Host Controller (rev a1)
IOMMU group 26:	[10de:1adb] 26:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU106 USB Type-C UCSI Controller (rev a1)

To ensure that GPU is not occupied by Unraid, I inserted pci-stub into syslinux: 

kernel /bzimage
append pcie_acs_override=multifunction isolcpus=0-3,6-9 pci-stub.ids=10de:1f06,10de:10f9,10de:1ada,10de:1adb initrd=/bzroot

I tried to download a 2060 super rom and remove the header using hex editor.  The configure of a new Windows 10 machine. 

2143012469_ScreenShot2020-06-04at11_03_06PM.thumb.png.59e94de4ee87dcd649d64772240ac970.png

Set the XML to multifunction to 'on' and change the virtual devices to the same slot with corresponding functions.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x26' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/2060s.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x26' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x26' slot='0x00' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x26' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x3'/>
    </hostdev>

After all these work, I login Windows 10 from server from RDP and installed the device driver. Windows is detecting the device. 

1435398987_ScreenShot2020-06-04at11_20_36PM.png.d263a7a36a2db81da12867973d238c28.png

However, Nvidia driver is not detecting the gpu! Pretty wired!

893955278_ScreenShot2020-06-04at11_21_51PM.png.0bc959aaaa6afe20b7b1caed87531776.png

 

Here are some error informations the might be useful.

VM-> Windows 10 -> Log I can get Qemu log with full of errors like,

2020-06-04T15:14:05.725314Z qemu-system-x86_64: vhost_region_add_section:Section rounded to 92206000 prior to previous 92206010
2020-06-04T15:14:05.726742Z qemu-system-x86_64: vhost_region_add_section:Section rounded to 92206000 prior to previous 92206010
2020-06-04T15:14:05.727011Z qemu-system-x86_64: vhost_region_add_section:Section rounded to 92206000 prior to previous 92206010
2020-06-04T15:14:05.727345Z qemu-system-x86_64: vhost_region_add_section:Section rounded to 92206000 prior to previous 92206010
2020-06-04T15:14:05.727888Z qemu-system-x86_64: vhost_region_add_section:Section rounded to 92206000 prior to previous 92206010
2020-06-04T15:14:05.728086Z qemu-system-x86_64: vhost_region_add_section:Section rounded to 92206000 prior to previous 92206010
2020-06-04T15:14:05.728246Z qemu-system-x86_64: vhost_region_add_section:Section rounded to 92206000 prior to previous 92206010

In Windows 10 Devices manager there is an unknown device and virtio-win does not contain its driver.

804749687_ScreenShot2020-06-04at11_25_18PM.png.7994245b5040b705c68fc2e6aa9fb394.png

 

I have worked three days on passing through the GPU. Any advice/idea is appreciated. 

If you need detailed information please tell me.

Thanks in advance.

 

~~~~~~

I think I have solved this problem by buying another graphic card for Unraid.

Edited by hackghost
solved
Link to comment
  • 9 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.