RX 580 passthrough hard hanging host


raun

Recommended Posts

I upgraded my working GTX 750 to a RX 580.  I have 1 Win10 VM that passes through the GPU.  The GTX 750 worked great.  The RX 580 has been a headache.

 

The RX 580 does not appear to have reset problems.  I can reboot the VM all day long and it has a 'reset' entry in /sys/kernel/iommu_groups/22

 

Quote

for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d);do echo "IOMMU group $(basename "$iommu_group")"; for device in $(\ls -1 "$iommu_group"/devices/); do if [[ -e "$iommu_group"/devices/"$device"/reset ]]; then echo -n "[RESET]"; fi; echo -n $'\t';lspci -nns "$device"; done; done

 

IOMMU group 22
[RESET] 17:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7)
        17:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [1002:aaf0]

 

First, the only driver that will install is the AMD driver currently in Windows Update 17.1.1.  Any other driver (18.1.1, 19.2.1, 19.2.2) will blank the screen, cause the VM to reboot and then hard hang the VM part way through the windows boot process.  I cant enter safe mode.  When the VM is hung like this, if I don't kill it manually within 2-3 minutes the unraid host will hard hang too.  The VM needs to be reimaged at this point, Windows seems completely gone (or I can remove the RX 580 pass through).

 

Second, just going with the 17.1.1 drivers:  Windows says there's "no speakers or headset connected"  device manager shows the HDMI audio driver is installed correctly, but for some reason I looks like the RX 580 never figures out there's an audio capable display attached.... odd, but not a deal breaker.  Some applications (discord for example) even show the HDMI monitor as an output device, but no sound will play.

 

Third, Using the 17.1.1 drivers everything appears to work.  I can run games, play with the over clocking/voltage settings.  Freesync works, etc.   If I leave the VM idle for some amount of time (like over night), the unraid host is hard hung the next morning.  I haven't recovered any logs or kernel messages as of yet.

 

This is a PowerColor Red Devil RX 580 with this bios: https://www.techpowerup.com/vgabios/198033/198033

 

Without seeing a kernel panic message I'm really debugging by guessing.  I haven't see this type of issue resolved in any forum yet.  Any thoughts?

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>001 - Windows 10</name>
  <uuid>2a62fac2-74fe-ad80-c885-196e74ee5ab8</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='11'/>
    <vcpupin vcpu='2' cpuset='3'/>
    <vcpupin vcpu='3' cpuset='13'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='16'/>
    <vcpupin vcpu='6' cpuset='8'/>
    <vcpupin vcpu='7' cpuset='18'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/2a62fac2-74fe-ad80-c885-196e74ee5ab8_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/disks/INTEL_SSDSA2CW512G3_CVPR129501XZ512NGN/vdisk001.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win10_1806_English_x64.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.160.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:4e:ae:de'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='5'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x17' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/disks/INTEL_SSDSA2CW512G3_CVPR129501XZ512NGN/amd580.dump2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x17' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0a5c'/>
        <product id='0x21e8'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1017'/>
        <product id='0x900a'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x258a'/>
        <product id='0x1006'/>
      </source>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x3842'/>
        <product id='0x2410'/>
      </source>
      <address type='usb' bus='0' port='4'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

Link to comment

Hey, How it came along?, I'm planing to buying the hardware to setup an unRaid home server for hosting 2 Windows 10 VMs with passing though GPU, so by reading different posts it seems that NVIDIA CPUs have some issues when it comes to install the drivers so I was thinking about buying two AMD GPUs, I wonder if you can recommend some? should those two be different model/brand or can be the same? thanks in advance

 

Link to comment
15 hours ago, CarlosCo said:

Hey, How it came along?, I'm planing to buying the hardware to setup an unRaid home server for hosting 2 Windows 10 VMs with passing though GPU, so by reading different posts it seems that NVIDIA CPUs have some issues when it comes to install the drivers so I was thinking about buying two AMD GPUs, I wonder if you can recommend some? should those two be different model/brand or can be the same? thanks in advance

 

Hi, I have used amd Rx580 8gb, Vega 64, and Nvidia 1070ti's in this setup. Various card manufacturers too. Frankly, all of them are a pain in the bum, but they all work. I'd say the Nvidia cards have been less trouble, although since the latest unraid update the vega works very well too besides the reset bug. I had the same issues as OP with my rx580's, and indeed switching to Q35 fixed them. Still can't install the latest driver though. I've never had issues installing the latest Nvidia drivers, but I do have very annoying hdmi audio issues lately. Probably fixable since I didn't have it before. I think you should just pick whatever gives the best performance for the price, I use the vega mainly because it's my most powerful gpu and I'm a total amd fanboy, but I would lean more to Nvidia if you're starting from scratch.

Link to comment
  • 9 months later...
On 2/22/2019 at 5:44 PM, thomas said:

Try to change the machine type to Q35 and try again. For my AMD that was the only way to fix it.

I have this same problem when its booting up an hangs before windows boots.. I'm using a xfx rx580 xxx... where is the Q35 opion?? Iv tryed everything I could fine somewhat I'm a true noob on linux and unraid.. 

Thanks in advance.

Link to comment
19 minutes ago, thomas said:

Q35 is under machine type, below the memory size and it has to be chosen instead of i440fx.

It will be best to create a new VM from scratch and choose Q35 and reinstall Windows.

Reusing the old VM is still possible by manually choosing the old VM's vdisk.

Ok awesome I will try that... one last thing for the GPU passthrough since i have the xrx rx580 xxx. So far the only way its letting me see anything on my monitor is if I add the bios dump file.. y is this? The videos say its only for nvidia GPUs.. thanks thomas

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.