MacOS GPU passthrough issues on New AMD X570 Build


Recommended Posts

Hi everyone,

 

I have just migrated my Intel i5 6500 build to a new Ryzen build with the below hardware and haven't been able to figure out why my Mac OS Catalina VM is not allowing me to pass through my GPU. 

 

Hardware:

ASRock X570 Taichi

AMD Ryzen 3950X

AMD Radeon RX 570

 

Unlike my previous Intel server, my monitor seems to show the UNRAID OS boot details on my monitor through my AMD Radeon RX 580 despite having video=efifb:off in my linuxsys configuration as follows:

kernel /bzimage
append pcie_acs_override=downstream isolcpus=6-15,22-31 initrd=/bzroot video=efifb:off

I used @SpaceInvaderOne 's amazing Macinabox Docker as I usually have to build the Mac OS VM but seem to get stuck. The VM boots but now video appears aside from a black screen and the following log appears:

2020-02-05 01:50:36.083+0000: Domain id=7 is tainted: custom-argv
2020-02-05 01:50:36.083+0000: Domain id=7 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2020-02-05T01:50:36.106188Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.108014Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.108708Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.109366Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.110033Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.110706Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.111406Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.112062Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.112908Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.113615Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.114287Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.114963Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.115624Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.116318Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.117019Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.117708Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.118377Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.119080Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.119757Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:36.120423Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
2020-02-05T01:50:37.720601Z qemu-system-x86_64: vfio_err_notifier_handler(0000:0e:00.1) Unrecoverable error detected. Please collect any data possible and then kill the guest
2020-02-05T01:50:37.720695Z qemu-system-x86_64: vfio_err_notifier_handler(0000:0e:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest

As you can see from the above, there is "vfio_err_notifier_handler" that appears to reference my GPU.

 

Here is my XML for my VM:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='4' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>MacinaboxCatalina</name>
  <uuid>16c4990b-e509-4928-ac06-72cc5857b010</uuid>
  <description>MacOS Catalina</description>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="default.png" os="Catalina"/>
  </metadata>
  <memory unit='KiB'>33030144</memory>
  <currentMemory unit='KiB'>33030144</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>20</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='6'/>
    <vcpupin vcpu='1' cpuset='22'/>
    <vcpupin vcpu='2' cpuset='7'/>
    <vcpupin vcpu='3' cpuset='23'/>
    <vcpupin vcpu='4' cpuset='8'/>
    <vcpupin vcpu='5' cpuset='24'/>
    <vcpupin vcpu='6' cpuset='9'/>
    <vcpupin vcpu='7' cpuset='25'/>
    <vcpupin vcpu='8' cpuset='10'/>
    <vcpupin vcpu='9' cpuset='26'/>
    <vcpupin vcpu='10' cpuset='11'/>
    <vcpupin vcpu='11' cpuset='27'/>
    <vcpupin vcpu='12' cpuset='12'/>
    <vcpupin vcpu='13' cpuset='28'/>
    <vcpupin vcpu='14' cpuset='13'/>
    <vcpupin vcpu='15' cpuset='29'/>
    <vcpupin vcpu='16' cpuset='14'/>
    <vcpupin vcpu='17' cpuset='30'/>
    <vcpupin vcpu='18' cpuset='15'/>
    <vcpupin vcpu='19' cpuset='31'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-3.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/16c4990b-e509-4928-ac06-72cc5857b010_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='20' threads='1'/>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/disks/Samsung_SSD_960_PRO_512GB_S3EWNX0JA08097W/MacinaboxCatalina/Clover.qcow2' index='3'/>
      <backingStore/>
      <target dev='hdc' bus='sata'/>
      <boot order='1'/>
      <alias name='sata0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/disks/Samsung_SSD_960_PRO_512GB_S3EWNX0JA08097W/MacinaboxCatalina/Catalina-install.img' index='2'/>
      <backingStore/>
      <target dev='hdd' bus='sata'/>
      <alias name='sata0-0-3'/>
      <address type='drive' controller='0' bus='0' target='0' unit='3'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/disks/Samsung_SSD_960_PRO_512GB_S3EWNX0JA08097W/MacinaboxCatalina/macos_disk.img' index='1'/>
      <backingStore/>
      <target dev='hde' bus='sata'/>
      <alias name='sata0-0-4'/>
      <address type='drive' controller='0' bus='0' target='0' unit='4'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:98:49:a6'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-4-MacinaboxCatalina/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0e' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
  <qemu:commandline>
    <qemu:arg value='-usb'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-kbd,bus=usb-bus.0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='************************'/>
    <qemu:arg value='-smbios'/>
    <qemu:arg value='type=2'/>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check'/>
  </qemu:commandline>
</domain>

Any help would be greatly appreciated to help get me up and running again in Mac OS.

 

Thanks in advance.

fosterserver-diagnostics-20200204-2222.zip

Edited by mafoster
wrong part mentioned
Link to comment

@david279 I have removed the +pcid from the VM XML and stubbed the GPU using the VFIO-PCI CFG plugin and also adjusted the linuxsys to the following:

 

label unRAID OS
  menu default
  kernel /bzimage
  append pcie_acs_override=downstream isolcpus=6-15,22-31 video=efifb:off pci=noaer initrd=/bzroot

I added pci=noaer in addition to moving the video=efifb:off before the initrt as you mentioned. 

 

I am still not getting it to work but the amount of errors are reducing:

 

-cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2020-02-06 01:22:11.894+0000: Domain id=2 is tainted: high-privileges
2020-02-06 01:22:11.894+0000: Domain id=2 is tainted: custom-argv
2020-02-06 01:22:11.894+0000: Domain id=2 is tainted: host-cpu

Any further ideas?

 

Attached is my latest diagnostics files.

fosterserver-diagnostics-20200205-2028.zip

Link to comment
  • 4 months later...
  • 2 weeks later...
  • 1 year later...

me 3.... my GPU passthrough works fine on multiple VMs just not the Macinabox it's an R9 280 which is supposed to work on bigsur hackintoshes. maybe this thread would be better served under the macinabox support thread, I'll start a new one there referencing this topic.

Link to comment
  • 3 weeks later...

Any luck on this yet? I'm having the exactly same problem with a NVIDIA Corporation GK110B [GeForce GTX TITAN Black] (rev a1) on my top slot with MAC bigsur. I did look it up and it looks compatible on the site mentioned above.  I did dump the vbios and checked it on a separate VM (windows10) and finally got that working. So I was very hopeful that when I tried the MacOS it would work. 

Link to comment
  • 2 years later...

Has anyone found a solution? I am running into the same issue after adding a rx 570 to a bigsur vm following spaceinvaders tutorial. 

 

Motherboard:ASRock B550 Taichi Razer Edition, 

Processor:AMD Ryzen 7 3700X 8-Core @ 3600 MHz

 

I have followed all the passthrough guidance (BIOS and system devices) that I could come across with dozens of reboots and two installs of the VM. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.