derpuma Posted June 19, 2020 Posted June 19, 2020 (edited) First I was struggeling but later today I was able to install a stable and reliable Windows 10 VM on a virtual Disk with full GPU passthrough. I tested it with some rounds of high FPS 3d Gaming and it was rock solid and stable! So my Hardware (Ryzen 3900X and Gigabyte Aorus Pro Z570) is able to deal with UNRAID. I splitted my IOMMU Groups via PCIe ACS override with the setting "Both", did not bound any IOMMU via VFIO-PCI Config and it worked great. To get this far, I used a new kernel from this thread... Unfortunatly my Installation of macOS was a complete fail. Installation was easy and booting via VNC was no problem at all. Touble started when I tried to GPU passthrough like I did with Windows. I was not able to get it together, Attatched you will find my IoMMU Groups and my XML setting of both, my working Windows 10 and my messy Catalina install. I tried different combinations with and without USB passthrough. Maybe someone could have a look at my files an give me some tips and help. In general I guess it should be possible, to get a working macOS too, if Windows 10 ist flawless? My passthrough GPU ist a Radon 5700 XT. My Unraid Host GPU an older RX480 set to first PCIe device after boot, to be alble to have the 5700 XT for the passthrough. If you need anything other background information, please let me know. IOMMU group 0:[1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 1:[1022:1483] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 2:[1022:1483] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 3:[1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 4:[1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 5:[1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 6:[1022:1483] 00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 7:[1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 8:[1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 9:[1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 10:[1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 11:[1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 12:[1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 13:[1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 14:[1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 IOMMU group 15:[1987:5012] 01:00.0 Non-Volatile memory controller: Phison Electronics Corporation E12 NVMe Controller (rev 01) IOMMU group 16:[1022:57ad] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream IOMMU group 17:[1022:57a3] 03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 18:[1022:57a4] 03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 19:[1022:57a4] 03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 20:[1022:57a4] 03:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 21:[8086:1539] 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 22:[1022:1485] 05:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP IOMMU group 23:[1022:149c] 05:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller IOMMU group 24:[1022:149c] 05:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller IOMMU group 25:[1022:7901] 06:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 26:[1022:7901] 07:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 27:[1002:1478] 08:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch (rev c1) IOMMU group 28:[1002:1479] 09:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch IOMMU group 29:[1002:731f] 0a:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] (rev c1) IOMMU group 30:[1002:ab38] 0a:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio IOMMU group 31:[1002:67df] 0b:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev c7) IOMMU group 32:[1002:aaf0] 0b:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] IOMMU group 33:[1022:148a] 0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function IOMMU group 34:[1022:1485] 0d:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP IOMMU group 35:[1022:1486] 0d:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP IOMMU group 36:[1022:149c] 0d:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller IOMMU group 37:[1022:1487] 0d:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller CPU Thread Pairings Pair 1:cpu 0 / cpu 12 Pair 2:cpu 1 / cpu 13 Pair 3:cpu 2 / cpu 14 Pair 4:cpu 3 / cpu 15 Pair 5:cpu 4 / cpu 16 Pair 6:cpu 5 / cpu 17 Pair 7:cpu 6 / cpu 18 Pair 8:cpu 7 / cpu 19 Pair 9:cpu 8 / cpu 20 Pair 10:cpu 9 / cpu 21 Pair 11:cpu 10 / cpu 22 Pair 12:cpu 11 / cpu 23 USB Devices Bus 001 Device 001:ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002:ID 048d:8297 Integrated Technology Express, Inc. ITE Device(8595) Bus 001 Device 003:ID 1e7d:38a4 ROCCAT ROCCAT Kave Headset Bus 001 Device 004:ID 05e3:0608 Genesys Logic, Inc. Hub Bus 002 Device 001:ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 003 Device 001:ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 002:ID 05e3:0608 Genesys Logic, Inc. Hub Bus 003 Device 003:ID 03f0:2b4a Hewlett-Packard Bus 003 Device 004:ID 090c:1000 Silicon Motion, Inc. - Taiwan (formerly Feiya Technology Corp.) Flash Drive Bus 004 Device 001:ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 005 Device 001:ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 005 Device 002:ID 0b05:18e1 ASUSTek Computer, Inc. ROG STRIX IMPACT II Bus 005 Device 003:ID 03eb:3301 Atmel Corp. at43301 4-Port Hub Bus 005 Device 004:ID 1038:0100 SteelSeries ApS Ideazon Zboard Bus 006 Device 001:ID 1d6b:0003 Linux Foundation 3.0 root hub SCSI Devices [0:0:0:0]disk /dev/sda 3.99GB [N:0:1:1]disk PNY CS3030 2TB SSD__1 /dev/nvme0n1 2.00TB Log of the failing macOS GPU Passthrough: -blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-3-storage","backing":null}' \ -device ide-hd,bus=ide.2,drive=libvirt-3-format,id=sata0-0-2,bootindex=1,write-cache=on \ -blockdev '{"driver":"file","filename":"/mnt/user/domains/MacinaboxCatalina/Catalina-install.img","node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-2-storage"}' \ -device ide-hd,bus=ide.3,drive=libvirt-2-format,id=sata0-0-3,write-cache=on \ -blockdev '{"driver":"file","filename":"/mnt/user/domains/MacinaboxCatalina/macos_disk.img","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \ -device ide-hd,bus=ide.4,drive=libvirt-1-format,id=sata0-0-4,write-cache=on \ -netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 \ -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:04:b9:bd,bus=pci.3,addr=0x0 \ -chardev pty,id=charserial0 \ -device isa-serial,chardev=charserial0,id=serial0 \ -chardev socket,id=charchannel0,fd=38,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \ -device usb-tablet,id=input0,bus=usb.0,port=1 \ -device vfio-pci,host=0000:0a:00.0,id=hostdev0,bus=pci.4,addr=0x0,romfile=/mnt/disk1/isos/Powercolor_5700XT.rom \ -device vfio-pci,host=0000:0a:00.1,id=hostdev1,bus=pci.5,addr=0x0 \ -usb \ -device usb-kbd,bus=usb-bus.0 \ -device '************************' \ -smbios type=2 \ -cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2020-06-19 16:56:10.708+0000: Domain id=1 is tainted: high-privileges 2020-06-19 16:56:10.708+0000: Domain id=1 is tainted: custom-argv 2020-06-19 16:56:10.708+0000: Domain id=1 is tainted: host-cpu char device redirected to /dev/pts/3 (label charserial0) 2020-06-19T16:56:10.775914Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17] 2020-06-19T16:56:10.777224Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17] 2020-06-19T16:56:10.778138Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17] 2020-06-19T16:56:10.778998Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17] 2020-06-19T16:56:10.779874Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17] 2020-06-19T16:56:10.780785Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17] 2020-06-19T16:56:10.781505Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17] 2020-06-19T16:56:10.782190Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17] 2020-06-19T16:56:17.038207Z qemu-system-x86_64: vfio: Cannot reset device 0000:0a:00.1, no available reset mechanism. 2020-06-19T16:56:17.042396Z qemu-system-x86_64: vfio: Cannot reset device 0000:0a:00.1, no available reset mechanism. 2020-06-19T16:56:31.836311Z qemu-system-x86_64: terminating on signal 15 from pid 8553 (/usr/sbin/libvirtd) 2020-06-19 16:56:32.236+0000: shutting down, reason=destroyed Win10_Working.xml MacinaboxCatalina.xml Edited June 20, 2020 by derpuma problem solved Quote
derpuma Posted June 20, 2020 Author Posted June 20, 2020 Just want to give a quick update. After hassling around the RX 5700 XT macOS is booting with the GPU right now. The pita was not the GPU or the Reset Bug, it was Clover and I had to set some boot args and deselect some preselected options while booting. 1) The GPU needs the boot-arg agdpmod=pikera 2) by default clover does ATI Inject, I deselected this 3) I selected load VBIOS in clover 4) I changed SMBIOS to MacPro1,1 (MacPro7,1 will also work) These are the main reasons, why I was not able to passthrough my GPU to macOS. If a reset bug happens, I use the script from spacinvador via User Scripts plugin: #!/bin/bash # #replace xx\:xx.x with the number of your gpu and sound counterpart # # echo "disconnecting amd graphics" echo "1" | tee -a /sys/bus/pci/devices/0000\:0a\:00.0/remove echo "disconnecting amd sound counterpart" echo "1" | tee -a /sys/bus/pci/devices/0000\:0a\:00.1/remove echo "entered suspended state press power button to continue" echo -n mem > /sys/power/state echo "reconnecting amd gpu and sound counterpart" echo "1" | tee -a /sys/bus/pci/rescan echo "AMD graphics card succesfully reset" I also updated to the latest Bios for my Gigabyte Z570 Aorus Pro (F20s). Iommu is like a charm and almost every device has his own IOMMU Group. Single USB passthrough also works. Next step is to passthrough the different USB Controllers to get USB hotplugable. Another thing I will try is to passthrough HDD and SSD devices and boot from their OS. So happy days! Quote
Coshy93 Posted July 25, 2020 Posted July 25, 2020 Would you be able to explain to a dummy like me how I make these changes go my unraid server / Mac os vm to pass through my 5700xt? Im unsure where to add those boot args and don't know how to perform the other steps with clover etc. But eager to learn! Quote
podlipny Posted July 25, 2020 Posted July 25, 2020 yeah some quick guide would be great. I am currently trying to migrate from bare metal Hackintosh to unRaid setup. But I m running i9 9900K Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.