nicfitzgerald Posted April 15, 2019 Share Posted April 15, 2019 (edited) First off, I'm brand new to Unraid. Second, I apologize if this has been asked previously. I have been looking for AGES on how to make this work. I'll start with my machine specs: AMD Ryzen R7 1700 PowerColor RX 590 Corsair Dominator 16gb @3000 I have Unraid running without a problem, and I have a Ubuntu Server VM running without issue. I'm trying to get a Windows VM running and I've followed just about every tutorial I can find. I have excluded the RX from the host, and have verified that with a syslinux change and the ACS override in VM settings. My VM Template: <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows</name> <uuid>4680f734-d50c-0edf-f250-fc41bc3f39c0</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>10485760</memory> <currentMemory unit='KiB'>10485760</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>10</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='8'/> <vcpupin vcpu='2' cpuset='1'/> <vcpupin vcpu='3' cpuset='9'/> <vcpupin vcpu='4' cpuset='2'/> <vcpupin vcpu='5' cpuset='10'/> <vcpupin vcpu='6' cpuset='4'/> <vcpupin vcpu='7' cpuset='12'/> <vcpupin vcpu='8' cpuset='6'/> <vcpupin vcpu='9' cpuset='14'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='10' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Win10_1809Oct_v2_English_x64.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:5f:72:e5'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x1f' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/Powercolor.RX590.8192.181106.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x1f' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1038'/> <product id='0x1730'/> </source> <address type='usb' bus='0' port='1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1b1c'/> <product id='0x1b49'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x2433'/> <product id='0xb200'/> </source> <address type='usb' bus='0' port='3'/> </hostdev> <memballoon model='none'/> </devices> </domain> The syslinux override: kernel /bzimage append iommu=pt iommu=1 vfio-pci.ids=1002:67df, 1002:aaf0 pcie_acs_override=downstream,multifunction vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot System devices: IOMMU group 25: [1002:67df] 1f:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev e1) IOMMU group 26: [1002:aaf0] 1f:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] With the GPU attached to Windows, it either won't boot at all, gives me the `127` error or it tells me that it's not able to power up and stuck in the d3 mode. Logs after trying to boot the VM: 2019-04-15T22:53:36.119948Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3 2019-04-15T22:53:36.120452Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3 2019-04-15T22:53:41.588646Z qemu-system-x86_64: vfio_err_notifier_handler(0000:1f:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest 2019-04-15T22:53:41.589405Z qemu-system-x86_64: vfio_err_notifier_handler(0000:1f:00.1) Unrecoverable error detected. Please collect any data possible and then kill the guest Are there any suggestions? Edited April 15, 2019 by nicfitzgerald Quote Link to comment
JonathanM Posted April 15, 2019 Share Posted April 15, 2019 21 minutes ago, nicfitzgerald said: With the GPU attached to Windows, it either won't boot at all, gives me the `127` error or it tells me that it's not able to power up and stuck in the d3 mode. Shot in the dark, but what happens if you start that VM first thing after a cold power cycle? As in disable any auto starting services, shut down the machine, remove power, power back up, start the problem VM before running anything else? Quote Link to comment
nicfitzgerald Posted April 15, 2019 Author Share Posted April 15, 2019 30 minutes ago, jonathanm said: Shot in the dark, but what happens if you start that VM first thing after a cold power cycle? As in disable any auto starting services, shut down the machine, remove power, power back up, start the problem VM before running anything else? I haven't tried without the Ubuntu server running, but I am currently trying to get a vbios dump of my own card instead of a third party source. Quote Link to comment
dk4dk4 Posted April 16, 2019 Share Posted April 16, 2019 (edited) I am also new to Unraid and I'm having the exact same issue. I have not purchased a license yet, running on trial... License purchase will depend on this working. Unraid Version: 6.7.0-rc7 AMD Ryzen 1700X Radeon RX 570 With the GPU attached to Windows 10 VM, it either won't boot at all, gives me the `127` error or it tells me that it's not able to power up and stuck in the D3 mode. jarvis-diagnostics-20190416-0400.zip Edited April 16, 2019 by dk4dk4 1 Quote Link to comment
nicfitzgerald Posted April 16, 2019 Author Share Posted April 16, 2019 1 hour ago, dk4dk4 said: I am also new to Unraid and I'm having the exact same issue. I have not purchased a license yet, running on trial... License purchase will depend on this working. Unraid Version: 6.7.0-rc7 AMD Ryzen 1700X Radeon RX 570 With the GPU attached to Windows 10 VM, it either won't boot at all, gives me the `127` error or it tells me that it's not able to power up and stuck in the D3 mode. jarvis-diagnostics-20190416-0400.zip I so badly want to enjoy this, but it's borderline impossible and I definitely don't want to just have a sitting paperweight until it's fixed 😔 Quote Link to comment
sadkisson Posted April 17, 2019 Share Posted April 17, 2019 I have a RX 590 passed through to windows 10. I had to convert my windows 10 VM to uefi boot mode to get it working properly. Also, I have observed that if I stop the VM I have to reboot the whole server or upon VM start my server freezes. I do not stop the VM very often however so it has not been much of an issue. I am unable to post the Unraid VM config at the moment but I will tomorrow. Quote Link to comment
nicfitzgerald Posted April 17, 2019 Author Share Posted April 17, 2019 24 minutes ago, sadkisson said: I have a RX 590 passed through to windows 10. I had to convert my windows 10 VM to uefi boot mode to get it working properly. Also, I have observed that if I stop the VM I have to reboot the whole server or upon VM start my server freezes. I do not stop the VM very often however so it has not been much of an issue. I am unable to post the Unraid VM config at the moment but I will tomorrow. Thank you so much 😭 I was at a complete Quote Link to comment
dk4dk4 Posted April 17, 2019 Share Posted April 17, 2019 57 minutes ago, sadkisson said: I have a RX 590 passed through to windows 10. I had to convert my windows 10 VM to uefi boot mode to get it working properly. Also, I have observed that if I stop the VM I have to reboot the whole server or upon VM start my server freezes. I do not stop the VM very often however so it has not been much of an issue. I am unable to post the Unraid VM config at the moment but I will tomorrow. My Win10 is already at UEFI (GPT) VM and still not working... FYI 2 Quote Link to comment
cybrnook Posted April 17, 2019 Share Posted April 17, 2019 Interested in this too Quote Link to comment
sadkisson Posted April 18, 2019 Share Posted April 18, 2019 Below is my configuration with a working RX 590 pass through to Windows 10. Hardware Motherboard Supermicro X9SCM-F (modified pci express slot to fit X16 card) latest BIOS Processor Intel Xeon E3-1200 Graphics Card XFX Radeon RX 590 Fatboy UnRaid Settings UnRaid booted in UEFI mode PCIe ACS override: Downstream VFIO allow unsafe interrupts: No Graphics pass through using vfio-pci.cfg method stored on flash drive Windows 10 VM config Machine: Q35-3.1 BIOS: OVMF Hyper-V: Yes Graphics Card: AMD Radeon RX 470/480/570/570X/580/590 VM Settings Windows 10 VM set to boot in UEFI mode Latest VirtIO drivers When I first tried to use the RX590 my Windows 10 VM was using Machine: i440fx and SeaBios. This resulted in my server giving an error when trying to start the VM. I had to convert my Windows 10 VM to boot via UEFI and then the VM would not boot with SeaBios. I had to recreate my UnRaid Windows 10 VM config to boot with Machine: OVMF and BIOS: Q35 and it loaded right up. Installed the AMD driver from their website and I was up and running. The only issue I have had with this setup is the known AMD reset bug. So if I have to stop the VM I have to reset the entire server to start the VM again or the whole server will freeze when trying to start the VM. From what I understand it has something to do with the AMD graphics card not properly releasing after VM shutdown. Please if you have any question on my setup ask and I can give you any information I might have left out. Hope you all get yours working as well. 1 Quote Link to comment
cybrnook Posted April 18, 2019 Share Posted April 18, 2019 Awesome info, will give it a spin! Quote Link to comment
dk4dk4 Posted April 18, 2019 Share Posted April 18, 2019 3 hours ago, sadkisson said: Below is my configuration with a working RX 590 pass through to Windows 10. Hardware Motherboard Supermicro X9SCM-F (modified pci express slot to fit X16 card) latest BIOS Processor Intel Xeon E3-1200 Graphics Card XFX Radeon RX 590 Fatboy UnRaid Settings UnRaid booted in UEFI mode PCIe ACS override: Downstream VFIO allow unsafe interrupts: No Graphics pass through using vfio-pci.cfg method stored on flash drive Windows 10 VM config Machine: Q35-3.1 BIOS: OVMF Hyper-V: Yes Graphics Card: AMD Radeon RX 470/480/570/570X/580/590 VM Settings Windows 10 VM set to boot in UEFI mode Latest VirtIO drivers When I first tried to use the RX590 my Windows 10 VM was using Machine: i440fx and SeaBios. This resulted in my server giving an error when trying to start the VM. I had to convert my Windows 10 VM to boot via UEFI and then the VM would not boot with SeaBios. I had to recreate my UnRaid Windows 10 VM config to boot with Machine: OVMF and BIOS: Q35 and it loaded right up. Installed the AMD driver from their website and I was up and running. The only issue I have had with this setup is the known AMD reset bug. So if I have to stop the VM I have to reset the entire server to start the VM again or the whole server will freeze when trying to start the VM. From what I understand it has something to do with the AMD graphics card not properly releasing after VM shutdown. Please if you have any question on my setup ask and I can give you any information I might have left out. Hope you all get yours working as well. I tried out your setup and still get the black screen on VM startup: RX 570 video card IOMMU group 15:[1002:67df] 0b:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev ff) [1002:aaf0] 0b:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] (rev ff) cat vfio-pci.cfg BIND="0b:00.0" VM log ending: 2019-04-18T03:24:07.735594Z qemu-system-x86_64: -device vfio-pci,host=0b:00.0,id=hostdev0,bus=pci.7,addr=0x1: Failed to mmap 0000:0b:00.0 BAR 0. Performance may be slow 2019-04-18T03:24:08.916579Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3 2019-04-18T03:24:08.920566Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3 2019-04-18T03:24:09.442392Z qemu-system-x86_64: vfio_err_notifier_handler(0000:0b:00.1) Unrecoverable error detected. Please collect any data possible and then kill the guest Syslog: Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: [1002:67df] type 00 class 0x030000 Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: reg 0x10: [mem 0xe0000000-0xefffffff 64bit pref] Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: reg 0x18: [mem 0xf0000000-0xf01fffff 64bit pref] Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: reg 0x20: [io 0xc000-0xc0ff] Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: reg 0x24: [mem 0xfce00000-0xfce3ffff] Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: reg 0x30: [mem 0xfce40000-0xfce5ffff pref] Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: BAR 0: assigned to efifb Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: supports D1 D2 Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.0: PME# supported from D1 D2 D3hot D3cold Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.1: [1002:aaf0] type 00 class 0x040300 Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.1: reg 0x10: [mem 0xfce60000-0xfce63fff 64bit] Apr 18 00:14:00 Jarvis kernel: pci 0000:0b:00.1: supports D1 D2 Quote Link to comment
GreenEyedMonster Posted May 1, 2019 Share Posted May 1, 2019 So I had this issue as well for the longest time. Switching it to: Machine: Q35-3.1 BIOS: OVMF Hyper-V: Yes Graphics Card: AMD Radeon RX 470/480/570/570X/580/590 Solved it for me... but I could only use Google Chrome Remote desktop with it. To play games I would ALWAYS get black screens and I could no longer login. What works for me. Is to set the first GPU as VNC then add the my RX 580 as my second GPU. I get two screens on my VM. I then do duplicated display and then disable the VNC from the manage device settings. It then works perfectly. Bit of a hassle but AMD cards are hit or miss apparently. Quote Link to comment
PrisonIndustry Posted November 22, 2020 Share Posted November 22, 2020 this is very intersting with problems the rx 500 cards. i have problems, i must be update my bios and i have other errors, rx are bad for vms. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.