Kewjoe Posted March 11, 2019 Share Posted March 11, 2019 (edited) I'm having a lot of trouble passing through my RX 580 (XFX Radeon RX 580 8GB GTS XXX Edition). My system was working fine with an old ATI 5450 card. Even with OVMF selected. What have I tried? all sorts of combinations of the following: - UEFI and Legacy boot mode for unraid - video=efifb:off in syslinux settings - stubbing the the card (and audio) in syslinux - both OVMF and SeaBIOS. SeaBIOS will post something, but often it will say "no bootable devices found" or "booting from hard drive" instead of booting from the install media - tried both i440 and Q35 - tried passing a bios file from tech powerup This is happening with both Windows 10 and Linux Mint 19.1. Fresh installs as well as existing installs I had from the 5450. I mostly just get a black screen. with Linux Mint, i've gotten a blinking cursor. What am I doing wrong? I'm running this on an E5-2699v4, 32g of ECC DDR4 and off an Intel P3600 1.6TB cache drive. Everything is the same as when things worked fine with my 5450. Just added the RX580 and doesn't matter what i do I cannot get it to work Edited March 12, 2019 by Kewjoe Quote Link to comment
Warrentheo Posted March 11, 2019 Share Posted March 11, 2019 (edited) Step 1 is to double check that the card works as intended on a baremetal boot... Can't pass broken hardware... Try turning off any motherboard legacy boot or CSM boot option, this will force UEFI boot for UnRaid and the GPU... Windows 10 should use OVMF for UEFI in the VM also... VBIOS passing is only a thing for NVidia cards, because currently they are the only ones that are encrypting their VBIOS and the connection between it and their drivers... video=efifb:off should not be needed... VFIO-PCI drivers for the GPU and its audio card (to be more specific, everything in its IOMMU group except the PCIe slot) in syslinux should help... I prefer the Q35 machine because it fixed some compatibility issues for me, but I have been told that i440fx is slightly faster and works for alot of people... Start there, and if that doesn't work, we will need the XML from the VM and the diagnostics file from the host... Edit: OOPS, vfio-pci drivers should never be used for "CPU"... Meant to say "GPU" >.< Edited March 12, 2019 by Warrentheo 1 Quote Link to comment
GreenEyedMonster Posted March 11, 2019 Share Posted March 11, 2019 So I'm having a similar issue with my 580. My 1070 would pass just fine but my 580 in slot 2 won't let me login. Here is my XML...https://justpaste.it/237zq and my diagnostics...media-diagnostics-20190311-1425.zip I have tried almost everything and still no go. It boots up fine under VNC. Any help would be greatly appreciated it! Quote Link to comment
Warrentheo Posted March 11, 2019 Share Posted March 11, 2019 1 hour ago, GreenEyedMonster said: So I'm having a similar issue with my 580. My 1070 would pass just fine but my 580 in slot 2 won't let me login. Here is my XML...https://justpaste.it/237zq and my diagnostics...media-diagnostics-20190311-1425.zip I have tried almost everything and still no go. It boots up fine under VNC. Any help would be greatly appreciated it! In the future, please create a new thread and monitor this one, we don't want to hijack the thread since they are unlikely to be the same exact issue... I noticed you had the UnRaid NVidia plugin installed, what do you use the 1070 for? p.s. you should answer that in your own thread... Quote Link to comment
GreenEyedMonster Posted March 11, 2019 Share Posted March 11, 2019 55 minutes ago, Warrentheo said: In the future, please create a new thread and monitor this one, we don't want to hijack the thread since they are unlikely to be the same exact issue... I noticed you had the UnRaid NVidia plugin installed, what do you use the 1070 for? p.s. you should answer that in your own thread... Noted and made. Quote Link to comment
Kewjoe Posted March 12, 2019 Author Share Posted March 12, 2019 10 hours ago, Warrentheo said: Step 1 is to double check that the card works as intended on a baremetal boot... Can't pass broken hardware... Try turning off any motherboard legacy boot or CSM boot option, this will force UEFI boot for UnRaid and the GPU... Windows 10 should use OVMF for UEFI in the VM also... VBIOS passing is only a thing for NVidia cards, because currently they are the only ones that are encrypting their VBIOS and the connection between it and their drivers... video=efifb:off should not be needed... VFIO-PCI drivers for the CPU and its audio card (to be more specific, everything in its IOMMU group except the PCIe slot) in syslinux should help... I prefer the Q35 machine because it fixed some compatibility issues for me, but I have been told that i440fx is slightly faster and works for alot of people... Start there, and if that doesn't work, we will need the XML from the VM and the diagnostics file from the host... Thanks for the reply. Here are the answers (and additional questions): - I pulled the RX580 out of a fully working bare metal setup. Was working fine there. It obviously displays the bios/boot/etc. I've even had it work a little bit in both windows and linux. But it wasn't sustainable (hence this thread). - I've tried both fully UEFI and legacy. When doing UEFI, i ensured it was done properly. It was displaying the unraid menu in 1080P and unraid confirmed that it was indeed in UEFI mode. - Yes, I am using OVMF for Windows 10. Seabios isn't working (although it made it to the install screen) - Yes, I'm aware. I was just desperate 😁 - I saw a thread in General Support that mentioned adding that. Thought I had found the golden ticket as I never came across it before. But it didn't work - Tell me more about the VFIO-PCI drivers for CPU and Audio? I'm not familiar with this. - I also prefer Q35 (it is what i used exclusively when i had working VM's with other video cards) The latest update since my post yesterday is that I bought a Zotac Geforce 730GT PCI-E 1x card today. I don't have any free x16 slots and I wanted to try having two video cards. I haven't had much time to play around to see if it makes a difference. My first boot, even though the 730GT is in PCIE slot 1, and the RX580 is in slot 2, it displayed the boot/bios/etc. on the AMD card and not the Nvidia. I can't find any options in the BIOS to change that behaviour (I have an Asrock X99 Taichi). Quote Link to comment
Kewjoe Posted March 12, 2019 Author Share Posted March 12, 2019 I'm not having any success with the added second GPU (GT 710). Exactly the same behaviour. My motherboard bios doesn't allow me to select what the primary GPU is, so it continues to default to the RX580. Attached is a XML for a windows 10 install i tried. Also my diagnostics. windows10.xml tower-diagnostics-20190312-0408.zip Quote Link to comment
Warrentheo Posted March 12, 2019 Share Posted March 12, 2019 13 hours ago, Kewjoe said: - Tell me more about the VFIO-PCI drivers for CPU and Audio? I'm not familiar with this. Stuff plugged into PCIe slots are not like USB ports, they require reset signals from the motherboard to reset themselves and connect to a new computer... The process of passing through a GPU to a VM is a bit like moving the GPU from one computer to another without powering anything down first, so we need the Host to not be using it at the time, and we need to be able to send a reset signal to it when we connect it to the VM instead... If you boot the linux kernel normally, it expects to be able to use every device it discovers, and so will load drivers for the GPU, and then use it to show stuff on the screen... To prevent UnRaid from using the GPU, we need to tell the linux kernel during boot to not hook into the GPU, we do that by telling it to use the vfio-pci driver for the GPU instead... This "driver" is more like a plug for something else to provide the driver instead, and just specifies the shape of that plug rather than do anything to the GPU... If you "stub" one item in an IOMMU group, you should stub them all, and pass all items in the group (except the PCIe slot) together... You are going to want to edit your syslinux to re-arrange somethings: BOOT_IMAGE=/bzimage vfio-pci.ids=1002:67df,1002:aaf0,1b73:1100,8086:10d6,8086:15a1,1b21:1242 video=efifb:off initrd=/bzroot The "initrd=/bzroot" should almost always be at the end, and the "video=efifb:off" also should not be needed... You will probably also want to add "10de:128b,10de:0e0f,19da:5360" for your NVidia card if you plan to pass it through... Unfortunately alot of this comes down to how your motherboard interacts with the GPU's you put in it, so some of this comes down to experimentation on your part to see what works for you... Quote Link to comment
Kewjoe Posted March 12, 2019 Author Share Posted March 12, 2019 13 minutes ago, Warrentheo said: Stuff plugged into PCIe slots are not like USB ports, they require reset signals from the motherboard to reset themselves and connect to a new computer... The process of passing through a GPU to a VM is a bit like moving the GPU from one computer to another without powering anything down first, so we need the Host to not be using it at the time, and we need to be able to send a reset signal to it when we connect it to the VM instead... If you boot the linux kernel normally, it expects to be able to use every device it discovers, and so will load drivers for the GPU, and then use it to show stuff on the screen... To prevent UnRaid from using the GPU, we need to tell the linux kernel during boot to not hook into the GPU, we do that by telling it to use the vfio-pci driver for the GPU instead... This "driver" is more like a plug for something else to provide the driver instead, and just specifies the shape of that plug rather than do anything to the GPU... If you "stub" one item in an IOMMU group, you should stub them all, and pass all items in the group (except the PCIe slot) together... You are going to want to edit your syslinux to re-arrange somethings: BOOT_IMAGE=/bzimage vfio-pci.ids=1002:67df,1002:aaf0,1b73:1100,8086:10d6,8086:15a1,1b21:1242 video=efifb:off initrd=/bzroot The "initrd=/bzroot" should almost always be at the end, and the "video=efifb:off" also should not be needed... You will probably also want to add "10de:128b,10de:0e0f,19da:5360" for your NVidia card if you plan to pass it through... Unfortunately alot of this comes down to how your motherboard interacts with the GPU's you put in it, so some of this comes down to experimentation on your part to see what works for you... I understand how stubbing works, i think i got confused because you said "CPU" instead of "GPU" and i was like "wait what?". But we're on the same page now as for my weird syslinux, it is a product of my experimenting to make it work. My initrd=/bzroot is usually at the end and i don't normally pass video=efifb:off. Just trying anything at this point. my syslinux is back to normal now: kernel /bzimage append vfio-pci.ids=1002:67df,1002:aaf0,1002:68e1,1002:aa68,1b73:1100,8086:15a1,1b21:1242 initrd=/bzroot Latest update: I took out the zotac 1x card and put in my old ATI 5450. But I had to remove my 4 port nic (future pfsense vm) to fit. With the 5450 on top and the RX580 in second slot, everything is working fine. Created a Windows VM with OVMF and it worked, created a Linux VM with OVMF and it worked. This wasn't my ideal setup as I wanted that 4 port NIC in here, but I'll just make due. I don't have integrated graphics with my CPU, but with the 5450 alone, it would work fine with OVMF. Which is funny because the 5450 is not a UEFI card. But the RX580 by itself, refused to work. Ah well. At least I got it working. @Warrentheo Thanks for taking the time to help! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.