sixkittens

Okay then, what version of Unraid are you running? These lockups seem to no longer occur on Unraid 7.3.

It sounds like you might be having a different issue to what is discussed here if you are unable to open the VM page in the Unraid UI. Have you tried rebooting your server?

Make sure that you do not have the GPU or sound device bound to VFIO at boot, this seems to invariably cause the GPU to lock up (I have no idea why). Then try to boot your VM and if you still get no video, check the VM log by clicking the VM's icon and selecting from the drop-down menu. The log should tell you if there are errors preventing video output.

Was this with or without CPU migration enabled? There's a known issue with Linux VM instability that occurs when CPU migration is turned off

Meanwhile I have been on 7.3.0 for a few days, shutting my VM down each night without the system hanging once so far. Hopefully the trend continues. I did have to enable CPU migration to prevent instability issues I was having but this seems to be exclusive to Linux VM's.

I recommend unbinding the GPU from VFIO, and testing again. Not sure how you got it to work before, but binding to VFIO (for me anyway) causes the GPU to get stuck in the wrong power state and there's no video output as a result. I intend to look further into what might be causing the lockup when bound, but I don't know enough about how these things work to have a good starting point. I have added a ROM that I dumped myself from GPU-Z, and updated to 7.3.0, which has a newer kernel, so it may yet prevent the lockups when restarting the vm.

I haven't had an opportunity to test this, but there is a known VM instability for hosts on 6.18 or newer where VM migration is disabled, so enabling migration may fix the issue. https://discussion.fedoraproject.org/t/kvm-guests-become-unstable-on-6-18-kernel/182870/11 Edit: to clarify, that's Linux kernel 6.18, not referring to an Unraid version Edit 2: I can confirm that enabling CPU migration has prevented the kernel panics for me, at least for the trigger that I was able to identify (KDE Discover checking for updates)

Similar issue here with 7.3 but uncertain if it's related. My Fedora VM boots fine, but after login I get a kernel panic. It appears to be related to the kernel version of Unraid, because I had the same issue a while back when I was testing a newer kernel to try and fix passthrough issues. Weirdly the kernel panic happens whenever KDE Discover tries to find updates. Does your VM try to call packagekit or something at boot time?

Hi Arie, Are you referring to a vbios? Any more information you can provide would be helpful as we have been struggling with this for months now (I had mostly given up and just never shut down my VM). I have a vbios file that I dumped directly from my card but it did not fix anything. So if you can confirm what the romfile is and where you got it, as well as any other steps you took to get things working that would be much appreciated.

Nothing definitive that I've been able to find. There's a few recommendations out there that work for some people, but I haven't found anything that makes an actual difference in my testing. I'm hopeful that the 7.3 update might have a positive effect, moving to the 6.18 kernel. In my case, there's about a 20% (guessing) chance of the GPU locking up after VM stop and crashing the server if the VM starts again, but none of the suggested fixes have made an improvement on that for me.

In your post you mention the AMD vendor-reset module/plugin which is only compatible with cards up to the Navi 14 architecture, i.e 5000 series cards, newer cards that experience the vendor reset bug when passing through are not supported by the module and, as far as I am aware, require fixes from the in-kernel drivers. Additionally, the issue being discussed in this thread isn't to do with being able to pass the card through as such, it's preventing the hard freeze that occurs once a VM using the card is shut down and a new VM session starts. If you are interested in reading up on what other people have written who have managed to prevent the reset bug on newer cards, I'll link some resources here: https://forum.level1techs.com/t/vfio-pass-through-working-on-9070xt/227194 https://forum.proxmox.com/threads/working-win11-vm-wont-boot-after-passthrough-of-gpu-vendor-reset-not-able-to-set-reset_method-solution-update-kernel-unbind-via-hook-script.166362/ https://forum.proxmox.com/threads/working-amd-rx-9070xt-support.163370/ https://www.reddit.com/r/unRAID/comments/1j52fkc/unable_to_pass_through_amd_rx_9070_xt/ So far I have not had much success following the steps in the threads linked above, but for whatever reason my server doesn't freeze every time, possibly because I am running a Linux VM rather than Windows. Since having a newer host kernel seems to play a role in other users' success, I am hoping for a stable custom kernel for Unraid 7.2 to be released (linked in the reddit thread) before I try anything further, as the unstable one caused my VM to repeatedly kernel panic. In all cases where people have it working, it appears to be necessary to manually unbind and rebind the GPU before starting a new VM session. Unfortunately these issues with 7000 and 9000 series cards don't seem to be well known outside of the hobby virtualization communities, and Limetech don't seem to have any ideas, so a definite fix might not be forthcoming any time soon.

Adding this line in the syslinux command line does not seem to prevent the GPU from binding to D3 when the GPU is bound to VFIO at boot (as mentioned before, it does not do it for me when it's unbound), nor do spaceinvaderone's suggestions of disabling the framebuffer and resizeable BAR. I decided to revisit this because despite not having the shutdown issue with my Linux VM, I am on rare occasion seeing the server freeze when I try to start the VM instead. I have not had much luck so far though. Edit: The weird thing is that the VM log says the GPU is stuck in D3 and won't restart if I shut it down, like I would expect if it was stuck in D3, but cat /sys/bus/pci/devices/0000:03:00.0/power_state always reports D0

In the qemu command line? Doesn't seem to help with the D3 power issue for me in my test just now when binding my GPU for the Linux VM. I can run the VM unbound but when the VM is off the displays don't power down so eventually I'd like to be able to bind the GPU again. Haven't tested windows yet Edit: nevermind, when unbound it does actually seem to fix the monitors staying on. How strange. I still get no video out when bound, which is indicative of some problem still existing, but that seems to do something at least. I'll try test windows tomorrow. Edit 2: Scratch that, it doesn't get saved if entered into the qemu args section. Guess the change I noticed was from binding and unbinding the GPU.

The issue does not occur with Linux-based VM's only Windows ones.

Unfortunately I haven't been able to set up a Windows VM to test the issue, for some reason the installation hangs for both Windows 10 and 11. That said I have had no issues with my Linux VM since updating my previous post. Edit: managed to install Win 11, but it's no go. "vfio: Unable to power on device, stuck in D3" showing up in the logs and a hard reset is still needed to stop the VM if the OS shuts down even after adding the line from my earlier message to the XML.