So I'm trying to get my windows VM to use my Tesla P40 video card. It was working fine in the past but now it is not. Nothing has changed between it working and not working (that i'm aware of). On a fresh restart of unraid I am able to boot the VM and use the video card as normal. After some time the video card becomes unusable. Windows inside the VM at this point will tell me everything is fine (the application using the card is the only one complaining) and device manager says everything is fine too. OR everything in Windows complains including device manager, saying something about error code 12.

If I attempt to restart the VM after the video card error. I get this message:

A search online said something about binding the video card to the VFIO on boot. So I did that and when I restarted unraid the entire server locked up when it attempted to launch the VM. I mean really weird stuff happened. SSH would fail but the webui would kind of load. sometimes not. commands on the console would take multiple minutes to execute. I managed to very quickly get into the VM settings and disable the VM service before the array could start allowing me to delete the libvirt.img and reset the VM settings. At that point I also disabled the VFIO binding not wanting to risk that again.

So that is where I am at right now.

If you need any more info let me know.

waffle-diagnostics-20260423-1218.zip

You said this use to work.

1. Do you possibly have to re-enable above 4g decoding in the bios?

2. Does that video card require audio to be passed thru as well, cuz i see no audio devices on ur xml. Many nvidia cards needs multifunction enabled to work properly.

3. According to your xml you dont' have the arguments used to set the memory allocation for larger nvidia cards, which if you don't can cause issues. I don't remember it exactly but essentially you have to make ur pci memory larger to accommodate the large nvidia card.

4. You are running windows 11 on i440x? That may be the whole issue right there, windows 11. If you are hardware limited, maybe use windows 10. Or goto q35 for windows 11. That may be the whole issue tho. I really think you need q35 and maybe make a new vm w/ that first.

I think if you google referenced into ai/google/ you would get the specific commands. But thats my initial take.

12 hours ago, zs893284 said:

You said this use to work.

1. Do you possibly have to re-enable above 4g decoding in the bios?

2. Does that video card require audio to be passed thru as well, cuz i see no audio devices on ur xml. Many nvidia cards needs multifunction enabled to work properly.

3. According to your xml you dont' have the arguments used to set the memory allocation for larger nvidia cards, which if you don't can cause issues. I don't remember it exactly but essentially you have to make ur pci memory larger to accommodate the large nvidia card.

4. You are running windows 11 on i440x? That may be the whole issue right there, windows 11. If you are hardware limited, maybe use windows 10. Or goto q35 for windows 11. That may be the whole issue tho. I really think you need q35 and maybe make a new vm w/ that first.

I think if you google referenced into ai/google/ you would get the specific commands. But thats my initial take.

1. I thought so too. but when i upgraded from unraid 6.x.x to 7.x.x and installed windows 11 (fresh install from windows 10). I didn't need above 4g xml settings anymore. BIOS has remained untouched.

2. IDK if the Tesla P40 has audio. Its a data center card of the GTX 1080 era.

3. I am aware of this, but as #1 above. For some reason I never had to do this with the new windows 11 VM and updated unraid.

4. This weird problem exists when using i440x and Q35.

Chatgpt wants to suggest that unraid or a docker is keeping control of the video card. Hence the VFIO binding attempt.

From my experience the above 4g decoding would just fail outright if it wasn't set correctly. Current conditions has the video card work for some while. Then after some time it'll show errors in windows. And a VM restart won't bind the card to the VM. It'll only work for a few hours my guess after a full server restart.

Edited April 24Apr 24 by offroadguy56

Minor update. I renabled the resize bar in the XML. Unfortunately I am away from the server and cannot check the BIOS. When the VM boots device manager says everything is ok. But when i try and execute CLI nvidia-smi. It tells me: unabel to determine device handle for 0000:09:05:0 GPU is lost. Reboot to recover GPU.

I tried rebinding the GPU to VFIO on boot and the GPU still isn't working correctly. Getting the same as above.

I guess now that I've complained about it it has stopped working entirely. I'll see if I can reseat the card and I'll take a peak in the BIOS to confirm above 4G decode and etc.

OK I'm pretty sure I found the problem. The AIO that I had on the P40 has leaked. I never caught it because the tubes ran outside the case and had leaked onto the carpet. But I caught it this time when I reseated the video card and saw a water droplet on the tube.

I'm going to assume that the card is either shutting off from thermal runaway or it has already fried.

I'll see if thermaltake will warranty the cooler. Thankfully they still sell coolers with the same mounting style.

Sorry for late reply, but seems like you may have found the issue. I apologize i wasn't able to provide the solution but glad you were able to get it fixed!