W11 vm hang under gpu load

[dcchillin46]

So im really at a loss at this point. I've spent the better part of a week trying to get a w11 vm up and running. Ive started at least 10 vms at this point trying every setting combo i can think of. I've got it functional, except whenever I go into a game the whole system hangs. Sometimes immediately, sometimes a few min into a benchmark run. The strange part is I've been able to run a 5min stress test through radeon software with 0 issue, but launching 3dmark right after causes it to hang.

 

It passes through the gpu and hdmi audio to vfio. I've used 1440fx, q35, hyper visor and non, spice and vnc. I'm using gpu 1 as virtual so the web portal will launch, then running gpu 2 as my rx580. Windows device manager sees the device fine, drivers install fine, and 3dmark even recognizes it to start the benchmark. I've tried using a vbios, but it seems to be more stable without. I've even tried running it as gpu 1 and connecting via steam link, but thats a no go. I've also tried parsec, with the same results.

 

I see all these posts and videos about the vm running close to bare metal, but I've put probably 15 hours into this so far and can't even get a benchmark to run. Upside is I've tried literally every setting in the unraid options, downside is I've tried every setting in the unraid options. it's so exhausting.

 

Anyone who can help with troubleshooting or something I may be missing would be greatly appreciated. Between this and being unable to get containers running for my 3d printer, unraid is becoming very frustrating over the last week or two.

 

Edit most recent attempt ends with this in the logs

 

`2023-09-14T23:41:25.917890Z qemu-system-x86_64: vfio_dma_map(0x14e477e5e600, 0x381000000000, 0x200000000, 0x14e26ec00000) = -22 (Invalid argument)`  

`2023-09-14T23:41:25.918364Z qemu-system-x86_64: VFIO_MAP_DMA failed: Invalid argument`  

`2023-09-14T23:41:25.918371Z qemu-system-x86_64: vfio_dma_map(0x14e477e5e600, 0x381200000000, 0x200000, 0x14e26d800000) = -22 (Invalid argument)`  

`2023-09-14T23:41:25.982628Z qemu-system-x86_64: VFIO_MAP_DMA failed: Invalid argument`

 

My xml: https://imgur.com/SAx6kFA