NVIDIA Passthrough Doing My Head In

[PeteyBoPetey]

Hi Guys,

^ (Every time I say that now, I have Space Invader One's voice and accent in my head)

 

I have 3 GPU's for 3 VM's with no iGPU. When I create a VM I don't get any error messages just a black screen.

 

• ACS overide enabled and GPU's, their sound and USB counter parts are in separate IOMMU groups
• Edited my Syslynix.cfg so unraid doesn't use any of the GPU's. It still boots into terminal though? Followed this video
• Downloaded vbios from techpowerup and removed header
• Tried both OVMF and SEAbios (whatever they are?)
• Tried Q35 machine (don't know what this is?)

 

The only other thing it could be as Space Invader One has pointed out, is that only the graphics portion of the card is passed through and the NVIDA card knows it's in a VM and subsequently having a dummy spit. When I was on Unraid 6.8.something the XML template added the .0 (video) and .1 (audio) to XML. Now I'm on Unraid 6.9.0 it only adds the .0 (video).

 

Questions:

1. How do I edit the XML to put the video, audio and USB 3 together in one PCIe virtual slot?
2. When editing the XML how do I know the number of the physical slot each of the 3 GPU's are in on the motherboard so I edit the XML correctly?. Motherboard is an ASUS X299 E II
3. When I place the NVIDA video, audio and USB 3 together into a PCIe slot on the virtual motherboard, do I need to continue the numbering logically. i.e if the last slot on the physical motherboard is 5, do I start the PCIe number for the virtual motherboard at 6?

 

I really need to get this up and running soon because University is about to start and I'm attending via online classes. I've transferred a lot of my drives/files to the Unraid machine and the PC I'm on now is on it's last legs. Is there anyone out there prepared to help me with editing the XML?. I've spent days trying to trouble shoot this, I'm at wits end.

 

Peter

[PeteyBoPetey]

-smp 24,sockets=1,dies=1,cores=12,threads=2 \
-uuid 43df6fc0-2d1d-8347-557e-673a5f9b66bb \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=31,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 10 Dads/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
-device virtio-blk-pci,bus=pci.0,addr=0x4,drive=libvirt-3-format,id=virtio-disk2,bootindex=1,write-cache=on \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Win10_20H2_v2_English_x64.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device ide-cd,bus=ide.0,unit=0,drive=libvirt-2-format,id=ide0-0-0,bootindex=2 \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.173-2.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device ide-cd,bus=ide.0,unit=1,drive=libvirt-1-format,id=ide0-0-1 \
-netdev tap,fd=33,id=hostnet0 \
-device virtio-net,netdev=hostnet0,id=net0,mac=52:54:00:8e:f9:3e,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=35,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device vfio-pci,host=0000:65:00.0,id=hostdev0,bus=pci.0,multifunction=on,addr=0x5,romfile=/mnt/cache/isos/Asus.RTX2080Ti.11264.180907_1.rom \
-device vfio-pci,host=0000:65:00.1,id=hostdev1,bus=pci.0,addr=0x5.0x1 \
-device usb-host,hostbus=1,hostaddr=8,id=hostdev2,bus=usb.0,port=1 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2021-03-03 10:57:11.148+0000: Domain id=2 is tainted: high-privileges
2021-03-03 10:57:11.148+0000: Domain id=2 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2021-03-03T10:57:15.001872Z qemu-system-x86_64: vfio: Cannot reset device 0000:65:00.1, depends on group 92 which is not owned.

[PeteyBoPetey]

Solved Guys.

 

I added a No VGA option in Unraids start menu in the syslynix.cfg file as shown in this Video but didn't remove 'default menu' under the first menu option. So I had 2 default options. Even if I selected the No VGA option Unraid still grabbed the GPU. So I removed the first default option and made 'No VGA' the only default option.