ytddewqf Posted January 18, 2021 Share Posted January 18, 2021 (edited) Hi, I've tried all day to get my 3090 playing nicely with UNRAID. So far I feel like I'm on the home stretch, but just need a little more help. When I try and run my Windows VM I get the execution error at the bottom of this post. For reference, my XML is; <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10</name> <uuid>XXXRedactedXXX</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>33030144</memory> <currentMemory unit='KiB'>33030144</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>10</vcpu> <cputune> <vcpupin vcpu='0' cpuset='5'/> <vcpupin vcpu='1' cpuset='15'/> <vcpupin vcpu='2' cpuset='6'/> <vcpupin vcpu='3' cpuset='16'/> <vcpupin vcpu='4' cpuset='7'/> <vcpupin vcpu='5' cpuset='17'/> <vcpupin vcpu='6' cpuset='8'/> <vcpupin vcpu='7' cpuset='18'/> <vcpupin vcpu='8' cpuset='9'/> <vcpupin vcpu='9' cpuset='19'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-5.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/d453c6c4-7f4b-0b7c-2bf4-a0569f800301_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='5' threads='2'/> <cache mode='passthrough'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Win10_20H2_v2_English_x64.iso'/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.190-1.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <interface type='bridge'> <mac address='52:54:00:10:74:ca'/> <source bridge='br0'/> <model type='virtio-net'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/vbios/rtx3090.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <memballoon model='none'/> </devices> </domain> Lastly, here is my Syslinux Configuration; Any help would be greatly appreciated. ***Edit*** I have tried Q35-5.1 however I get the following error instead; Edited January 19, 2021 by LoneTraveler Quote Link to comment
runamuk Posted January 18, 2021 Share Posted January 18, 2021 (edited) machine='pc-i440fx-5.1 /// switch to Q35 by making a whole new VM with the same settings and selecting Q35 prior to ever starting it; You get that PCI controller error when there is still PC-i440Fx setup in the XML. I'm not sure what motherboard you have but did you enable IMMOU in your bios? Something in your XML is off(missing) can you take a screenshot of your VM page Edited January 18, 2021 by runamuk Quote Link to comment
ytddewqf Posted January 19, 2021 Author Share Posted January 19, 2021 (edited) 10 hours ago, runamuk said: machine='pc-i440fx-5.1 /// switch to Q35 by making a whole new VM with the same settings and selecting Q35 prior to ever starting it; You get that PCI controller error when there is still PC-i440Fx setup in the XML. I'm not sure what motherboard you have but did you enable IMMOU in your bios? Something in your XML is off(missing) can you take a screenshot of your VM page Hi, Thank you for taking the time to reply. Below are two screenshots of my VM Manager page and System Information; I have an MSI Z490 Godlike motherboard. I have scoured the bios and enabled everything I can find relating to virtualisation. ***EDIT*** I have tried all four options under; "PCIe ACS override". Edited January 19, 2021 by LoneTraveler Quote Link to comment
SimonF Posted January 19, 2021 Share Posted January 19, 2021 2 hours ago, LoneTraveler said: find relating to virtualisation. Hi, Have you bound all parts of the card, for example my rtx2060 has a usb controller even thou there is not a physical port. This is on 6.8.3 with vfio plugin. Can you show the output from the system devices in tools? Looks like your are on 6.9.x should look similar to the below. I dont have a GPU on my test system but this is the bmc graphics card. Quote Link to comment
ytddewqf Posted January 19, 2021 Author Share Posted January 19, 2021 (edited) 36 minutes ago, SimonF said: Hi, Have you bound all parts of the card, for example my rtx2060 has a usb controller even thou there is not a physical port. This is on 6.8.3 with vfio plugin. Can you show the output from the system devices in tools? Looks like your are on 6.9.x should look similar to the below. I dont have a GPU on my test system but this is the bmc graphics card. Hi, Thank you for replying. I could only see two bindings for the 3090, 15 & 16. I'm not sure if the 760 card (17 & 18) are causing some sort of conflict. Edited January 19, 2021 by LoneTraveler Quote Link to comment
SimonF Posted January 19, 2021 Share Posted January 19, 2021 @LoneTraveler I cannot see anything obvious. Are you able to see in the bios which device is being used as the primary display? I don't use a rom for my rtx2060 or gt720 and both are connected to two windows VMs, but again I do have graphics from onboard BMC. Guessing you have already tried without a rom, can you passthru gt760 does that work? Quote Link to comment
ytddewqf Posted January 19, 2021 Author Share Posted January 19, 2021 (edited) On 3/13/2017 at 3:27 PM, 1812 said: It's is probably complaining because you're trying to only pass through 1 device from the group when allowing unsafe interrupts (which i won't allow,) and conversely not allowing unsafe interrupts when only passing the single devices which isn't working because you are not using the correct device id's from your specific cards. Try this in your syslinux.cfg: append vfio-pci.ids=XXXX:XXXX,YYYY:YYYY vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot XXXX:XXXX is your video card id YYYY:YYYY is your sound card id you can obtain your video/sound card id's by going to tools>system devices, and look under pci device for 09:00.0 and 09.00.1 the device id will be at the end of he description in the brackets. replace the X's and Y's above with those numbers, reboot, and try again. If it doesn't work, then post your server diagnostics and xml for the vm. After a bit more searching, the above worked for me. I edited my syslinux.cfg file and added the following line; vfio_iommu_type1.allow_unsafe_interrupts=1 Thank you @SimonF @1812 @runamuk @Maor for your assistance. Edited January 19, 2021 by LoneTraveler 2 Quote Link to comment
SimonF Posted January 19, 2021 Share Posted January 19, 2021 7 minutes ago, LoneTraveler said: Thank you No worries, you may not need the ACS overide, unless you need that for other reasons. Glad you got it working, I didnt know about the unsafe_ints as don't need it on my system but will remember in case I need in future, thanks for posting your find/resolution. 1 Quote Link to comment
ytddewqf Posted January 19, 2021 Author Share Posted January 19, 2021 I thought I hade spoke too soon. Upon restart of the server, I received a; "can't reserve error". The solution however was to run the following; echo 0 > /sys/class/vtconsole/vtcon0/bind echo 0 > /sys/class/vtconsole/vtcon1/bind echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind This needs to be run upon each restart of UNRAID and so a quick script sorts that out. Quote Link to comment
KptnKMan Posted January 26, 2021 Share Posted January 26, 2021 Hey @LoneTraveler I just upgraded my passthrough 1080Ti to a 3090 this past weekend. I used the method by SpaceInvaderOne but I made a guide here: I made my 3090 work using the exact same method, and I'm using it right now, rock solid, multiple restarts, everything working. I'd advise taking a look at the HOWTO I wrote above, try the instructions precisely, and let me know if it helped you. 1 Quote Link to comment
KptnKMan Posted January 26, 2021 Share Posted January 26, 2021 @LoneTraveler can you mention what exact 3090 model you're using? I can't find it anywhere in this thread. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.