Chenhi Posted January 5, 2021 Share Posted January 5, 2021 Hi guys, I managed to do gpu passthrough to my Win10 VM with RTX 2070 Super but after I restart the Unraid Server, it will not work anymore. I tried to delete the vm and reinstall again it works but still, after I restarted ,it will give me the same error again. Please help. Quote 2021-01-05T14:29:21.054639Z qemu-system-x86_64: vfio: Cannot reset device 0000:09:00.1, depends on group 31 which is not owned. <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10 2</name> <uuid>6bfcea46-158d-4390-4f91-6917f17eefed</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>16777216</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>12</vcpu> <cputune> <vcpupin vcpu='0' cpuset='2'/> <vcpupin vcpu='1' cpuset='10'/> <vcpupin vcpu='2' cpuset='3'/> <vcpupin vcpu='3' cpuset='11'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='12'/> <vcpupin vcpu='6' cpuset='5'/> <vcpupin vcpu='7' cpuset='13'/> <vcpupin vcpu='8' cpuset='6'/> <vcpupin vcpu='9' cpuset='14'/> <vcpupin vcpu='10' cpuset='7'/> <vcpupin vcpu='11' cpuset='15'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-5.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/6bfcea46-158d-4390-4f91-6917f17eefed_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='6' threads='2'/> <cache mode='passthrough'/> <feature policy='require' name='topoext'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10 2/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Windows10pro.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.190-1.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x8'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x9'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0xa'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0xb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:6d:72:88'/> <source bridge='br0'/> <model type='virtio-net'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/gtx2070.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </hostdev> <memballoon model='none'/> </devices> </domain> Quote Link to comment
Chenhi Posted January 6, 2021 Author Share Posted January 6, 2021 Anyone can help? Quote Link to comment
derpuma Posted January 6, 2021 Share Posted January 6, 2021 10 hours ago, Chenhi said: Anyone can help? <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x0' multifunction='on'/> </source> <rom file='/mnt/user/isos/gtx2070.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/> </hostdev> Try multifunction! Quote Link to comment
Chenhi Posted January 7, 2021 Author Share Posted January 7, 2021 <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/gtx2070.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/> i changed and still the same problem. the VM can start but still the same error message. can you advise me? thanks Quote Link to comment
salaroli Posted January 8, 2021 Share Posted January 8, 2021 You need to edit your YML and include the device on group 31 Could you please post your IOMMU groups? Quote Link to comment
salaroli Posted January 8, 2021 Share Posted January 8, 2021 (edited) <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/gtx2070.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x2'/> </hostdev> Will problably look something like this. Edited January 8, 2021 by salaroli Quote Link to comment
Chenhi Posted January 10, 2021 Author Share Posted January 10, 2021 IOMMU group 21: [1022:43c6] 02:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge (rev 01) IOMMU group 22: [1022:43c7] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) IOMMU group 23: [1022:43c7] 03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) IOMMU group 24: [1022:43c7] 03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) IOMMU group 25: [1022:43c7] 03:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) IOMMU group 26: [1022:43c7] 03:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) IOMMU group 27: [8086:1539] 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 28: [1b4b:9215] 08:00.0 SATA controller: Marvell Technology Group Ltd. Device 9215 (rev 11) [9:0:0:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sdf 4.00TB [10:0:0:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sdg 4.00TB [11:0:0:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sdh 4.00TB [12:0:0:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sdi 4.00TB IOMMU group 29: [10de:1e84] 09:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER] (rev a1) IOMMU group 30: [10de:10f8] 09:00.1 Audio device: NVIDIA Corporation TU104 HD Audio Controller (rev a1) IOMMU group 31: [10de:1ad8] 09:00.2 USB controller: NVIDIA Corporation TU104 USB 3.1 Host Controller (rev a1) Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub IOMMU group 32: [10de:1ad9] 09:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU104 USB Type-C UCSI Controller (rev a1) IOMMU group 33: [10de:1cb1] 0a:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P1000] (rev a1) IOMMU group 34: [10de:0fb9] 0a:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1) IOMMU group 35: [1cc1:8201] 0b:00.0 Non-Volatile memory controller: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive (rev 03) [N:1:1:1] disk ADATA SX8200PNP__1 /dev/nvme1n1 1.02TB Hi @salaroli, please refer to the above iommu group that i have. Quote Link to comment
Chenhi Posted January 10, 2021 Author Share Posted January 10, 2021 @salaroli somehow it works now although it still give me error but pointing to another group now. thank you so much 2021-01-10 05:13:27.866+0000: Domain id=2 is tainted: high-privileges 2021-01-10 05:13:27.866+0000: Domain id=2 is tainted: host-cpu char device redirected to /dev/pts/1 (label charserial0) 2021-01-10T05:13:28.712750Z qemu-system-x86_64: vfio: Cannot reset device 0000:09:00.2, depends on group 32 which is not owned. Quote Link to comment
Chenhi Posted January 10, 2021 Author Share Posted January 10, 2021 and its broken again.... Quote Link to comment
salaroli Posted January 10, 2021 Share Posted January 10, 2021 You need to pass every group that they are asking for, in the XML Quote Link to comment
Chenhi Posted January 11, 2021 Author Share Posted January 11, 2021 @salaroli I updated the xml with the groups and the errors are gone but there's no gpu passthrough. Just saw your own post with the similar problem. Hopefully there's someone who can advise us. Quote Link to comment
salaroli Posted January 11, 2021 Share Posted January 11, 2021 If I figure out in mine I will try to help you :) Quote Link to comment
bastl Posted January 11, 2021 Share Posted January 11, 2021 22 hours ago, salaroli said: You need to pass every group that they are asking for, in the XML @Chenhi Be careful with this. Don't blindly passthrough what is shown. If the GPU for example is in the same group with one of your NVME drives or an storage controller this will cause your server to crash and in the worste case a data loss. Only passthrough devices which really belong to the device which you want for example the USB controller and the audio part of your card. Sometimes it might help to use another PCIe slot for the device you wanna passthrough. Quote Link to comment
Chenhi Posted January 12, 2021 Author Share Posted January 12, 2021 12 hours ago, bastl said: @Chenhi Be careful with this. Don't blindly passthrough what is shown. If the GPU for example is in the same group with one of your NVME drives or an storage controller this will cause your server to crash and in the worste case a data loss. Only passthrough devices which really belong to the device which you want for example the USB controller and the audio part of your card. Sometimes it might help to use another PCIe slot for the device you wanna passthrough. I have another gpu Quadro P1000 in another slot for plex hw transcoding. Ill try to switch slots for both gpu and see if it helps. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.