Hi guys, I have strange issue with passing through my Nvidia GTX 580 GPU to Win 10 VM.

 

System details:

Model: Custom
M/B: ASUSTeK COMPUTER INC. RAMPAGE IV FORMULA Version Rev 1.xx - s/n: 113693900000034
BIOS: American Megatrends Inc. Version 5001. Dated: 12/05/2014
CPU: Intel® Core™ i7-3820 CPU @ 3.60GHz
HVM: Enabled
IOMMU: Enabled
Cache: 32 KiB, 256 KiB, 10240 KiB
Memory: 16 GiB DDR3 (max. installable capacity 96 GiB)
Network: bond0: fault-tolerance (active-backup), mtu 1500
 eth0: 1000 Mbps, full duplex, mtu 1500
Kernel: Linux 4.19.94-Unraid x86_64

Only one GPU available for Unraid and VM

GPU - Nvidia Asus ROG Direct CU GTX580

Bios - Direct CU Bios from techpowerup

GPU and Audio in the same IOMMU group

All required setiings enabled in BIOS to enable GPU passthrough

IOMMU group 18:	[10de:1080] 01:00.0 VGA compatible controller: NVIDIA Corporation GF110 [GeForce GTX 580] (rev a1)
[10de:0e09] 01:00.1 Audio device: NVIDIA Corporation GF110 High Definition Audio Controller (rev a1)

 

Issue description:

So when I passthrough GPU using spaceinvader video manual I`m able to start Win 10 VM and can see VGA adapter. But after installation of lates Nvidia drivers for this GPU, Win 10 VM unable to boot. I`m unable to ping it and connect via RDP.

 

Have tried to install different drivers, one was from ASUS web site, one from Windows, with no success. Also passing through without BIOS give the same issue. Have tried Win 7 VM but again the same result received. 

 

Win 10 VM logs below:

ErrorWarningSystemArrayLogin


-m 8192 \
-overcommit mem-lock=off \
-smp 6,sockets=1,cores=3,threads=2 \
-uuid 5934f64e-9065-4564-a79b-82ba2db94883 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.0,addr=0x7 \
-device ahci,id=sata0,bus=pci.0,addr=0x4 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Windows 10 x64/Windows 10 version 1909 En-Ru x64 6-In-1 [02.2020].iso","node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw","file":"libvirt-3-storage"}' \
-device ide-cd,bus=sata0.0,drive=libvirt-3-format,id=sata0-0-0,bootindex=2 \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/VirtIO Drivers ISO/virtio-win-0.1.171.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device ide-cd,bus=sata0.1,drive=libvirt-2-format,id=sata0-0-1 \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 10/vdisk1.img","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \
-device ide-hd,bus=sata0.2,drive=libvirt-1-format,id=sata0-0-2,bootindex=1,write-cache=on \
-netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=37 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:9c:75:9c,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=36,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0,bus=usb.0,port=2 \
-device 'vfio-pci,host=0000:01:00.0,id=hostdev0,bus=pci.0,multifunction=on,addr=0x3,romfile=/mnt/user/Distr/Drivers/Nvidia/BIOS ROM/Direct CU/Asus.GTX580.1536.110311.rom' \
-device vfio-pci,host=0000:01:00.1,id=hostdev1,bus=pci.0,addr=0x3.0x1 \
-device usb-host,hostbus=1,hostaddr=3,id=hostdev2,bus=usb.0,port=1 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2020-04-13 03:19:15.365+0000: Domain id=1 is tainted: high-privileges
2020-04-13 03:19:15.365+0000: Domain id=1 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)

 

Also I have no video signal on my monitor attached via HDMI, but i`m suppose that this is because of drivers was not installed correctly. But i have seen Unraid boot screen on the same monitor when started the server.

 

Have similar issue that described here. Mybe the reason of this issue is that I have only one GPU without integrated to motherboard and when unraid boot up it utilize GTX580 so that it can`t be passthrough normally to the VM? Can somebody tell if it is possible to disable unraid from utilizing GPU so that it can be connected as free one to the VM?

12 hours ago, ionedji said:

Have similar issue that described here. Mybe the reason of this issue is that I have only one GPU without integrated to motherboard and when unraid boot up it utilize GTX580 so that it can`t be passthrough normally to the VM? Can somebody tell if it is possible to disable unraid from utilizing GPU so that it can be connected as free one to the VM?

No. What display to boot with (or none at all) is a functionality of the motherboard BIOS.

 

There are 3 different "Direct CU" vbios on the website. Are you sure you got the right one?

If you can dump your own vbios, that would be best. Otherwise, there's no way to know for sure your symptom isn't due to a wrong vbios.

 

The GTX 580 is also a very old card. It could just not be happy with passing through.