Alirgion Posted May 3, 2019 Share Posted May 3, 2019 Hi, For a few days now, me and my friend have been working on a project very similar to Linus's two gamers one CPU. First one was a success, however the second one is a total bust since it's not booting into Windows installation. I've tried different BIOS, switching Hyper-V, different drives, we tried different pci-e slots on motherboard and we are not able to fix it so I come for help here. Our specifications: Asus Prime X399-A AMD Threadripper 1950X 32 GB RAM NVidia GT 710 (host) NVidia GTX 1080TI (guest (working)) NVidia GTX 650 (guest (not working)) Log's of VM in question: Quote 2019-05-03 12:28:00.036+0000: starting up libvirt version: 4.7.0, qemu version: 3.0.0, kernel: 4.18.20-unRAID, hostname: Tower LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10 GTX 650,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-Windows 10 GTX 650/master-key.aes' -machine pc-i440fx-3.0,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=none -m 8192 -realtime mlock=off -smp 6,sockets=1,cores=6,threads=1 -uuid 45de98e2-1618-4f20-3686-d5b8a5f3e6fb -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=27,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-hpet -no-shutdown -boot strict=on -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.0,addr=0x7 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive 'file=/mnt/user/domains/Windows 10 GTX 650/vdisk1.img,format=raw,if=none,id=drive-virtio-disk2,cache=writeback' -device virtio-blk-pci,scsi=off=6,id=hostdev3,bus=usb.0,port=2 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on 2019-05-03 12:28:00.036+0000: Domain id=2 is tainted: high-privileges 2019-05-03 12:28:00.036+0000: Domain id=2 is tainted: host-cpu 2019-05-03T12:28:00.082470Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/0 (label charserial0) libusb: error [udev_hotplug_event] ignoring udev action bind IOMMUs Quote IOMMU group 0:[1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 1:[1022:1453] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 2:[1022:1453] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 3:[1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 4:[1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 5:[1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 6:[1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 7:[1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 8:[1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 9:[1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 10:[1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 11:[1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 12:[1022:1460] 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:19.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:19.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 13:[1022:43ba] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller (rev 02) IOMMU group 14:[1022:43b6] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller (rev 02) IOMMU group 15:[1022:43b1] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge (rev 02) IOMMU group 16:[1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 17:[1022:43b4] 02:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 18:[1022:43b4] 02:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 19:[1022:43b4] 02:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 20:[1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 21:[1022:43b4] 02:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 22:[8086:1539] 05:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 23:[1b21:2142] 08:00.0 USB controller: ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller IOMMU group 24:[10de:0fc6] 09:00.0 VGA compatible controller: NVIDIA Corporation GK107 [GeForce GTX 650] (rev a1) IOMMU group 25:[10de:0e1b] 09:00.1 Audio device: NVIDIA Corporation GK107 HDMI Audio Controller (rev a1) IOMMU group 26:[1022:145a] 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 27:[1022:1456] 0a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 28:[1022:145c] 0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller IOMMU group 29:[1022:1455] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 30:[1022:7901] 0b:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 31:[1022:1457] 0b:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller IOMMU group 32:[1022:1452] 40:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 33:[1022:1453] 40:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 34:[1022:1452] 40:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 35:[1022:1452] 40:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 36:[1022:1453] 40:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 37:[1022:1452] 40:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 38:[1022:1452] 40:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 39:[1022:1454] 40:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 40:[1022:1452] 40:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 41:[1022:1454] 40:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 42:[10de:1b06] 41:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1) IOMMU group 43:[10de:10ef] 41:00.1 Audio device: NVIDIA Corporation GP102 HDMI Audio Controller (rev a1) IOMMU group 44:[10de:128b] 42:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 710] (rev a1) IOMMU group 45:[10de:0e0f] 42:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1) IOMMU group 46:[1022:145a] 43:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 47:[1022:1456] 43:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 48:[1022:145c] 43:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller IOMMU group 49:[1022:1455] 44:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 50:[1022:7901] 44:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) Any additional info I'll gladly provide. Thank you in advance for any help Quote Link to comment
jordanmw Posted May 3, 2019 Share Posted May 3, 2019 I am guessing it is the GTX 650 that is having the issue. I would try swapping slots on those cards and testing to see if the 650 can provide video for any of the VMs. I know that some of the older nvidia cards are a real pain to get working. May I suggest that you run headless instead of dedicating a card to unraid? It's kind of a waste to dedicate a GPU to unraid since the only real advantage is managing the server without a network. That is how I am currently setup- my last VM is setup to take over unraid's host card when it boots. Then you just manage unraid from the web gui or ssh. I currently have 2x gtx960 and 2x rtx2070s setup with 4 game stations on a 1920x with no issues. Quote Link to comment
Alirgion Posted May 3, 2019 Author Share Posted May 3, 2019 4 minutes ago, jordanmw said: I am guessing it is the GTX 650 that is having the issue. I would try swapping slots on those cards and testing to see if the 650 can provide video for any of the VMs. I know that some of the older nvidia cards are a real pain to get working. May I suggest that you run headless instead of dedicating a card to unraid? It's kind of a waste to dedicate a GPU to unraid since the only real advantage is managing the server without a network. That is how I am currently setup- my last VM is setup to take over unraid's host card when it boots. Then you just manage unraid from the web gui or ssh. I currently have 2x gtx960 and 2x rtx2070s setup with 4 game stations on a 1920x with no issues. My ultimate goal is to have two people playing simultaneously on both VMs, and since TR doesn't have on board gpu i need to pass both of those "gaming" gpus through GT710. And one of those gpus is working actually I can play on 1080ti without any issues however my machine with gtx650 passed through is "starting" but it's just not showing anything on monitor. So I don't exactly know how to change the situation. Also I've booted unraid on GTX 650 just to check if it's working at all and it is because unRAID gui showed with no problem, however GT710 is definitely not a card I would use for gaming. So in summary and much needed shortcut, I need some kind of fix for this. Quote Link to comment
jordanmw Posted May 3, 2019 Share Posted May 3, 2019 Why don't you try to blacklist the GPU and pass a bios file to it for that machine?- that may work better. I had to do that for the GPU that unraid wanted to use for itself. Just a thought. Also, you may need to pass a modified bios to that machine even without blacklisting it. Quote Link to comment
Alirgion Posted May 3, 2019 Author Share Posted May 3, 2019 I do apologize for my lack of knowledge I've just started to look into unRAID and I am not an experienced user. Could you explain or direct me to some kind of tutorial where I could see how you do this? 2 hours ago, jordanmw said: Why don't you try to blacklist the GPU and pass a bios file to it for that machine?- that may work better. I had to do that for the GPU that unraid wanted to use for itself. Just a thought. Also, you may need to pass a modified bios to that machine even without blacklisting it. Quote Link to comment
jordanmw Posted May 3, 2019 Share Posted May 3, 2019 These are the basics- Pay attention to the bios dumping and editing. It's kind of an advanced hack since it requires a hex editor- but not too bad. Really important process for picky GPUs. Quote Link to comment
Alirgion Posted May 3, 2019 Author Share Posted May 3, 2019 The problem is that the first step of this tutorial is to power on the machine and get to device manager. I however can't do that because I do not have a working display output. Quote Link to comment
jordanmw Posted May 3, 2019 Share Posted May 3, 2019 You do need to boot a machine with that GPU installed to dump the bios from the card. I did this by installing windows 10 on a separate drive that is dedicated to a bare metal install. You can always wipe the drive and add it back to the array when you are done. Just install win 10 bare metal and dump your bios with GPUZ. Alternately you might be able to find one on techpowerup.com that will work. For that matter, for burning in the machine- I usually stress test with bare metal before I start an unraid build. Once you have the bios dumped, you can remove the nvidia header from it, then save it as a .rom that you can point to that file in your xml for the VM. Quote Link to comment
Alirgion Posted May 3, 2019 Author Share Posted May 3, 2019 Do you think that if I install windows by VNC the card will work? Quote Link to comment
jordanmw Posted May 3, 2019 Share Posted May 3, 2019 Some have had a lot of luck doing that- but it never worked for me- always gave me a code 43 error in device manager if I installed with VNC, then removed it- and added a GPU. You could give it a shot. I have installed windows 10 maybe 100 times while testing different setup processes- would have saved me a bunch of time if I had just sys-prepped a vm image. My experience was that if at any point I added VNC as the primary video- the GPU would give me code 43 when I added it. Quote Link to comment
Alirgion Posted May 3, 2019 Author Share Posted May 3, 2019 Okay, I will try it. And if it doesn’t work I’ll try dumping bios. Although I still don’t understand why do you need to open firstly this card on a machine. I mean this is a working card it was in my old computer so I honestly don’t know what is turning it in a system on is supposed to give Quote Link to comment
jordanmw Posted May 3, 2019 Share Posted May 3, 2019 It is purely to dump the bios from the card. As I said- sometimes you can go to https://www.techpowerup.com/vgabios/ and look for your bios there to save yourself some work. Quote Link to comment
Alirgion Posted May 3, 2019 Author Share Posted May 3, 2019 So I've tried dumping with this tutorial and unfortunately no luck. I still can't see anything on my screen. Obviously my VNC plan was a bust too. Any other ideas? Quote Link to comment
Alirgion Posted May 3, 2019 Author Share Posted May 3, 2019 Logs: Quote 2019-05-03 20:58:31.406+0000: starting up libvirt version: 4.7.0, qemu version: 3.0.0, kernel: 4.18.20-unRAID, hostname: Tower LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10 GTX 650,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-Windows 10 GTX 650/master-key.aes' -machine pc-i440fx-3.0,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=none -m 8192 -realtime mlock=off -smp 12,sockets=1,cores=12,threads=1 -uuid f8c8ec8b-ea3e-f55c-1a91-3ded5d1cab98 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=27,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-hpet -no-shutdown -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus0 -device vfio-pci,host=09:00.0,id=hostdev0,x-vga=on,bus=pci.0,addr=0x5,romfile=/mnt/user/isos/gtx650.dump -device vfio-pci,host=09:00.1,id=hostdev1,bus=pci.0,addr=0x6 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on 2019-05-03 20:58:31.407+0000: Domain id=10 is tainted: high-privileges 2019-05-03 20:58:31.407+0000: Domain id=10 is tainted: host-cpu 2019-05-03T20:58:31.458310Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/2 (label charserial0) Quote Link to comment
jordanmw Posted May 3, 2019 Share Posted May 3, 2019 It may just be incompatibility with that card. As I said- some others have had major issues getting some older nvidia cards to work. If you can get your hands on a newer card- to test- you may prove that to be true. I don't have anything that old to test with unfortunately. Sorry. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.