Second Machine not working


Recommended Posts

Hi,

For a few days now, me and my friend have been working on a project very similar to Linus's two gamers one CPU. First one was a success, however the second one is a total bust since it's not booting into Windows installation. I've tried different BIOS, switching Hyper-V, different drives, we tried different pci-e slots on motherboard and we are not able to fix it so I come for help here.

 

Our specifications:

Asus Prime X399-A

AMD Threadripper 1950X

32 GB RAM

NVidia GT 710 (host)

NVidia GTX 1080TI (guest (working))

NVidia GTX 650 (guest (not working))

 

Log's of VM in question:

Quote

2019-05-03 12:28:00.036+0000: starting up libvirt version: 4.7.0, qemu version: 3.0.0, kernel: 4.18.20-unRAID, hostname: Tower
LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10 GTX 650,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-Windows 10 GTX 650/master-key.aes' -machine pc-i440fx-3.0,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=none -m 8192 -realtime mlock=off -smp 6,sockets=1,cores=6,threads=1 -uuid 45de98e2-1618-4f20-3686-d5b8a5f3e6fb -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=27,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-hpet -no-shutdown -boot strict=on -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.0,addr=0x7 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive 'file=/mnt/user/domains/Windows 10 GTX 650/vdisk1.img,format=raw,if=none,id=drive-virtio-disk2,cache=writeback' -device virtio-blk-pci,scsi=off=6,id=hostdev3,bus=usb.0,port=2 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
2019-05-03 12:28:00.036+0000: Domain id=2 is tainted: high-privileges
2019-05-03 12:28:00.036+0000: Domain id=2 is tainted: host-cpu
2019-05-03T12:28:00.082470Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/0 (label charserial0)
libusb: error [udev_hotplug_event] ignoring udev action bind

 

IOMMUs

Quote

IOMMU group 0:[1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 1:[1022:1453] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge

IOMMU group 2:[1022:1453] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge

IOMMU group 3:[1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 4:[1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 5:[1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 6:[1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 7:[1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B

IOMMU group 8:[1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 9:[1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B

IOMMU group 10:[1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59)

[1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)

IOMMU group 11:[1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0

[1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1

[1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2

[1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3

[1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4

[1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5

[1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6

[1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7

IOMMU group 12:[1022:1460] 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0

[1022:1461] 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1

[1022:1462] 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2

[1022:1463] 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3

[1022:1464] 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4

[1022:1465] 00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5

[1022:1466] 00:19.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6

[1022:1467] 00:19.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7

IOMMU group 13:[1022:43ba] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller (rev 02)

IOMMU group 14:[1022:43b6] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller (rev 02)

IOMMU group 15:[1022:43b1] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge (rev 02)

IOMMU group 16:[1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

IOMMU group 17:[1022:43b4] 02:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

IOMMU group 18:[1022:43b4] 02:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

IOMMU group 19:[1022:43b4] 02:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

IOMMU group 20:[1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

IOMMU group 21:[1022:43b4] 02:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

IOMMU group 22:[8086:1539] 05:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)

IOMMU group 23:[1b21:2142] 08:00.0 USB controller: ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller

IOMMU group 24:[10de:0fc6] 09:00.0 VGA compatible controller: NVIDIA Corporation GK107 [GeForce GTX 650] (rev a1)

IOMMU group 25:[10de:0e1b] 09:00.1 Audio device: NVIDIA Corporation GK107 HDMI Audio Controller (rev a1)

IOMMU group 26:[1022:145a] 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a

IOMMU group 27:[1022:1456] 0a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor

IOMMU group 28:[1022:145c] 0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller

IOMMU group 29:[1022:1455] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455

IOMMU group 30:[1022:7901] 0b:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)

IOMMU group 31:[1022:1457] 0b:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller

IOMMU group 32:[1022:1452] 40:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 33:[1022:1453] 40:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge

IOMMU group 34:[1022:1452] 40:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 35:[1022:1452] 40:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 36:[1022:1453] 40:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge

IOMMU group 37:[1022:1452] 40:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 38:[1022:1452] 40:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 39:[1022:1454] 40:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B

IOMMU group 40:[1022:1452] 40:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge

IOMMU group 41:[1022:1454] 40:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B

IOMMU group 42:[10de:1b06] 41:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1)

IOMMU group 43:[10de:10ef] 41:00.1 Audio device: NVIDIA Corporation GP102 HDMI Audio Controller (rev a1)

IOMMU group 44:[10de:128b] 42:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 710] (rev a1)

IOMMU group 45:[10de:0e0f] 42:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1)

IOMMU group 46:[1022:145a] 43:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a

IOMMU group 47:[1022:1456] 43:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor

IOMMU group 48:[1022:145c] 43:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller

IOMMU group 49:[1022:1455] 44:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455

IOMMU group 50:[1022:7901] 44:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)

 

 

Any additional info I'll gladly provide.

 

Thank you in advance for any help

Link to comment

I am guessing it is the GTX 650 that is having the issue.  I would try swapping slots on those cards and testing to see if the 650 can provide video for any of the VMs.  I know that some of the older nvidia cards are a real pain to get working.  May I suggest that you run headless instead of dedicating a card to unraid? It's kind of a waste to dedicate a GPU to unraid since the only real advantage is managing the server without a network. That is how I am currently setup- my last VM is setup to take over unraid's host card when it boots.  Then you just manage unraid from the web gui or ssh.  I currently have 2x gtx960 and 2x rtx2070s setup with 4 game stations on a 1920x with no issues.

Link to comment
4 minutes ago, jordanmw said:

I am guessing it is the GTX 650 that is having the issue.  I would try swapping slots on those cards and testing to see if the 650 can provide video for any of the VMs.  I know that some of the older nvidia cards are a real pain to get working.  May I suggest that you run headless instead of dedicating a card to unraid? It's kind of a waste to dedicate a GPU to unraid since the only real advantage is managing the server without a network. That is how I am currently setup- my last VM is setup to take over unraid's host card when it boots.  Then you just manage unraid from the web gui or ssh.  I currently have 2x gtx960 and 2x rtx2070s setup with 4 game stations on a 1920x with no issues.

 

My ultimate goal is to have two people playing simultaneously on both VMs, and since TR doesn't have on board gpu i need to pass both of those "gaming" gpus through GT710. And one of those gpus is working actually I can play on 1080ti without any issues however my machine with gtx650 passed through is "starting" but it's just not showing anything on monitor. So I don't exactly know how to change the situation. Also I've booted unraid on GTX 650 just to check if it's working at all and it is because unRAID gui showed with no problem, however GT710 is definitely not  a card I would use for gaming.

 

So in summary and much needed shortcut, I need some kind of fix for this.

Link to comment

Why don't you try to blacklist the GPU and pass a bios file to it for that machine?- that may work better.  I had to do that for the GPU that unraid wanted to use for itself.  Just a thought.  Also, you may need to pass a modified bios to that machine even without blacklisting it.    

Link to comment

I do apologize for my lack of knowledge I've just started to look into unRAID and I am not an experienced user. Could you explain or direct me to some kind of tutorial where I could see how you do this?

2 hours ago, jordanmw said:

Why don't you try to blacklist the GPU and pass a bios file to it for that machine?- that may work better.  I had to do that for the GPU that unraid wanted to use for itself.  Just a thought.  Also, you may need to pass a modified bios to that machine even without blacklisting it.    

 

Link to comment

You do need to boot a machine with that GPU installed to dump the bios from the card.  I did this by installing windows 10 on a separate drive that is dedicated to a bare metal install.  You can always wipe the drive and add it back to the array when you are done.  Just install win 10 bare metal and dump your bios with GPUZ.  Alternately you might be able to find one on techpowerup.com that will work.

 

For that matter, for burning in the machine- I usually stress test with bare metal before I start an unraid build.  Once you have the bios dumped, you can remove the nvidia header from it, then save it as a .rom that you can point to that file in your xml for the VM.

Link to comment

Some have had a lot of luck doing that- but it never worked for me- always gave me a code 43 error in device manager if I installed with VNC, then removed it- and added a GPU.  You could give it a shot.  I have installed windows 10 maybe 100 times while testing different setup processes- would have saved me a bunch of time if I had just sys-prepped a vm image.

 

My experience was that if at any point I added VNC as the primary video- the GPU would give me code 43 when I added it.

Link to comment

Okay, I will try it. And if it doesn’t work I’ll try dumping bios. Although I still don’t understand why do you need to open firstly this card on a machine. I mean this is a working card it was in my old computer so I honestly don’t know what is turning it in a system on is supposed to give 

Link to comment

Logs:

 

Quote

2019-05-03 20:58:31.406+0000: starting up libvirt version: 4.7.0, qemu version: 3.0.0, kernel: 4.18.20-unRAID, hostname: Tower
LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10 GTX 650,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-Windows 10 GTX 650/master-key.aes' -machine pc-i440fx-3.0,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=none -m 8192 -realtime mlock=off -smp 12,sockets=1,cores=12,threads=1 -uuid f8c8ec8b-ea3e-f55c-1a91-3ded5d1cab98 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=27,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-hpet -no-shutdown -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus0 -device vfio-pci,host=09:00.0,id=hostdev0,x-vga=on,bus=pci.0,addr=0x5,romfile=/mnt/user/isos/gtx650.dump -device vfio-pci,host=09:00.1,id=hostdev1,bus=pci.0,addr=0x6 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
2019-05-03 20:58:31.407+0000: Domain id=10 is tainted: high-privileges
2019-05-03 20:58:31.407+0000: Domain id=10 is tainted: host-cpu
2019-05-03T20:58:31.458310Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/2 (label charserial0)

 

Link to comment

It may just be incompatibility with that card.  As I said- some others have had major issues getting some older nvidia cards to work.  If you can get your hands on a newer card- to test- you may prove that to be true.  I don't have anything that old to test with unfortunately.  Sorry.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.