jordanmw Posted October 2, 2018 Share Posted October 2, 2018 (edited) It seems that no matter what I do, I can't get the code 43 error to go away with passthrough GPUs. I have 4 GTX960s and am trying to get a 4 gamers 1 cpu build going with my threadripper build. Here is what I have: 1920x threadripper X399 Asrock TaiChi (bios 3.20) 1kw EVGA PSU 64 Gb vengeance corsair RAM 4x EVGA GTX960 SSC I have dumped the bios from those cards and am selecting the .rom file within the machine setup. It does recognize the card- but will not load the driver with a code 43. It obviously has something to do with not loading because it sees the virtual machine. I need some help getting things going- and am willing to pay if needed. This is driving me a little crazy since I went through the guide: https://forums.unraid.net/topic/69670-guide-fix-nvidia-code-43-issue-on-nvidia-gpu/ and even got an older image of windows 10- but am still having problems. I ran through the process in the guide with versions 388 and 411 but am having no luck installing- code 43 every time. System info: IOMMU group 0:[1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 1:[1022:1453] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 2:[1022:1453] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 3:[1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 4:[1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 5:[1022:1453] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 6:[1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 7:[1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 8:[1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 9:[1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 10:[1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 11:[1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 12:[1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 13:[1022:1460] 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:19.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:19.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 14:[1022:43ba] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller (rev 02) IOMMU group 15:[1022:43b6] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller (rev 02) IOMMU group 16:[1022:43b1] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge (rev 02) IOMMU group 17:[1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 18:[1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 19:[1022:43b4] 02:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 20:[1022:43b4] 02:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 21:[1022:43b4] 02:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) IOMMU group 22:[8086:1539] 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 23:[8086:24fb] 05:00.0 Network controller: Intel Corporation Dual Band Wireless-AC 3168NGW [Stone Peak] (rev 10) IOMMU group 24:[8086:1539] 06:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 25:[10de:1401] 08:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) IOMMU group 26:[10de:0fba] 08:00.1 Audio device: NVIDIA Corporation GM206 High Definition Audio Controller (rev a1) IOMMU group 27:[10de:1401] 09:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) IOMMU group 28:[10de:0fba] 09:00.1 Audio device: NVIDIA Corporation GM206 High Definition Audio Controller (rev a1) IOMMU group 29:[1022:145a] 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 30:[1022:1456] 0a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 31:[1022:145c] 0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller IOMMU group 32:[1022:1455] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 33:[1022:7901] 0b:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 34:[1022:1457] 0b:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller IOMMU group 35:[1022:1452] 40:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 36:[1022:1453] 40:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 37:[1022:1452] 40:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 38:[1022:1452] 40:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 39:[1022:1453] 40:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 40:[1022:1452] 40:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 41:[1022:1452] 40:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 42:[1022:1454] 40:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 43:[1022:1452] 40:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 44:[1022:1454] 40:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 45:[10de:1401] 41:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) IOMMU group 46:[10de:0fba] 41:00.1 Audio device: NVIDIA Corporation GM206 High Definition Audio Controller (rev a1) IOMMU group 47:[10de:1401] 42:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) IOMMU group 48:[10de:0fba] 42:00.1 Audio device: NVIDIA Corporation GM206 High Definition Audio Controller (rev a1) IOMMU group 49:[1022:145a] 43:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 50:[1022:1456] 43:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 51:[1022:145c] 43:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller IOMMU group 52:[1022:1455] 44:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 53:[1022:7901] 44:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) CPU Thread Pairings Pair 1:cpu 0 / cpu 12 Pair 2:cpu 1 / cpu 13 Pair 3:cpu 2 / cpu 14 Pair 4:cpu 3 / cpu 15 Pair 5:cpu 4 / cpu 16 Pair 6:cpu 5 / cpu 17 Pair 7:cpu 6 / cpu 18 Pair 8:cpu 7 / cpu 19 Pair 9:cpu 8 / cpu 20 Pair 10:cpu 9 / cpu 21 Pair 11:cpu 10 / cpu 22 Pair 12:cpu 11 / cpu 23 USB Devices Bus 001 Device 001:ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002:ID 8087:0aa7 Intel Corp. Bus 001 Device 003:ID 2433:b200 Bus 002 Device 001:ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 003 Device 001:ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 002:ID 046d:c52b Logitech, Inc. Unifying Receiver Bus 004 Device 001:ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 005 Device 001:ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 005 Device 002:ID 045e:0745 Microsoft Corp. Nano Transceiver v1.0 for Bluetooth Bus 006 Device 001:ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 006 Device 002:ID 0781:5583 SanDisk Corp. Ultra Fit SCSI Devices [0:0:0:0]disk SanDisk Ultra Fit 1.00 /dev/sda 30.7GB [1:0:0:0]disk ATA WDC WD30PURZ-85G 0A80 /dev/sdb 3.00TB [2:0:0:0]disk ATA WDC WD30EZRX-00M 0A80 /dev/sdc 3.00TB [5:0:0:0]disk ATA PLEXTOR PX-512S2 1.04 /dev/sdd 512GB [7:0:0:0]disk ATA WDC WD30EZRX-00M 0A80 /dev/sde 3.00TB VM XML: <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10-Lance</name> <uuid>d6c004a2-4090-d2c2-eeda-2ef31aae118c</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>10485760</memory> <currentMemory unit='KiB'>2097152</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>4</vcpu> <cputune> <vcpupin vcpu='0' cpuset='6'/> <vcpupin vcpu='1' cpuset='18'/> <vcpupin vcpu='2' cpuset='7'/> <vcpupin vcpu='3' cpuset='19'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/d6c004a2-4090-d2c2-eeda-2ef31aae118c_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='1234567890ab'/> </hyperv> <kvm> <hidden state='on'/> </kvm> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='4' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10-Lance/vdisk1.img'/> <target dev='hdc' bus='scsi'/> <boot order='1'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Win10_1607_English_x64.iso'/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.141.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='scsi' index='0' model='virtio-scsi'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:48:96:de'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </source> <rom file='/mnt/disk1/isos/GM2061304.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </memballoon> </devices> </domain> System Log: ct 2 11:51:50 Tower avahi-daemon[2580]: Leaving mDNS multicast group on interface vnet0.IPv6 with address fe80::fc54:ff:fe48:96de. Oct 2 11:51:50 Tower kernel: br0: port 3(vnet0) entered disabled state Oct 2 11:51:50 Tower kernel: device vnet0 left promiscuous mode Oct 2 11:51:50 Tower kernel: br0: port 3(vnet0) entered disabled state Oct 2 11:51:50 Tower avahi-daemon[2580]: Withdrawing address record for fe80::fc54:ff:fe48:96de on vnet0. Oct 2 11:51:51 Tower kernel: vfio-pci 0000:08:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Oct 2 11:52:29 Tower kernel: vfio-pci 0000:08:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Oct 2 11:52:29 Tower kernel: br0: port 3(vnet0) entered blocking state Oct 2 11:52:29 Tower kernel: br0: port 3(vnet0) entered disabled state Oct 2 11:52:29 Tower kernel: device vnet0 entered promiscuous mode Oct 2 11:52:29 Tower kernel: br0: port 3(vnet0) entered blocking state Oct 2 11:52:29 Tower kernel: br0: port 3(vnet0) entered listening state Oct 2 11:52:31 Tower kernel: vfio_ecap_init: 0000:08:00.0 hiding ecap 0x1e@0x258 Oct 2 11:52:31 Tower kernel: vfio_ecap_init: 0000:08:00.0 hiding ecap 0x19@0x900 Oct 2 11:52:31 Tower avahi-daemon[2580]: Joining mDNS multicast group on interface vnet0.IPv6 with address fe80::fc54:ff:fe48:96de. Oct 2 11:52:31 Tower avahi-daemon[2580]: New relevant interface vnet0.IPv6 for mDNS. Oct 2 11:52:31 Tower avahi-daemon[2580]: Registering new address record for fe80::fc54:ff:fe48:96de on vnet0.*. Oct 2 11:52:31 Tower kernel: br0: port 3(vnet0) entered learning state Oct 2 11:52:33 Tower kernel: br0: port 3(vnet0) entered forwarding state Oct 2 11:52:33 Tower kernel: br0: topology change detected, propagating Oct 2 13:45:06 Tower avahi-daemon[2580]: Interface vnet0.IPv6 no longer relevant for mDNS. Oct 2 13:45:06 Tower avahi-daemon[2580]: Leaving mDNS multicast group on interface vnet0.IPv6 with address fe80::fc54:ff:fe48:96de. Oct 2 13:45:06 Tower kernel: br0: port 3(vnet0) entered disabled state Oct 2 13:45:06 Tower kernel: device vnet0 left promiscuous mode Oct 2 13:45:06 Tower kernel: br0: port 3(vnet0) entered disabled state Oct 2 13:45:06 Tower avahi-daemon[2580]: Withdrawing address record for fe80::fc54:ff:fe48:96de on vnet0. Oct 2 13:45:07 Tower kernel: vfio-pci 0000:08:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Oct 2 13:46:16 Tower nginx: 2018/10/02 13:46:16 [error] 2634#2634: *87587 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 10.0.0.102, server: , request: "GET /plugins/dynamix.plugin.manager/scripts/plugin&arg1=install&arg2=http://lime-technology.com/forum/index.php?topic=36543.0 HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.0.147", referrer: "http://10.0.0.147/Plugins" Oct 2 13:48:25 Tower emhttpd: req (3): cmd=/plugins/dynamix.plugin.manager/scripts/plugin&arg1=install&arg2=https://raw.github.com/bergware/dynamix/master/unRAIDv6/dynamix.system.info.plg&csrf_token=**************** Oct 2 13:48:25 Tower emhttpd: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin install https://raw.github.com/bergware/dynamix/master/unRAIDv6/dynamix.system.info.plg Oct 2 13:48:54 Tower root: plugin: running: anonymous Oct 2 13:48:54 Tower root: plugin: creating: /boot/config/plugins/dynamix.system.info/dynamix.system.info.txz - downloading from URL https://raw.githubusercontent.com/bergware/dynamix/master/archive/dynamix.system.info.txz Oct 2 13:48:55 Tower root: plugin: checking: /boot/config/plugins/dynamix.system.info/dynamix.system.info.txz - MD5 Oct 2 13:48:55 Tower root: plugin: running: /boot/config/plugins/dynamix.system.info/dynamix.system.info.txz Oct 2 13:48:55 Tower root: plugin: running: anonymous Oct 2 14:12:18 Tower kernel: kvm: already loaded the other module I did add some hyper-v and kvm edits to no avail. Any help would be very much appreciated. tower-diagnostics-20181002-1344.zip Edited October 2, 2018 by jordanmw 1 Quote Link to comment
jonp Posted October 3, 2018 Share Posted October 3, 2018 A few things to try. First, in the motherboard BIOS, do you have an option to enable 4G decoding? If so, please enable that. Another thing to try would be to turn on VFIO allow unsafe interrupts. Also, are you booting using a OVMF or SeaBIOS VM right now? Try changing that and reinstall windows. Quote Link to comment
jordanmw Posted October 3, 2018 Author Share Posted October 3, 2018 I have it setup with OVMF. 4G decoding? is that in the PCIE settings? I have the AMD PBS settings in advanced with a generation 2, ect setting. Quote Link to comment
jordanmw Posted October 3, 2018 Author Share Posted October 3, 2018 Okay, so I tried several different settings- enabled VFIO and tried to boot with only a gpu passed through(no vnc). It just makes windows refuse to boot. Quote Link to comment
bastl Posted October 4, 2018 Share Posted October 4, 2018 Did you modified the dumped VBIOS of your cards or do you pass them trough without manual edit them first removing the NVIDIA header? @jordanmw I already posted the video from SpaceInvader in the other forum. A dumped VBIOS without the small modifications never worked for me. No matter if I dumped it on my own or used a VBIOS from TechPowerUP. And i only have to do this for the card in the first slot. Get your VM up and running with VNC and install some sort of remote management software like Teamviewer, Anydesk or enable RDP. Make sure you install the VIRTIO drivers especially for the network nic. Check what IP you get assigned to the VM. Test if you can connect and than remove VNC. It might look on the first boot that windows hangs. Watch the CPU load on the Unraid main page and ping the VM to check if the network comes up. 1 Quote Link to comment
jordanmw Posted October 4, 2018 Author Share Posted October 4, 2018 No, I haven't edited the bios first. I am doing as you suggested though- I install teamviewer and can connect to the machine until I add the gpu and reboot. Then the machine just never boots. If I add vnc back in with the video card too- it goes to windows setup repair. I am adding the virtio drivers after installing the os and give it a static ip. It never responds to ping after a reboot with any of my gpus attached. Maybe I should hex edit my vbios. I did dump them from the actual cards- but didn't think I had to edit if I did that. Quote Link to comment
jordanmw Posted October 4, 2018 Author Share Posted October 4, 2018 I am also seeing an unidentified device in device manager before changing from vnc to passthrough GPU- anyone know if this could cause issues? Quote Link to comment
jordanmw Posted October 4, 2018 Author Share Posted October 4, 2018 (edited) Followed spaceinvader's guide exactly to setup a new machine- after removing vnc and adding passthrough gpu with rom that was dumped and hexedited- windows does not boot. Teamviewer and rdp enabled and machine never comes back on the network- checked with ping to IP also. Anything else that I can post, to help figure this out>? I also tried one with seabios and it did the same thing. Machine XML: <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10</name> <uuid>6d351a55-5f4c-d492-618e-48d0dd42f29e</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>10485760</memory> <currentMemory unit='KiB'>10485760</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>4</vcpu> <cputune> <vcpupin vcpu='0' cpuset='10'/> <vcpupin vcpu='1' cpuset='22'/> <vcpupin vcpu='2' cpuset='11'/> <vcpupin vcpu='3' cpuset='23'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/6d351a55-5f4c-d492-618e-48d0dd42f29e_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='4' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Win10_1607_English_x64.iso'/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <interface type='bridge'> <mac address='52:54:00:02:39:d8'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x41' slot='0x00' function='0x0'/> </source> <rom file='/mnt/disk1/isos/GM2061279.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x41' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc52b'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </memballoon> </devices> </domain> Edited October 4, 2018 by jordanmw Quote Link to comment
bastl Posted October 4, 2018 Share Posted October 4, 2018 Did you tried to passthrough another card? In some situations the card in the first PCIE slot can be a bit picky. And don't expect if you have 4 similiar cards that the bios of all of them are the same. Also i would recomend to test one of the BIOS from TechPowerUp. The BIOS i used is from this site. Choose the manufacturer of your card, the specific model and download maybe 2 or 3 versions of the BIOS available. For me it never worked if i dump it on my own and modify it for the EVGA 1050ti. The second one from TechPowerUp i modified and passed through worked. 3 hours ago, jordanmw said: I am also seeing an unidentified device in device manager check this: Quote Link to comment
jordanmw Posted October 4, 2018 Author Share Posted October 4, 2018 The device manager hid issue fix is great, thanks. I have grabbed the techpowerup bios for my card and tried that- still won't boot windows after I add one card. Since I have 4 gpus- I have tried each of them and know which one unraid has- based on my screen going black if I assign and start with it attached to a vm. This is driving me a little crazy- maybe have installed windows 100+ times and just can understand where it is failing. I literally just started using unraid and bought the pro license thinking this was an easier system to work with than some of the other distros with the same components.... starting to doubt making that call. I have worked with hyper-v, esxi, and messed around with tons of Linux virtualization- just not with pass through GPUs-- this is starting to drive me a little nuts. 1 Quote Link to comment
jordanmw Posted October 5, 2018 Author Share Posted October 5, 2018 (edited) On 10/2/2018 at 8:37 PM, jonp said: A few things to try. First, in the motherboard BIOS, do you have an option to enable 4G decoding? If so, please enable that. Another thing to try would be to turn on VFIO allow unsafe interrupts. Also, are you booting using a OVMF or SeaBIOS VM right now? Try changing that and reinstall windows. So, when I enable 4g decoding in the boot menu, the computer will not post. I ended up resetting the bios multiple times and even with no other bios tweaks, it won't post with it enabled. I did turn on vfio- what acs setting should I be using? Here are my current cards: I have dumped each of their bios and hex edited the header. I am removing all but 2 of the cards, the one that has always been in the top slot and unraid will choose, and the one faster card of the 4- ftw edition. I can load windows without unraid and put full loads on all cards and p95 the cpu for hours on end with no thermal throttling- so this thing should be rock solid if I can just get unraid working. Still getting the same result for 2 cards installed. VM Windows won't boot without the vnc card added. It will still load if I add a card, and usually let me install drivers, but on reboot, it won't boot. Edited October 5, 2018 by jordanmw Quote Link to comment
jonp Posted October 5, 2018 Share Posted October 5, 2018 Ok, a few more comments to put you on the right path: 1) If enabling 4G Decoding causes the system to not post, you may have defective hardware. I would open a case with ASRock as that is fairly basic motherboard functionality. 2) What happens if you only have 2 GPUs in the system? Can the one that Unraid isn't using work correctly? 3) Have you tried any alternate GPUs? There is no reason the EVGA GeForce 960 shouldn't work because I have that exact same card and it worked fine on my system. 4) Previously I had asked a question and provided a suggestion that I don't see a response to: On 10/2/2018 at 9:37 PM, jonp said: Also, are you booting using a OVMF or SeaBIOS VM right now? Try changing that and reinstall windows. SeaBIOS is definitely worth trying as for many, it resolves the issue. Edit: One last thing I may have missed here, but have you updated your motherboard BIOS to latest? Quote Link to comment
jordanmw Posted October 5, 2018 Author Share Posted October 5, 2018 Nope, having some success now- no bad hardware- thank god. Here is what I have found... If I install vnc for graphics, it will not install the graphics card drivers, and nvidia detecting virtual systems is for real. I'll get a code 43 if I install the regular drivers but if I get wdk and use the powershell script for hacking the drivers with version 388.0, I can install those- then upgrade to the 411 branch with no issues. I just have to do the full install on hardware without ever adding the vnc video. Never had to pass a rom doing that- just install on physical monitor and thnings go better. Bios is latest, OVMF for bios- may try some additional testing to see if I can pin down the issues. Quote Link to comment
jordanmw Posted October 5, 2018 Author Share Posted October 5, 2018 I got all 4 up and running! Now I'm on to the audio issues.... demon sound? Seems like a couple of scripts put together the right way could iron some of this out... did the msi edit, but no dice. Quote Link to comment
bastl Posted October 5, 2018 Share Posted October 5, 2018 For the demon sound their is a solution. I think SpaceInvader also did a video on that. Here are the Unraid wiki entry: https://wiki.unraid.net/index.php/UnRAID_6/VM_Guest_Support#Enable_MSI_for_Interrupts_to_Fix_HDMI_Audio_Support The MSI Util v2 for Windows you can find here: https://forums.guru3d.com/threads/windows-line-based-vs-message-signaled-based-interrupts.378044/ Quote Link to comment
jordanmw Posted October 5, 2018 Author Share Posted October 5, 2018 Yeah, I did that registry edit on all 4 machines but it seems like it didn't help. Do I need to re-install the hd audio driver after I do the edit? Quote Link to comment
JonathanM Posted October 5, 2018 Share Posted October 5, 2018 29 minutes ago, jordanmw said: Yeah, I did that registry edit on all 4 machines but it seems like it didn't help. Do I need to re-install the hd audio driver after I do the edit? Are you sure that you enabled MSI on ALL the passed through bits? I seem to remember looking at a list when I did mine and there was an item hiding not sorted with the rest. Quote Link to comment
jordanmw Posted October 5, 2018 Author Share Posted October 5, 2018 We are just talking about a single edit on each machine, right? Everything that I have followed just suggested finding the one device with the by type view. I find it- shows HD audio- grab the vendor code, then edit the registry with the corresponding hklm\ccs key to a 1, from a 0. Am I missing something there? Quote Link to comment
JonathanM Posted October 5, 2018 Share Posted October 5, 2018 57 minutes ago, jordanmw said: Am I missing something there? It's been a while so I may be misremembering something, but I thought you needed to apply the change to all the pieces of the card, not just the audio portion. Quote Link to comment
jordanmw Posted October 5, 2018 Author Share Posted October 5, 2018 Is that true? None of the documentation I have read explicitly stated that, but I do see where he mentions doing it for all his other devices. I'll give that a shot. Quote Link to comment
JonathanM Posted October 5, 2018 Share Posted October 5, 2018 57 minutes ago, jordanmw said: I do see where he mentions doing it for all his other devices. I don't think all the devices need it, just those directly associated. Quote Link to comment
jordanmw Posted October 5, 2018 Author Share Posted October 5, 2018 I just had something really strange happen, I resized one of the VM disks and after booting- it went back to a code 43 error!! I reinstalled the old drivers and got it working again, but still- this is crazy. Quote Link to comment
jonp Posted October 5, 2018 Share Posted October 5, 2018 13 hours ago, jordanmw said: Nope, having some success now- no bad hardware- thank god. Here is what I have found... How did you rule out bad hardware? Were you able to get the above 4G decoding to work? That is pretty important for larger system setups. We needed that in the video where we did 7 gamers on one machine. 13 hours ago, jordanmw said: If I install vnc for graphics, it will not install the graphics card drivers, and nvidia detecting virtual systems is for real. This is actually not your issue. While NVIDIA makes attempts to detect for virtualization, we have implemented patches that negate that completely. There are literally TONS of users in our community using NVIDIA GPUs with Unraid VMs and have no problems. When you get a Code 43 error, it is because of one of a few things. The most obvious thing is a problem with the hardware, but that's not always the reason. That said, Code 43 will almost always happen if you try to do GPU pass through where VNC is the primary display and the GPU is a secondary. The big question here is what happens when you try to install windows without VNC (just the GPU). This should work with both OVMF and SeaBIOS, but if it doesn't work with OVMF, you might want to try SeaBIOS. Another thing to check is to contact EVGA and see if there are any firmware updates for your card. EVGA doesn't just have them available for download from their website, you actually need to contact them to check if there are real firmware updates available and they will ask you to send them the mode/serial number for your device(s). They are VERY responsive to these requests, so I would definitely do it. 13 hours ago, jordanmw said: I'll get a code 43 if I install the regular drivers but if I get wdk and use the powershell script for hacking the drivers with version 388.0, I can install those- then upgrade to the 411 branch with no issues. I just have to do the full install on hardware without ever adding the vnc video. Never had to pass a rom doing that- just install on physical monitor and thnings go better. Bios is latest, OVMF for bios- may try some additional testing to see if I can pin down the issues. The amount of jumping through hoops you are having to go through to make this work is extremely unusual. We have a threadripper system in our labs with a GTX 1080 Ti that Eric uses in a Windows VM and it works flawlessly. Not sure why you're having problems here, but I STRONGLY recommend you attempt to build your VMs using SeaBIOS instead of OVMF. OVMF should work with any GPUs that support UEFI (which the EVGA 960s do), but just in case, try SeaBIOS to see if that resolves your issue. It may also solve other issues you're experiencing. 1 Quote Link to comment
jordanmw Posted October 5, 2018 Author Share Posted October 5, 2018 Thanks jonp. It did feel like I was jumping through way more hoops than I should have to. I didn't enable the 4G setting and was still able to get everything going if I just pretended that each machine was a separate bare metal install. I think you are right about getting bios updates for my cards- I will try to get that done this weekend and maybe play around with some of the other things you suggested. But right now, I have 4 windows 10 installs with 4 cores each and 10Gb of RAM each running p95 and heaven to test things out. Has been stable for the last 8 hours and no thermal throttling. Quote Link to comment
jordanmw Posted October 9, 2018 Author Share Posted October 9, 2018 Didn't get as much time as I would like to playing around with settings and rebuilding vms this weekend but did get some quality gaming time on it with the wife. Things are running pretty well just using ovmf without passing gpu bios roms and have the demon audio issues worked out- just passing the audio from the graphics cards works with the msi fix- didn't apply it to any devices but that. I think I mentioned that I had a windows 10 install that I can boot from to test overall stability- so I know things are pretty rock solid with the hardware. It does seem like there is some weirdness with re-assigning usb devices but without dedicated controllers to pass through- I can't complain too much. I have booted a couple times and seen an error 43 but usually remove and re-install the driver- then it starts working again. Still experimenting on settings, will post back here with my experiences. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.