darianf Posted February 25, 2017 Share Posted February 25, 2017 @gridrunner - thank you for the awesome tutorials @rahool - i assume your friend has unraid already (or similar meanings to dump), so yes... The rom is specific to the GPU model/vendor, so it's independent of your/your friend's PC. No harm, it will either not work to pass through the GPU, or will not display the video output [properly], or worst case will freeze the unraid box and you'll probably need to powercycle/reset it (e.g. power your PC down, then power it up). My suggestion is to try first what i dumped, before attempting to do it yourself... Just so that you know, my card is the mini version (exact item model number ZT-P10610A-10L) and based on your SFF intention i assume this is the one you want. https://www.newegg.com/Product/Product.aspx?Item=N82E16814500405&cm_re=zotac_1060_mini_3g-_-14-500-405-_-Product Saying this, because there are other 3gb versions available, like AMP (at least i see this model in europe market), but who knows, probably they have same rom... -d Quote Link to comment
rahool Posted February 25, 2017 Share Posted February 25, 2017 thanks @darianf and @gridrunner for your help. @darianf i have the exact model as you have got, as you said i would first try out the dump. once again, thanks for your help. Quote Link to comment
hblockx Posted February 28, 2017 Share Posted February 28, 2017 Is it possible to change primary card of Unraid6 if you have two graphic cards? I have a watercooled setup and it is hard for me to change their positions to get the second card in the upper slot... The whole thing should be still possible when i manage to use the second card for unraid first?! Thanks for help! Quote Link to comment
darianf Posted February 28, 2017 Share Posted February 28, 2017 @hblockx - some motherboards allow selection of primary GPU, from a bios option. Some add this option only in newer versions of the bios. Mine does not (asus x99-m ws) yet and as a result, the slot 1 card is used to output the boot vga, and this is the card where unraid console is displayed. Once i start a VM using this primary card, the console output is taken for displaying the video output of that VM. And it's never returned back to unraid, even if VM stopped, until the unraid gets restarted. Quote Link to comment
hblockx Posted March 1, 2017 Share Posted March 1, 2017 (edited) I do not think i am going to have the option, too. Is there a way to do this with a different linux? Maybe a linux which does not even need a graphics card for terminal (just SSH)? Without any way i cannot get this done Edited March 1, 2017 by hblockx Quote Link to comment
SpaceInvaderOne Posted March 2, 2017 Author Share Posted March 2, 2017 @hblockx I don't know of a way of booting the system with a graphics card present in the machine and for the card not to be shadowed. If you cant move the cards I see you have two options. 1. Post what GPU you have here and hopefully someone with the same GPU as you would dump the rom for you to use. 2 Un seat both GPUs. Use a PCIe riser cable to be able to plug the primary card into the secondary slot. Then use another cable to plug any other card into the first slot. This way you wouldn't have to move the cards much at all. No idea if it would work but maybe? Quote Link to comment
hblockx Posted March 2, 2017 Share Posted March 2, 2017 I also thought about this. I guess i am going to drain my system, getting the card in the second slot, with a borrowed small card in the primary. Then afterwards draining again and changing to old system. Hopefully it works then Thank you anyways and wish me luck, draining and changing a custom-watercooling two times in a row sucks :'D Quote Link to comment
SpaceInvaderOne Posted March 2, 2017 Author Share Posted March 2, 2017 4 hours ago, hblockx said: I also thought about this. I guess i am going to drain my system, getting the card in the second slot, with a borrowed small card in the primary. Then afterwards draining again and changing to old system. Hopefully it works then Thank you anyways and wish me luck, draining and changing a custom-watercooling two times in a row sucks :'D Ah best of luck. I would hate to have to do that! What GPU do you use. Maybe someone else here has the same who has already dumped the rom? Would save a lot of hastle? Quote Link to comment
hblockx Posted March 2, 2017 Share Posted March 2, 2017 Palit 980ti Super Jetstream, it would :'D Quote Link to comment
Kudjo Posted March 4, 2017 Share Posted March 4, 2017 Good news: adding "append vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot" to the syslinux.cfg file worked for getting the VM to boot with the card! Another snag, though. I get the following message whenever I run the unbind/bind command: "-bash: echo: write error: No such device". The BIOS dump only produces a 126KB file, so I don't think it's happening properly. So, two questions: 1) remind me what we need the bios file for again if we can get the card to boot using the UnRaid GUI? (I'm sure we DO, I'm just unclear what the difference/advantage is between bios dumped and added to XML vs not) 2) Is there a way to verify the validity of my bios dump other than filesize? (And if filesize is the only way to check, how do I know I have the right filesize) Almost there! I'm so grateful for your great videos that help me get the most out of unraid. I feel I'm close, I just want to understand what's going on. (Teach me to "fish") Thank you, again! Quote Link to comment
heratic Posted April 1, 2017 Share Posted April 1, 2017 Got a 1080ti bios to add to the collection https://drive.google.com/open?id=0B9xjk7ZlI7QiYnBTbkFFcl80azQ Quote Link to comment
SpaceInvaderOne Posted April 4, 2017 Author Share Posted April 4, 2017 On 01/04/2017 at 9:33 PM, heratic said: Got a 1080ti bios to add to the collection https://drive.google.com/open?id=0B9xjk7ZlI7QiYnBTbkFFcl80azQ Thanks ive added to the op. Which 1080ti is this from please? Quote Link to comment
heratic Posted April 5, 2017 Share Posted April 5, 2017 On 4/4/2017 at 10:02 AM, gridrunner said: Thanks ive added to the op. Which 1080ti is this from please? EVGA founders edition but pretty sure it would work for any reference card. 1 Quote Link to comment
CloseTurkey Posted July 6, 2017 Share Posted July 6, 2017 Does anyone have a gtx 980 rom dump? Or is there any way to get a rom dump in my setup without a 2nd temporary GPU? Quote Link to comment
SpaceInvaderOne Posted July 8, 2017 Author Share Posted July 8, 2017 On 06/07/2017 at 11:35 PM, CloseTurkey said: Does anyone have a gtx 980 rom dump? Or is there any way to get a rom dump in my setup without a 2nd temporary GPU? I dont have a 980 vbios but i made a video a while back about how to get a working vbios without a second gpu. 1 Quote Link to comment
CloseTurkey Posted July 17, 2017 Share Posted July 17, 2017 I'm having no luck getting my gtx 980 passed through. Im struggling to locate a second card perform a vbios dump like you demonstrated in the video. I've even put together an old rig only to find out it doesn't support IOMMU. So i've ended up trying multiple different vbios from the techpowerup.com to no avail. I'm really at a loss here and pulling my hair out, my free trial runs out in a few days and i'd really like to get my windows vm with gpu passthrough working so i can continue with my project. I've posted my machine spec and xml below in case I've done something glaringly obviously wrong, any help appreciated, thanks. Machine Spec: Model: Custom M/B: ASUSTeK COMPUTER INC. - X99-DELUXE II CPU: Intel® Core™ i7-6800K CPU @ 3.40GHz HVM: Enabled IOMMU: Enabled Cache: 384 kB, 1536 kB, 15360 kB Memory: 32 GB (max. installable capacity 512 GB) Network: bond0: fault-tolerance (active-backup), mtu 1500 eth0: 1000 Mb/s, full duplex, mtu 1500 eth1: not connected Kernel: Linux 4.9.30-unRAID x86_64 XML <domain type='kvm'> <name>Windows 10</name> <uuid>943beee4-6867-a88e-a068-e5aed337923e</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>22020096</memory> <currentMemory unit='KiB'>22020096</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='3'/> <vcpupin vcpu='4' cpuset='6'/> <vcpupin vcpu='5' cpuset='7'/> <vcpupin vcpu='6' cpuset='8'/> <vcpupin vcpu='7' cpuset='9'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/943beee4-6867-a88e-a068-e5aed337923e_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='4' threads='2'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Win10_1607_English_x64.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.126-2.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk1.img'/> <target dev='hdc' bus='sata'/> <boot order='1'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <controller type='usb' index='0' model='nec-xhci'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:ea:e7:f3'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/Drivers/gtx980-141103.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc52e'/> </source> <address type='usb' bus='0' port='1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0b05'/> <product id='0x180a'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x2433'/> <product id='0xb200'/> </source> <address type='usb' bus='0' port='3'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </memballoon> </devices> </domain> Quote Link to comment
SpaceInvaderOne Posted July 17, 2017 Author Share Posted July 17, 2017 (edited) Hi @CloseTurkey When you used the techpowerup vbioses did you remove the Nvidia header as in my video I linked to you before? Please let me know what errors you have. Or does the vm just start and you have a black screen? Anyway, I have edited an Evga 980 4 gig vbios for you to try linked here. Give this a try. Your XML looks ok EVGA.GTX980.4096.rom Edited July 17, 2017 by gridrunner Quote Link to comment
CloseTurkey Posted July 17, 2017 Share Posted July 17, 2017 Hi @gridrunner I feel so stupid, i watched it on my iPhone when you posted it and inadvertently watched the posts original video again. Come to think of it i did think it was strange to post the same video twice ?? Thanks for your prompt response, i will try the rom you attached tomorrow and let you know how i get on. Quote Link to comment
CloseTurkey Posted July 18, 2017 Share Posted July 18, 2017 (edited) No luck so far, I'm getting the error log below. VM starts up and stuck with black screen LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-Windows 10/master-key.aes' -machine pc-i440fx-2.7,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/943beee4-6867-a88e-a068-e5aed337923e_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 21504 -realtime mlock=off -smp 8,sockets=1,cores=4,threads=2 -uuid 943beee4-6867-a88e-a068-e5aed337923e -display none -no-user-config -nodefaults -chardev 'socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-1-Windows 10/monitor.sock,server,nowait' -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device nec-usb-xhci,i0,server,nowait' -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x5,romfile=/mnt/user/drivers/EVGA.GTX980.4096.rom -device vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x6 -device usb-host,hostbus=1,hostaddr=6,id=hostdev2,bus=usb.0,port=2 -device usb-host,hostbus=1,hostaddr=7,id=hostdev3,bus=usb.0,port=3 -device usb-host,hostbus=1,hostaddr=2,id=hostdev4,bus=usb.0,port=4 -device usb-host,hostbus=1,hostaddr=5,id=hostdev5,bus=usb.0,port=1.1 -device usb-host,hostbus=1,hostaddr=3,id=hostdev6,bus=usb.0,port=1.2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=onDomain id=1 is tainted: high-privilegesDomain id=1 is tainted: host-cpuchar device redirected to /dev/pts/0 (label charserial0) Edited July 18, 2017 by CloseTurkey correcting logs Quote Link to comment
SpaceInvaderOne Posted July 18, 2017 Author Share Posted July 18, 2017 (edited) Um @CloseTurkeyThis is strange. You have a similar server to me with an x99 board and Nvidia GPU. If it's a black screen then it is going to be the vbios of the card. Something is wrong with the file. Maybe I edited it wrong as I was tired when I did it. Try editing a vbios yourself from techpowerup. What is the exact model of the gpu you are using. Also please post your iommu groups and device here. Another thing to try. Please make a new windows 10 vm with seabios as bios type and passthrough the romfile. I expect you will get the same result. But on the off chance the vbios you are using doesn't support UEFI boot that would cause a black screen when using ovmf. Also, your card is definitely a 980 not a 980ti as the 980 has 4 gigs of ram and 980 ti 6 gigs , the bios is different? Anyway heres another 980 bios. 980vbios Ps Contact Limetech and they will give you an extension on your trial Edited July 18, 2017 by gridrunner Quote Link to comment
CloseTurkey Posted July 19, 2017 Share Posted July 19, 2017 (edited) Hi gridrunner, Managed to get passthrough working tonight, the frustration got too much last night and i purchased a GT 710 on prime now to arrive today ? Still not sure what was causing the passthrough to fail with a single gpu. Tried editing several vbios from techpowerup with no success and I'm wondering if it was the vbios or the actual passthrough? I'm not really sure where to go with diagnosing this issue but so far I've taken a dump straight from the GTX 980 so I may take out the GT 710 - I jumped the gun here, dump size is 4 bytes and try passthrough with a single card again, it would be nice to just have the one in the rig Quote What is the exact model of the gpu you are using. Also please post your iommu groups and device here. EVGA GeForce GTX 980 SC ACX2.0 Not sure how to get my IOMMU groups and device, I'm still pretty new to all of this Quote Ps Contact Limetech and they will give you an extension on your trial Thanks, i'll contact them in the next couple days. Now the real work starts now I'm partially over that hurdle Edited July 19, 2017 by CloseTurkey update Quote Link to comment
SpaceInvaderOne Posted July 20, 2017 Author Share Posted July 20, 2017 16 hours ago, CloseTurkey said: Hi gridrunner, Managed to get passthrough working tonight, the frustration got too much last night and i purchased a GT 710 on prime now to arrive today ? Still not sure what was causing the passthrough to fail with a single gpu. Tried editing several vbios from techpowerup with no success and I'm wondering if it was the vbios or the actual passthrough? I'm not really sure where to go with diagnosing this issue but so far I've taken a dump straight from the GTX 980 so I may take out the GT 710 - I jumped the gun here, dump size is 4 bytes and try passthrough with a single card again, it would be nice to just have the one in the rig EVGA GeForce GTX 980 SC ACX2.0 Not sure how to get my IOMMU groups and device, I'm still pretty new to all of this Thanks, i'll contact them in the next couple days. Now the real work starts now I'm partially over that hurdle To check the iommu groups is really easy. Just goto tools system devices. Here i have 2 GPUs in my rig a 1080 (group 32) and a 750 ti (group 30). You will see each is in its own iommu group with its sound counterpart. If you have more than this in an iommu that your GPU is in passthrough will not work correctly. That's why I wanted to see yours when you have just the one GPU installed. Quote Link to comment
CloseTurkey Posted July 21, 2017 Share Posted July 21, 2017 ok so here is what I've found with one gpu lspci - v gives me this 04:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 980] (rev a1) (prog-if 00 [VGA controller]) Subsystem: eVga.com. Corp. GM204 [GeForce GTX 980] Flags: fast devsel, IRQ 11, NUMA node 0 Memory at f8000000 (32-bit, non-prefetchable) [disabled] Memory at b0000000 (64-bit, prefetchable) [disabled] Memory at c0000000 (64-bit, prefetchable) [disabled] I/O ports at d000 [disabled] Expansion ROM at f9000000 [disabled by cmd] Capabilities: [60] Power Management version 3 Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [78] Express Legacy Endpoint, MSI 00 Capabilities: [100] Virtual Channel Capabilities: [250] Latency Tolerance Reporting Capabilities: [258] L1 PM Substates Capabilities: [128] Power Budgeting <?> Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?> Capabilities: [900] #19 04:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1) Subsystem: eVga.com. Corp. GM204 High Definition Audio Controller Flags: bus master, fast devsel, latency 0, IRQ 10, NUMA node 0 Memory at f9080000 (32-bit, non-prefetchable) Capabilities: [60] Power Management version 3 Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [78] Express Endpoint, MSI 00 and IOMMU groups are this IOMMU group 26 [8086:8d47] 00:1f.0 ISA bridge: Intel Corporation C610/X99 series chipset LPC Controller (rev 05) [8086:2822] 00:1f.2 RAID bus controller: Intel Corporation SATA Controller [RAID mode] (rev 05) [8086:8d22] 00:1f.3 SMBus: Intel Corporation C610/X99 series chipset SMBus Controller (rev 05) IOMMU group 27 [10de:13c0] 04:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 980] (rev a1) [10de:0fbb] 04:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1) IOMMU group 28 [1b21:1242] 05:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller IOMMU group 29 [1b21:1242] 06:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller Logs SeaBIOS Jul 21 11:44:29 Tower kernel: input: Logitech USB Optical Mouse as /devices/pci0000:00/0000:00:14.0/usb1/1-9/1-9.3/1-9.3:1.0/0003:046D:C077.000A/input/input14Jul 21 11:44:29 Tower kernel: hid-generic 0003:046D:C077.000A: input,hidraw3: USB HID v1.11 Mouse [Logitech USB Optical Mouse] on usb-0000:00:14.0-9.3/input0Jul 21 11:44:41 Tower kernel: vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+memJul 21 11:44:41 Tower kernel: br0: port 2(vnet0) entered blocking stateJul 21 11:44:41 Tower kernel: br0: port 2(vnet0) entered disabled stateJul 21 11:44:41 Tower kernel: device vnet0 entered promiscuous modeJul 21 11:44:41 Tower kernel: br0: port 2(vnet0) entered blocking stateJul 21 11:44:41 Tower kernel: br0: port 2(vnet0) entered forwarding stateJul 21 11:44:42 Tower kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e@0x258Jul 21 11:44:42 Tower kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900Jul 21 11:44:42 Tower kernel: vfio-pci 0000:00:1b.0: enabling device (0000 -> 0002)Jul 21 11:44:42 Tower acpid: input device has been disconnected, fd 6Jul 21 11:44:42 Tower acpid: input device has been disconnected, fd 7Jul 21 11:44:44 Tower kernel: kvm: zapping shadow pages for mmio generation wraparoundJul 21 11:44:49 Tower kernel: usb 1-9.3: reset low-speed USB device number 11 using xhci_hcdJul 21 11:44:50 Tower kernel: usb 1-9.3: reset low-speed USB device number 11 using xhci_hcdJul 21 11:44:50 Tower kernel: usb 1-9.2: reset low-speed USB device number 6 using xhci_hcdJul 21 11:44:51 Tower kernel: usb 1-9.2: reset low-speed USB device number 6 using xhci_hcdJul 21 11:44:51 Tower kernel: kvm: zapping shadow pages for mmio generation wraparound VM Logs 2017-07-21 18:44:41.766+0000: starting up libvirt version: 2.4.0, qemu version: 2.7.1, hostname: TowerLC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10 Gaming,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-Windows 10 Gaming/master-key.aes' -machine pc-i440fx-2.7,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -m 8192 -realtime mlock=off -smp 4,sockets=1,cores=2,threads=2 -uuid 3e3a2e1e-bfa3-59e0-2bde-cab34c96cd5f -display none -no-user-config -nodefaults -chardev 'socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-2-Windows 10 Gaming/monitor.sock,server,nowait' -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device nec-usb-xhci,id=usb,bus=pci.0,addr=0x7 -device ahci,id=sata0,bus=pci.0,addr=0x3 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive 'file=/mnt/user/domains/Windows 10 Gaming/vdisk1.img,format=r=0x6,romfile=/mnt/user/drivers/test.dump -device vfio-pci,host=00:1b.0,id=hostdev1,bus=pci.0,addr=0x8 -device usb-host,hostbus=1,hostaddr=11,id=hostdev2,bus=usb.0,port=1 -device usb-host,hostbus=1,hostaddr=6,id=hostdev3,bus=usb.0,port=2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x9 -msg timestamp=onDomain id=2 is tainted: high-privilegesDomain id=2 is tainted: host-cpuchar device redirected to /dev/pts/0 (label charserial0) LOGS OVMF Jul 21 11:52:33 Tower kernel: usb 1-9.3: USB disconnect, device number 13Jul 21 11:52:34 Tower kernel: usb 1-9.3: new low-speed USB device number 14 using xhci_hcdJul 21 11:52:34 Tower kernel: input: Logitech USB Optical Mouse as /devices/pci0000:00/0000:00:14.0/usb1/1-9/1-9.3/1-9.3:1.0/0003:046D:C077.0010/input/input20Jul 21 11:52:34 Tower kernel: hid-generic 0003:046D:C077.0010: input,hidraw3: USB HID v1.11 Mouse [Logitech USB Optical Mouse] on usb-0000:00:14.0-9.3/input0Jul 21 11:52:57 Tower kernel: vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+memJul 21 11:52:57 Tower kernel: br0: port 2(vnet0) entered blocking stateJul 21 11:52:57 Tower kernel: br0: port 2(vnet0) entered disabled stateJul 21 11:52:57 Tower kernel: device vnet0 entered promiscuous modeJul 21 11:52:57 Tower kernel: br0: port 2(vnet0) entered blocking stateJul 21 11:52:57 Tower kernel: br0: port 2(vnet0) entered forwarding stateJul 21 11:52:59 Tower kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e@0x258Jul 21 11:52:59 Tower kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900Jul 21 11:52:59 Tower kernel: vfio-pci 0000:00:1b.0: enabling device (0000 -> 0002)Jul 21 11:52:59 Tower acpid: input device has been disconnected, fd 6Jul 21 11:52:59 Tower acpid: input device has been disconnected, fd 7Jul 21 11:53:06 Tower kernel: usb 1-9.3: reset low-speed USB device number 14 using xhci_hcdJul 21 11:53:07 Tower kernel: usb 1-9.3: reset low-speed USB device number 14 using xhci_hcdJul 21 11:53:08 Tower kernel: usb 1-9.2: reset low-speed USB device number 6 using xhci_hcdJul 21 11:53:08 Tower kernel: usb 1-9.2: reset low-speed USB device number 6 using xhci_hcdJul 21 11:53:10 Tower kernel: kvm: zapping shadow pages for mmio generation wraparoundJul 21 11:53:10 Tower kernel: kvm: zapping shadow pages for mmio generation wraparound VM Logs 2017-07-21 18:52:57.121+0000: starting up libvirt version: 2.4.0, qemu version: 2.7.1, hostname: TowerLC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-Windows 10/master-key.aes' -machine pc-i440fx-2.7,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/988a76c3-d21b-e56b-8e8c-a0f7c78f1c24_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 20480 -realtime mlock=off -smp 8,sockets=1,cores=4,threads=2 -uuid 988a76c3-d21b-e56b-8e8c-a0f7c78f1c24 -display none -no-user-config -nodefaults -chardev 'socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-Windows 10/monitor.sock,server,nowait' -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device nec-usb-xhci,i,nowait' -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x6,romfile=/mnt/user/drivers/test.dump -device vfio-pci,host=00:1b.0,id=hostdev1,bus=pci.0,addr=0x8 -device usb-host,hostbus=1,hostaddr=14,id=hostdev2,bus=usb.0,port=1 -device usb-host,hostbus=1,hostaddr=6,id=hostdev3,bus=usb.0,port=2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x9 -msg timestamp=onDomain id=3 is tainted: high-privilegesDomain id=3 is tainted: host-cpuchar device redirected to /dev/pts/0 (label charserial0)2017-07-21T18:54:12.473689Z qemu-system-x86_64: libusb_release_interface: -4 [NO_DEVICE] I performed a cat of the rom with the GTX980 in the second pci slot which worked successfully (attached). I then tested that rom by editing the xml on 2 seperate vms. one with SeaBIOS and the other OVMF, both of which gave me a black screen. Ive tried this with the GTX 980 in the primary PCI slot and the secondary slot with the same results. The vms tested above do work with passthrough when there are 2 GFX cards in the rig, when GT 710 is in the primary PCI slot and theGTX 980 in the secondary . Sorry for the lengthy post. TL;DR Cannot get to work with 1 GPU and rom dump. Will work with 2 GPU's GTX980test.dump Quote Link to comment
lionceau Posted July 26, 2017 Share Posted July 26, 2017 It looks like somebody found a way to patch the Nvidia driver to remove the virtualization check that causes Error 43. Perhaps this is helpful for some edge cases where people can't dump their BIOS for whatever reason. https://github.com/sk1080/nvidia-kvm-patcher I have not tried this, attempt at your own risk. Quote Link to comment
Matoking Posted August 1, 2017 Share Posted August 1, 2017 (edited) I was pointed towards this thread when I had trouble isolating my 1070 for PCI passthrough. Long story short, I tried dumping my vBIOS like instructed in the video, but couldn't do so (the `cat` command printed I/O errors instead). Instead, I resorted to dumping the full vBIOS under Windows and using a hex editor to splice the relevant part of the ROM into a new file, using some of the partial vBIOS files uploaded here as samples. This finally allowed me to pass the GPU to the Windows VM! --- Anyway, I wrote a Python script that should automate this process (you give it a full ROM from techPowerUp or one you dumped using nvflash under Windows), and it should create a patched ROM that you can use to make GPU passthrough work. I passed a few ROMs I downloaded from techPowerUp through the script and compared them to what you guys uploaded here, and so far the Pascal vBIOS files appeared to match, bit by bit. Still, I can't stress it enough that this script is based on guesswork, so it may end up bricking your GPU if you're unlucky. It does a few rudimentary sanity checks, but I would recommend dumping the partial ROM yourself if you can. Still, for those who are pulling your hair out over not being able to do that, this may be a lifesaver. https://github.com/Matoking/NVIDIA-vBIOS-VFIO-Patcher Edited August 1, 2017 by Matoking 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.